You are here

  • Home
  • Archive
  • Volume 13, Issue 1
  • Implementation of a quality improvement programme using the Active Patient Link call and recall system to improve timeliness and equity of childhood vaccinations: protocol for a mixed-methods evaluation

Article Text

Article menu

Download PDFPDF

Immunology (including allergy)
Protocol
Implementation of a quality improvement programme using the Active Patient Link call and recall system to improve timeliness and equity of childhood vaccinations: protocol for a mixed-methods evaluation
  1. Milena Marszalek1,
  2. Meredith K D Hawking1,
  3. Ana Gutierrez1,
  4. Isabel Dostal1,
  5. Zaheer Ahmed1,
  6. Nicola Firman1,
  7. John Robson1,
  8. Helen Bedford2,
  9. Anna Billington1,
  10. Ngawai Moss1,
  11. Carol Dezateux1
  1. 1Wolfson Institute of Population Health, Yvonne Carter Building, Queen Mary University, London, UK
  2. 2Great Ormond Street Institute of Child Health, University College London, London, UK
  1. Correspondence to Dr Milena Marszalek; m.marszalek{at}qmul.ac.uk

Abstract

Introduction Call and recall systems provide actionable intelligence to improve equity and timeliness of childhood vaccinations, which have been disrupted during the COVID-19 pandemic. We will evaluate the effectiveness, fidelity and sustainability of a data-enabled quality improvement programme delivered in primary care using an Active Patient Link Immunisation (APL-Imms) call and recall system to improve timeliness and equity of uptake in a multiethnic disadvantaged urban population. We will use qualitative methods to evaluate programme delivery, focusing on uptake and use, implementation barriers and service improvements for clinical and non-clinical primary care staff, its fidelity and sustainability.

Methods and analysis This is a mixed-methods observational study in 284 general practices in north east London (NEL). The target population will be preschool-aged children eligible to receive diphtheria, tetanus and pertussis (DTaP) or measles, mumps and rubella (MMR) vaccinations and registered with an NEL general practice. The intervention comprises an in-practice call and recall tool, facilitation and training, and financial incentives. The quantitative evaluation will include interrupted time Series analyses and Slope Index of Inequality. The primary outcomes will be the proportion of children receiving at least one dose of a DTaP-containing or MMR vaccination defined, respectively, as administered between age 6 weeks and 6 months or between 12 and 18 months of age. The qualitative evaluation will involve a ‘Think Aloud’ method and semistructured interviews of stakeholders to assess impact, fidelity and sustainability of the APL-Imms tool, and fidelity of the implementation by facilitators.

Ethics and dissemination The research team has been granted permission from data controllers in participating practices to use deidentified data for audit purposes. As findings will be specific to the local context, research ethics approval is not required. Results will be disseminated in a peer-reviewed journal and to stakeholders, including parents, health providers and commissioners.

  • IMMUNOLOGY
  • Public health
  • Community child health
  • Paediatric infectious disease & immunisation
  • Quality in health care
  • PREVENTIVE MEDICINE
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2022-064364

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    STRENGTHS AND LIMITATIONS OF THIS STUDY

    • The intervention will be applied to an entire population eligible to receive vaccinations without charge, indicating the impact in a socioeconomically and ethnically diverse setting.

    • Availability of high-quality, real-time data for the evaluation and mixed-methods design enabling process evaluation as well as insights into effective facilitation.

    • This evaluation will provide specific as well as generic learning about the adoption of data-enabled innovative tools and their use in practice.

    • While the study is observational and not randomised, robust and established methods for observational studies will be used to compare preimplementation and postimplementation phases and to assess equity.

    • Potential limitations include risk of selection bias in the qualitative evaluation, which may be influenced by contextual factors.

    Introduction

    Childhood vaccinations are among the most effective public health interventions and protect children from a range of infections and their consequences.1 The WHO has specified childhood vaccination coverage targets, which are to be met in order to ensure population immunity, protect children from infections and enable elimination of infections such as measles.2

    These targets have not been achieved in England in recent years.3 There are marked and avoidable geographic and sociodemographic inequalities in coverage of childhood vaccinations, most notably for measles, mumps and rubella (MMR) vaccinations.4 The UK WHO measles elimination status, awarded in 2017, was withdrawn in 20192 following a sharp rise in 2018 in confirmed measles cases in England.5 Of the 971 children who contracted measles in the 2018 outbreak, 40% lived in London.5 This has been further exacerbated by the COVID-19 pandemic,6 with the latest evidence suggesting that—on average—MMR vaccine coverage measured at 24 months fell by 0.3% in England to 90.3%.3 7 This is consistent with findings from a study of general practices in England, however, this did not include many practices in London, which has historically the lowest childhood vaccination rates in the UK.8 We have recently assessed the impact of the COVID-19 pandemic in north east London (NEL), and found a 4% decrease in MMR timeliness in 2020 compared with the previous year, and marked inequalities between children living in the least and most deprived areas.9

    The COVID-19 pandemic has highlighted the limitations of the national Cover of Vaccination Evaluated Rapidly system, which is a retrospective measure of coverage at defined ages. This is not a suitable metric for outbreaks and pandemics, which depend on rapid assessment of the impact on current vaccination status and of the effectiveness of interventions to target low uptake.3 There is increasing interest in using timeliness as a more actionable and time-sensitive measure of vaccine uptake, as this assesses whether children are optimally protected,10 11 and their current status.11 When used in real time, routine electronic health records providing coded data on current vaccination status can be integrated into call and recall systems to identify children who are partially vaccinated or completely unvaccinated.10 In recognition of this, in 2021, National Health Service (NHS) England announced new targets for childhood vaccination and immunisation services as part of the general practice Quality and Outcomes Framework (QOF): from April 2021 these incentivised administration of preschool childhood vaccinations.12 13 These nationally set targets are challenging for regions with historically low update, such as London. The introduction of this QOF also coincides with a major NHS reorganisation. Previous large scale health system reorganisations have been shown to result in fragmentation of roles and responsibilities and to impact negatively on the effective delivery of childhood vaccination services in England.14

    In combination, these factors risk widening existing inequity in routine childhood vaccinations in London. There is strong evidence that call and recall systems are effective in increasing uptake and reducing inequalities in uptake.15 In the UK, childhood vaccinations are delivered by primary healthcare staff working in general practice and recorded in real time in electronic patient records.16 The Clinical Effectiveness Group (CEG) at Queen Mary University of London has had notable success in supporting general practice teams in NEL to provide equitable access to effective healthcare for its ethnically diverse and disadvantaged population through data-enabled and facilitated quality improvement (QI) programmes.17 18

    There are a variety of reasons for non-vaccination, with vaccine hesitancy due to social and cultural attitudes often cited. However, research among the Charedi Orthodox Jewish community in Hackney, east London, found that access to, and convenience of, immunisation services were important factors in facilitating vaccine uptake.19 Two earlier studies in NEL have demonstrated improvements in child and adult vaccination uptake through innovative use by healthcare providers of data-enabled in-practice call and recall systems.20 21 Recent innovations include the use of Active Patient Link (APL) call and recall tools which—together with standardised data entry templates to ensure high-quality data are captured at the point of care, training and facilitation—actively support population health management in real time.22 We have adapted this concept and developed the APL-Immunisation (APL-Imms) in-practice call and recall tool, which is designed to support timely preschool routine childhood vaccination in general practices within the context of a childhood vaccination QI programme. We describe this programme—which focuses on improving vaccine provider services for preschool childhood immunisations—and present a protocol to evaluate its implementation in general practices in NEL as part of a London-wide health data strategy pathfinder project.23

    Purpose of the study

    Our overarching aim is to assess the effectiveness of a data-enabled childhood vaccination QI programme focused on provider-related factors and using the APL-Imms call and recall tool in improving timeliness and reducing inequalities in childhood vaccinations for the population of children registered with all general practices in NEL. Specifically, we will assess whether this programme improves the timeliness, and reduces ethnic and socioeconomic inequalities, in receipt of at least one dose of diphtheria, tetanus and pertussis (DTaP) containing or MMR vaccinations. We will use qualitative methods to evaluate the delivery of the programme, focusing on the uptake and use, implementation barriers and service improvements for clinical and non-clinical primary care staff of the APL-Imms QI programme and its fidelity and sustainability.

    Vaccination QI programme: target audience, components and implementation plan

    Target audience

    NEL comprises a population of around 2 million24–26 served by 284 general practices which together provide vaccination services to approximately 150 000 children under the age of 5 years living in one of eight localities: Newham, City and Hackney, Tower Hamlets, Waltham Forest, Barking and Dagenham, Havering, and Redbridge.24 Approximately, 29 000 babies who will require vaccination are born each year to women registered with these practices. This is an ethnically diverse population with language barriers and socioeconomic deprivation predisposing to vaccination inequalities.24 25

    The childhood vaccination QI programme

    The QI programme comprises three elements: an APL-Imms call and recall tool which enables practices to stratify preschool aged children on the practice register according to their current vaccination status and to undertake a ‘virtual patient review’ of individual children17; training and educational materials delivered by facilitators to support practices in adopting and using the tool; and financial incentives to encourage practices to deliver timely vaccinations. The target audience for the APL-Imms tool is the primary care team. Specific staff engaging with the programme will vary according to local general practice staffing arrangements and resources since routine recall is performed by different staff members in different localities and practices. The overall user group is expected to include clinical (nurses, healthcare assistants) and non-clinical (reception and administrative staff, practice managers) staff responsible for arranging appointments for routine childhood vaccinations.25

    APL-Imms call and recall tool

    The APL-Imms call and recall tool was developed by the CEG data analyst (ID) and senior informatician (ZA) with input from CEG clinical staff and facilitators. Initial versions were tested and refined with practitioners and facilitators to ensure that user requirements were met. It is designed to work on the two main electronic record provider systems in use across the UK (Egton Medical Information Systems (EMIS) Web, EMIS Health; SystmOne, The Phoenix Partnership). At the time of writing, the EMIS version has been released and is available from the CEG website (https://www.qmul.ac.uk/blizard/ceg/realhealth/software-tools/aplimms/) with the SystmOne version to follow.

    The first step requires practice staff to run a standardised search each week of their identifiable practice patient records held in the practice electronic patient record system and to import the results of this search into the APL-Imms tool, which is installed on the practice computer system. The APL-Imms tool displays this up-to-date information on vaccine status of patients in two dashboards. The first enables practices to create and display lists of children who are due or overdue any selected vaccines on a particular date by using age and specific vaccine date filters (figure 1) and the second enables a virtual patient review of any child selected from the first dashboard. Using the first dashboard, practice staff can prioritise vaccination appointments based on timeliness, for example, by identifying all children who are due a specific vaccination in the coming week, or who are not protected because their vaccination is delayed. From this dashboard, staff can export lists of children due appointments for import into automated text messaging systems used by the practice to contact parents or carers with appointment reminders or other information. They can also export a more detailed set of patient level information into a csv file, including previous immunisations, recorded declines or adverse reactions, ethnic group, safeguarding flags and contact details such as mobile phone numbers. This can be used to manage phoning of patients or for the team meetings to review other actions for children who have not been brought to their appointments on several occasions. From the patient list displayed on the first dashboard, practice staff can open a second ‘virtual patient review’ page for any selected child in the first dashboard without leaving the tool. This second dashboard summarises the entire vaccination record for that child including: vaccinations given and due, vaccination declines, adverse reactions or contraindications, age ineligibility for vaccinations and appropriate time periods between vaccinations within the routine schedule (figure 2). It also identifies missing demographic information and provides up-to-date links to national vaccination policies and advice.

    Figure 1
    Figure 1

    APL-Imm call and recall tool—front page dashboard. APL-Imm, Active Patient Link Immunisation.

    Figure 2
    Figure 2

    APL-Imm call and recall tool—virtual patient review dashboard. APL-Imm, Active Patient Link Immunisation.

    Training and education materials

    Training and education materials have been developed to train facilitators and childhood vaccination coordinators who provide information and practical support for practices. These include printed and web-based documents and video materials, the latter available from the CEG website and CEG YouTube channel (https://www.qmul.ac.uk/blizard/ceg/childhood-immunisations/apl-imms-tool-guidance/). CEG facilitators flag the availability of the QI programme and APL-Imms tool to practices at their periodic routine contacts and offer training and support for practice-based staff and through educational online meetings held in each locality. Childhood immunisation coordinators also provide support with practice visits, for example, by targeting support to practices with lower vaccination uptake (figure 3). Further information, including on any updates to the tool, are also communicated to practice staff through the CEG weekly borough-specific email bulletin to all practices.

    Figure 3
    Figure 3

    Summary of phased approach to quality improvement programme implementation.

    Financial incentives

    From April 2021, routine childhood vaccinations are being incentivised financially through NHS England’s QOF using three mechanisms. The first is an item of service (IoS) payment for each vaccination or immunisation, given regardless of age at receipt. The second awards payments on a sliding scale depending on thresholds for targets related to three preschool vaccination indicators (table 1). Payments require a target of 90% to be met for two of the indicators, and 87% for the third. The third penalises practices with uptake below 80% who will forfeit IoS payments equivalent to half of their eligible population. Further details of the payment scheme can be in found in reference.26 These targets are generally only attained by the highest achieving practices and, as most practices in NEL are unlikely to meet thresholds for payment, at the time of writing a local incentive scheme is being considered for NEL, which aims to incentivise practices to reach more attainable targets.

    View this table:
    Table 1

    Quality outcomes framework for childhood vaccinations and immunisations: target definitions and payment thresholds12 26

    Implementation

    The programme will be implemented using a phased approach across all areas of NEL. All practices have been contacted to flag the release date and availability of the APL-Imms tool and invited to attend user webinars and to download the APL-Imms tool from the CEG website. When a user requests the tool, they submit a form which is recorded on an online platform, recording all submissions and requests. The facilitators can request a visit through the same form. CEG facilitators and childhood immunisation coordinators provide support with practice visits (figure 3).

    Methods and analysis

    Study sample

    We will create monthly cohorts of children eligible to receive their first DTaP and MMR vaccinations in that month between 1 January 2019 to July 2023 and extract data on primary outcomes from their primary care electronic health records.

    Primary outcomes

    The primary outcomes for the quantitative evaluation are timely receipt of at least one dose of a DTaP containing vaccine and MMR vaccine defined, respectively, as administered between age 6 weeks and 6 months and between 12 and 18 months of age. Children not receiving at least one dose of a DTaP containing vaccine by 6 months will be unable to meet the first QOF target (VI001) as 1 month must elapse between successive doses. The qualitative evaluation will assess the fidelity and sustainability of the APL-Imms tool for local stakeholders and the fidelity of the implementation process of the APL-Imms tool QI programme by facilitators.

    Data source, extraction and postprocessing

    We will extract data from the NEL Discovery Programme Compass database, which comprises deidentified coded primary care data from all 284 general practices providing routine childhood vaccination services in the geographically contiguous areas within NEL. Compass includes coded information on routine childhood vaccinations due in the first 5 years of life together with age at vaccination (to nearest week in first year of life and to nearest month thereafter) and is updated daily.

    Data for the primary outcomes will be extracted on the first day of each calendar month (referred to as the run date) from January 2018 to June 2023 inclusive for all children registered with any of the NEL general practices on the run date and eligible to be immunised. For DTaP and MMR, eligible children are those turning 6 and 18 months, respectively, in the calendar month preceding the run date. The following records will be excluded from analyses:

    • Records with an MMR clinical code but no date.

    • Records with a DTaP clinical code but no date.

    • Exact duplicate records.

    • Duplicate records where the date is the same, but MMR clinical code differs.

    • Duplicate records where the date is the same but DTaP clinical code differs.

    • Latter of multiple MMR events

    • Latter of multiple DTaP events.

    The programme implementation commenced in February 2022 with release of the EMIS version of the APL-Imms tool and will run over a 2-year period initially. The preimplementation period will start in January 2018 and the postimplementation phase from the point at which 80% of practices have downloaded the tool (estimated to be May 2023). We will create monthly observations for the preimplementation, implementation and postimplementation phases, respectively. The date of intervention will be the recorded date of APL-Imms download for each practice.

    Explanatory variables

    We will assign a 2015 Index of Multiple Deprivation (IMD) score, an area-based measure of socioeconomic deprivation27 obtained by linking information from the Decennial Census, to the 2011 Lower Super Output Area (LSOA) of registered patient addresses. We will allocate each LSOA a ranked status based on the IMD, with quintile 1 being the most, and quintile 5 the least, deprived. We will use NHS Digital Organisation Data Service codes to identify individual general practices. We will assign ethnicity based on parent/carer report of child ethnicity coded using Office for National Statistics codes28 and grouped into six categories (white, black African/Caribbean/black British, South Asian (Bangladeshi, Pakistani, Indian)), other (including mixed race and Chinese), not reported or missing.

    Statistical analysis plan

    We will describe the baseline demographic and clinical characteristics of the study sample (table 2). We will conduct an interrupted time series (ITS) analysis to compare primary outcomes in the preimplementation and postimplementation periods.29 Trends over time in the proportion of children receiving timely vaccinations will be modelled using Joinpoint regression, which finds the best fit for points of change in trend.30 We will examine consistency of findings across these two approaches. Predicted values will be generated based on a Poisson segmented regression model and plotted to create the a priori model. Joinpoint regression will also be used to guide the number of inflection points within the a priori model.29 The counterfactual scenario will be plotted by creating a data frame as if the programme had never been implemented. Predictions will be generated under the counterfactual scenario and added to the plot.29 We will assess autocorrelation and allow for any overdispersion using a quasi-Poisson model. We will check the unadjusted and adjusted prediction models by plotting residuals against time and by examining autocorrelation and partial autocorrelation functions. We will use harmonic terms specifying the number of sin and cosine pairs and period length to adjust for time-varying confounders including seasonality.

    View this table:
    Table 2

    Description of preimplementation characteristics by vaccination cohorts

    We will perform a sensitivity analysis by lowering the age threshold of the primary outcomes to 14 months for MMR and 12 weeks for DTaP.

    We will assess the impact of the QI programme on inequalities in timely receipt of vaccines by measuring the Slope Index of Inequality for the primary outcomes by deprivation quintile and testing for significant variation between quintiles.31–33 All analyses and data visualisations will be coded in R34 and made available on publication. We estimate, using simulation-based power calculations previously reported for ITS studies35 36 that our study will have sufficient sample size and data points to provide 80% power to detect a 10% difference following implementation significant at the 5% level with an autocorrelation coefficient of 0.

    Qualitative evaluation

    The implementation strategy for this QI programme is underpinned by normalisation process theory,37 which describes how practices can become routinely engrained within a social context and identifies aspects of an intervention that make it feasible for stakeholders.38 It is based on two elements: ‘normalisation’ that refers to the routine embedding of an organisational practice into everyday life and ‘process’ that refers to patterns of interactions between objects that give capabilities, agents that make contributions, and contexts that confer capacity and potential.37

    The qualitative evaluation comprises three components and will explore how the QI programme is implemented as part of an existing service and any facilitators or barriers that impact its use in clinical care in NEL. We will recruit practices via the CEG weekly email newsletter and contacts within the CEG user group. Practices will be purposively sampled from a range of primary care networks, ensuring representation of practices with a diverse experience of QI projects. Within those practices, purposive sampling of staff to include clinical and non-clinical roles will be performed with the intention that participants will take part in all three stages of data collection. The number of participants recruited for this evaluation will be based on skill-sets relevant to APL-Imms tool use and variety in location to capture variation in local organisational processes. We will draw on concepts of information power to guide our sample size aiming to recruit 8–10 participants.39

    The first stage comprises a face-to-face exercise lasting about 30 min using the ‘Think Aloud’ method to test the APL-Imms tool with practice staff who are using the tool for the first time. The ‘Think Aloud’ method is a method in which participants speak aloud any words in their mind as they complete a task. This gives insight into the processes of working memory and the thought processes of the participants completing the task and captures real time reactions to each stage of the intervention as detailed feedback on usability and usefulness.40 The second stage involves observing and recording an implementational webinar and capturing the queries and perspectives of the stakeholders on implementation of the tool in field notes. The third stage comprises semistructured interviews with the recruited staff members 6 months after the first stage when service delivery has become more embedded in routine practice. These will take around 30 min and will be performed remotely via telephone or video call.

    Data will be transcribed, and analysed iteratively, using thematic analysis techniques.41 Full coding will be performed by one member of the team with a second member analysing 10% of transcripts and with themes agreed between team members. Field notes and a reflexive diary will be kept. The findings will be triangulated against the quantitative findings. Additionally, the analysis will be compared with the ‘Big Tent’ framework, to assess for quality.42 The qualitative findings will be triangulated with the findings from the ITS analysis to give a complete picture of the success of the programme and to identify any improvements that may need to be made to the service.

    Patient and public involvement

    We have established an independent advisory panel which includes parents, carers, health professionals and other stakeholders. This group has reviewed this protocol. Public involvement and engagement will conform to best practice and principles defined in the UK Standards for Public Involvement.43 The project steering group includes a parent coinvestigator who is a member of Katie’s Team,44 an NEL patient and public advisory group of mothers and members of the public with a connection to evaluations relevant to childbirth, pregnancy and reproduction.

    Ethics and dissemination

    Permission has been granted by the NEL Discovery Programme Board acting on behalf of the data controllers to access deidentified data for this evaluation. As it is an evaluation of a service improvement using routinely collected and deidentified data and interviews of service providers, research ethics approval is not required.45 Funders will have no role in the decision to publish. The protocol adheres to the SQUIRE checklist for reporting QI Protocols.46 Results will be disseminated in a peer-reviewed journal and to stakeholders, including patients, health providers and commissioners.

    Discussion

    Globally, since 2016 and preceding the pandemic there has been an increase in measles cases driven mainly by outbreaks occurring in multiple countries including the UK and reflecting failure to vaccinate. There is clear evidence of a further detrimental impact of the pandemic on measles vaccination programmes, including in the UK. Mulholland et al had emphasised the importance of public health actions to deliver effective vaccination programmes and prepare for measles outbreaks.47 The proportion of children receiving measles vaccinations on time or at all in London is the lowest in Europe and on a par with many low income countries, with an average that conceals even wider inequalities at smaller geographic scales.48 Support for effective delivery of childhood vaccinations in general practice is key. The currently proposed national financial incentives are insufficient to achieve the momentum needed in London to avert a second pandemic of this highly contagious and deadly disease.12 There is strong evidence that call and recall systems are effective15 and a clinical effectiveness approach combining data-enabled practice-facing tools, training and facilitation to support healthcare providers and further financial incentives are key components to their success.

    This programme and its evaluation will make a vital contribution to the understanding around provider-led interventions that can successfully improve both timeliness and equity in childhood vaccination. The focus on both robust quantitative and qualitative methodology will give insight into how human and organisational factors contribute to changes in equitable vaccination uptake and can be sustained in practice. This programme has been supported through the London Health Data Strategy Programme, which aims to develop a data-enabled learning health system for health improvement and equity through pathfinder projects that are scalable across London. This evaluation will provide specific as well as generic learning about the adoption of data-enabled innovative tools and their use in practice for health improvement. Inequalities in childhood vaccination are not confined to London, and our findings will have relevance to policy and practice in other parts of the UK as well internationally. The in-practice tools we have developed for use in the two main primary care electronic patient record systems in use in the England will be made available without charge and, together with this evaluation, will enable improvement in the timeliness and equity of childhood vaccination services.

    Challenges to the success of the programme include the lack, at the time of writing, of meaningful financial incentives in areas with pre-existing low vaccination uptake, such as London. Strengths of our study include availability of high-quality, real-time data for the evaluation and use of a mixed-methods study design enabling process evaluation as well as insights into effective facilitation. Potential limitations include risk of selection bias in the qualitative evaluation, influenced by current clinical pressures on NHS staff.49 We also acknowledge that this QI programme does not specify how and with what information practice teams contact parents or the timing, location or ease of making appointments which are important factors in increasing access to services. We will take appropriate measures to ensure diversity of staff locations and backgrounds. The ethnic diversity of NEL may impact generalisability to areas with a different demography. The study is observational and not randomised, however, we will use robust and established methods for observational study designs to compare preimplementation and postimplementation phases.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    We are grateful to patients and general practitioners in East London for supporting this programme and the Discovery Programme data controllers for approving access to data. We would like to acknowledge the contributions of stakeholders, members of Katie’s Team and the CEG facilitators and analysts. This work uses data provided by patients and collected by the NHS as part of their care and support.

    References

    1. ↵
      1. The Lancet Infectious Diseases
      . The imperative of vaccination. Lancet Infect Dis 2017;17:1099. doi:10.1016/S1473-3099(17)30590-X
      OpenUrl
    2. ↵
      1. World Health Organization
      . Seventh meeting of the european regional verification commission for measles and rubella elimination (RVC); 2018.
    3. ↵
      1. Public Health England
      . Quarterly vaccination coverage statistics for children aged up to 5 years in the UK (COVER programme): April to June 2021. Contract no: 16; 2021.
    4. ↵
      1. Public Health England
      . National immunisation programme: health equity audit. London,
    5. ↵
    6. ↵
      1. Saxena S,
      2. Skirrow H,
      3. Bedford H
      . Routine vaccination during covid-19 pandemic response. BMJ 2020;369:m2392. doi:10.1136/bmj.m2392
    7. ↵
      1. NHS Digital
      . Childhood vaccination coverage statistics. 2021. Available: https://digital.nhs.uk/data-and-information/publications/statistical/nhs-immunisation-statistics
    8. ↵
      1. Public Health England
      . Impact of COVID-19 on childhood vaccination counts to week 4 in 2021, and vaccine coverage to decemeber 2020 in england: interim analyses. Contract no: 3; 2021.
    9. ↵
      1. Firman N,
      2. Marszalek M,
      3. Gutierrez A, et al
      . Impact of the COVID-19 pandemic on timeliness and equity of measles, mumps and rubella vaccinations in north east london: a longitudinal study using electronic health records. BMJ Open 2022;12:e066288. doi:10.1136/bmjopen-2022-066288
    10. ↵
      1. Walton S,
      2. Cortina-Borja M,
      3. Dezateux C, et al
      . Measuring the timeliness of childhood vaccinations: using cohort data and routine health records to evaluate quality of immunisation services. Vaccine 2017;35:7166–73. doi:10.1016/j.vaccine.2017.10.085
      OpenUrlCrossRefPubMed
    11. ↵
      1. Tiley KS,
      2. White JM,
      3. Andrews N, et al
      . Inequalities in childhood vaccination timing and completion in london. Vaccine 2018;36:6726–35. doi:10.1016/j.vaccine.2018.09.032
      OpenUrlPubMed
    12. ↵
      1. British Medical Association
      2. National Health Service
      . Quality and outcomes framework guidance for 2021/22. 2021. Available: https://www.england.nhs.uk/wp-content/uploads/2021/03/B0456-update-on-quality-outcomes-framework-changes-for-21-22-.pdf
    13. ↵
      1. NHS Digital
      . Update on vaccination and immunisation changes for 2021/22. London, Available: https://www.england.nhs.uk/wp-content/uploads/2021/03/B0434_Update-on-vaccination-and-immunisation-changes-for-202122-v4.pdf
    14. ↵
      1. Chantler T,
      2. Lwembe S,
      3. Saliba V, et al
      . “ I’’s a complex mesh”-how large-scale health system reorganisation affected the delivery of the immunisation programme in england: a qualitative study. BMC Health Serv Res 2016;16:489. doi:10.1186/s12913-016-1711-0
      OpenUrl
    15. ↵
      1. Crocker-Buque T,
      2. Edelstein M,
      3. Mounier-Jack S
      . Interventions to reduce inequalities in vaccine uptake in children and adolescents aged <19 years: a systematic review. J Epidemiol Community Health 2017;71:87–97. doi:10.1136/jech-2016-207572
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Edelstein M,
      2. Crocker-Buque T,
      3. Tsang C, et al
      . Extracting general practice data for timely vaccine coverage estimates: the england experience. Vaccine 2017;35:5110–4. doi:10.1016/j.vaccine.2017.08.015
      OpenUrlCrossRefPubMed
    17. ↵
      1. Queen Mary University of London Clinical Effectiveness Group
      . Software tools to improve clinical management and recording. 2022. Available: https://www.qmul.ac.uk/blizard/ceg/resources/
    18. ↵
      1. Robson J,
      2. Hull S,
      3. Mathur R, et al
      . Improving cardiovascular disease using managed networks in general practice: an observational study in inner London. Br J Gen Pract 2014;64:e268–74. doi:10.3399/bjgp14X679697
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Letley L,
      2. Rew V,
      3. Ahmed R, et al
      . Tailoring immunisation programmes: using behavioural insights to identify barriers and enablers to childhood immunisations in a jewish community in london, UK. Vaccine 2018;36:4687–92. doi:10.1016/j.vaccine.2018.06.028
      OpenUrlCrossRefPubMed
    20. ↵
      1. Cockman P,
      2. Dawson L,
      3. Mathur R, et al
      . Improving MMR vaccination rates: herd immunity is a realistic goal. BMJ 2011;343:bmj.d5703. doi:10.1136/bmj.d5703
    21. ↵
      1. Hull S,
      2. Hagdrup N,
      3. Hart B, et al
      . Boosting uptake of influenza immunisation: a randomised controlled trial of telephone appointing in general practice. Br J Gen Pract 2002;52:712–6.
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Chahal JK,
      2. Antoniou S,
      3. Earley M, et al
      . Preventing strokes in people with atrial fibrillation by improving ABC. BMJ Open Qual 2019;8:e000783. doi:10.1136/bmjoq-2019-000783
    23. ↵
      1. One London
      . Pathfinder projects to tackle london’s health challenges using the power of data at scale. 2022. Available: https://www.onelondon.online/pathfinder-projects-to-tackle-londons-health-challenges-using-the-power-of-data-at-scale/
    24. ↵
      1. Office for National Statistics
      . Estimates of the population for the UK, england, wales, scotland and northern ireland - office for national statistics (ons.gov.UK); 2020.
    25. ↵
      1. Crocker-Buque T,
      2. Edelstein M,
      3. Mounier-Jack S
      . A process evaluation of how the routine vaccination programme is implemented at GP practices in england. Implement Sci 2018;13:132. doi:10.1186/s13012-018-0824-8
      OpenUrl
    26. ↵
      1. NHS England
      . General medical services statement of financial entitlements directions. 2021. Available: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/975395/GMS_SFE_2021.pdf
    27. ↵
      1. Ministry of Housing Communities & Local Government
      . The english indices of deprivation 2019 - frequently asked questions (FAQs). 2016. Available: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/853811/IoD2019_FAQ_v4.pdf
    28. ↵
      1. Office for National Statistics
      . Research report on population estimates by characteristics; 2020.
    29. ↵
      1. Lopez Bernal J,
      2. Soumerai S,
      3. Gasparrini A
      . A methodological framework for model selection in interrupted time series studies. J Clin Epidemiol 2018;103:82–91. doi:10.1016/j.jclinepi.2018.05.026
      OpenUrlCrossRefPubMed
    30. ↵
      1. Statistical Methodology and Applications Branch
      . Joinpoint regression program. In: Surveillance Research Program. National Cancer Institute, 2020.
    31. ↵
      1. Public Health England
      . PHEIndicatorMethods; 2021.
    32. ↵
      1. Hosseinpoor AR,
      2. Schlotheuber A,
      3. Nambiar D, et al
      . Health equity assessment toolkit plus (heat plus): software for exploring and comparing health inequalities using uploaded datasets. Glob Health Action 2018;11:1440783. doi:10.1080/16549716.2018.1440783
      OpenUrl
    33. ↵
      1. Public Health Wales
      . Measuring inequalities - trends in mortality and life expectancy in wales. 2011. Available: https://phw.nhs.wales/services-and-teams/observatory/data-and-analysis/measuring-inequalities-2011/
    34. ↵
      1. RStudio Team
      . RStudio: integrated development for R. Boston, MA: PBC, 2020.
    35. ↵
      1. Liu W,
      2. Ye S,
      3. Barton BA, et al
      . Simulation-based power and sample size calculation for designing interrupted time series analyses of count outcomes in evaluation of health policy interventions. Contemp Clin Trials Commun 2020;17:100474. doi:10.1016/j.conctc.2019.100474
      OpenUrl
    36. ↵
      1. Hawley S,
      2. Ali MS,
      3. Berencsi K, et al
      . Sample size and power considerations for ordinary least squares interrupted time series analysis: a simulation study. Clin Epidemiol 2019;11:197–205. doi:10.2147/CLEP.S176723
      OpenUrlPubMed
    37. ↵
      1. Murray E,
      2. Treweek S,
      3. Pope C, et al
      . Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010;8:63. doi:10.1186/1741-7015-8-63
      OpenUrlCrossRefPubMed
    38. ↵
      1. Davidoff F,
      2. Dixon-Woods M,
      3. Leviton L, et al
      . Demystifying theory and its use in improvement. BMJ Qual Saf 2015;24:228–38. doi:10.1136/bmjqs-2014-003627
      OpenUrlAbstract/FREE Full Text
    39. ↵
      1. Malterud K,
      2. Siersma VD,
      3. Guassora AD
      . Sample size in qualitative interview studies: guided by information power. Qual Health Res 2016;26:1753–60. doi:10.1177/1049732315617444
      OpenUrlCrossRefPubMed
    40. ↵
      1. Eccles DW,
      2. Arsal G
      . The think aloud method: what is it and how do I use it? Qual Res Sport Exerc Health 2017;9:514–31. doi:10.1080/2159676X.2017.1331501
      OpenUrl
    41. ↵
      1. Braun V,
      2. Clarke V
      . Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. doi:10.1191/1478088706qp063oa
      OpenUrlCrossRef
    42. ↵
      1. Tracy SJ
      . Qualitative quality: eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry 2010;16:837–51. doi:10.1177/1077800410383121
      OpenUrlCrossRefWeb of Science
    43. ↵
      1. UK Public Involvement Standards Development Partnership
      . UK standards for public involvement: better public involvement for better health and social care research; 2019.
    44. ↵
      1. Barts Research Centre for Women’s Health
      . Katie’s team. 2022. Available: https://www.barc-research.org/katies-team
    45. ↵
      1. NHS
      2. Health Research Authority
      . HRA approval. 2020. Available: https://www.hra.nhs.uk/approvals-amendments/what-approvals-do-i-need/hra-approval/
    46. ↵
      1. Ogrinc G,
      2. Davies L,
      3. Goodman D, et al
      . Squire 2.0 (standards for quality improvement reporting excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2016;25:986–92. doi:10.1136/bmjqs-2015-004411
      OpenUrlAbstract/FREE Full Text
    47. ↵
      1. Mulholland K,
      2. Kretsinger K,
      3. Wondwossen L, et al
      . Action needed now to prevent further increases in measles and measles deaths in the coming years. Lancet 2020;396:1782–4. doi:10.1016/S0140-6736(20)32394-1
      OpenUrl
    48. ↵
      1. European Centre for Disease Prevention and Control
      . Vaccination coverage for second dose of a measles-containing vaccine, EU/EEA, 2018. 2021 Available: https://www.ecdc.europa.eu/en/publications-data/vaccination-coverage-second-dose-measles-containing-vaccine-eueea-2018
    49. ↵
      1. Alderwick H
      . Is the NHS overwhelmed? BMJ 2022;376:51. doi:10.1136/bmj.o51
      OpenUrl

    Footnotes

    • Twitter @milmarsz, @meredithkdh, @ANAGCEG

    • Contributors As per CrEdit accreditation, significant contributions to: Conceptualisation (MM, MKDH, AG, JR, HB and CD), Methodology (MKDH, AG, NF and CD), Resources (MM, AG, AB, NM and CD), Software (MM, AG, ID and ZA), Data Curation (MM, AG, NF, ID, ZA and CD), writing-original draft (MM, MKDH and CD), review and editing (MM, MKDH, AG, NF, ID, ZA, JR, HB, AB, NM and CD), project administration (AB and AG), project management (AG), supervision (CD), funding acquisition (CD).

    • Funding MM is a general practice trainee and a locally funded National Institute of Health Research accredited Academic Clinical Fellow. All other authors are funded by their respective institutions through grants or core funding. The Clinical Effectiveness Group is funded by NEL Commissioners to support quality improvement in primary care. The QI programme and its evaluation is funded by grants from the NEL Digital First Programme, Barts Charity (ref MGU0419) and the London Health Data Strategy Programme. This work was also supported by Health Data Research UK (award reference: LOND1), which is funded by the UK Medical Research Council and a consortium of funders.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

    • Provenance and peer review Not commissioned; externally peer reviewed.