You are here

Article Text

Article menu

Download PDFPDF

XML
Public health
Protocol
Evaluation of the digital diabetes prevention programme pilot: uncontrolled mixed-methods study protocol
  1. Elizabeth Murray1,
  2. Kerry Daff2,
  3. Anthi Lavida3,
  4. William Henley4,
  5. Jenny Irwin2,
  6. Jonathan Valabhji5,6
  1. 1 Research Department of Primary Care and Population Health, University College London, London, UK
  2. 2 RSM UK Consulting LLP, Belfast, UK
  3. 3 Primary Care and Population Health, University College London, London, UK
  4. 4 Institute of Health Research, University of Exeter Medical School, Exeter, UK
  5. 5 Medical Directorate, NHS England, London, UK
  6. 6 Diabetes and Endocrinology, Imperial College Healthcare NHS Trust, London, UK
  1. Correspondence to Professor Elizabeth Murray; elizabeth.murray{at}ucl.ac.uk

Abstract

Introduction The prevalence of type 2 diabetes is rising steeply. National Health Service England (NHSE) is exploring the potential of a digital diabetes prevention programme (DDPP) and has commissioned a pilot with embedded evaluation.

Methods and analysis This study aims to determine whether, and if so, how, should NHSE implement a national DDPP, using a mixed-methods pretest and post-test design, underpinned by two theoretical frameworks: the Coventry, Aberdeen and London - Refined (CALO-RE) taxonomy of behavioural change techniques for the digital interventions and the Consolidated Framework for Implementation Research (CFIR) for implementation processes. In eight pilot areas across England, adults with non-diabetic hyperglycaemia (NDH) (glycated haemoglobin (HbA1c) 42–47 mmol/mol or fasting plasma glucose 5.5–6.9 mmol/L) and adults without NDH who are overweight (body mass index (BMI) >25 kg/m2) or obese (BMI >30 kg/m2) will be referred to one of five digitally delivered diabetes prevention interventions. The primary outcomes are reduction in HbA1c and weight (for people with NDH) and reduction in weight (for people who are overweight or obese) at 12 months. Secondary outcomes include use of the intervention, satisfaction, physical activity, patient activation and resources needed for successful implementation. Quantitative data will be collected at baseline, 6 months and 12 months by the digital intervention providers. Qualitative data will be collected through semistructured interviews with commissioners, providers, healthcare professionals and patients. Quantitative data will be analysed descriptively and using generalised linear models to determine whether changes in outcomes are associated with demographic and intervention factors. Qualitative data will be analysed using framework analysis, with data pertaining to implementation mapped onto the CFIR.

Ethics and dissemination The study has received ethical approval from the Public Health England Ethics and Research Governance Group (reference R&D 324). Dissemination will include a report to NHSE to inform future policy and publication in peer-reviewed journals.

  • diabetes mellitus
  • type 2
  • health promotion
  • primary prevention
  • eHealth
  • internet
  • digital divide
  • health policy

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2018-025903

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This will be the first large-scale evaluation of digital diabetes prevention programmes internationally, and will provide data on effectiveness, uptake rates and on resources required for effective implementation, allowing a realistic determination of potential population impact.

    • It benefits from real-world experience and data, providing strong external validity.

    • The lack of a comparator or any randomisation means that any changes in outcomes observed during the study cannot be said to be due to the interventions offered. Changes observed may be due to the impact of identification and measurement, the interventions offered, regression to the mean or some other unmeasured confounder.

    Introduction

    Diabetes is a national and international priority for health services, with a steeply rising prevalence. Globally, it affects over 400 million people or around 9% of the adult population,1 and of these, over 90% have type 2 diabetes mellitus (T2DM). There were an estimated 3.7 million people with diabetes in England in 2016–2017.2 People with diabetes are at risk of complications including cardiovascular disease, nephropathy, retinopathy and neuropathy. The costs of treating diabetes and its complications are estimated at 10% of the total National Health Service (NHS budget, or some £10 billion per annum.3 4

    Risk factors for developing T2DM include diet, lack of physical activity, obesity, genetic factors and deprivation. For many people, T2DM is a preventable illness, with prevention focused on the modifiable factors of diet, physical activity and weight. There is high-quality international evidence that face-to-face programmes focusing on these three factors can reduce the rate of progression to T2DM in high-risk individuals.5–15 To be successful, these programmes appear to require intensive sustained input over a prolonged period of time.7 In light of this evidence NHS England (NHSE) launched Healthier You: The NHS Diabetes Prevention Programme (NHS DPP) in 2015, initially in seven demonstrator sites, and subsequently rolled out across England. All programmes within the NHS DPP must offer at least 16 hours of face-to-face contact, spread over 13 sessions, with the total programme lasting at least 9 months.16

    NHSE is also considering a national digital diabetes prevention programme (DDPP), and in order to generate the evidence to inform future policy in this area, launched a pilot DDPP in 2017, called ‘Healthier You: NHS DPP digital stream’.17 The reasons cited by NHSE for exploring the potential of a digital DPP are twofold: first, digital delivery may overcome some of the challenges affecting face-to-face programmes; and second, systematic review evidence that digital health interventions can be effective in increasing physical activity, changing diets and promoting weight loss,18–27 all behaviours which are effective in preventing T2DM. Challenges affecting the population impact of face-to-face programmes include problems of acceptability, as their intensive nature may make it difficult for people who work or have other commitments in their lives to attend; and there may be perceived stigma in attending a programme aimed at prevention of T2DM. Finally, the face-to-face programmes are costly, particularly when implemented at scale and a digital programme could potentially be easier to deliver at scale and more cost-effective.

    However, there is little evidence to support these potential advantages,28 and there are some well-documented challenges in the delivery of digital health interventions. Three of the most important of these challenges are problems with engagement and adherence; concerns around the ‘digital divide’; and well-documented problems with implementation. Many digital health interventions show low rates of initial uptake as well as high rates of subsequent attrition from the intervention, which limits their potential for population impact.29 Moreover, there is uncertainty as to how best to improve engagement and adherence—although there are data which suggest that a certain amount of human input, for example, in the form of supportive or coaching telephone calls, can improve engagement, the requirement for human input can impact on the scaleability and costs of digital health interventions.30–32 There is real concern that the ‘digital divide’ (the divide between those who do and do not make regular use of digital technology) will exacerbate health inequalities, as many of those with greatest health needs (older people, people with long-term health problems and those with low incomes) are also people who make less use of digital technology.33–37 Third, the challenges of successful implementation of digital health interventions are well known,38 39 with few examples of successful integration of digital health interventions into routine healthcare and considerable uncertainty as to how best to achieve such integration.

    In the light of this potential, coupled with these major areas of uncertainty, NHSE commissioned a pilot DDPP with an associated evaluation to run alongside the national DPP. In the initial tender document, NHSE specified that the goal of the pilot and associated evaluation was to inform future policy in this area, and as such, the overarching aim of the evaluation was to determine: ‘Whether, and if so, how, should NHSE roll out a national DDPI at the end of the pilot?’ Specific areas of interest were around uptake, use and impact on weight and glycated haemoglobin (HbA1c); the likely impact of a digital programme on health inequalities; and how the interventions should be integrated into NHS workflows, including determining the resource requirements for successful implementation. Although a formal comparison of the effectiveness of different interventions was out of scope, NHSE was interested in whether specific features of different interventions appeared to be associated with variation in observed uptake, use or impact. The tender specified that a formal health economic analysis was out of scope. The evaluation is due to report in 2020.

    The specific objectives of the evaluation can be grouped into three areas: uptake, use and impact of the interventions; the extent to which uptake, use and impact vary by socioeconomic status (SES) as an indicator of likely impact on health inequalities; and factors relating to implementation.

    Objectives pertaining to uptake, use and impact:

    1. Determine uptake and use of the digital diabetes prevention interventions (DDPI) by people referred to them through the DDPP.

    2. Determine the effects of DDPI on people referred to them through the DDPP.

    3. Explore the extent to which these benefits vary according to differences in key features of the selected DDPI.

    4. Explore user views about the acceptability of DDPI, including perceptions relating to use/non-use and impact on relevant behaviours.

    Objectives pertaining to health inequalities:

    1. Explore the extent to which uptake, use and effects vary by SES.

    Objectives pertaining to implementation:

    1. Describe the various implementation strategies applied in the eight demonstrator sites.

    2. Determine the costs associated with implementing and delivering a DDPP, from an NHS perspective.

    3. Explore commissioner, healthcare professional and provider views about key factors influencing implementation, uptake and impact of the DDPI.

    4. Summarise and synthesise these data in a report which can be used to inform the policy decisions about whether, and if so, how, to roll out a DDPP across England.

    Methods

    Design

    Mixed-methods pretest and post-test design, underpinned by theory.

    Theoretical frameworks

    This evaluation will be underpinned by two theoretical frameworks: one pertaining to the effectiveness of the DDPI and one pertaining to the implementation processes.

    Understanding the likely and observed effectiveness of the selected DDPI will be promoted by applying the Coventry, Aberdeen and London - Refined (CALO-RE) taxonomy of behaviour change techniques40 and describing interventions using the Template for Intervention Description and Replication (TiDIER) framework41 (online supplementary appendix 1).

    Supplemental material

    To help with describing and understanding the implementation processes, we will use the Consolidated Framework for Implementation Research (CFIR).42 This specifies that the key constructs which determine whether an intervention is successfully implemented or not are: intervention characteristics; outer setting; inner setting; individual characteristics and the implementation process (for details see online supplementary appendix 2).

    Patient and public involvement

    The board overseeing the NSHE programme in diabetes and diabetes prevention is made up of a triumvirate of NHSE, Public Health England (PHE) and Diabetes UK (DUK). DUK is the largest charity representing the voice of people with diabetes in the UK. Patient and public involvement (PPI) in this study was, therefore, provided by DUK, through their membership of the board. This board determined how the DDPP should be piloted and evaluated, what the requirements for the evaluation were, including overall design (numbers of demonstrator sites, DDPI, pretest and post-test design, primary outcomes and duration of study). The board also provides oversight of the conduct and progress of the study and will receive the reports of the study. Hence, there was PPI input into determining the research questions, outcome measures, study design and dissemination.

    Setting

    Eight demonstrator sites were selected by NHSE in parallel with the selection of the evaluator team, who had no input into site selection. Sites volunteered to participate in the digital pilot, and were selected to achieve a range of geographies and demographies, including rural, semirural, urban and metropolitan areas, with widely varying proportions of people from black and minority ethnic backgrounds, SES and pre-existing levels of digital readiness and engagement with diabetes prevention.

    Populations and participants

    There were three populations specified by NHSE: (1) adults with non-diabetic hyperglycaemia (NDH) defined as having had an HbA1c measurement of 42–47 mmol/mol, or a fasting glucose measurement of 5.5–6.9 mmol/L, in the 12 months prior to referral; (2) adults who are overweight (body mass index (BMI) of 25 to <30 kg/m2) without NDH and (3) adults who are obese (BMI of 30 kg/m2 or over) without NDH. It is the responsibility of the eight demonstrator sites to determine how they will identify people who fell into these three populations, how general practitioner (GPs) and patients will be informed about the programme, and how patients will be referred to a DDPI. In most sites, patients will be referred to the interventions by their healthcare professionals, usually their GPs. In some areas, identification and referral may be undertaken by alternative providers contracted to undertake assessment of cardiovascular risk as part of the NHS Health Check Programme, which includes a two-stage assessment that aims to identify NDH and undiagnosed T2DM.43 Referring healthcare professionals are responsible for discussing the referral with patients, to ensure patients understand their diagnosis, the type of intervention they are being referred to and the expected benefits. Referring healthcare professionals are responsible for sending the patient’s name and contact details to the relevant DDPI provider; that provider is then responsible for contacting the patient and onboarding them to the intervention. These processes are identical to those used in the national face-to-face DPP,44 with the only difference being that the provider is offering a digital, rather than a face-to-face intervention.

    Interventions

    A total of five DDPI were selected following a multistage selection process undertaken by an independent assessor (Our Mobile Health) under contract to NHSE. Following widespread advertising of the opportunity, 84 companies registered an interest and underwent initial screening against six criteria. These were: (1) that the intervention supported behavioural change; (2) was available by June 2017; (3) was localised for the English market; (4) was underpinned by an evidence-based approach; (5) did not require any further integration with existing health information technology (IT) systems before launching; (6) had a pricing system in place (although cost itself was not a criterion). Thirty providers met these criteria and progressed to a self-assessment exercise which explored eight criteria: safety; privacy and security; pricing; evidence based or indicators of effectiveness; usability and accessibility; technical stability; change management and interoperability. The self-assessment questionnaire can be found at https://developer.nhs.uk/daq.45 This led to a shortlist of 14 interventions, which were reviewed by subject matter experts including behaviour change theory experts, clinical safety officers, GPs, diabetologists, diabetes specialist nurses and dieticians. The final sample was selected to vary on factors known to be important in influencing uptake, use and effectiveness of digital health interventions, namely: the delivery platform (smartphone vs not); the amount of human interaction to promote uptake; and inclusion (or not) of wearables. Within this, interventions with better pre-existing evidence for uptake, use and effectiveness as well as capabilities and infrastructure to implement at scale, were prioritised.

    All five interventions focused on dietary intake, weight loss and physical activity, and all interventions set personalised goals and provided feedback on progress towards these goals.

    Intervention A (Buddi Nujjer) is a smartphone app, which links to a wristband for monitoring physical activity. The participants log their eating habits and activities via the programme, receive three motivational messages from the app per day and complete a total of 22 learning modules over the course of the 12-month programme. Onboarding is done by an initial phone call and email for registration. A customer services team is available for those who have technical difficulties, but apart from that, the service is entirely digital.

    Intervention B (Hitachi) is smartphone (Android and Apple), tablet and desktop compatible. The solution provides a hybrid of digital and non-digital interactions with a website and a series of phone calls with an experienced health advisor, whose advice follows NHS guidelines. Participants and health advisors set an action plan at the start of the programme. Participants then self-report their outcomes and log their lifestyle on the website to understand their progress against key milestones and objectives. This information goes back to the health advisor team who prompt on guidance and can interact with participants. Health advisors contact participants monthly for the first 6 months and have one more step-down call at the 8-month mark. Pedometers and scales can be provided on request. Onboarding is done through an initial brief phone call, at which point a longer, goal setting telephone call is arranged.

    Intervention C (Liva) is smartphone (Android and Apple), tablet and desktop compatible. There is an initial face-to-face appointment with a health coach for onboarding and goal setting, followed by 26 coaching sessions with the same coach, delivered weekly for the first 12 weeks and then tapering off in frequency. There is an online peer support group.

    Intervention D (Ourpath) is smartphone (Android and Apple), tablet and desktop compatible and includes wireless weighing scales and a wearable activity tracker. Participants are entered into a peer group of up to 10 other people with similar goals who live locally. Groups interact by group messaging, and group targets are set as well as individual ones. The programme has three stages: the ‘Core’ programme with daily education content received through the app for the first 6 weeks, the ‘Sustain’ programme with weekly education content until the 6 month mark and the final 6 months where the user will have completed all the education modules but still have access to the dietician and group support. Onboarding is done through two phone calls —an initial introductory one, and a second one to set the participant up with the programme and group.

    Intervention E (Oviva) is an app (Android and Apple compatible) with supporting material (learning materials, podcasts, recipes) delivered through an online portal. For patients without a smartphone, the content can be delivered via phone calls. The app allows users to track their activity, weight, and food and drink intake (using a photo food diary). The programme is a mix of digital and non-digital interactions with a series of phone calls accompanying the app. The programme is more intense at the start with a weekly phone call in the first 8 weeks to cover the 16-topic curriculum, tapering to a monthly phone call thereafter. The phone calls are all conducted by the same dietician who is a specialist diabetes dietician with at least 2 years experience. Onboarding is done by the dietician in the first phone call.

    The interventions will be described according to the TiDIER Framework for describing complex interventions41 and the CALO-RE Behaviour Change Technique taxonomy.40

    Outcomes

    The primary outcomes were prespecified by NHS England, and were change between baseline and 12 months in HbA1c for the population with NDH and weight for all three populations. Changes over 6 months are considered secondary outcomes.

    Secondary and explanatory outcomes have been selected according to our pathway of action model, which posits that the DDPI achieves its goal of reducing a user’s risk of diabetes by promoting behavioural change, specifically, promoting dietary change and an increase in physical activity. Taken together these behavioural changes result in reduced HbA1c and reduced weight. To achieve these changes requires the user to: register with the intervention, use the intervention, initiate behaviour change and sustain behaviour change. Effects at each stage will be moderated by intervention factors and by patient factors (see figure 1). The context and implementation process will also affect the overall population uptake and impact.

    Figure 1
    Figure 1

    Theory of change/pathway of action for effects of intervention. NB: all taking place within different contexts (local health economies) and with different implementation processes. HbA1c, glycated haemoglobin; SES, socioeconomic status.

    Secondary outcomes are listed in table 1 and include intervention uptake, use and satisfaction; behavioural and clinical outcomes; and costs related to implementation, although it should be noted that a formal health economic analysis was excluded from the brief. Data will be collected at baseline, 6 months and 12 months.

    View this table:
    Table 1

    Quantitative outcomes

    Measures used are the Friends and Family Test (FFT)46 to measure satisfaction with care, the International Physical Activity Questionnaire (IPAQ)47 to measure physical activity and the 13-item Patient Activation Measure (PAM-13).48

    Demographic characteristics are collected at baseline only and will be used as explanatory factors. Demographic data to be collected includes age (date of birth), gender, ethnicity, postal code (to be used for determining SES by mapping against the Index of Multiple Deprivation (IMD)) and highest level of education attained.

    Qualitative and explanatory outcomes

    Reasons for observed differences in quantitative outcomes will be explored through qualitative interviews. These will be undertaken with a range of stakeholders, including commissioners/leads for the DDPP in selected local health economies (LHE); healthcare professionals (GPs, practice nurses, diabetes nurses, healthcare assistants, practice managers); digital diabetes intervention providers; the implementation teams charged with implementing the DDPP in the selected LHE and patients.

    Interviewees will be selected through purposive sampling, aiming for variation across geographical area, digital diabetes intervention provider, disciplinary backgrounds and areas of high and low uptake. Recruitment of interviewees will continue until (1) all digital diabetes intervention providers and representatives from each demonstrator site have been interviewed and (2) data saturation is achieved.

    Interviews will be conducted using semistructured topic guides, by trained interviewers, working for RSM. The topic guides will vary according to the background of the stakeholder, with the main areas covered summarised in table 2. Interviews will be undertaken in waves, with wave 1 taking place during initial implementation and set-up, wave 2 once the programme is well established and wave 3 towards the end of the pilot.

    View this table:
    Table 2

    Qualitative and explanatory outcomes

    Data collection

    Referral forms will include basic demographic and clinical data, including HbA1c and weight recorded in the previous 3 months to confirm eligibility.

    Quantitative data

    The DDPI providers are responsible for collecting all quantitative data. Baseline data will be obtained from referral forms and supplemented with data obtained during onboarding interviews with patients. Follow-up self-report data (FFT, IPAQ and PAM-13) will be collected online. HbA1c and weights will be measured by the DDPI providers or by patients’ general practices, with the method and site of measurement recorded.

    HbA1c measurements may be done on either venous or capillary blood samples, using either registered NHS laboratories or validated point-of-care testing kits. Whichever measurement process is used at baseline should be used at follow-up. Weights will be recorded using calibrated scales, with patients wearing light indoor clothing.

    Data analysis

    Quantitative data

    The baseline characteristics of the three cohorts (NDH, overweight and obese) will be summarised with respect to sociodemographic characteristics, intervention uptake, behavioural and clinical outcomes and economic outcomes. Continuous data will be summarised in terms of the mean, SD and number of observations or, where skewed, median and IQR. Binary/categorical data will be summarised in terms of frequency counts and percentages. Descriptive statistics will also be used to explore differences in baseline characteristics across the eight demonstrator sites and five DDPIs.

    The primary analysis will be based on participants with complete data. For continuous outcomes, the overall effectiveness of the programme will be assessed in pre–post analyses by comparing the mean outcomes in each cohort from baseline to 6 months, and from baseline to 12 months (presented with 95% CIs for the estimated change in outcomes). The statistical significance of any changes will be assessed using a paired t-test. For categorical outcomes, pre–post analyses will be conducted using McNemar’s test.

    Multivariable generalised linear models will be used to determine whether changes in outcomes are associated with demographic factors, adjusting for baseline outcome scores. Where necessary, continuous outcomes will be transformed to ensure good regression model fit. Exploratory analysis of the influence of potential mediators will be conducted by adding variables relating to usage and features of the DDPI to the regression models. The potential for clustering effects by demonstrator site will be considered by the inclusion of random effects for the demonstrator site in the generalised linear models. Three-level models accounting for clustering by GP practice within demonstrator sites will also be explored.

    Reasons for missing data will be documented and the baseline characteristics of those with and without missing data compared. Although the primary analysis will be based on participants with complete data, we will undertake sensitivity analyses using various imputation models. The potential for bias due to non-random attrition will be addressed by fitting a propensity score model to account for drop-out on the basis of baseline characteristics and then using inverse probability weighting based on the propensity score to fit the treatment effectiveness model.49 No formal adjustment for multiple significance testing will be applied.

    Sample size

    Target referral and registration numbers were preset by NHSE as part of the tender at 3500 registrations for the NDH population and 1500 for the overweight/obese population. We estimated minimum detectable effect sizes at 90% power and a 5% significance level for the key research questions, given these fixed sample sizes. Assuming a 25% completion rate (at 12 months), it will be possible to detect standardised effect sizes of d=0.11 and d=0.17 when assessing overall effectiveness in the NDH and overweight/obese groups respectively, assuming clustering is ignorable. This compares favourably with a weighted mean effect size of d=0.22 (95% CI 0.20 to 0.23) estimated in a meta-analysis by Johnson et al 50 for behaviour change interventions targeting eating and physical activity. Further power analysis allowing for clustering effects by demonstrator site (with an intraclass correlation coefficient (ICC) of 0.02 based on a median estimate of 0.0185 in a study of ICCs in adults with diabetes in primary care practices51) gave minimum detectable effect sizes of d=0.18 and 0.22 in the NDH and overweight/obese groups, respectively, assuming a 25% completion rate at 12 months. For the purpose of analysis, completion is defined as obtaining data on weight and HbA1c at 12 months.

    Qualitative data

    Interviews will be recorded, transcribed verbatim and anonymised prior to analysis. Transcripts will be analysed using framework analysis52 which is well suited to policy-relevant research, with specific questions and a priori issues. The five steps of framework analysis are (1) familiarisation, (2) identifying a thematic framework; (3) indexing; (4) charting and (5) mapping and interpretation. Familiarisation will be achieved by reading and rereading transcripts, with an a priori framework based on the CFIR used to index and chart the data. Data that cannot be coded using CFIR will be noted. Mapping and interpretation will take place in multidisciplinary data clinics where interpretations can be proposed, discussed and refined.

    Ethics, research governance and data security

    PHE is the sponsor for this research.

    Data security and information governance

    Data will be handled according to the principles of the General Data Protection Regulation, the European Union framework for data protection which became law in the UK in May 2018.

    Quantitative data

    The DDPI providers will be responsible for obtaining and pseudonymising the quantitative data. No personally identifiable data will be handled by RSM. Postal code mapping for IMD will be undertaken by the LHE (with support from RSM).

    Qualitative data

    1. Patients. Patients will be invited to participate in the interviews via the digital diabetes prevention providers and/or the LHE and will opt in to providing fully informed consent for interviews.

    2. Healthcare professionals, commissioners and providers. RSM has the contact details for these individuals as RSM is also responsible for the implementation of the DDPI. Written informed consent will be obtained prior to undertaking interviews.

    Interview tapes will be stored securely on RSM servers. Only anonymised transcripts will be shared with the evaluation team outside RSM.

    Dissemination

    Contracted outputs include a 12-month report to the funder, based on 6-month follow-up data, and a final report, based on 12-month follow-up data. The findings will inform the scale of future provision of digital approaches within the NHS DPP.

    Academic dissemination will also be undertaken in the form of conference presentations and publications in peer-reviewed journals. These presentations and publications will require advance approval from NHS England. Approval will not be unreasonably withheld, but academic dissemination may have to be delayed till after major policy decisions have been taken and made public.

    References

    1. 1.↵
      International Diabetes Federation.. IDF Diabetes Atlas: 2014 update. 2015 http://www.idf.org/diabetesatlas
    2. 2.↵
    3. 3.↵
      2. Hex N ,
      3. Bartlett C ,
      4. Wright D , et al
      . Estimating the current and future costs of Type 1 and Type 2 diabetes in the UK, including direct health costs and indirect societal and productivity costs. Diabet Med 2012;29:855–62.doi:10.1111/j.1464-5491.2012.03698.x
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Alva ML ,
      3. Gray A ,
      4. Mihaylova B , et al
      . The impact of diabetes-related complications on healthcare costs: new results from the UKPDS (UKPDS 84). Diabet Med 2015;32:459–66.doi:10.1111/dme.12647
      OpenUrlPubMed
    5. 5.↵
      2. Aguiar EJ ,
      3. Morgan PJ ,
      4. Collins CE , et al
      . Efficacy of interventions that include diet, aerobic and resistance training components for type 2 diabetes prevention: a systematic review with meta-analysis. Int J Behav Nutr Phys Act 2014;11:2.doi:10.1186/1479-5868-11-2
      OpenUrl
    6. 6.↵
      2. Ali MK ,
      3. Echouffo-Tcheugui J ,
      4. Williamson DF
      . How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Aff 2012;31:67–75.doi:10.1377/hlthaff.2011.1009
      OpenUrlAbstract/FREE Full Text
    7. 7.↵
      2. Ashra NB ,
      3. Spong R ,
      4. Carter P
      . 2015. A systematic review and meta-analysis assessing the effectiveness of pragmatic lifestyle interventions for the prevention of type 2 diabetes mellitus in routine practice: Public Health England. Report No.: PHE publications gateway number: 2015280.
    8. 8.↵
      2. Aziz Z ,
      3. Absetz P ,
      4. Oldroyd J , et al
      . A systematic review of real-world diabetes prevention programs: learnings from the last 15 years. Implement Sci 2015;10:172.doi:10.1186/s13012-015-0354-6
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. Baker MK ,
      3. Simpson K ,
      4. Lloyd B , et al
      . Behavioral strategies in diabetes prevention programs: a systematic review of randomized controlled trials. Diabetes Res Clin Pract 2011;91:1–12.doi:10.1016/j.diabres.2010.06.030
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. Dunkley AJ ,
      3. Bodicoat DH ,
      4. Greaves CJ , et al
      . Diabetes prevention in the real world: effectiveness of pragmatic lifestyle interventions for the prevention of type 2 diabetes and of the impact of adherence to guideline recommendations: a systematic review and meta-analysis. Diabetes Care 2014;37:922–33.doi:10.2337/dc13-2195
      OpenUrlAbstract/FREE Full Text
    11. 11.↵
      2. McCurley JL ,
      3. Crawford MA ,
      4. Gallo LC
      . Prevention of Type 2 Diabetes in U.S. Hispanic Youth: A Systematic Review of Lifestyle Interventions. Am J Prev Med 2017;53:519–32.doi:10.1016/j.amepre.2017.05.020
      OpenUrl
    12. 12.↵
      2. Neamah HH ,
      3. Sebert Kuhlmann AK ,
      4. Tabak RG
      . Effectiveness of Program Modification Strategies of the Diabetes Prevention Program: A Systematic Review. Diabetes Educ 2016;42:153–65.doi:10.1177/0145721716630386
      OpenUrlCrossRefPubMed
    13. 13.↵
      2. Pronk NP ,
      3. Remington PL
      . Community Preventive Services Task Force. Combined diet and physical activity promotion programs for prevention of diabetes: community preventive services task force recommendation statement. Ann Intern Med 2015;163:465–8.doi:10.7326/M15-1029
      OpenUrlCrossRefPubMed
    14. 14.↵
      2. Subramanian K ,
      3. Midha I ,
      4. Chellapilla V
      . Overcoming the Challenges in Implementing Type 2 Diabetes Mellitus Prevention Programs Can Decrease the Burden on Healthcare Costs in the United States. J Diabetes Res 2017;2017:1–5.doi:10.1155/2017/2615681
      OpenUrlCrossRef
    15. 15.↵
      2. Sun Y ,
      3. You W ,
      4. Almeida F , et al
      . The effectiveness and cost of lifestyle interventions including nutrition education for diabetes prevention: a systematic review and meta-analysis. J Acad Nutr Diet 2017;117:404–21. e36.doi:10.1016/j.jand.2016.11.016
      OpenUrlCrossRefPubMed
    16. 16.↵
      NHS England. NHS Diabetes Prevention Programme (NHS DPP). https://www.england.nhs.uk/diabetes/diabetes-prevention/.
    17. 17.↵
    18. 18.↵
      2. Harris J ,
      3. Felix L ,
      4. Miners A , et al
      . Adaptive e-learning to improve dietary behaviour: a systematic review and cost-effectiveness analysis. Health Technol Assess 2011;15:1–160.doi:10.3310/hta15370
      OpenUrlCrossRefPubMed
    19. 19.↵
      2. Norman GJ ,
      3. Zabinski MF ,
      4. Adams MA , et al
      . A review of eHealth interventions for physical activity and dietary behavior change. Am J Prev Med 2007;33:336–45.doi:10.1016/j.amepre.2007.05.007
      OpenUrlCrossRefPubMedWeb of Science
    20. 20.↵
      2. Afshin A ,
      3. Babalola D ,
      4. Mclean M , et al
      . Information Technology and Lifestyle: A Systematic Evaluation of Internet and Mobile Interventions for Improving Diet, Physical Activity, Obesity, Tobacco, and Alcohol Use. J Am Heart Assoc 2016;5.doi:10.1161/JAHA.115.003058
    21. 21.↵
      2. Seo DC ,
      3. Niu J
      . Evaluation of Internet-Based Interventions on Waist Circumference Reduction: A Meta-Analysis. J Med Internet Res 2015;17:e181.doi:10.2196/jmir.3921
      OpenUrl
    22. 22.↵
      2. Sherrington A ,
      3. Newham JJ ,
      4. Bell R , et al
      . Systematic review and meta-analysis of internet-delivered interventions providing personalized feedback for weight loss in overweight and obese adults. Obes Rev 2016;17:541–51.doi:10.1111/obr.12396
      OpenUrlCrossRefPubMed
    23. 23.↵
      2. Schembre SM ,
      3. Liao Y ,
      4. Robertson MC , et al
      . Just-in-Time Feedback in Diet and Physical Activity Interventions: Systematic Review and Practical Design Framework. J Med Internet Res 2018;20:e106.doi:10.2196/jmir.8701
      OpenUrl
    24. 24.↵
      2. Spruijt-Metz D ,
      3. Wen CK ,
      4. O’Reilly G , et al
      . Innovations in the Use of Interactive Technology to Support Weight Management. Curr Obes Rep 2015;4:510–9.doi:10.1007/s13679-015-0183-6
      OpenUrl
    25. 25.↵
      2. Bardus M ,
      3. Smith JR ,
      4. Samaha L , et al
      . Mobile and Web 2.0 interventions for weight management: an overview of review evidence and its methodological quality. Eur J Public Health 2016;26:602–10.doi:10.1093/eurpub/ckw090
      OpenUrlCrossRefPubMed
    26. 26.↵
      2. Muellmann S ,
      3. Forberger S ,
      4. Möllers T , et al
      . Effectiveness of eHealth interventions for the promotion of physical activity in older adults: a systematic review. Prev Med 2018;108:93–110.doi:10.1016/j.ypmed.2017.12.026
      OpenUrl
    27. 27.↵
      2. Direito A ,
      3. Carraça E ,
      4. Rawstorn J , et al
      . mHealth Technologies to Influence Physical Activity and Sedentary Behaviors: Behavior Change Techniques, Systematic Review and Meta-Analysis of Randomized Controlled Trials. Ann Behav Med 2017;51:226–39.doi:10.1007/s12160-016-9846-0
      OpenUrl
    28. 28.↵
      2. Joiner KL ,
      3. Nam S ,
      4. Whittemore R
      . Lifestyle interventions based on the diabetes prevention program delivered via eHealth: A systematic review and meta-analysis. Prev Med 2017;100:194–207.doi:10.1016/j.ypmed.2017.04.033
      OpenUrl
    29. 29.↵
      2. Murray E ,
      3. White IR ,
      4. Varagunam M , et al
      . Attrition revisited: adherence and retention in a web-based alcohol trial. J Med Internet Res 2013;15:e162.doi:10.2196/jmir.2336
      OpenUrlCrossRefPubMed
    30. 30.↵
      2. Yardley L ,
      3. Spring BJ ,
      4. Riper H , et al
      . Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016;51:833–42.doi:10.1016/j.amepre.2016.06.015
      OpenUrl
    31. 31.↵
      2. McNamee P ,
      3. Murray E ,
      4. Kelly MP , et al
      . Designing and Undertaking a Health Economics Study of Digital Health Interventions. Am J Prev Med 2016;51:852–60.doi:10.1016/j.amepre.2016.05.007
      OpenUrl
    32. 32.↵
      2. O’Connor S ,
      3. Hanlon P ,
      4. O’Donnell CA , et al
      . Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak 2016;16:120.doi:10.1186/s12911-016-0359-3
      OpenUrlPubMed
    33. 33.↵
      2. Dutton WH ,
      3. Blank G ,
      4. Groselj D
      . Cultures of the Internet: The Internet in Britain. Oxford Internet Survery 2013. Oxford: Oxford Internet Institute 2013.
    34. 34.↵
      2. Gordon NP ,
      3. Hornbrook MC
      . Older adults' readiness to engage with eHealth patient education and self-care resources: a cross-sectional survey. BMC Health Serv Res 2018;18:220.doi:10.1186/s12913-018-2986-0
      OpenUrl
    35. 35.↵
      2. Levy H ,
      3. Janke AT ,
      4. Langa KM
      . Health literacy and the digital divide among older Americans. J Gen Intern Med 2015;30:284–9.doi:10.1007/s11606-014-3069-5
      OpenUrlCrossRefPubMed
    36. 36.↵
      2. Estacio EV ,
      3. Whittle R ,
      4. Protheroe J
      . The digital divide: Examining socio-demographic factors associated with health literacy, access and use of internet to seek health information. J Health Psychol 2017:135910531769542.doi:10.1177/1359105317695429
    37. 37.↵
      2. Latulippe K ,
      3. Hamel C ,
      4. Giroux D
      . Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res 2017;19:e136.doi:10.2196/jmir.6731
      OpenUrl
    38. 38.↵
      2. Wachter RM
      . 2016. Making IT Work: Harnessing the Power of HealthInformation Technology to Improve Care in England. Report of the National Advisory Group on Health Information Technology in England. London: Department of Health and Social Care.
    39. 39.↵
      2. Ross J ,
      3. Stevenson F ,
      4. Lau R , et al
      . Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016;11:146.doi:10.1186/s13012-016-0510-7
      OpenUrl
    40. 40.↵
      2. Michie S ,
      3. Ashford S ,
      4. Sniehotta FF , et al
      . A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychol Health 2011;26:1479–98.doi:10.1080/08870446.2010.540664
      OpenUrlCrossRefPubMedWeb of Science
    41. 41.↵
      2. Hoffmann TC ,
      3. Glasziou PP ,
      4. Boutron I , et al
      . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687.doi:10.1136/bmj.g1687
      OpenUrlAbstract/FREE Full Text
    42. 42.↵
      2. Damschroder LJ ,
      3. Aron DC ,
      4. Keith RE , et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50.doi:10.1186/1748-5908-4-50
      OpenUrlCrossRefPubMed
    43. 43.↵
      Public Health England. 2016. NHS Health check best practice guidance: London. Report No.: 2015710.
    44. 44.↵
      2. Barron E ,
      3. Clark R ,
      4. Hewings R , et al
      . Progress of the Healthier You: NHS Diabetes Prevention Programme: referrals, uptake and participant characteristics. Diabet Med 2018;35:513–8.doi:10.1111/dme.13562
      OpenUrl
    45. 45.↵
      Digital NHS Health Developer Network. Digital Assessment Questions - Beta London: NHS Digital. 2018 https://developer.nhs.uk/digital-tools/daq/
    46. 46.↵
      NHS England. Friends and family test: NHS England, 2014.
    47. 47.↵
      2. Craig CL ,
      3. Marshall AL ,
      4. Sjöström M , et al
      . International physical activity questionnaire: 12-country reliability and validity. Med Sci Sports Exerc 2003:35.
    48. 48.↵
      2. Hibbard JH ,
      3. Mahoney ER ,
      4. Stockard J , et al
      . Development and testing of a short form of the patient activation measure. Health Serv Res 2005;40(6 Pt 1):1918–30.doi:10.1111/j.1475-6773.2005.00438.x
      OpenUrlCrossRefPubMedWeb of Science
    49. 49.↵
      2. Cole SR ,
      3. Hernán MA
      . Constructing inverse probability weights for marginal structural models. Am J Epidemiol 2008;168:656–64.doi:10.1093/aje/kwn164
      OpenUrlCrossRefPubMedWeb of Science
    50. 50.↵
      2. Johnson BT ,
      3. Scott-Sheldon LA ,
      4. Carey MP
      . Meta-synthesis of health behavior change meta-analyses. Am J Public Health 2010;100:2193–8.doi:10.2105/AJPH.2008.155200
      OpenUrlCrossRefPubMedWeb of Science
    51. 51.↵
      2. Littenberg B ,
      3. MacLean CD
      . Intra-cluster correlation coefficients in adults with diabetes in primary care practices: the Vermont Diabetes Information System field survey. BMC Med Res Methodol 2006;6:20.doi:10.1186/1471-2288-6-20
      OpenUrlCrossRefPubMed
    52. 52.↵
      2. Ritchie J ,
      3. Spencer L
      . Qualitative data for applied policy research. In: Bryman A , Burgess R , eds. Analysing qualitative data. London: Routledge, 1994:173–94.

    Footnotes

    • Contributors EM and JI developed the initial outline evaluation plan in response to the NHSE tender; this was developed, refined and operationalised by EM, KD, WH and AL. WH led on the statistical analysis plan. JV chairs the NHSE Programme Board which developed and oversaw the tender. EM wrote the first draft of the protocol paper; all authors have read and commented on drafts and have approved the final version. PPI involvement was provided by Diabetes UK membership of the Board overseeing this work.

    • Funding This work was supported by NHS England, as part of the overall contract for implementation and evaluation of the Digital Diabetes Prevention Programme. EM is part funded by the NIHR School for Primary Care Research and the NIHR Collaboration for Leadership in Applied Health Research and Care, North Thames. AL is funded by the HEE Deanery (North Thames).

    • Competing interests JI is a partner and KD a management consultant at RSM. RSM hold the contract from NHSE to implement and evaluate the Digital Diabetes Prevention Programme. EM and WH receive consultancy fees for their work on the evaluation. EM is managing director of a not-for-profit Community Interest Company, HeLP-Digital, which exists to disseminate a digital diabetes self-management programme, HeLP-Diabetes, across the NHS. JV is the National Clinical Director for Diabetes and Obesity at NHS England.

    • Ethics approval Ethical approval has been granted by the Public Health England Research Ethics and Governance Group, reference R&D 324.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Patient consent for publication Not required.