Article Text
PDF
Abstract
Objective To predict the diabetes-related outcomes of people undertaking a type 2 Diabetes Self-Management Education (DSME) programme from their baseline data.
Design A mixed-methods longitudinal experimental study. 6 practice nurses and 2 clinical academics undertook blind assessments of all baseline and process data to predict clinical, behavioural and psychological outcomes at 6 months post-DSME programme.
Setting Primary care.
Participants –31 people with type 2 diabetes who had not previously undertaken DSME.
Intervention All participants undertook the Diabetes Manual 1:1 self-directed learning 12-week DSME programme supported by practice nurses trained as Diabetes Manual facilitators.
Outcome variables Glycated haemoglobin (HbA1c), diabetes knowledge, physical activity, waist circumference, self-efficacy, diabetes distress, anxiety, depression, demographics, change talk and treatment satisfaction. These variables were chosen because they are known to influence self-management behaviour or to have been influenced by a DSME programme in empirical evidence.
Results Baseline and 6-month follow-up data were available for 27 participants of which 13 (48%) were male, 22 (82%) white British, mean age 59 years and mean duration of type 2 diabetes 9.1 years. Significant reductions were found in HbA1c t(26)=2.35, p=0.03, and diabetes distress t(26)=2.30, p=0.03, and a significant increase in knowledge t(26)=−2.06, p=0.05 between baseline and 6 months. No significant changes were found in waist circumference, physical activity, anxiety, depression or self-efficacy. Accuracy of predictions varied little between clinical academics and practice nurses but greatly between outcome (0–100%). The median and mode accuracy of predicted outcome was 66.67%. Accuracy of prediction for the key outcome of HbA1c was 44.44%. Diabetes distress had the highest prediction accuracy (81.48%).
Conclusions Clinicians in this small study were unable to identify individuals likely to achieve improvement in outcomes from DSME. DSME should be promoted to all patients with diabetes according to guidelines.
- Self-management education
- Predicting outcomes
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
Strengths and limitations of the study
Thirteen diabetes-related clinical, behavioural and psychological outcomes were assessed for each participant.
Data from quantitative and qualitative sources were used.
Participants were new to diabetes self-management education.
Selection bias regarding psychological outcomes was possible.
In clinical practice, nurses have access to non-verbal clues and patient history in making assessments.
Introduction
Diabetes Self-Management Education (DSME) is advocated for people with diabetes by major diabetes organisations across the developed world.1–4 Outcomes of DSME trials have been equivocal with most programmes demonstrating some effect on a range of outcomes including glycaemic control,5 ,6 smoking cessation and illness beliefs,7 diabetes distress and self-efficacy,8 and quality of life.6 However, not all have demonstrated effects on the outcome of greatest clinical importance, namely glycaemic control.7–9 This has contributed to variability in healthcare professionals’ (HCPs) views of DSME, and the extent to which DSME is commissioned and delivered in health economies.2 UK National Institute for Health and Care Excellence (NICE) guidance advocates DSME (which in the UK is generally referred to as Diabetes Structured Patient Education) for all people with diabetes,10–12 with education being considered a key priority in the management of type 2 diabetes:Offer structured education to every person and/or their carer at and around the time of diagnosis, with annual reinforcement and review. Inform people and their carers that structured education is an integral part of diabetes care. (ref. 12, p.6)
However, in 2012–2013, in England only 6.0% of all people with type 2 diabetes, and 16.7% of those newly diagnosed, had been offered a DSME programme.13 Uptake of DSME was even lower; only 1.6% of all people with type 2 diabetes, and 3.6% of the newly diagnosed, were recorded as attending DSME.13 In addition to ambivalence towards DSME, some HCPs have explained low referral rates to DSME by arguing that they can anticipate who will benefit from such programmes and will only refer those for whom advantages are perceived.8 ,14 For example, HCPs have described reasons for low referrals based on their perceptions of patients’ ability to understand the content and awareness of the need for DSME.14 To address HCP ambivalence to refer patients to DSME, in England provision of DSME (Structured Education) became a Quality and Outcomes Framework15 item (pay for performance) in 2013 with the aim that more people will be referred. Nonetheless, referral does not guarantee uptake and attendance, and primary care professionals continue to have an important role in communicating the importance of DSME in improving a range of patient outcomes and encouraging attendance. The opportunity remains for HCPs to decide who to encourage based on their perceptions of likely patient benefit. Our objective was to assess the reliability of the argument offered by HCPs that they know whom to offer DSME. This paper presents findings from a broader study whose aims were to assess the feasibility of an enhanced Diabetes Manual programme. The research question addressed in this paper is: ‘Is it possible to predict the diabetes-related outcomes of people undertaking a type 2 DSME programme from their baseline and process data set?’.
Methods
Design
A mixed-method longitudinal experimental research design was employed between 2010 and 2012. Patients completed data collection when they consented to participate in the Diabetes Manual programme at the study baseline and at 6 months follow-up. Minimal important difference, that is, the smallest difference in outcome for a patient that is perceived to be meaningful,16 was used to measure change in outcomes for each individual. This paper presents a person-centred analysis.17 The detailed longitudinal data set enabled the construction of individual narratives examining how baseline variables are linked to specific outcomes.18
The DSME intervention
The Diabetes Manual is an evidence-based 1:1 DSME programme for type 2 diabetes largely self-directed with support from practice nurses who have been trained as Diabetes Manual facilitators (DMF) to elicit behavioural changes and to provide psychological support. The Diabetes Manual consists of a workbook, relaxation audio components and a minimum of three face-to-face or telephone DMF contacts as preferred by the participant. The Diabetes Manual is designed to take 3 months to complete, involving approximately an hour a day for participants. Examples of how this hour may be spent includes reading the Diabetes Manual, taking physical activity, reading food labels, cooking a healthy meal, blood glucose monitoring or listening to the relaxation audio programme. The Diabetes Manual is evidence based8 ,19 ,20 and available for commissioning in the National Health Service (NHS) or direct purchase by people with type 2 diabetes.21
Participants
Six practice nurses employed at general practices were recruited through the Primary Care Research Network and opportunistic sampling and two nurses were recruited from a hospital diabetes clinic. All were trained to become DMFs. The nurses each were asked to recruit up to 10 patients with type 2 diabetes, the ability to read English, glycated haemoglobin (HbA1c) >7.4% (57.4 mmol/mol) and who had not attended diabetes self-management education. Our sample size was based on the aims of the broader study in which we aimed to recruit 50 participants to give 80% confidence for assessing changes in diabetes management self-efficacy. After 2 months, two DMFs withdrew due to time pressures without recruiting patients. The remaining six DMFs consented 31 patients, four of whom subsequently withdrew.
Procedures
Following completion of the training, DMFs arranged an appointment with participants to obtain informed consent, introduce them to the Diabetes Manual programme and collect baseline clinical assessments. The DMFs also gave the participants psychological outcome questionnaires and an accelerometer with instructions for use, which were collected from the participant 7 days later by the researcher. Within 2 months of recruiting their first patient, the DMFs were observed in two or three consultations by a trained facilitator for quality assurance (QA) purposes as per NICE guidance.11 A total of 17 QA consultations took place and were recorded. After their QA consultation, each patient took part in a brief interview with a researcher about their experiences in the consultation. Follow-up data collection took place 6 months postbaseline. Participants attended an appointment with their DMF where clinical measurements were taken. The psychological measures and accelerometer were sent to patients a week before their follow-up appointment. These measures were completed prior to their appointment. Finally, all participants took part in a follow-up interview to discuss their experiences of the DM programme, and any areas where the DSME had impacted on their diabetes management. The clinical, psychological, behavioural, process and demographic data known to impact on, or be impacted by DSME, and collected for the main study, represents the data set used to develop the outcome assessment framework.
Outcome assessment framework
Our earlier Diabetes Manual randomised controlled trial (RCT)8 had found improvements in diabetes distress and self-efficacy. Subsequent meta-analyses22 had identified reduction in HbA1c with psychological intervention. We hypothesised therefore that with the addition of psychological care components to the Diabetes Manual, we would find clinically and statistically significant reductions in HbA1c. We further hypothesised that the more suboptimal the outcome in each patient, the greater the improvement would be although we did not develop hypotheses regarding DMF’s outcome prediction accuracy. We did not involve participants in making self-assessments regarding their anticipated outcomes of participating in the study. Each variable used in the outcome assessment framework is described, and inclusion justified, in table 1. Variables were included in the framework if they were (1) known to influence self-management behaviour (eg, ethnicity;23 depression24) or (2) known to have been influenced by a DSME programme in empirical evidence (eg, HbA1c;5 ,6 diabetes distress8 ,25). Data were collected at baseline and 6-month follow-up. Process indicator data, such as any patient change talk or value verbalised about the DSME, were collected during the QA process. Baseline measures and process indicators formed the data set used to ask the question, ‘Based on this patient’s data do I think that engaging with DSME will result in any changes in HbA1c, waist circumference, exercise levels, anxiety, depression and distress in 6 months’ time for this patient?’ With each patient and each outcome, the DMFs made a prediction through recording one of three expectations (1) the outcome would improve by a minimally important difference,16 (2) the outcome would deteriorate or (3) there would be no change. Specifically for prediction purposes, data were presented in a table format alongside clinical guidelines relevant to each outcome. All data in the framework could be available to nurses during routine consultations if they chose to access the information.
Variables within the predictive framework
Methods of predicting outcome
Two clinical academic research team members (JS, a nurse and FG, a general practitioner (GP)) developed and pilot tested the prediction method using the outcome assessment framework. They independently examined individual patient baseline and process data and made predications pertaining to 6-month follow-up outcome. This iterative process was piloted with seven patients and then applied with the study population. Following individual predictions, discussion took place to reach agreement. On a separate occasion, six DMFs used the framework to predict outcomes on each other's anonymised patient data (ie, participants not known to them) in a recorded focus group. Patient data were randomly allocated to individual DMFs, ensuring that each DMF examined a unique subset of different patients, and that all patients’ data were examined by two different DMFs. Initially the DMFs made individual predictions for their own subset of 9 patients, then collectively they discussed and made predictions for 14 patients. Therefore, each participant data set was individually assessed by two DMFs (different for each patient) to produce specific predictions and a brief outcome narrative (see table 2, eg, data and predictions). Once all predictions were completed the 6-month follow-up data were entered into the framework to enable analysis. The standard of assessment for determining a positive change in outcomes for each patient was the minimally important difference unique to each outcome (as listed in table 1). During analysis, where no prediction was given for change or stability, this was classed as ‘no change’ predicted. Where there were disagreements about the prediction, this was noted and the prediction made by the majority was used in analysis. Accuracy is described per patient (ie, how many of the 6 outcome predictions per patient were accurate), and per outcome (ie, what percentage of the 27 predictions made for each outcome were accurate).
Example prediction framework data
Results
Quantitative data on clinical, psychological and behavioural outcomes were collected at baseline from 30 patients and at 6 months from 27 patients. Of the latter, 13 (48%) were male, 22 (82%) were white British, ages ranged from 39 to 81 years (mean 59.2 years) and duration of type 2 diabetes was 3 months to 34 years (mean 9.1 years). See table 3 for participant clinical and psychological characteristics at baseline and follow-up. Among the whole group, there were significant reductions in HbA1c% t(26)=2.35, p=0.03, and diabetes distress t(26)=2.30, p=0.03, and a significant increase in knowledge t(26)=−2.06, p=0.05 between baseline and 6 months (see table 3). No significant changes were found in waist circumference t(26)=−0.43, p=0.67; physical activity t(26)=0.99, p=0.33; anxiety t(26)=−1.39, p=0.18; depression t(26)=−0.38, p=0.71; or self-efficacy t(26)=−1.83, p=0.08.
Clinical and psychological characteristics of participants
Researcher predictions (pilot)
The accuracy of the researcher predictions for change in the key clinical, behavioural and psychological outcomes for the seven pilot patients was examined. Accuracy of predictions ranged from no accurate outcome predictions to all six outcome predictions accurate. Accuracy of predictions per outcome varied from 42.86% to 71.43% (ie, of the seven predictions made for each outcome (one for each patient), between three and five were accurate). Prediction agreement between the two researchers was 96%.
DMF predictions
The accuracy of the DMF predictions for key clinical, behavioural and psychological outcomes for the whole data set was examined. Although there were disagreements in individual predictions, an overall reduction in HbA1c was predicted for all patients. Change was less widely predicted for other variables. Clinically important outcomes not captured by clinical, psychological and behavioural data but revealed during interview include dietary changes and reduction in alcohol intake (n=7), increased medication adherence (n=1) and smoking cessation (n=1).
Accuracy per patient
Accuracy of the six outcome predictions per patient ranged from one to six, with the median and mode being four out of six predictions. There were between one and six prediction agreements per patient, with an average of 4.12 prediction agreements in outcomes per patient. Overall, prediction agreement for the DMFs was 68%.
Accuracy per outcome
Accuracy for predictions by outcome varied from 44.44% for HbA1c to 81.48% for diabetes distress. The median and mode accuracy was 66.67%. Table 4 shows the nature of the change predicted for each outcome and the percentage of patients in which changes were predicted for each outcome, the nature of actual change observed and the percentage of accurate predictions made. Many predictions were correct because no change was predicted and this was accurate.
DMF predictions as to the impact of the DSME on clinical outcomes, and the accuracy of these predictions
Discussion
Summary of main findings
Findings from this study identified that accuracy for predicting change in HbA1c by the nurses was low with their accuracy for reductions in diabetes distress higher. We found that the majority of accurate predictions related to an anticipation of no change (ie, the nurses thought the patients 6-month outcomes would not change) in relation to waist circumference, physical activity levels, anxiety and depression. Overall, there was greater accuracy in predictions about lack of change, than in identifying individuals who would achieve improvement in outcome. Predictions for change were made most frequently for measures that showed the greatest room for change (ie, those outcomes in those patients with high baseline scores). The Diabetes Manual DSME programme has continued to demonstrate improvements in glycaemic control and diabetes distress in this new population.8 High HbA1c was an inclusion criterion for participation in the study and there was consistency in the prediction that the DSME intervention would reduce this, suggesting that health professionals can find it difficult to predict in what ways people will benefit from DSME. Health professionals appear to believe that their patients will change (optimistic bias) where scope for change is evident, and where they believe in the efficacy of their treatment endeavours. Prediction agreement within the group of nurses was much lower than for the researchers. This could indicate the lack of experience in undertaking this exercise in contrast to the researchers who had developed the process (so were more practiced in applying it). It was notable that, where available, the qualitative data were heavily drawn on by the nurses to inform predictions. Qualitative evidence was used to provide insight into underlying motivations for behaviour change and attitudes towards the intervention, consistent with the narrative development process for narratives exploring how baseline variables are linked to specific outcomes.18
Strengths and limitations of the study
The research intervention used a tried and tested DSME programme which enabled the DMFs to focus on the research questions presented and not on DSME programme evaluation. The nurses were aware of the earlier RCT findings which may have influenced their decision-making. This research benefits from the assessment of a wide range of clinical and psychological measures, so that the anticipated impact of DSME on a variety of outcomes could be assessed. However, it is limited by the lack of a measure of eating habits. This is a key behavioural change targeted by the DSME, and which has the potential to significantly improve HbA1c.26 The baseline characteristics show that our participants’ anxiety and depression levels were low, so few changes were expected. This may indicate a participant selection bias, as people with depression could have been less likely to consent, or be offered the opportunity, to participate in the study. Several outcomes in our framework were self-report, and completing these could have been interventional in themselves by causing the participant to reflect on their mood, coping, confidence or knowledge. The DMFs were all experienced at working with people with type 2 diabetes, and so were able to draw on their clinical experience of this patient group when making predictions. The process and outcome narratives were, however, produced in a fairly artificial setting. In consultations, health professionals often have prior knowledge of their patients and draw on non-verbal data to make their assessments. Despite this, the health professionals did articulate why they would expect particular outcomes, making explicit the clinical evaluation process. In relation to study eligibility criteria, the DMFs struggled to first identify, and then recruit, patients who had not previously attended locally offered DSME and who could read English. This could indicate that patients in the settings served by these nurses had already been extensively offered DSME in contrast to the national average of 6.0%.13
Comparison with wider literature
Nurses tend to draw heavily on their experience when interpreting the different sources of information available during routine consultations experience.27 ,28 In this study, the DMFs used their experience to make inferences from the available data. Such inferences demonstrate how previous clinical experiences inform (or even bias) current evaluations.29 ,30 Physicians’ clinical experience and knowledge have long been viewed as the ‘quintessential skills’ that they have to offer (ref. 31, p.657). Used alone, however, professional opinion has been described as the ‘least reliable and valid form of evidence’ on which to base clinical decisions (ref. 30, p.232), below hierarchies of research evidence. One systematic review found that physicians with more experiential knowledge were less likely to adhere to appropriate standards of care, a finding which they describe as inconsistent with the notion that experience enhances knowledge and skills, leading to better patient care.32 Other authors have claimed that experienced clinicians form hypothesises and diagnostic plans more quickly and to a higher standard than inexperienced clinicians.33 The benefit of clinical experience in making evaluations and decisions is somewhat contested then. In this study, where nurses had to make clinical judgements without physically seeing a patient, they had few other resources to draw on other than their experience with similar patients. This wider literature and the findings of this study have implications for the training of clinicians who refer people to DSME. If this study were repeated with GPs the findings may have been different.
Conclusions
Our research indicates that while clinicians draw on their extensive clinical experience in assessing the benefit of DSME, in our relatively modest group of 6 practice nurses and outcome data on 27 people with diabetes, it was not possible for them to reliably and accurately determine outcomes utilising 10–12 pieces of data per person on which to make these assessments. Our results indicate that all people with diabetes should continue to be offered DSME programmes according to national2 ,10 ,11 and international guidelines.1 ,3 ,4 Furthermore, research should explore the clinical decision-making process, to make explicit the process through which clinicians make judgements on the potential benefit (or not) of DSME for different patients. Further exploration of this topic could highlight how clinical experience is used to interpret data within current situations, and the outcome this has for the patient in their access to care.
Acknowledgments
The authors would like to thank the practice nurses and people with diabetes who took part in this study, NHS Coventry and Warwickshire for supporting the study and the Warwick Research and Education User Group for their Patient and Public Involvement in the study design and execution. The research study has been supported by a grant from the National Institute for Health Research (NIHR) Research for Patient Benefit programme (PB-PG-0808-17085). This article presents independent research funded by the NIHR.
References
Footnotes
Twitter Follow Jackie Sturt at @jackie_sturt
Contributors JS developed the Diabetes Manual DSME programme. JS, JD, FG, IC, RW and JPO designed and managed the study and interpreted the findings. RW and IC carried out the DMF training. CH carried out the data collection and outcome analysis. JS, FG and CH designed the predictions framework, undertook predictions and CH analysed the data. All authors contributed to the development of the manuscript.
Funding The research study has been supported by a grant from the National Institute for Health Research (NIHR) Research for Patient Benefit programme (PB-PG-0808-17085). This article presents independent research funded by the NIHR.
Competing interests JS and RW have licence to distribute the Diabetes Manual programme within the NHS for profit. RW runs a small business to which this licence has been offered by University of Warwick. JS has IP rights associated with her development of the Diabetes Manual and has a royalty sharing agreement with University of Warwick.
Ethics approval Coventry and Warwickshire Research Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.