You are here

  • Home
  • Archive
  • Volume 5, Issue 6
  • Lessons learnt from a cluster-randomised trial evaluating the effectiveness of Self-Management Support (SMS) delivered by practice nurses in routine diabetes care

Article Text

Article menu

Download PDFPDF

XML
General practice / Family practice
Research
Lessons learnt from a cluster-randomised trial evaluating the effectiveness of Self-Management Support (SMS) delivered by practice nurses in routine diabetes care
  1. Anneke van Dijk-de Vries1,2,
  2. Marloes A van Bokhoven1,
  3. Bjorn Winkens3,
  4. Berend Terluin4,
  5. J André Knottnerus1,
  6. Trudy van der Weijden1,
  7. Jacques Th M van Eijk2
  1. 1Department of Family Medicine, Maastricht University, CAPHRI School for Public Health and Primary Care, Maastricht, The Netherlands
  2. 2Department of Social Medicine, Maastricht University, CAPHRI School for Public Health and Primary Care, Maastricht, The Netherlands
  3. 3Department of Methodology and Statistics, Maastricht University, CAPHRI School for Public Health and Primary Care, Maastricht, The Netherlands
  4. 4Department of General Practice and Elderly Care Medicine, VU University Medical Centre, EMGO Institute for Health and Care Research, Amsterdam, The Netherlands
  1. Correspondence to Anneke van Dijk-de Vries; anneke.vandijk{at}maastrichtuniversity.nl

Abstract

Objective To evaluate the effectiveness of biopsychosocial Self-Management Support (SMS) delivered by practice nurses in routine diabetes care.

Design A pragmatic cluster-randomised controlled trial within a hybrid effectiveness-implementation study design. Practice nurses were cluster-randomised.

Setting A regional care group in the Netherlands consisting of 77 family practices. The study involved practice nurses (n=40) providing care to approximately 4000 patients with diabetes.

Participants Patients with type 2 diabetes (n=264) selected by a self-administered questionnaire aimed at measuring emotional distress and diabetes-related reduced daily functioning.

Intervention Practice nurses in the intervention arm (n=19) were trained to integrate SMS into their routine consultations. SMS included detection of patients with emotional distress and reduced daily functioning, and supporting them when needed through problem solving and reattribution techniques. Practice nurses in the control arm (n=21) provided usual care.

Main outcome measures The primary outcome measure was a dichotomised score on a Visual Analogue Scale that measured the perceived effect of diabetes on daily functioning. Secondary measures included patients’ diabetes-related distress, quality of life, autonomy and participation, self-efficacy, self-management and glycaemic control. Outcomes were measured at baseline and at 4-month and 12-month follow-ups.

Results Only 16 of the 117 patients in the intervention arm (14%) who were found eligible by the posted research-driven screening questionnaire were detected by their practice nurses. Extra consultations for the self-management support were delivered to only 11 study participants. In the control arm, 147 patients received usual care. Multilevel analyses showed no significant differences in outcomes between the intervention and control arms.

Conclusions SMS in its present form was not effective. The research-driven screening to select trial participants appeared to be inconsistent with nurse-led detection in routine practice. Adequate follow-up moments need to be built in to overcome barriers resulting from tension between the implementation and effectiveness parts of hybrid studies.

Trial registration number Current Controlled Trials NTR2764.

  • PUBLIC HEALTH
  • PRIMARY CARE

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2014-007014

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • A thorough cluster-randomised trial to evaluate the effectiveness of an evidence-based intervention integrated into routine primary care.

    • Limited interference of the researcher in the routine care setting.

    • Low exposure of study participants to the complete intervention.

    • Absence of a pilot of the detection method prior to implementation.

    Introduction

    Patients with chronic diseases such as type 2 diabetes mellitus have to deal with challenging day-to-day management tasks regarding the medical, emotional and social consequences of their chronic condition.1 ,2 Research has shown a reciprocal relationship between the emotional health status of people with diabetes and their medical self-management: emotional distress may interfere with control over the disease, whereas poor control over the disease can lead to emotional distress.3–6 Studies highlight the need to support the medical as well as the emotional and role management tasks in newly diagnosed patients and patients who face the longer-term consequences of their chronic condition.7–9

    In European countries, most patients with diabetes receive follow-up care in the primary care setting by nurses.10 Practice nurses (PNs) in the Netherlands work according to guidelines that focus on medical and behavioural management.11 ,12 Furthermore, the financial reimbursement of care and patients’ health outcomes are determined by biomedical targets.13 Consequently, the psychosocial aspects of diabetes care are not systematically incorporated in clinical practice.10 ,13–15

    In collaboration with a regional care group of general practitioners (GPs) in the Netherlands and a health insurer, the SMS (‘Self-Management Support’) implementation project was initiated to realise a shift from biomedically oriented care towards a biopsychosocial approach in diabetes care.16 The starting point was a nurse-led minimal psychological intervention. A previous randomised trial has shown that this intervention was cost-effective for patients with diabetes with minor to moderate depression: 9 months after receiving the intervention, depressive symptoms were significantly lower, there was a positive effect on patients’ quality of life, and patients experienced less anxiety, possessed more self-efficacy skills, demonstrated better glycaemic control and showed more participation in comparison to control patients.17–19 There was an implementation momentum. The health insurer promised to pay the costs for the extra care, and the care group gave a commitment to take care of training facilities and integration of SMS parameters into Electronic Medical Records. Some adjustments to the nurse-led intervention were crucial. Where the original intervention had been delivered at patients’ homes by specifically trained nurses from the research team who selected eligible patients by means of an elaborate diagnostic procedure, in SMS it was provided by PNs as a structural part of their consultations delivered in the family practice. For the identification of eligible patients, a simple detection method became an inherent part of SMS. The eligibility criteria changed from having a mild to moderate depression towards suffering from both emotional distress and interference in their daily functioning due to the burden of diabetes. This focus on daily functioning instead of a diagnosis of depression was supposed to fit the primary care setting. Patients who were detected by the PN received the nurse-led minimal psychological intervention or were referred to the GP to see whether more specialised care would be required, depending on the severity of the symptoms.

    SMS was evaluated by means of a type 2 hybrid effectiveness-implementation study design, in which the regional implementation strategies and the effectiveness of SMS were evaluated simultaneously.20 This paper will focus on the effectiveness part of the SMS project, though it is closely connected to the implementation part. This paper reports the effectiveness of SMS integrated by PNs into routine diabetes consultations regarding patients’ daily functioning, emotional health, quality of life, autonomy and participation, self-efficacy and self-management skills, and blood glucose levels.

    Methods

    Study design

    A two-arm, pragmatic, cluster-randomised controlled trial was conducted with PNs as the unit of randomisation and their patients with type 2 diabetes mellitus as the unit of analysis. Measurements were at baseline, at 4 months and at 12 months. The study was conducted between November 2011 and February 2013 and was registered in a Dutch public trial registry [NTR2764]. The study protocol has been published elsewhere.16

    Study participants

    All the family practices of one regional organisation of GPs (a so-called ‘care group’) in the South of the Netherlands were eligible to participate in the study. GPs were asked to participate in the SMS project. Their PNs for diabetes care were randomly assigned to an intervention or control arm. PNs in the intervention arm were trained in SMS, whereas PNs in the control arm provided usual diabetes care, conforming to the Dutch guidelines. Owing to the variety of PNs working solo versus in a team, and in one versus more practices, PNs were stratified into working alone in a practice, working in a team and working in different settings. The randomisation was performed by an independent research assistant who used a random number seed computer program to assign PNs to study arms, assuming an allocation ratio of 1:1. PNs working together in a practice were clustered for being randomised to the same trial arm to avoid the risk of contamination if the SMS and usual care would be delivered in the same family practice. Regarding the analysis, a team of PNs was seen as one cluster if patients could receive diabetes care from these PNs alternately.

    In the selection of patients for the effectiveness trial, the following issues were crucial. First, we wanted to select patients who would receive the complete intervention including detection and follow-up. Furthermore, patients from the intervention and control arms should be selected in a similar way, without interfering in their routine care. We set up a screening procedure by means of written questionnaires sent to patients’ home addresses to identify patients with actual problems of daily functioning and emotional distress. Patients with a clinically established diagnosis of type 2 diabetes mellitus were sent a letter by their GPs to introduce the SMS project. The enclosed self-administered questionnaire consisted of the screening instruments that intervention PNs would also apply for SMS in their routine practice. It included the ‘Daily Functioning Thermometer’ (DFT), which is a Visual Analogue Scale to measure how suffering under the burden of diabetes affects patients’ perceived functioning in everyday life. Patients were asked to indicate a position between 0 (no burden at all) and 10 (extreme burden) at a continuous vertical 10 cm line. A score of 4 was chosen to differentiate between patients who could benefit from support in their self-management skills (score >4) and patients who had found a satisfactory way to live with the consequences of their diabetes (score ≤4). The DFT has been developed for the purpose of this study. It is comparable to the Distress Thermometer, a validated questionnaire in the care for patients with cancer,21 and a validated Visual Analogue Scale regarding the worst (0) and the best possible life (10) for adolescents with type 1 diabetes mellitus.22 Next to the DFT, patients were asked to complete the three-item Distress Screener (DS), which is a quick-scan instrument for emotional distress and an indicator of potential underlying severe mental health problems.23 Patients with score DFT >4 and DS >3 were eligible for trial participation. The time interval between the researcher-driven self-administered screening questionnaire and the face-to-face nurse-led detection procedure in the consultation room needed to be as short as possible. However, intervention patients should complete the baseline measurement before SMS is applied. Owing to the logistics, patients received the posted screening questionnaire 4–6 weeks before a planned diabetes consultation. Reminders were sent at least 3 weeks before a consultation. Patients were asked to return the completed questionnaire to the research centre and to give informed consent to be approached if they were found to be eligible for research follow-up measurements. Within a week after receiving the screening questionnaire at our research centre, eligible patients were sent an invitation letter together with an informed consent form for trial participation and the baseline measurement. Patients who gave informed consent knew whether they would receive an addition to their usual care or not. No details were given about the content of the intervention.

    Intervention: SMS in routine care

    During three 8 h training sessions, PNs in the intervention arm were trained to integrate the detection and follow-up phase of SMS into their daily practice. The training sessions were followed up by booster sessions to maintain and improve PNs’ skills in SMS. PNs attended these booster sessions three or four times during the year of follow-up.

    After the training sessions, PNs started to integrate SMS into their routine care practice. PNs were blinded regarding the outcomes of the recruitment procedure and study participation of their patients. They applied SMS in all their consultations with patients with diabetes. Financial reimbursement for the extra time spent on SMS was added to the bundled payment arrangement for diabetes care.24

    SMS included a detection and follow-up phase. The flow chart of SMS is presented in figure 1. The detection phase of SMS started by exploring whether a patient experienced problems in daily life. PNs then applied the DFT by showing patients a vertical line ranging from 0 to 10 with 0 indicating no burden at all and 10 indicating extreme burden, and asking them to rate the burden of diabetes in their daily life. PNs also verbally administered the three questions of the DS.23 Patients with score DS >3, which indicates high risk for underlying mental health problems, were asked to complete the Four-Dimensional Symptom Questionnaire (4DSQ)25 immediately after the consultation. This self-report instrument is widely used in Dutch primary care to distinguish non-specific distress from depression, anxiety and somatisation. It differentiates between mild, moderate or severe symptoms. The completed 4DSQ was returned to the PN, who computed the sum scores on each domain.

    Figure 1
    Figure 1

    Self-Management Support (SMS) as implemented in routine care. DFT, Daily Functioning Thermometer; DS, Distress Screener; 4DSQ, Four-Dimensional Symptom Questionnaire; GP, general practitioner.

    The follow-up phase of SMS was based on the outcomes of the DFT and 4DSQ. Patients registering a ‘severe’ score on at least one of the four subscales of the 4DSQ were referred to the GP for further diagnostics and treatment.

    Patients who experienced problems of daily functioning (DFT >4) and emotional health problems (moderate scores on at least one subscale of the 4DSQ) were offered consultations for SMS. These extra consultations delivered by PNs were aimed at supporting patients in their day-to-day management of diabetes and its emotional and social consequences. The intervention strategy derived from the principles of learning theory has been described elsewhere.26 PNs supported patients in the processes of defining problems and finding solutions themselves, by applying problem-solving and reattribution techniques. Problem-solving consists of seven stages that efficiently address problems and their possible solutions.27 The reattribution technique was applied to challenge patients to link feelings and cognition to consequent behaviour. Patients could use information from a diary in which they recorded symptoms, thoughts, worries, feelings, and behaviour. Both problem solving and reattribution techniques were intended to result in action plans indicating how patients would achieve their personal goals.28

    Measurements effectiveness trial

    We used postal questionnaires for patient measurements. Patient characteristics regarding their age, sex, year of diagnosis, treatment of diabetes (insulin therapy or tablets), marital status, work status and educational level were assessed at baseline.

    The glycaemic control of patients was measured during diabetes consultations. These measurements were extracted from the electronic patient databases.

    In the year following the training, PNs in the intervention arm recorded process measures and outcomes of SMS on a specific registration form. These data were used to evaluate whether patients were exposed to SMS.

    Outcomes

    The effect of SMS on patients’ daily functioning was measured by means of the DFT. The DFT was dichotomised to compare the number of patients who were improved in their daily functioning and had apparently found a satisfactory way to live with the consequences of their diabetes (DFT ≤4) with patients who still perceived a burden of diabetes on their daily functioning (DFT >4).

    The secondary outcome measure ‘diabetes-related emotional distress’ was measured by the 20-item Problem Areas in Diabetes questionnaire (PAID).29–31 Scores were transformed to a 0–100 scale, with 0 being the most favourable outcome. Missing items were imputed using patients’ individual mean score if at least 50% of items were available.

    Participation and autonomy were measured by means of the Impact on Participation and Autonomy (IPA) questionnaire.32 The response options of the 32 items ranged from 0 to 4 with higher scores representing poorer participation and autonomy. Five subscale scores were computed: autonomy indoors (range 0–28), family role (range 0–28), autonomy outdoors (range 0–20), social relationships (range 0–28) and work and education (range 0–24). In accordance with the manual, missing items were imputed using the individual mean scores of items that were not missing in patients for whom at least 75% of items were available.33

    Patients’ self-management knowledge and behaviours regarding their diabetes were measured using the Dutch version of the Partners in Health scale (PIH-NL).34 The 12 items covered four domains of patients’ competency in relation to their self-management (knowledge, coping, management of condition and adherence to treatment). We used the total sum score.

    The 12-item Short-Form Health Survey (SF-12) measured the quality of life.35 A physical component summary and a mental component summary were computed by using item weights and regression constants for the Dutch population derived from the oblique rotation method.36 ,37 The norm-based summary scores have a mean of 50 and an SD of 10 in the Dutch population.37 Higher scores reflect better outcomes on the quality of life.

    The General Self-Efficacy Scale (GSES-12) assessed patients’ belief in their ability to organise and engage in certain behaviours.38 ,39 A higher sum score (range 12–60) reflects higher levels of self-efficacy. Missing items were imputed using patients’ individual mean score if at least 50% of items were available.

    The glycated haemoglobin in mmol/mol was measured during consultations. For the baseline measure, we used data from the first consultation after the patient's baseline measurement (with a maximum interval of 3 months). The consultation following this first consultation was used as the 4-month follow-up measurement. The last measurement was the assessment approximately 1 year after the patient's baseline measurement (at least more than 6 months after the first consultation).

    Statistical analysis

    The power calculation was based on the dichotomous DFT.16 On the basis of a group size of 46 PNs, we determined in advance that a sample size of 232 patients (at least 5 patients per PN) would have 90% power and an α of 0.05 to detect an improvement in perceived daily functioning (defined as DFT ≤4) at 12 months measurement occurring in 20% of patients in the intervention arm versus 5% of those in the control arm. In this power calculation for cluster-randomised trials, an intraclass correlation coefficient of 0.04 was used. Assuming that not all positively screened patients would give informed consent for trial participation, and a 30% loss to follow-up, we planned to invite 10 eligible patients for each PN.

    Baseline variables were compared by means of independent-samples t tests and χ2 tests to detect statistically significant differences between both groups at baseline. For the effect evaluation, linear and logistic multilevel models were used with an unstructured covariance structure for repeated measures, and patients and PNs as random factors. These models account for the correlation between the repeated measurements for each patient and also for the hierarchical structure of the data, with measurements (level-1) clustered within patients (level-2) who in turn were clustered within PNs (level-3). Type of practice, that is, the stratification variable, time of measurement (categories: 0, 4 and 12) and the interaction between group and time of measurement were included in the models as fixed effects. Analyses were performed on an intention-to-treat basis.

    To ensure that all participants of the intervention and control arms started with a score DFT >4, indicating a perceived burden of diabetes, we used the dichotomised research-led screening outcomes at baseline. Consequently, the model only included 4-month and 12-month follow-ups as the baseline value was the same for all participants.

    A two-sided p value <0.05 was considered statistically significant. Statistical analyses were conducted using the software packages IBM SPSS Statistics V.21.0 for Windows and SAS V.9.3.

    Results

    Participants

    From the 77 family practices that were approached between April and June 2011, 40 agreed to participate. Their PNs (n=41) were by randomisation assigned to the intervention arm (20 PNs) and the control arm (21 PNs). After randomisation, but before patient recruitment, one family practice in the intervention arm withdrew from study participation due to the heavy workload of the PN. This left 19 PNs who received training in SMS and integrated it into their daily practice, and 21 PNs in the control arm who provided usual care.12 As some PNs worked together in a team, 15 units of analysis were left in the intervention arm and 19 units of analysis in the control arm.

    A total number of 3822 patients with diabetes were sent a screening questionnaire in the period from October 2011 to March 2012. The response rate was 50% in the intervention arm and 44% in the control arm. As screening questionnaires were sent by GPs, we could not compare responders with non-responders with regard to demographic characteristics.

    From all 1805 patients who participated in the screening, 357 (20%) met the detection criteria and were invited to take part in the trial. A total of 264 patients (74%) gave informed consent to participate in the trial. Their research-led screening outcomes were not significantly different from outcomes from eligible patients who refused trial participation.

    From the 117 intervention patients and 147 control patients who participated in the trial, 80% completed all of the follow-up measurements. In 10% of the sample, one follow-up measurement was missing. Three patients did not complete the baseline measurement and gave informed consent at the 4-month follow-up measurement. Another 23 patients completed only the baseline measurement. We found no baseline variables that were significantly related to incompleteness of measurements. Figure 2 illustrates the flow of recruitment and randomisation of PNs, screening of patients and measurements of trial participants.

    Figure 2
    Figure 2

    Flow of participants through the Self-Management Support (SMS) trial. DFT, Daily Functioning Thermometer; PNs, practice nurses.

    Baseline characteristics

    Table 1 illustrates the baseline characteristics of participants in the SMS trial.

    View this table:
    Table 1

    Baseline characteristics of study participants in intervention and control arms

    The sample comprised 46% female patients with the majority having a lower level of education. The average age of the participants was 65. Of all 121 participants younger than 65 years, 41% had paid employment. The proportion of patients with a paid job was significantly higher in the intervention arm than in the control arm. On average, patients had been diagnosed with diabetes for 8–9 years. The majority were being treated with oral hypoglycaemic medication. The number of patients being treated with insulin was significantly higher in the intervention group (33%) compared to patients in the control group (16%). At the time of baseline measurement, 84% were not receiving psychosocial care from a health professional.

    Patients of both groups were comparable for the primary and secondary outcomes at the baseline measurement except for the sum score on the PIH scale. This reflected better self-management skills of intervention patients regarding knowledge, symptom management and adherence to treatment. Both groups were comparable regarding the subdomain ‘coping’ that included three items about dealing with the effects of diabetes on daily life.

    Study participants’ exposure to SMS

    An unexpected outcome was the very low exposure to SMS of the study participants in the intervention arm. It is presented in table 2. PNs did not register any data about SMS for 17 participants (15%) in the patient file during the year of follow-up. For the other 100 participants, between 1 and 5 diabetes consultations were registered with data on the DFT and DS. The combination of DFT >4 and DS >3 was found in only 16 participants (14%).

    View this table:
    Table 2

    Exposure to SMS of study participants in the intervention arm (n=117)

    A total of 46 study participants scored neither DFT >4 nor DS >3 during consultations. The low detection rates also implied low exposure to the follow-up phase of SMS.

    From 30 participants who scored DS >3 and were therefore eligible for further diagnostics, 10 patients did not receive the 4DSQ because (1) they were already undergoing psychological treatment, or (2) they were not interested, or (3) a specific stressor was clearly causing the distress. Extra consultations for SMS were provided for 11 participants, of whom there were only two patients with a nurse-led detection score of DFT >4 and DS >3. From the other 14 participants with positive scores for DFT and DS, six patients were referred to the GP and one patient to a mental nurse because severe mental health problems were recorded using the 4DSQ. Five participants with score DFT >4 and DS >3 did not want to receive follow-up care, and two participants were already receiving psychological treatment.

    Primary outcome

    As illustrated in table 3, the number of patients who improved in their daily functioning (score DFT ≤4) was lower in the intervention group than in the control group at 4-month follow-up, but this was not a statistically significant difference (24% vs 32%; OR: 0.505 (95% CI 0.213 to 1.201)). At the 12-month follow-up, more patients in the intervention group improved in their daily functioning compared to patients in the control group. However, this was not statistically significant either (33% vs 26%, OR 1.754 (95% CI 0.742 to 4.148)).

    View this table:
    Table 3

    Multilevel analyses for differences between intervention and control arms regarding improvement on patients’ daily functioning reflected by the dichotomous primary outcome (DFT ≤4) at 4-month follow-up and at 12-month follow-up

    Secondary outcomes

    For the secondary outcome measures, the intervention group perceived significantly more participation than the control group regarding autonomy indoors and family role (p<0.05) after 4 months. The same applied, with nearly statistically significant differences, for autonomy outdoors (p=0.06) and social relationships (p=0.05). However, these effects disappeared at 12-month follow-up. As illustrated in table 4, we found no other statistically significant differences for the secondary outcome measures at 4 and 12 months.

    View this table:
    Table 4

    Multilevel analyses for differences between intervention arm and control arm for secondary outcome measures at 4-month and 12-month follow-ups

    Discussion

    This study, with a low exposure of study participants to the complete intervention, could not demonstrate any effect of SMS on emotional, social or biomedical parameters, except for autonomy and participation with regard to activities indoors and the family role after 4 months. This effect in favour of the patients in the intervention arm had disappeared after 12 months.

    The critical issue of the implementation of SMS was the addition of a detection procedure as an inherent and integrated part of SMS. PNs were trained to identify eligible patients for the follow-up phase of SMS by applying screening questions during diabetes consultations. The DFT and DS were considered to be simple, patient-centred indicators of patients’ daily functioning and their emotional distress. The 4DSQ would enable the PNs and GPs to get more insight into the presence and severity of the emotional problems. For the effectiveness trial, a self-administered postal screening procedure was inserted into the study design to select patients from both the intervention and control arms similarly. To perform a pragmatic trial, we minimised the interference of the researchers in clinical practice as much as possible. The outcomes of DFT and DS on the research-driven screening were consistent with our assumptions based on the literature that approximately 20% of the diabetic population would be eligible for the follow-up phase of SMS. However, these efforts to select patients with emotional problems, aimed at increasing the contrast between patients in the intervention and control arms, appeared to be useless as the added detection method in routine practice did not function as expected. This was not a problem of PNs’ adherence to the SMS protocol, as the screening tools were integrated into diabetes consultations of more than 85% of the study participants. The problem was that the majority of study participants who scored above the cut-off values on the postal questionnaires did not meet the required detection criteria when screened face to face by the PNs. These patients were therefore not exposed to the follow-up phase of SMS. We have no reason to assume that non-respondents on the research-driven screening were those most in distress, who would have been the easiest to be detected by the PNs, as low detection rates were reported for the whole diabetic population in the intervention practices.

    Differences between the self-administered screening outcomes and the nurse-led detection could be explained by the fluctuation in distress symptoms and the phenomenon of regression towards the mean. This may have resulted in study participants scoring less extreme values at nurse-led detection compared to values at research-driven screening. Furthermore, tests may function differently across settings and administration methods.40 Owing to patients’ difficulty in expressing emotional problems during consultations and lack of recognition by health professionals of emotional problems experienced by patients with chronic physical illness, the chance of positive screening outcomes seems to have been higher in anonymous research-driven screening than in clinical practice.41 ,42 Nonetheless, we did not expect such a large discrepancy between research-driven screening and nurse-led detection.

    For 17 of the 117 study participants, no data on SMS activities were registered, suggesting that these patients were exposed neither to the detection phase nor the follow-up phase of SMS. The reasons for this phenomenon could be twofold. First, PNs had to integrate SMS into all consultations while the effects were assessed in only a few patients per PN. PNs did not know which patients participated in the trial. As PNs’ integration of SMS into consultations could have fluctuated during follow-up, they may have missed study participants. Second, registration bias may have occurred. PNs needed to open an extra data file to record the process and outcomes of SMS. This step might have created a barrier to their compliance. This underlines the need for a thorough process evaluation of how and under what conditions the intervention procedure was carried out. In this regard, the simultaneous evaluation of the implementation process will reveal important issues. The outcomes of this process evaluation will be presented elsewhere.

    In the planning of the study, we considered avoiding research-driven screening by inserting the nurse-led detection phase of SMS into both study arms. However, monitoring and discussing psychological well-being as part of routine care may already have positive effects on patients’ moods.43 Moreover, from an ethical perspective, patients in the control arm who would be detected positively for mental health problems would need to be referred for further diagnosis or psychosocial treatment, thus changing the usual care. Using research-led screening was therefore assumed to be the most adequate solution for the selection of participants for the effectiveness part of the study. In retrospect, a pilot study should have been part of our implementation project. It was not planned for several reasons. There was an implementation moment. Besides, we could use the experiences from the former study, in which the treatment phase was evaluated. PNs would receive comprehensive training, and they would ask common questions about psychosocial functioning. Furthermore, the outcomes of the screening did not give rise to questions about the instruments: as expected, 20% of the respondents met the detection criteria.

    Although collaboration with regional stakeholders resulted in organisational and financial benefits for family practices willing to participate, this incentive did not appear to be a decisive factor in the process of considering whether or not to participate. For robust multilevel modelling, it is necessary that sufficient clusters are recruited and sufficient patients are available per cluster.44 Our intention was to include 46 PNs, but only 40 PNs participated. Furthermore, the final total number of clusters for analysis was actually only 33 as some patients were seen alternately by PNs working in a team. Owing to the regional approach of the SMS project, we were confined to the family practices within the region under study. The number of patients within clusters ranged from 1 to 19. The loss to follow-up of patients was less than expected, which led to an acceptable total number of study participants. However, the availability of clusters as well as the balance between clusters needs attention when designing a cluster-randomised trial that will be performed in an implementation setting.

    In implementation studies, formative evaluation methods can be used to adapt or optimise the implementation or intervention itself during the study.20 Owing to the trial protocol and time and research funding limitations, we did not further optimise the intervention or implementation strategies other than to inform PNs after 3-month follow-up to use the criteria DFT >4 and DS >3 less rigidly. The planning of a more flexible design of the effectiveness trial within the hybrid design would have been helpful in overcoming implementation problems. This calls for inclusion of specific evaluation moments to decide about the need for adjustments in the intervention itself, in implementation activities or with regard to follow-up measurements.

    In conclusion, the adjusted nurse-led intervention in its present form was not effective. There was a dilution of the contrast between intervention and control patients. This is because outcomes of the detection method, added for implementation purposes, were neither optimal nor congruent with outcomes of the screening method added for research purposes. We do not know whether SMS would be effective if PNs used other methods to identify patients in need for support in their emotional and role management skills. Blending effectiveness and implementation studies may have the potential to facilitate a rapid adoption of evidence-based care innovations in routine practice and to provide useful information for decision-makers about effects and implementation issues,20 but researchers who are planning a hybrid effectiveness-implementation study need to be aware of the necessity to build in adequate follow-up moments and to take action either to remove identified barriers or to adapt the intervention to an acceptable level.

    Acknowledgments

    The authors thank the patients and health professionals who participated in the study. The authors are grateful to Anuska Muyres for her administrative support. The authors also thank Samira Guerrouj and Ron Pijpers for their cooperation in the development of the SMS registration system and data extraction. The authors would also like to thank Esther Stoffers and Jeroen Doorenbos (patient representatives) as well as Job Metsemakers, Karianne Jonkers and Peter Verhaak for their participation in the advisory committee.

    References

    1. ↵
      1. Fisher EB,
      2. Thorpe CT,
      3. DeVellis BM, et al
      . Healthy coping, negative emotions, and diabetes management. Diabetes Educ 2007;33:1080–103. doi:10.1177/0145721707309808
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Holman H,
      2. Lorig K
      . Patient self-management: a key to effectiveness and efficiency in care of chronic disease. Public Health Rep 2004;119:239–43. doi:10.1016/j.phr.2004.04.002
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Penninx BWJH,
      2. Kritchevsky SB,
      3. Yaffe K, et al
      . Inflammatory markers and depressed mood in older persons: results from the health, aging and body composition study. Biol Psychiatry 2003;54:566–72. doi:10.1016/S0006-3223(02)01811-5
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Fisher L,
      2. Mullan JT,
      3. Arean P, et al
      . Diabetes distress but not clinical depression or depressive symptoms is associated with glycemic control in both cross-sectional and longitudinal analyses. Diabetes Care 2010;33:23–8. doi:10.2337/dc09-1238
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. DiMatteo MR,
      2. Lepper HS,
      3. Croghan TW
      . Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence. Arch Int Med 2000;160:2101–7. doi:10.1001/archinte.160.14.2101
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Aikens JE
      . Prospective associations between emotional distress and poor outcomes in type 2 diabetes. Diabetes Care 2012;35:2472–8. doi:10.2337/dc12-0181
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Macdonald W,
      2. Rogers A,
      3. Blakeman T, et al
      . Practice nurses and the facilitation of self-management in primary care. J Adv Nurs 2008;62:191–9. doi:10.1111/j.1365-2648.2007.04585.x
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Van Houtum L,
      2. Rijken M,
      3. Heijmans M, et al
      . Self-management support needs of patients with chronic illness: do needs for support differ according to the course of illness? Patient Educ Couns 2013;93:626–32. doi:10.1016/j.pec.2013.08.021
      OpenUrl
    9. ↵
      1. Rane K,
      2. Wajngot A,
      3. Wändell PE, et al
      . Psychosocial problems in patients with newly diagnosed diabetes: number and characteristics. Diabetes Res Clin Pract 2011;93:371–8. doi:10.1016/j.diabres.2011.05.009
      OpenUrlPubMed
    10. ↵
      1. Elissen A,
      2. Nolte E,
      3. Knai C, et al
      . Is Europe putting theory into practice? A qualitative study of the level of self-management support in chronic care management approaches. BMC Health Serv Res 2013;13:117. doi:10.1186/1472-6963-13-117
      OpenUrlCrossRefPubMed
    11. ↵
      Netherlands Diabetes Federation. NDF Care Standard. Transparancy and quality of diabetes care for people with type 2 diabetes. Amersfoort: Nederlandse Diabetes Federatie (NDF), 2007.
    12. ↵
      1. Rutten GEHM,
      2. De Grauw WJC,
      3. Nijpels G, et al.
      NHG-Standaard Diabetes mellitus type 2 (Tweede herziening) [NHG Practice Guideline Diabetes mellitus type 2 (Second revision)]. Huisarts Wet 2006;49:137–52
      OpenUrl
    13. ↵
      1. Tsiachristas A,
      2. Hipple-Walters B,
      3. Lemmens KMM, et al
      . Towards integrated care for chronic conditions: Dutch policy developments to overcome the (financial) barriers. Health Policy 2010;101:122–32. doi:10.1016/j.healthpol.2010.10.013
      OpenUrl
    14. ↵
      1. Jansen D,
      2. Spreeuwenberg P,
      3. Heijmans M
      . Ontwikkelingen in de zorg voor chronisch zieken: rapportage 2012 [Developments in chronic care: report 2012]. Utrecht: Netherlands Institute for Health Services Research (NIVEL), 2012.
    15. ↵
      1. Heiligers PJM,
      2. Noordman J,
      3. Korevaar JC, et al.
      Kennisvraag: praktijkondersteuners in de huisartspraktijk (POH's) klaar voor de toekomst? [Knowledge base: Practice nurses in the GP practices, ready for the future?]. Utrecht: Netherlands Institute for Health Services Research (NIVEL), 2012.
    16. ↵
      1. Van Dijk-de Vries A,
      2. van Bokhoven MA,
      3. Terluin B, et al
      . Integrating nurse-led Self-Management Support (SMS) in routine primary care: design of a hybrid effectiveness-implementation study among type 2 diabetes patients with problems of daily functioning and emotional distress: a study protocol. BMC Fam Pract 2013;14:77. doi:10.1186/1471-2296-14-77
      OpenUrl
    17. ↵
      1. Lamers F,
      2. Jonkers CCM,
      3. Bosma H, et al
      . A minimal psychological intervention in chronically ill elderly patients with depression: a randomised trial. Psychother Psychosom 2010;79:217–26. doi:10.1159/000313690
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Jonkers CCM,
      2. Lamers F,
      3. Bosma H, et al
      . The effectiveness of a minimal psychological intervention on self-management beliefs and behaviors in depressed chronically ill elderly persons: a randomized trial. Int Psychogeriatr 2012;24:288–97. doi:10.1017/S1041610211001748
      OpenUrlCrossRefPubMed
    19. ↵
      1. Lamers F,
      2. Jonkers CCM,
      3. Bosma H, et al
      . Treating depression in diabetes patients: does a minimal psychological intervention affect diabetes-specific quality of life and glycemic control? A randomized controlled trial. J Adv Nur 2011;67:788–99. doi:10.1111/j.1365-2648.2010.05540.x
      OpenUrl
    20. ↵
      1. Curran GM,
      2. Bauer M,
      3. Mittman B, et al
      . Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012;50:217–26. doi:10.1097/MLR.0b013e3182408812
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Tuinman MA,
      2. Gazendam-Donofrio SM,
      3. Hoekstra-Weebers JE
      . Screening and referral for psychosocial distress in oncologic practice. Cancer 2008;113:870–8. doi:10.1002/cncr.23622
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Maas-van Schaaijk NM,
      2. Odink RJ,
      3. Ultee K, et al
      . Can one question be a useful indicator of psychosocial problems in adolescents with diabetes mellitus? Acta Paediatrica 2011;100:708–11. doi:10.1111/j.1651-2227.2011.02132.x
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Braam C,
      2. van Oostrom SH,
      3. Terluin B, et al
      . Validation study of a distress screener. J Occup Rehabil 2009;19:231–7. doi:10.1007/s10926-009-9178-z
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Struijs JN,
      2. Baan CA
      . Integrating care through bundled payments—lessons from the Netherlands. N Engl J Med 2011;364:990–1. doi:10.1056/NEJMp1011849
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Terluin B,
      2. van Marwijk HWJ,
      3. Adèr HJ, et al
      . The Four-Dimensional Symptom Questionnaire (4DSQ): a validation study of a multidimensional self-report questionnaire to assess distress, depression, anxiety and somatization. BMC Psychiatry 2006;6:34. doi:10.1186/1471-244X-6-34
      OpenUrlCrossRefPubMed
    26. ↵
      1. Van Eijk JTM,
      2. Diederiks JPM,
      3. Kempen GIJM, et al
      . Development and feasibility of a nurse administered strategy on depression in community-dwelling patients with a chronic physical disease. Patient Educ Couns 2004;54:87–94. doi:10.1016/S0738-3991(03)00201-5
      OpenUrlPubMed
    27. ↵
      1. Mynors-Wallis L
      . Problem solving treatment in general psychiatric practice. Advan Psychiatr Treat 2001;7:417–25. doi:10.1192/apt.7.6.417
      OpenUrlFREE Full Text
    28. ↵
      1. Lamers F,
      2. Jonkers CCM,
      3. Bosma H, et al
      . Effectiveness and cost-effectiveness of a minimal psychological intervention to reduce non-severe depression in chronically ill elderly patients: the design of a randomised controlled trial. BMC Public Health 2006;6:161. doi:10.1186/1471-2458-6-161
      OpenUrlPubMed
    29. ↵
      1. Polonsky WH,
      2. Anderson BJ,
      3. Lohrer PA, et al
      . Assessment of diabetes-related distress. Diabetes Care 1995;18:754–60. doi:10.2337/diacare.18.6.754
      OpenUrlAbstract/FREE Full Text
    30. ↵
      1. Snoek FJ,
      2. Pouwer F,
      3. Welch GW, et al
      . Diabetes-related emotional distress in Dutch and US diabetic patients: cross-cultural validity of the problem areas in diabetes scale. Diabetes Care 2000;23:1305–9. doi:10.2337/diacare.23.9.1305
      OpenUrlAbstract/FREE Full Text
    31. ↵
      1. Welch G,
      2. Weinger K,
      3. Anderson B, et al
      . Responsiveness of the Problem Areas In Diabetes (PAID) questionnaire. Diabet Med 2003;20:69–72. doi:10.1046/j.1464-5491.2003.00832.x
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      1. Cardol M,
      2. de Haan RJ,
      3. de Jong BA, et al
      . Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210–16. doi:10.1053/apmr.2001.18218
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Kersten P
      . Impact on Participation and Autonomy (IPA) Manual to the English version: IPA. http://www.nivel.nl/sites/default/files/bestanden/INT-IPA-Manual.pdf (accessed 18 Jun 2015).
    34. ↵
      1. Petkov J,
      2. Harvey P,
      3. Battersby M
      . The internal consistency and construct validity of the partners in health scale: validation of a patient rated chronic condition self-management measure. Qual Life Res 2010;19:1079–85. doi:10.1007/s11136-010-9661-1
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Ware JE,
      2. Kosinski M,
      3. Keller S
      . A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220–33. doi:10.1097/00005650-199603000-00003
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Fleishman JA,
      2. Selim AJ,
      3. Kazis LE
      . Deriving SF-12v2 physical and mental health summary scores: a comparison of different scoring algorithms. Qual Life Res 2010;19:231–41. doi:10.1007/s11136-009-9582-z
      OpenUrlPubMedWeb of Science
    37. ↵
      1. Mols F,
      2. Pelle AJ,
      3. Kupper N
      . Normative data of the SF-12 health survey with validation using postmyocardial infarction patients in the Dutch population. Qual Life Res 2009;18:403–14. doi:10.1007/s11136-009-9455-5
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Bosscher RJ,
      2. Smit JH
      . Confirmatory factor analysis of the General Self-Efficacy Scale. Behav Res Ther 1998;36:339–43. doi:10.1016/S0005-7967(98)00025-4
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Sherer M,
      2. Maddux JE,
      3. Mercandante B, et al
      . The self-efficacy scale: construction and validation. Psychol Rep 1982;51:663–71. doi:10.2466/pr0.1982.51.2.663
      OpenUrlCrossRefPubMedWeb of Science
    40. ↵
      1. De Waal MW,
      2. van der Weele GM,
      3. van der Mast RC, et al
      . The influence of the administration method on scores of the 15-item Geriatric Depression Scale in old age. Psychiatry Res 2012;197: 280–4. doi:10.1016/j.psychres.2011.08.019
      OpenUrlPubMed
    41. ↵
      1. Cape J,
      2. McCulloch Y
      . Patients’ reasons for not presenting emotional problems in general practice consultations. Br J Gen Pract 1999;49:875–9.
      OpenUrlAbstract/FREE Full Text
    42. ↵
      1. Mitchell AJ,
      2. Rao S,
      3. Vaze A
      . Can general practitioners identify people with distress and mild depression? A meta-analysis of clinical accuracy. J Affect Disord 2011;130:26–36. doi:10.1016/j.jad.2010.07.028
      OpenUrlCrossRefPubMed
    43. ↵
      1. Pouwer F,
      2. Snoek FJ,
      3. van der Ploeg HM, et al
      . Monitoring of psychological well-being in outpatients with diabetes. Diabetes Care 2001;24:1929–35. doi:10.2337/diacare.24.11.1929
      OpenUrlAbstract/FREE Full Text
    44. ↵
      1. Campbell MK,
      2. Mollison J,
      3. Steen N, et al
      . Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract 2000;17:192–6. doi:10.1093/fampra/17.2.192
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributors JvE initiated the SMS project. AvDV, BW, LvB, TvdW, BT and JvE were involved in developing and refining the study design. AvD and BW performed statistical analyses and interpretation of data and take responsibility for the integrity of data and accuracy of data analysis. All authors were involved in interpretation of outcomes. AvD drafted the manuscript. All authors critically reviewed draft revisions and read and approved the final version.

    • Funding This research was supported by the Dutch Diabetes Research Foundation (Diabetes Fonds) with grant No. 2010.13.1366 (Voice of the Patient programme), and by the ‘Annadal Foundation’ in Maastricht, an independent financial support fund in the field of healthcare. Both the training of practice nurses and operation of the system for registration of SMS were facilitated by the ‘HOZL’ group of collaborating family practices in the eastern part of the Southern Limburg region. During the SMS project, CZ Health Insurance included a fee for SMS in the bundled payment arrangement for diabetes care.

    • Competing interests None declared.

    • Ethics approval The study was approved by the Medical Ethics Committee of Maastricht University/University Hospital Maastricht.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Additional data from the study database are available on request from the corresponding author at anneke.vandijk@ maastrichtuniversity.nl