Article Text
PDF
Abstract
Objectives Rehabilitation interventions for older adults are complex as they involve a number of interacting components, have multiple outcomes of interest and are influenced by a number of contextual factors. The importance of rigorous intervention development prior to formal evaluation has been acknowledged and a number of frameworks have been developed. This review explored which frameworks have been used to guide the development of rehabilitation interventions for older adults.
Design Systematic scoping review.
Setting Studies were not limited for inclusion based on setting.
Participants Studies were included that featured older adults (>65 years of age).
Interventions Studies were included that reported the development of a rehabilitation intervention.
Primary and secondary outcome measures Data were extracted on study population, setting, type of intervention developed and frameworks used. The primary outcome of interest was the type of intervention development framework.
Results Thirty-five studies were included. There was a range of underlying medical conditions including mild cognitive impairment and dementia (n=5), cardiac (n=4), stroke (n=3), falls (n=3), hip fracture (n=2), diabetes (n=2), breast cancer (n=1), Parkinson’s disease (n=1), depression (n=1), chronic health problems (n=1), osteoarthritis (n=1), leg ulcer (n=1), neck pain (n=1) and foot problems (n=1). The intervention types being developed included multicomponent, support based, cognitive, physical activities, nursing led, falls prevention and occupational therapy led. Twelve studies (34%) did not report using a framework. Five frameworks were reported with the Medical Research Council (MRC) framework for developing and evaluating complex interventions being the most frequently cited (77%, n=17).
Conclusion At present, the MRC framework is the most popular for developing rehabilitation interventions for older adults. Many studies do not report using a framework. Further, specific guidance to assist this complex field of rehabilitation research is required.
- older adults
- rehabilitation
- intervention development
- rehabilitation medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Rigorous intervention development prior to formal evaluation is important.
A number of frameworks have been developed but are inconsistently used or inconsistently reported.
Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews guidelines were followed for this systematic scoping review.
An electronic database search aimed to capture all studies reporting intervention development.
Studies involving older people and specifically reporting the development of a rehabilitation intervention were included.
Introduction
‘Rehabilitation is concerned with lessening the impact of disabling conditions’ (Young, p677)[1] and is a complex process requiring a holistic approach that considers physical, social and psychological function. Rehabilitation interventions for older adults are complex as they involve a number of interacting components, are often tailored to individual needs, have multiple outcomes of interest and are influenced by a number of environmental and contextual factors.2
The need to develop a robust evidence base for complex rehabilitation interventions has led to an increased focus on developing and evaluating these interventions. Interventions initially showing promise in small-scale testing are often ineffective when scaled into large multicentre randomised controlled trials (RCTs). For example, an inpatient falls prevention programme that was effective during an observational study,3 failed to prevent falls to a significant degree compared with a control in a multisite RCT.4 A review, including this example, explored the reasons for the difference in outcomes, citing different contextual factors (staffing, length of stay).5 However, while intervention development was reported by this example, a framework was not used and may have resulted in a lack of sound theoretical underpinning and understanding of the intervention mechanisms of action. The importance of rigorous intervention development prior to formal evaluation has been acknowledged by healthcare researchers in other fields6 and a number of frameworks have been developed. These frameworks include the Medical Research Council (MRC) guidance for developing and evaluating complex interventions,7 Criteria for Reporting the Development and Evaluation of Complex Interventions (CReDECI),8 intervention mapping9 and the 6 Steps in Quality Intervention Development (6SQuID).10 Although there are a number of intervention development frameworks, the lack of methodological detail and specificity to rehabilitation interventions may mean that researchers are using the frameworks in different ways or not using the frameworks at all.
Therefore, the aims of this review were to (1) to ascertain if intervention development frameworks are being used in older people rehabilitation research, (2) to document which frameworks have been used and (3) to explore how those frameworks are being used, what methods are employed, and how much detail is provided. This review will help researchers and clinicians to consider a range of frameworks for their studies and is the first step towards establishing more detailed guidance.
Methods
Review design
Systematic scoping review. This study was initially designed as a systematic review but was adapted at a late stage due to advice from reviewers and the editors.
Inclusion criteria
Target population of intervention
Studies were included if their participants were older people who were >65 years (either through study inclusion criteria, mean sample age of study population or are described as older or elderly).
Intervention
The interventions being developed or described focused on rehabilitation. The definition of rehabilitation used was ‘the process of returning to a healthy or good way of life, or the process of helping someone to do this after they have been in prison, been very ill or the process of returning something to a good condition.’11 To be a rehabilitation intervention, the paper had to report that the intervention: involved the individual(s) being rehabilitated; consisted of more than one session to indicate a process; aimed to create a change in the individual(s)’ state or ability from doing the intervention; took place either after something or to prevent something (eg, an incident/illness) and was described or labelled as ‘rehabilitation’ by the authors.
Types of studies
Studies were included if they stated an aim or intent to either report the intervention that had been developed or to document the process or synthesis as justification or background for the next stage of intervention testing. This included mixed-method studies, RCTs, controlled clinical trials, experimental studies, qualitative based analysis studies, cohort, cross-sectional and case–control studies. Systematic reviews (all types) were considered for inclusion so reference lists could be explored for further studies that may not have been identified in the search strategy. Types of publications were also considered. Study protocols were considered for inclusion, however, abstracts, thesis, dissertations and conference proceedings were excluded due to the level of detail characteristic of these manuscripts (eg, limited word counts with abstracts and significant word counts with thesis). Where possible if studies were part of a series of publications, the other material available was sought and the most prominent paper detailing the intervention development process included.
Types of data and outcomes
Studies were included if they reported or described ‘intervention development’ or ‘developing an intervention’. Studies without a framework were included but only if they met the predetermined criteria that sufficient information and detail on the intervention development process or methods was presented. Studies that claimed to have completed an intervention development process but did not include any information on the process or method were excluded due to lack of data. All studies were assessed for inclusion by two authors and any discrepancy on the decision of a paper was discussed by all authors to reach a group consensus.
Studies were not limited nor selected according to their outcomes.
Search methods for identification of studies
The search strategy aimed to find both published and unpublished studies. A phased search strategy included search terms: ‘developing and evaluating complex interventions’ (all fields), ‘development’ OR ‘develop*’ (title), ‘intervention’ (title), ‘older’ OR ‘old*’ OR ‘elderly’ (all fields).
Electronic searches
Initially, a limited search of MEDLINE and CINAHL was undertaken to identify and refine index terms used to describe relevant articles. Index terms and keywords were taken from known studies that reported their intervention development process and the search strategy refined to ensure these papers were captured.
A second full search using all identified keywords and index terms was then undertaken across relevant databases, including: The Cochrane Central Register of Controlled Trials (The Cochrane Library, latest issue), MEDLINE, EMBASE, AMED, CINAHL and PsycINFO. The search was completed in October 2017. A full search strategy is presented in online supplementary material 1.
Supplementary file 1
Searching other resources
The reference lists of full-text studies were searched for related material that could be included or were more relevant for data extraction.
Data collection and analysis
Selection of studies
Each study identified for inclusion was considered independently by two reviewers at all stages: title screening, abstract screening and full paper review for inclusion. Discrepancies between reviewer’s decisions were recorded and discussed between the other authors to achieve an outcome.
Data extraction and management
Data were extracted from the included papers using a bespoke data extraction tool, the main categories of which were: study population, setting, type of intervention developed and frameworks used. If a framework was cited then a more detailed review of the components used was completed. Microsoft Excel was used as the data management software and compiled into a single database once agreement of included studies and data extraction had been completed. The review has been reported according to Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines and a checklist completed.12
Assessment of methodological quality in included studies
Included studies were not assessed for methodological quality. However, each study was critiqued according to the latest reporting standards for the development and evaluation of complex interventions in healthcare (CReDECI2).13 Assessment of the reporting standard of the studies would not influence their inclusion in the review.
Data synthesis
Data were collated and narratively described using tables and text.
Patient and public involvement
Patients were not involved.
Results
Thirty-five studies were included in the review for data extraction.14–48 The flow diagram depicting the number of studies identified and excluded at each stage is provided in figure 1.
Flow diagram depicting the number of studies identified and excluded at each stage (each stage of the review process is depicted by a box in a sequential design as suggested by PRISMA-ScR12). PRISMA-ScR, Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews.
Description of the included studies
The most common population descriptor was community dwelling or older adults of a certain age.17 19 27 29 31 44 45 Underlying conditions included mild cognitive impairment and dementia (n=5),18 28 30 32 34 cardiac conditions (n=4),15 24–26 stroke (n=3),22 38 40 falls and fear of falling (n=3),21 35 48 hip fracture (n=2),37 39 diabetes (n=2),14 41 42 breast cancer,20 Parkinson’s disease,16 depression,23 chronic health problems,33 osteoarthritis,36 leg ulcer,43 neck pain46 and foot problems.47
The types of interventions reported were varied and included multicomponent (n=12),14 19 23 24 26–29 36 39 42 45 support based (n=5),15 16 18 30 41 cognitive interventions (n=5),31 33 34 40 48 physical activities (n=3),17 32 46 nursing (n=2),25 43 falls prevention,21 occupational therapy,22 poststroke care,38 podiatry47 and dietary advice.37
The included studies were from the UK (n=17),14 15 17 19 20 22 31 34–42 47 Netherlands (n=6),26 27 30 44 45 48 USA (n=4),18 24 25 28 Canada,16 India,23 Germany,29 46 Hong Kong,32 Italy,33 and Belgium.43 Ten studies were linked to other publications reporting the same intervention or other aspects of the development process.22 25 26 28 29 31 35 39 42 43
The reporting standard of the included studies was mixed with an average score of 4.4 (range=1–13) out of 13. All reported elements of the development and pilot phase of the checklist with only four studies reporting the evaluation stage.27 35 36 44 A table of the reporting standards for all included studies is provided in online supplementary material 2.
Supplementary file 2
What frameworks were reported
Thirteen studies did not report using a framework to assist their intervention development.18 21 23–25 28 32 34 35 37 46 48 In total five frameworks were reported. The MRC guidance was the most frequently used (77%, n=17).15 19 20 22 26 27 29 30 33 36 38–42 47 The other frameworks were intervention mapping (n=3),16 44 45 conceptual modelling (n=1),31 intervention/programme theory (n=1)17 and the Van Meijel model (n=1).43 Descriptions and key references for the frameworks are provided in online supplementary material 3.
Supplementary file 3
What methods were used for the different framework sections
MRC guided studies
A variety of different methods were used in the different stages of the MRC guidance within the included studies (see table 1). Most15 22 27 30 38 40–42 reported their intervention development process according to the three MRC framework stages. These are: (1) identifying the evidence base, (2) developing theory and (3) modelling processes and outcomes. Some only referenced the guidance and did not report the stages as distinct phases19 20 29 36 39 47 or described their own stages (such as evidence exploration, tune-up with experts and fine-tuning with patients).33 Three papers adapted and added a fourth stage their development process.14 26 40
Presentation of the methods used for each element of the Medical Research Council (MRC) framework
All except the study by Wylie et al 47 reported using a literature review in their development work. The literature review was most commonly used to identify relevant evidence or theories to underpin the intervention being developed (n=11). Other methods used included: expert consultation (n=2), qualitative interviews with either clinicians or patients (n=7) and observations or surveying patients (n=8).
A variety of terms were used to describe the second stage of their development process, with some categorising this as theoretical development, whereas others were focusing on modelling. There was a wide range of research methods reported in this second stage, including literature reviews (n=4), expert consultations (n=3), qualitative interviews and focus groups (n=4), observations (n=2), and pilot studies (n=5).
Nine studies then described a feasibility or modelling stage.14 15 22 26 27 30 33 38 40 This phase included pilot studies (n=3), qualitative focus groups and interviews (n=6), where data were collected. One study27 reported eight different research methods at this stage including a Delphi consensus process.
The four studies that added a fourth stage into their development processes varied in terms used to describe it, including ‘pilot study’,14 ‘face validity’26 and ‘assessing feasibility of the intervention’.40 Two of the studies reported completing a pilot or feasibility study within this stage14 40 whereas the third included expert meetings.26
Other framework guided studies
Six studies used a variety of intervention development frameworks.16 17 31 43–45 Reporting of the research methods used in these studies were varied even when the same framework was used (table 2). Intervention mapping9 was used in three studies, one of which provided no detail on the methods used in each section,16 whereas the other two reported very detailed processes and methods.44 45 Table 2 describes the different intervention development frameworks and the research methods used within each framework.
Presentation of the methods used for each element of the other intervention development frameworks
Discussion
Principal findings
Thirty-five studies were included in the review for data extraction. Twelve studies did not cite a framework to assist their intervention development. Five frameworks were reported by the 23 studies who did use one. The MRC guidance was the most frequently used with 77% (n=17). The other four frameworks were intervention mapping, conceptual modelling, intervention/programme theory and the Van Meijel model but these were only used in a small number of studies. Of the numerous potential frameworks, researchers could be using this study highlights that most researchers felt that the MRC, is at present, the most appropriate for their use. Although the quality of the studies in this review was not measured against any standardised measure, the studies that used the MRC guidance provided considerably more details about the components of intervention development than the studies using other frameworks providing a greater degree of confidence that the results had been rigorously collected and not biased. This may indicate that the MRC is written in a way that helps researchers follow a process more easily. However, three studies also adapted and added to the MRC process, indicating that there are further aspects to consider that are not addressed in that guidance.
Strengths and weaknesses of this study
This review was conducted in line with PRISMA-ScR guidelines following a systematic process, using predefined eligibility criteria and independent assessment by two reviewers at each stage. As with all reviews, there may have been studies that were missed due to the parameters of the review, such as, the definition of rehabilitation that was used. Data extraction was completed using a standardised spreadsheet by all authors and despite regular review meetings, there was discrepancy in the interpretation of research methods and the level of detail extracted. For example, what is counted as a ‘literature review’ could for one study be a Cochrane review while for another it is a non-systematised narrative description of the field of research.
The findings of this review are limited by the information available about intervention development within the identified literature. It is acknowledged that many journals prefer to publish detail on the intervention content with little focus on the development process and this was evident in this review. Intervention development frameworks are a relatively recent development and studies conducted before the MRC guidance was introduced in 2000 may have had limited methodological literature to guide their intervention development. This review searched all literature from the date of inception of the electronic database and this search strategy may have biased the number of studies not reporting the use of a framework. It is anticipated that over the coming years there will be many more studies reporting the use of a framework and providing more details on that process. Details on the intervention development may not be through journal publications, but through online supplementary material, discussion series, study or institution websites, or online data repositories.
This review did not report on the quality of the studies. While quality assessments are standard practice in systematic reviews,49 there is not yet a quality assessment tool for intervention development studies. The intention of this review was to make comment on the current state of the literature relating to intervention development. Studies were not included or excluded based on their quality, but on their detail of the intervention development process and methods. A critique against the reporting standards was included as a compromise and to compare the included studies to the recognised publishing standards.
Strengths and weaknesses in relation to other studies
To our knowledge, this is the first review of intervention development frameworks used in developing rehabilitation interventions for older people.
The MRC guidance7 from the UK provides a structure to the development and evaluation process for complex interventions. However, the MRC guidance is brief and has been criticised for not dealing well with the complexity of complex interventions.50–52 Although the MRC framework was the most commonly cited framework, the included papers provided varying levels of detail over how the framework was used, and a lack of clarity over whether all three stages of development were explored. The lack of consistency and detail may be a result of the limited practical guidance offered by the MRC framework. There were, however, common approaches used in the papers citing the MRC framework which included: literature reviews, consultation with stakeholders, interviews with patients and clinicians, consensus methodologies and pilot work. It is clear from this review that there is not a consistent approach to developing rehabilitation interventions for older adults and further work is needed to establish how, and which, research methods should be used within the different stages of intervention development.
Other frameworks to support intervention development include the 6SQUID which was based on the experiences of Wight et al 10 in developing public health interventions. Although this framework provides more detail, there is still a lack of methodological detail on how to undertake each element. It also has a public health focus which may not consider all aspects needed in the development of a complex rehabilitation intervention. In providing a rationale for the development of the 6SQUID framework, Wight et al provided a summary and appraisal of existing intervention development frameworks in public health and included both the MRC framework and intervention mapping which were identified in this review. Intervention mapping is an involved and detailed process,9 which may account for it being referred to in only three papers in this review. Möhler et al 8 published CReDECI through a three-stage consensus process. This aimed to improve quality of the reporting on the underlying theory of an intervention, the components and interactions of an intervention as well as any contextual factors. While its merits are acknowledged, the primary focus was on the evaluation phase and the criteria provide little detail on how to undertake the process of intervention development. The capability, opportunity, motivation and behaviour (COM-B) model and theoretical domains framework53 is another intervention development framework that is becoming increasingly popular in the behavioural change literature but has not widely been used in rehabilitation research as yet.
Meaning and implications
Many studies did not use an intervention framework and in those that did, there was a lack of consistent detail regarding the intervention development process. Rigorous intervention development is necessary to avoid costly trials of underdeveloped interventions that have no theoretical basis, however, there is a distinct lack of practical guidance to help researchers determine when an intervention is sufficiently developed. It is acknowledged that each rehabilitation intervention is by its very nature complex and therefore reliant on the experience of the individuals developing it, as well as the context and circumstances it is to be delivered in. A rigid framework that dictates exactly how an intervention should be developed may, therefore, not be appropriate as it would not allow for the nuances of each individual intervention and the different approaches that may be more pertinent to their circumstances. Nevertheless, there does appear a need to provide researchers with further detail on the indicators of good practice and what to consider when undertaking quality intervention development.
Recommendations
Following this review, a number of recommendations can be made, including:
Researchers should carefully plan and clearly detail the process of developing rehabilitation interventions for older people using a recognised framework such as MRC.
Rehabilitation journals need to welcome further detail on the intervention development process using online supplementary material.
A consensus process is needed to depict best practice and provide guidance on developing a rehabilitation intervention for older adults.
Conclusion
The MRC guidance is the most popular framework being used by researchers developing rehabilitation interventions for older adults. However, many studies do not report using a framework to guide their development. Further, specific guidance to help researchers choose and use the best framework for their intervention is needed.
Acknowledgments
The authors would like to thank the Division of Rehabilitation and Ageing, School of Medicine and Health Sciences, University of Nottingham, for supporting this publication. This research was funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (East Midlands) (NIHR CLAHRC East Midlands).
References
Footnotes
Contributors The original idea for the review was provided by KRR and VB. VB and KRR completed the electronic searches. VB, KRR, VH-M and JEH completed data extraction. KRR and VB drafted the paper. PL, JEH and VH-M edited the paper. All authors contributed to the final paper in intellectual content, design and writing.
Funding VB was supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care East Midlands (NIHR CLAHRC EM).
Disclaimer The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data can be accessed through correspondence with the lead author.
Patient consent for publication Not required.