Article Text
PDF
Abstract
Objectives Implementation studies are often poorly reported and indexed, reducing their potential to inform the provision of healthcare services. The Standards for Reporting Implementation Studies (StaRI) initiative aims to develop guidelines for transparent and accurate reporting of implementation studies.
Methods An international working group developed the StaRI guideline informed by a systematic literature review and e-Delphi prioritisation exercise. Following a face-to-face meeting, the checklist was developed iteratively by email discussion and critical review by international experts.
Results The 27 items of the checklist are applicable to the broad range of study designs employed in implementation science. A key concept is the dual strands, represented as 2 columns in the checklist, describing, on the one hand, the implementation strategy and, on the other, the clinical, healthcare or public health intervention being implemented. This explanation and elaboration document details each of the items, explains the rationale and provides examples of good reporting practice.
Conclusions Previously published reporting statements have been instrumental in improving reporting standards; adoption by journals and authors may achieve a similar improvement in the reporting of implementation strategies that will facilitate translation of effective interventions into routine practice.
- Dissemination and implementation research
- EQUATOR Network
- Implementation Science
- Reporting standards
- Organisational innovation
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- Dissemination and implementation research
- EQUATOR Network
- Implementation Science
- Reporting standards
- Organisational innovation
Strengths and limitations of this study
We followed recommended methodology for developing health research reporting guidelines, including a literature review, an e-Delphi exercise, an international face-to-face consensus meeting and inviting expert feedback on draft versions of the Standards for Reporting Implementation Studies (StaRI) checklist.
Implementation science is a broad field, and although the e-Delphi, working group and expert feedback enabled input from experts from a range of implementation science-related disciplines, we may have missed some perspectives.
Distance and financial constraints limited the geographical spread of representatives at the face-to-face consensus working group, but we invited feedback on the penultimate draft from experts from across the world.
Although our initial feedback suggests general agreement with the underlying concepts, the StaRI guidelines will need to be refined in the light of authors' and editors' practical experience of using the checklist.
Implementation science bridges the gap between developing and evaluating effective interventions and implementation in routine practice to improve patient and population health.1 Implementation studies are however often poorly reported and indexed,2 ,3 reducing their potential to inform the provision of healthcare services and improve health outcomes.4 The Standards for Reporting Implementation Studies (StaRI) initiative aimed to develop standards for transparent and accurate reporting of implementation studies. The StaRI statement describing the scope and conceptual underpinning is published in the BMJ;5 this elaboration document provides detailed explanation of the individual items.
Methods
Following established guidelines,6 ,7 we convened a consensus working group in London at which 15 international multidisciplinary delegates considered candidate items identified by a previous systematic literature review and an international e-Delphi prioritisation exercise,8 in the context of other published reporting standards and the panel's expertise in implementation science. The resultant checklist was subsequently developed iteratively by email discussion, and feedback on the penultimate draft guideline sought from colleagues working in implementation science.
Scope of the StaRI reporting standards
Implementation research is the scientific study of methods to promote the systematic uptake of evidence-based interventions into practice and policy and hence improve health.9–11 The discipline encompasses a broad range of methodologies applicable to improving the dissemination and implementation of clinical, healthcare, global health and public health interventions.12–14 The StaRI checklist focuses primarily on standards for reporting studies that evaluate implementation strategies developed to enhance the adoption, implementation and sustainability of interventions,15 but some items may be applicable to other study designs used in implementation science.
The StaRI reporting guidelines
Unlike most reporting guidelines that apply to a specific research methodology, StaRI is applicable to the broad range of study designs employed in implementation science. Authors are referred to other reporting standards for advice on reporting specific methodological features. In an evolving field, in which there is a range of study designs, terminology is neither static nor used consistently.16 For clarity, we have adopted specific terms in this paper; table 1 defines these terms and lists some of the alternative or related terminology.
Terminology, definitions and resources
Underpinning the StaRI reporting standards are the dual strands of describing, on the one hand, the implementation strategy and, on the other, the clinical, healthcare or public health intervention being implemented.17 These strands are represented as two columns in the checklist (see table 2). The primary focus of implementation science is the implementation strategy15 (column 1) and the expectation is that this will always be fully completed. The impact of the intervention on the target population should always be considered (column 2) and either health outcomes reported or robust evidence cited to support a known beneficial effect of the intervention on the health of individuals or populations. While all items are worthy of consideration, not all items will be applicable to or feasible in every study; a fully completed StaRI checklist may thus include a number of ‘not applicable’ items. For example, studies simultaneously testing a clinical intervention and an implementation strategy (Hybrid type 2 designs) would need to fully address both strands, whereas studies testing a clinical intervention while gathering information on its potential for implementation (Hybrid type 1) or testing an implementation strategy while observing the clinical outcomes (Hybrid Type 3) would focus primarily on items in the clinical intervention or implementation strategy columns, respectively.14
Standards for Reporting Implementation Studies (StaRI): the StaRI checklist
Elaboration on individual checklist items
Example of a table describing* an implementation strategy compiled from Kilbourne et al19 description of the implementation of life goals (LG): a clinical intervention for patients with mood disorders
Identification as an implementation study, and description of the methodology in the title and/or keywords
Identification as an implementation study, including a description of the implementation strategy to be tested, the evidence-based intervention being implemented, and defining the key implementation and health outcomes.
Description of the problem, challenge or deficiency in healthcare or public health that the intervention being implemented aims to address.
The aims of the study, differentiating between implementation objectives and any intervention objectives.
The design and key features of the evaluation (cross referencing to any appropriate methodology reporting standards) and any changes to study protocol, with reasons
The context in which the intervention was implemented. (Consider social, economic, policy, healthcare, organisational barriers and facilitators that might influence implementation elsewhere).
Example of a table describing* an implementation strategy compiled from Kilbourne et al19 description of the implementation of life goals (LG): a clinical intervention for patients with mood disorders
Example of a timeline describing an implementation strategy (compiled from Pinnock et al18 description of the implementation of a telephone service for providing asthma reviews). Note: The three-arm implementation study is illustrated in the centre of this schema with the preceding usual care, randomisation on 1 January 2004, the 15-month intervention and subsequent roll-out. The context (specifically the introduction of the Quality and Outcome Framework) is shown at the top of the schema. Below the three-arms of the study are the components of the implementation strategy from set-up and training, ongoing service provision and maintenance and adoption into routine practice.
⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓ ⇓
Any subgroups recruited for additional research tasks, and/or nested studies are described
Summary of outcomes and the related items in the Standards for Reporting Implementation Studies checklist.
Process evaluation objectives and outcomes related to the mechanism by which the strategy is expected to work.
Rationale for sample sizes (including sample size calculations, budgetary constraints, practical considerations, data saturation, as appropriate)
Methods of analysis (with reasons for that choice)
Any a priori sub-group analyses (e.g. between different sites in a multicentre study, different clinical or demographic populations), and sub-groups recruited to specific nested research tasks
Process data related to the implementation strategy mapped to the mechanism by which the strategy is expected to work.
Representativeness and outcomes of subgroups including those recruited to specific research tasks
Contextual changes (if any) which may have affected the outcomes
All important harms or unintended effects in each group.
Summary of findings, strengths and limitations, comparisons with other studies, conclusions and implications.
Include statement(s) on regulatory approvals (including, as appropriate, ethical approval, confidential use of routine data, governance approval), trial/study registration (availability of protocol), funding and conflicts of interest.
supplementary file
Conclusions
The StaRI standards are registered with the EQUATOR Network (http://www.equator-network.org) and the checklist is available from this journal and BMJ.com. We invite editors of journals publishing interventional implementation studies to consider requiring submission of a StaRI checklist, and authors reporting such studies to adopt the checklist. We have presented the concepts and sought feedback from delegates at workshops, conference discussions and implementation project steering groups; in general, the concepts seem to resonate with colleagues. In the future, we would like to work with authors as they apply the checklist to their papers, ‘road testing’ the standards and enabling iterative development.
We are particularly interested in whether the concept of the dual strands (implementation strategy/intervention) resonates with authors and readers of implementation studies. Is it practical to expect authors to consult other methodological checklists for reporting design-specific aspects of their study? We look forward to learning about innovative solutions to providing adequate descriptions of context, implementation strategies and interventions that accommodate the requirements of journals, needs of authors as well as preferences of readers.
Previously published statements have been instrumental in improving reporting standards,10 and our hope is that StaRI will achieve a similar improvement in the reporting of implementation strategies that will facilitate translation of effective interventions into routine practice.
Acknowledgments
Members of the PRISMS team (Eleni Epiphaniou, Gemma Pearce and Hannah Parke) supported the underpinning literature work, and the e-Delphi was handled by ClinVivo. The authors are grateful to a colleagues (implementation science experts, healthcare researchers, clinicians, PhD students) who reviewed the penultimate draft of the StaRI statement and provided a reality check and constructive feedback: Helen Ashdown, David Chambers, Louise Craig, Clarisse Dibao-Dina, Peter Hanlon, Roger Jones, Rachel Jordan, Chris del Mar, Brian McKinstry, Susan Morrow, John Ovretveit, David Price, Kamran Siddiqui, Rafael Stelmach, Paul Stephenson, Shaun Treweek, Bryan Weiner. The authors also thank Melissa Goodbourn and Allison Worth who arranged feedback from the Edinburgh Clinical Research Facility Patient Advisory Panel (Stephanie Ashby, Alison Williams) and Steven Towndrow who coordinated feedback from the Patient and Public Involvement representatives of the NIHR CLAHRC North Thames (Ben Wills-Eve, Rahila Bashir, Julian Ashton, Colleen Ewart, Karen Williams).
References
Footnotes
Collaborators The StaRI Group includes: MB, CRC, Peter Craig, SE, Eleni Epiphaniou, GG, JR-M, PM, Brian Mittman, EM, AP, Gemma Pearce, HP, AS and SJCT.
Contributors HP initiated the idea for the study and with SJCT led the development of the protocol, securing of funding, study administration, workshop and writing of the paper. AS, CJG and SE advised on the development of the protocol, and data analysis. All authors participated in the StaRI international working group along with GP, BM and MG. HP wrote the initial draft of the paper, to which all the authors contributed. HP is the study guarantor.
Funding The StaRI initiative and workshop was funded by contributions from the Asthma UK Centre for Applied Research (AC-2012-01); Chief Scientist Office, Scottish Government Health and Social Care Directorates (PCRCA_08_01); the Centre for Primary Care and Public Health, Queen Mary University of London; and with contributions in kind from the PRISMS team (NIHR HS&DR Grant ref: 11/1014/04). SJCT was (in part) supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart's Health NHS Trust. AS is supported by the Farr Institute.
Disclaimer The funding bodies had no role in the design, in the collection, analysis, and interpretation of data; in the writing of the manuscript; nor in the decision to submit the manuscript for publication. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare: research grants from Chief Scientist Office (HP), Asthma UK (AS, HP, SJCT), Farr Institute (AS), NIHR HS&DR (HP, SJCT), NIHR CLAHRC (SJCT) for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; CRC is the Deputy Editor-in-Chief for Academic Emergency Medicine and on the editorial boards for the Journal of the American Geriatrics Society and Annals of Internal Medicine's ACP Journal Club and serves as paid faculty for Emergency Medical Abstracts, JR-M is the Director of the NIHR HS&DR Programme, no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.