Article Text

PDF

Standards for Reporting Implementation Studies (StaRI): explanation and elaboration document
  1. Hilary Pinnock1,
  2. Melanie Barwick2,3,
  3. Christopher R Carpenter4,
  4. Sandra Eldridge5,
  5. Gonzalo Grandes6,
  6. Chris J Griffiths7,
  7. Jo Rycroft-Malone8,
  8. Paul Meissner9,
  9. Elizabeth Murray10,
  10. Anita Patel7,
  11. Aziz Sheikh1,
  12. Stephanie J C Taylor7
  13. for the StaRI Group
    1. 1Asthma UK Centre for Applied Research, Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, Edinburgh, UK
    2. 2Research Institute, Hospital for Sick Children, Toronto, Ontario, Canada
    3. 3Department of Psychiatry and Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    4. 4Division of Emergency Medicine, Washington University School of Medicine in St Louis, St Louis, USA
    5. 5Pragmatic Clinical Trials Unit, Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK
    6. 6Primary Care Research Unit of Bizkaia, Basque Health Service, Bilbao, Spain
    7. 7Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK
    8. 8Bangor Institute for Health and Medical Research, Bangor University, Bangor, UK
    9. 9Montefiore Medical Center, The University Hospital for Albert Einstein College of Medicine, New York, USA
    10. 10Department of Primary Care and Population Health, University College London, London, UK
    1. Correspondence to Professor Hilary Pinnock; hilary.pinnock{at}ed.ac.uk

    Abstract

    Objectives Implementation studies are often poorly reported and indexed, reducing their potential to inform the provision of healthcare services. The Standards for Reporting Implementation Studies (StaRI) initiative aims to develop guidelines for transparent and accurate reporting of implementation studies.

    Methods An international working group developed the StaRI guideline informed by a systematic literature review and e-Delphi prioritisation exercise. Following a face-to-face meeting, the checklist was developed iteratively by email discussion and critical review by international experts.

    Results The 27 items of the checklist are applicable to the broad range of study designs employed in implementation science. A key concept is the dual strands, represented as 2 columns in the checklist, describing, on the one hand, the implementation strategy and, on the other, the clinical, healthcare or public health intervention being implemented. This explanation and elaboration document details each of the items, explains the rationale and provides examples of good reporting practice.

    Conclusions Previously published reporting statements have been instrumental in improving reporting standards; adoption by journals and authors may achieve a similar improvement in the reporting of implementation strategies that will facilitate translation of effective interventions into routine practice.

    Statistics from Altmetric.com

    Strengths and limitations of this study

    • We followed recommended methodology for developing health research reporting guidelines, including a literature review, an e-Delphi exercise, an international face-to-face consensus meeting and inviting expert feedback on draft versions of the Standards for Reporting Implementation Studies (StaRI) checklist.

    • Implementation science is a broad field, and although the e-Delphi, working group and expert feedback enabled input from experts from a range of implementation science-related disciplines, we may have missed some perspectives.

    • Distance and financial constraints limited the geographical spread of representatives at the face-to-face consensus working group, but we invited feedback on the penultimate draft from experts from across the world.

    • Although our initial feedback suggests general agreement with the underlying concepts, the StaRI guidelines will need to be refined in the light of authors' and editors' practical experience of using the checklist.

    Implementation science bridges the gap between developing and evaluating effective interventions and implementation in routine practice to improve patient and population health.1 Implementation studies are however often poorly reported and indexed,2 ,3 reducing their potential to inform the provision of healthcare services and improve health outcomes.4 The Standards for Reporting Implementation Studies (StaRI) initiative aimed to develop standards for transparent and accurate reporting of implementation studies. The StaRI statement describing the scope and conceptual underpinning is published in the BMJ;5 this elaboration document provides detailed explanation of the individual items.

    Methods

    Following established guidelines,6 ,7 we convened a consensus working group in London at which 15 international multidisciplinary delegates considered candidate items identified by a previous systematic literature review and an international e-Delphi prioritisation exercise,8 in the context of other published reporting standards and the panel's expertise in implementation science. The resultant checklist was subsequently developed iteratively by email discussion, and feedback on the penultimate draft guideline sought from colleagues working in implementation science.

    Scope of the StaRI reporting standards

    Implementation research is the scientific study of methods to promote the systematic uptake of evidence-based interventions into practice and policy and hence improve health.9–11 The discipline encompasses a broad range of methodologies applicable to improving the dissemination and implementation of clinical, healthcare, global health and public health interventions.12–14 The StaRI checklist focuses primarily on standards for reporting studies that evaluate implementation strategies developed to enhance the adoption, implementation and sustainability of interventions,15 but some items may be applicable to other study designs used in implementation science.

    The StaRI reporting guidelines

    Unlike most reporting guidelines that apply to a specific research methodology, StaRI is applicable to the broad range of study designs employed in implementation science. Authors are referred to other reporting standards for advice on reporting specific methodological features. In an evolving field, in which there is a range of study designs, terminology is neither static nor used consistently.16 For clarity, we have adopted specific terms in this paper; table 1 defines these terms and lists some of the alternative or related terminology.

    Table 1

    Terminology, definitions and resources

    Underpinning the StaRI reporting standards are the dual strands of describing, on the one hand, the implementation strategy and, on the other, the clinical, healthcare or public health intervention being implemented.17 These strands are represented as two columns in the checklist (see table 2). The primary focus of implementation science is the implementation strategy15 (column 1) and the expectation is that this will always be fully completed. The impact of the intervention on the target population should always be considered (column 2) and either health outcomes reported or robust evidence cited to support a known beneficial effect of the intervention on the health of individuals or populations. While all items are worthy of consideration, not all items will be applicable to or feasible in every study; a fully completed StaRI checklist may thus include a number of ‘not applicable’ items. For example, studies simultaneously testing a clinical intervention and an implementation strategy (Hybrid type 2 designs) would need to fully address both strands, whereas studies testing a clinical intervention while gathering information on its potential for implementation (Hybrid type 1) or testing an implementation strategy while observing the clinical outcomes (Hybrid Type 3) would focus primarily on items in the clinical intervention or implementation strategy columns, respectively.14

    Table 2

    Standards for Reporting Implementation Studies (StaRI): the StaRI checklist

    Elaboration on individual checklist items

    Table 3

    Example of a table describing* an implementation strategy compiled from Kilbourne et al19 description of the implementation of life goals (LG): a clinical intervention for patients with mood disorders

    Item 1. Title

    Identification as an implementation study, and description of the methodology in the title and/or keywords

    Item 2. Abstract

    Identification as an implementation study, including a description of the implementation strategy to be tested, the evidence-based intervention being implemented, and defining the key implementation and health outcomes.

    Item 3. Introduction (Identify the problem)

    Description of the problem, challenge or deficiency in healthcare or public health that the intervention being implemented aims to address.

    Item 4. Introduction (Rationale: implementation strategy and intervention)

    Item 5. Aims and objectives

    The aims of the study, differentiating between implementation objectives and any intervention objectives.

    Item 6. Methods: study design

    The design and key features of the evaluation (cross referencing to any appropriate methodology reporting standards) and any changes to study protocol, with reasons

    Item 7. Methods: context

    The context in which the intervention was implemented. (Consider social, economic, policy, healthcare, organisational barriers and facilitators that might influence implementation elsewhere).

    Item 8. Methods: Targeted sites and populations
    Item 9. Methods: Description
    Table 3

    Example of a table describing* an implementation strategy compiled from Kilbourne et al19 description of the implementation of life goals (LG): a clinical intervention for patients with mood disorders

    Figure 1

    Example of a timeline describing an implementation strategy (compiled from Pinnock et al18 description of the implementation of a telephone service for providing asthma reviews). Note: The three-arm implementation study is illustrated in the centre of this schema with the preceding usual care, randomisation on 1 January 2004, the 15-month intervention and subsequent roll-out. The context (specifically the introduction of the Quality and Outcome Framework) is shown at the top of the schema. Below the three-arms of the study are the components of the implementation strategy from set-up and training, ongoing service provision and maintenance and adoption into routine practice.

    Item 10. Methods (subgroups)

    Any subgroups recruited for additional research tasks, and/or nested studies are described

    Item 11. Methods: Outcomes
    Figure 2

    Summary of outcomes and the related items in the Standards for Reporting Implementation Studies checklist.

    Item 12. Methods: Process evaluation

    Process evaluation objectives and outcomes related to the mechanism by which the strategy is expected to work.

    Item 13. Methods: Economic evaluation
    Item 14. Methods: sample size

    Rationale for sample sizes (including sample size calculations, budgetary constraints, practical considerations, data saturation, as appropriate)

    Item 15. Methods: analysis

    Methods of analysis (with reasons for that choice)

    Item 16. Methods: Sub-group analyses

    Any a priori sub-group analyses (e.g. between different sites in a multicentre study, different clinical or demographic populations), and sub-groups recruited to specific nested research tasks

    Item 17. Results: Populations
    Item 18. Results: Outcomes
    Item 19. Results: Process evaluation

    Process data related to the implementation strategy mapped to the mechanism by which the strategy is expected to work.

    Item 20. Results: economic evaluation
    Item 21. Results: subgroups

    Representativeness and outcomes of subgroups including those recruited to specific research tasks

    Item 22. Results: Fidelity and adaptation
    Item 23. Results: context

    Contextual changes (if any) which may have affected the outcomes

    Item 24. Results: harms

    All important harms or unintended effects in each group.

    Item 25: Discussion: summary

    Summary of findings, strengths and limitations, comparisons with other studies, conclusions and implications.

    Item 26. Discussion: Implications
    Item 27. General

    Include statement(s) on regulatory approvals (including, as appropriate, ethical approval, confidential use of routine data, governance approval), trial/study registration (availability of protocol), funding and conflicts of interest.

    Conclusions

    The StaRI standards are registered with the EQUATOR Network (http://www.equator-network.org) and the checklist is available from this journal and BMJ.com. We invite editors of journals publishing interventional implementation studies to consider requiring submission of a StaRI checklist, and authors reporting such studies to adopt the checklist. We have presented the concepts and sought feedback from delegates at workshops, conference discussions and implementation project steering groups; in general, the concepts seem to resonate with colleagues. In the future, we would like to work with authors as they apply the checklist to their papers, ‘road testing’ the standards and enabling iterative development.

    We are particularly interested in whether the concept of the dual strands (implementation strategy/intervention) resonates with authors and readers of implementation studies. Is it practical to expect authors to consult other methodological checklists for reporting design-specific aspects of their study? We look forward to learning about innovative solutions to providing adequate descriptions of context, implementation strategies and interventions that accommodate the requirements of journals, needs of authors as well as preferences of readers.

    Previously published statements have been instrumental in improving reporting standards,10 and our hope is that StaRI will achieve a similar improvement in the reporting of implementation strategies that will facilitate translation of effective interventions into routine practice.

    Acknowledgments

    Members of the PRISMS team (Eleni Epiphaniou, Gemma Pearce and Hannah Parke) supported the underpinning literature work, and the e-Delphi was handled by ClinVivo. The authors are grateful to a colleagues (implementation science experts, healthcare researchers, clinicians, PhD students) who reviewed the penultimate draft of the StaRI statement and provided a reality check and constructive feedback: Helen Ashdown, David Chambers, Louise Craig, Clarisse Dibao-Dina, Peter Hanlon, Roger Jones, Rachel Jordan, Chris del Mar, Brian McKinstry, Susan Morrow, John Ovretveit, David Price, Kamran Siddiqui, Rafael Stelmach, Paul Stephenson, Shaun Treweek, Bryan Weiner. The authors also thank Melissa Goodbourn and Allison Worth who arranged feedback from the Edinburgh Clinical Research Facility Patient Advisory Panel (Stephanie Ashby, Alison Williams) and Steven Towndrow who coordinated feedback from the Patient and Public Involvement representatives of the NIHR CLAHRC North Thames (Ben Wills-Eve, Rahila Bashir, Julian Ashton, Colleen Ewart, Karen Williams).

    References

    View Abstract

    Footnotes

    • Collaborators The StaRI Group includes: MB, CRC, Peter Craig, SE, Eleni Epiphaniou, GG, JR-M, PM, Brian Mittman, EM, AP, Gemma Pearce, HP, AS and SJCT.

    • Contributors HP initiated the idea for the study and with SJCT led the development of the protocol, securing of funding, study administration, workshop and writing of the paper. AS, CJG and SE advised on the development of the protocol, and data analysis. All authors participated in the StaRI international working group along with GP, BM and MG. HP wrote the initial draft of the paper, to which all the authors contributed. HP is the study guarantor.

    • Funding The StaRI initiative and workshop was funded by contributions from the Asthma UK Centre for Applied Research (AC-2012-01); Chief Scientist Office, Scottish Government Health and Social Care Directorates (PCRCA_08_01); the Centre for Primary Care and Public Health, Queen Mary University of London; and with contributions in kind from the PRISMS team (NIHR HS&DR Grant ref: 11/1014/04). SJCT was (in part) supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart's Health NHS Trust. AS is supported by the Farr Institute.

    • Disclaimer The funding bodies had no role in the design, in the collection, analysis, and interpretation of data; in the writing of the manuscript; nor in the decision to submit the manuscript for publication. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

    • Competing interests All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare: research grants from Chief Scientist Office (HP), Asthma UK (AS, HP, SJCT), Farr Institute (AS), NIHR HS&DR (HP, SJCT), NIHR CLAHRC (SJCT) for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; CRC is the Deputy Editor-in-Chief for Academic Emergency Medicine and on the editorial boards for the Journal of the American Geriatrics Society and Annals of Internal Medicine's ACP Journal Club and serves as paid faculty for Emergency Medical Abstracts, JR-M is the Director of the NIHR HS&DR Programme, no other relationships or activities that could appear to have influenced the submitted work.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.

    Request permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.