You are here

  • Home
  • Archive
  • Volume 10, Issue 6
  • Implementation of e-mental health interventions for informal caregivers of adults with chronic diseases: a protocol for a mixed-methods systematic review with a qualitative comparative analysis

Article Text

Article menu

Download PDFPDF

XML
Mental health
Protocol
Implementation of e-mental health interventions for informal caregivers of adults with chronic diseases: a protocol for a mixed-methods systematic review with a qualitative comparative analysis
  1. Chelsea Coumoundouros1,
  2. Louise von Essen1,
  3. Robbert Sanderman2,3,
  4. Joanne Woodford1
  1. 1Clinical Psychology in Healthcare, Department of Women’s and Children’s Health, Uppsala University, Uppsala, Sweden
  2. 2Department of Health Psychology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
  3. 3Department of Psychology, Health and Technology, University of Twente, Enschede, The Netherlands
  1. Correspondence to Ms Chelsea Coumoundouros; chelsea.coumoundouros{at}kbh.uu.se

Abstract

Introduction Informal caregivers provide the majority of care to individuals with chronic health conditions, benefiting the care recipient and reducing use of formal care services. However, providing informal care negatively impacts the mental health of many caregivers. E-mental health interventions have emerged as a way to provide accessible mental healthcare to caregivers. Much attention has been given to reviewing the effectiveness and efficacy of such interventions, however, factors related to implementation have received less consideration. Therefore, this mixed-methods systematic review will aim to examine factors associated with the effectiveness and implementation of e-mental health interventions for caregivers.

Methods and analysis Eligible studies published since 1 January 2007 will be searched for in several electronic databases (CINAHL Plus with Full Text, the Cochrane Library, EMBASE, PsycINFO, PubMed and Web of Science), clinical trial registries and OpenGrey, with all screening steps conducted by two independent reviewers. Studies will be included if they focus on the implementation or effectiveness of e-mental health interventions designed for informal adult caregivers of adults with cancer, heart disease, stroke, diabetes, dementia or chronic obstructive pulmonary disease. Pragmatic randomised controlled trials quantitatively reporting on caregiver anxiety, depression, psychological distress or stress will be used for a qualitative comparative analysis to identify combinations of conditions that result in effective interventions. Qualitative and quantitative data on implementation of e-mental health interventions for caregivers will be integrated in a thematic synthesis to identify barriers and facilitators to implementation. These results will inform future development and implementation planning of e-mental health interventions for caregivers.

Ethics and dissemination Ethical approval is not required for this study as no primary data will be collected. Results will be disseminated in the form of a scientific publication and presentations at academic conferences and plain language summaries for various stakeholders.

PROSPERO registration number CRD42020155727.

  • mental health
  • World Wide Web technology
  • primary care
  • depression & mood disorders
  • anxiety disorders
https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/bmjopen-2019-035406

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The mixed-method design of this review will ensure a wide variety of data on implementation is captured and interpretations account for both qualitative and quantitative research findings.

    • The peer-reviewed, comprehensive search strategy with all selection steps completed by two independent reviewers will ensure a thorough search of the literature and reduce bias in study selection.

    • High heterogeneity across studies in terms of implementation or intervention features is easily accommodated in a qualitative comparative analysis.

    • Crisp set qualitative comparative analysis produces concrete results, increasing the usability of findings for healthcare professionals and decision-makers.

    • However, crisp set qualitative comparative analysis dichotomises all variables including the outcome, therefore, a more detailed understanding of the strength of the effect size may be lost in this process.

    Introduction

    Informal caregivers (hereafter referred to as caregivers) are family members or friends who provide unpaid support and care to individuals with healthcare needs. Caregivers play a vital societal role in healthcare systems worldwide, providing up to 80% of care to individuals with long-term care needs.1 Informal care provision can include emotional support, assistance with household tasks (eg, cooking, cleaning), medical care, transportation, managing finances and advocacy on behalf of the care recipient.2 3 Demand for caregivers is expected to increase in the future as the proportion of older adults in populations around the world increases and healthcare policies favour deinstitutionalisation and outpatient care.4 5 As societal dependence on informal care continues to grow, it is becoming increasingly important to implement programmes and policies to support individuals who become caregivers.2 6

    Caregivers can experience both positive and negative outcomes over the course of their time as a caregiver.3 7 8 Caregiving can lead to an improved relationship between caregiver and care recipient, feelings of personal development and a sense of accomplishment related to obtaining skills and recognising the impact of the care they provide.8 However, caregivers may also experience negative outcomes related to the caregiving role, such as financial strain and poor physical and mental health.3 7 9 Indeed, the rate of depression and anxiety among caregivers exceeds that of the general population.10 The prevalence of depressive symptoms in cancer and stroke caregivers is often above 40% and the prevalence of anxiety ranges from 21% to over 40%.11 12 Mental health problems can result in large personal and societal costs related to increased morbidity and reduced productivity.13–15 Additionally, poor caregiver mental health negatively impacts distress levels in the care recipient16 17 and the quality of care provided by the caregiver.18–21

    There is a clear need to develop effective interventions and resources to prevent or reduce the mental health burden experienced by caregivers. However, caregivers have reported various barriers to accessing mental health services such as lack of knowledge regarding available services, financial barriers, stigma and prioritisation of the caregiving role over self-care.22 E-health technologies have emerged as an accessible way to provide support and information to caregivers23–25 and can be designed to achieve various goals such as, improving communication, teaching skills or reducing depression.26 27 Numerous systematic reviews and meta-analyses have examined e-health interventions for caregivers of adults with chronic health conditions, showing the potential for e-health interventions to improve caregiver well-being.23 25 28–40

    E-mental health interventions, that is, mental health interventions delivered via the internet or using mobile technologies,27 41 42 represent a subset of e-health interventions. E-mental health solutions offer a means to improve mental health service access globally43 44 by eliminating many barriers to mental health service access (eg, transportation, stigma, time)25 42 45 and are often more cost-effective than traditional therapies.42 45 Meta-analyses show reductions in caregiver’s depression and anxiety in response to e-mental health interventions.37 39 However, as many reviews focus on intervention efficacy and effectiveness,23 28–32 34 36–40 46 gaps remain in our understanding of factors related to the intervention and the implementation context that make e-mental health interventions effective among caregivers.

    Wider literature suggests that the implementation of e-health programmes in real-world settings often encounters many barriers,24 33 47 48 preventing effective interventions from being made available to those who need them. Few reviews have examined factors related to implementation of e-health interventions for caregivers,33 35 with no current reviews, to the best of our knowledge, focusing on implementation of e-mental health interventions for caregivers exclusively. Evaluating the implementation of an intervention is essential to gain insights into why interventions succeed or fail when put into practice. Factors influencing implementation can be related to the intervention itself, the participants, the implementation setting and wider societal factors (eg, regional policies).49 Trials with a more pragmatic design may be better suited to investigating factors potentially associated with implementation given real-world conditions are more closely reflected in pragmatic trials.50 51 However, systematic reviews and meta-analyses do not often distinguish between pragmatic and explanatory (also referred to as efficacy) trials despite the different conditions (eg, setting, recruitment methods, eligibility criteria, control of adherence to and delivery of the intervention) under which interventions are evaluated.50 52 Identifying trials with a pragmatic design may be a valuable factor to consider when interpreting results of reviews to inform implementation.

    The aim of this review is to examine factors related to the effectiveness and implementation of e-mental health interventions for caregivers of adults with chronic diseases. Two approaches will be used to investigate this. First, studies with more pragmatic designs will be used exclusively to determine which combinations of intervention or implementation characteristics are associated with effectiveness using a qualitative comparative analysis. Second, reports regarding the implementation of e-mental health interventions will be thematically synthesised to establish the common barriers and facilitators to e-mental health implementation. Findings from this review can be used to guide the development of effective e-mental health interventions to support caregivers and ensure the successful implementation of these interventions within real-world healthcare settings.

    Methods and analysis

    The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA) checklist53 (online supplementary appendix 1) and the Joanna Briggs methodology for mixed-methods systematic reviews54 were used to guide the development of this protocol. Any protocol amendments will be recorded in PROSPERO.

    Study eligibility criteria

    The eligibility criteria used to inform study inclusion and exclusion are outlined using population, interventions, comparators, outcomes and study design (PICOS).55 56

    Population

    Unpaid adult caregivers (aged 18 years or older) of adults with either cancer, chronic obstructive pulmonary disease (COPD), dementia, diabetes, heart disease or stroke. Care recipient’s chronic health conditions eligible for inclusion were selected as, globally, they are the largest sources of disability adjusted life years due to physical chronic diseases in adults57 and often require informal care.58 No restrictions will be placed on the frequency or amount of care provided for someone to be considered a caregiver. Studies exclusively focusing on caregivers with severe mental health conditions (eg, psychosis or bipolar disorder) will be excluded, as the focus of this review is on e-mental health interventions targeting psychological health difficulties associated with the provision of informal care, for example anxiety or depression, as opposed to targeting severe mental health conditions. Studies with interventions that solely focus on caregivers providing care to non-community dwelling care recipients will be excluded, given caregivers of individuals who do not live in the community may spend less time providing informal care59 and generally experience lower levels of depression.60 61 Additionally, studies of interventions designed specifically for caregivers of individuals at the end-of-life (eg, within a few months of death) will be excluded, as end-of-life caregiving is associated with additional needs and burdens, for example difficulties related to grief and bereavement.62

    Interventions

    Interventions will use internet technology, such as web-based platforms or mobile-based applications, to deliver a mental health intervention to caregivers.27 41 E-mental health interventions can encompass many types of mental health support such as screening, prevention, treatment or service delivery.41 This review will focus on interventions targeting the treatment of common caregiver psychological health difficulties (anxiety, depression, psychological distress or stress). This can include any type of mental health treatment, including psychoeducation. Psychoeducation is defined as the provision of information regarding common psychological health difficulties and can be delivered passively (eg, an information website) or actively (eg, an information website with therapist support, homework or exercises).63 The majority of therapeutic materials within the e-mental health intervention must be internet based, however, this may be supplemented with additional forms of support (such as telephone contact, face-to-face support or video conferencing). There are no restrictions on the amount of support provided within the e-mental health intervention. Interventions delivered via telephone, CD-ROM (Compact Disc-Read Only Memory) or video (including Skype) alone will be excluded.

    Comparators

    As it is necessary to determine effect sizes for the qualitative comparative analysis,64 only studies of pragmatic randomised controlled trials with non-active controls will be included in this analysis. Non-active controls include: no treatment, wait-list control, treatment as usual, non-specific treatment component control (eg, control for attention) or education on the care recipient’s condition.65 Studies using psychoeducation or active controls (eg, controls using specific treatment components or studies comparing two therapies) will be excluded.

    For thematic synthesis of barriers and facilitators to implementation, studies of any design (eg, randomised controlled trials, process evaluations, focus groups) will be included in the analysis, regardless of the presence or absence of a control.

    Outcomes

    For the qualitative comparative analysis, studies must report on caregiver mental health outcomes, specifically anxiety, depression, psychological distress or stress, measured using an instrument with at least acceptable reliability (Cronbach’s alpha ≥0.7).66 Reliability of outcome measures will be assessed based on the main validation paper of the relevant measurement instrument, as this review will likely include studies with different caregiver populations, ages, genders and languages, the combination of which may not have been validated. Examples of eligible measurement instruments include the Centre for Epidemiologic Studies-Depression Scale,67 the Hospital Anxiety and Depression Scale68 or the Perceived Stress Scale.69

    For the thematic synthesis, studies will report on barriers and/or facilitators to intervention implementation. This may include qualitative (eg, interviews or focus groups) or quantitative (eg, Normalisation Measure Development questionnaire70) data. Barriers or facilitators can include factors related to any aspect of the Consolidated Framework for Implementation Research49 or the implementation outcome framework developed by Proctor et al.71 The Consolidated Framework for Implementation Research consists of five domains related to implementation, namely (1) intervention characteristics (eg, adaptability, complexity); (2) outer setting (eg, external policies, patient needs and resources); (3) inner/implementation setting (eg, culture within the organisation, readiness for implementation); (4) characteristics of individuals (eg, self-efficacy, individual stage of change) and (5) process (eg, planning, engaging).49 The implementation outcome framework broadly classifies measurable implementation outcomes which includes acceptability, adoption, feasibility, fidelity, reach, appropriateness, implementation cost and sustainability.71

    Study designs

    Studies included for the qualitative comparative analysis must be pragmatic randomised controlled trials (also referred to as effectiveness trials). Pragmatic trials will be identified using the validated PRagmatic Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool.50 PRECIS-2 was developed with input from clinicians, researchers and policy-makers to allow trialists to assess how pragmatic or explanatory their trial design is across nine domains: eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome and primary analysis.50 Trials with a pragmatic design will be defined as any trial with a mean score of 3 or higher using the PRECIS-2 tool.50 PRECIS-2 has been used with this cut-off score to categorise studies in another systematic review,72 although to our knowledge it has not previously been used to exclude studies from a systematic review. Using a cut-off score of 3 should ensure generous inclusion of trials containing at least a mixture of pragmatic and explanatory design features.50

    To assess barriers and facilitators to implementation, any study type with quantitative and/or qualitative data will be eligible for inclusion.

    Search strategy

    Comprehensive literature searches will be conducted in multiple electronic databases (CINAHL Plus with Full Text, the Cochrane Library, EMBASE, PsycINFO, PubMed and Web of Science). Clinical trial registries (www.clinicaltrials.gov and www.who.int/trialsearch/) will be searched for relevant completed clinical trials and the resulting publications will be found and screened for inclusion. Searches for grey literature will be performed using OpenGrey (http://www.opengrey.eu/), a database of grey literature in Europe such as research reports and conference papers.

    The search strategy was developed in consultation with Agnes Kotka, a librarian at Uppsala University and was reviewed by Professor Mariët Hagedoorn and Truus van Ittersum (University Medical Centre Groningen, University of Groningen) and Dr. Nathan Davies (University College London) following the PRESS peer review guidelines73 (online supplementary appendix 2). The search was constructed using terms related to (1) caregivers; (2) the chronic health conditions of interest (cancer, COPD, dementia, diabetes, heart disease and stroke); (3) e-health/information and communication technology; (4) mental health and (5) psychological therapies (see online supplementary appendix 3). Included terms were informed by existing reviews focusing on the population and/or intervention of interest to this review.9 33 39 74–79 Search terms were refined based on feedback from the peer-review process, resulting in the addition of more truncations to search terms, elimination of repetitive search terms that did not retrieve additional records and the addition of an abbreviation missed prior to the peer-review process. The search will include relevant Medical Subject Headings when possible and terms will be searched for in the title/abstract of publications. Included studies will be restricted to those published in English, Dutch, German or Swedish. Literature produced from January 2007 onwards will be eligible for inclusion. Technologies from work published prior to 2007 may be outdated and other reviews have shown that production of publications involving e-health began to rise from 2007 onwards.33 35 Electronic searches will be rerun prior to reporting of results to ensure the search is as up to date as possible.

    On final inclusion of any studies, their references, results from forward citation searches and from the first three pages of the ‘find similar’ search function in PubMed will be used to check for any additional studies of interest. Experts in the field will be contacted to identify further studies for inclusion.

    Study selection

    Results of database searches will be imported into EndNote for deduplication following the procedures outlined by Bramer et al.80 Remaining records will be imported into the online screening software Rayyan.81 Titles, abstracts and full texts will be screened independently by two reviewers. Conflicts will be discussed and a third reviewer will be consulted if consensus cannot be reached. Study selection will be based on the criteria outlined by the PICOS, with reasons for study exclusion being recorded at the full-text screening stage. Full texts will be checked against each sub-section of the PICOS, recording which sub-sections are or are not met by each study, with an overall reason for exclusion being reported in the PRISMA flow diagram. This will facilitate detailed discussions regarding study exclusion when conflicts arise. If studies do not contain enough information to decide on inclusion, the original authors will be contacted at most twice over a 1-month period to obtain information to determine study eligibility. If the original authors do not respond, the study will be excluded. Abstracts, theses, books, commentaries, editorials and letters to the editor will be excluded. Reviews and study protocols will also be excluded, however, the references of related reviews will be checked for additional studies of interest, published results of relevant study protocols will be obtained and if protocol results are unpublished, authors will be contacted to determine whether access to unpublished results is possible.

    Records retrieved from searches of clinical trial registries and OpenGrey will be screened for eligibility by one reviewer. When relevant clinical trial registries are identified, any resulting publications will be retrieved and screened for inclusion, unless already captured by the electronic database searches. If results from relevant trial registries are unpublished, authors will be contacted to determine if they are able to share details of any available results. Authors of grey literature records that do not contain enough information to assess eligibility will also be contacted for additional study details.

    Exclusion of studies on the basis of adopting a more explanatory, as opposed to pragmatic, trial design will be conducted as a final step during the full-text screening process. This screening step will only be applied to trials eligible for the qualitative comparative analysis. Studies will be scored using the PRECIS-2 tool by two independent reviewers and studies with a mean score below 3 will be excluded.50 72

    Assessment of methodological quality

    Methodological quality of studies included in the qualitative comparative analysis will be evaluated using the Cochrane Risk of Bias 2.0 tool for randomised controlled trials.82 83 This evaluation will facilitate the identification of selection, performance, measurement, attrition and reporting bias.83 Authors will be contacted if more information is required to complete the quality assessment. Reporting bias will be explored by comparing outcomes measures described in study protocols to the outcome measures reported in the methods and results sections of the corresponding completed trial. In response to any identified inconsistencies, authors will be contacted to determine potential causes of this. Study assessment will be conducted by two independent reviewers, followed by discussion of any discrepancies, consulting a third reviewer as needed. Studies will not be excluded based on methodological quality, however the results of the Cochrane Risk of Bias 2.0 evaluation will be reported descriptively.

    Data extraction

    Data from included full texts will be extracted into Microsoft Excel (2016), using a data extraction form developed for this review based on the Centre for Reviews and Dissemination guidelines.56 Extracted information will include data pertaining to study participants, study design, the intervention and relevant outcomes (full details in online supplementary appendix 4). Data used in the qualitative comparative analysis and thematic synthesis will be extracted independently by two reviewers, with resulting extractions compared for accuracy and completion. All other data will be extracted by one reviewer and verified by a second reviewer. If conflicts arise, the original publication will be referred to in order to resolve misunderstandings and a third reviewer will be consulted if necessary. Authors will be contacted at most twice to obtain additional data and/or clarification as needed. Qualitative results pertaining to implementation will be transferred into NVivo V.10 software84 for thematic synthesis.

    Data synthesis

    Data related to the characteristics of each included study, such as the sample (eg, sample size, participant demographics) or intervention (eg, duration, type of support provided, delivery mode) characteristics, will be reported in summary tables. Further data synthesis will involve two analysis methods. Pragmatic randomised controlled trials with quantitative mental health outcome data will be included in the qualitative comparative analysis. Publications of any study design reporting on implementation will be included in the thematic synthesis, taking an integrative approach to synthesise both qualitative and quantitative findings.

    Qualitative comparative analysis

    A crisp set qualitative comparative analysis will be conducted to determine sets of conditions that result in effective e-mental health interventions for caregivers.64 Crisp set qualitative comparative analysis involves dichotomising outcome data (eg, effective or not effective) and conditions (eg, present or absent) selected for inclusion in the analysis into distinct categories.64 A crisp set analysis approach was selected over a fuzzy set analysis as the results will be more clearly interpretable and easier for decision-makers to use.85

    The first step of a qualitative comparative analysis is to build a data table containing information regarding the effectiveness of each study and conditions related to the intervention and its implementation (see online supplementary appendix 5).64 Conditions to include in the data table will be based on important factors related to intervention components (eg, uses goal-setting, homework), intervention delivery methods (eg, mobile app, computer) and implementation (eg, acceptability, feasibility). By restricting this analysis to pragmatic trials, which are designed to more closely reflect real-world settings, implementation conditions are more likely to be reported. Conditions selected will be adjusted given the need to ensure adequate heterogeneity is present.64 Qualitative comparative analysis requires diversity among studies in terms of conditions present and intervention effectiveness in order to determine the combination of factors sufficient for interventions to be effective.64 Therefore, adjustments to outcome classification and conditions selected for analysis will be needed after data collection is completed.

    Intervention effectiveness will be measured as the standardised mean effect size between control and comparator groups’ mental health outcomes, calculated using Hedges’ g and the Comprehensive Meta-Analysis (V.3) software. Effect sizes will be calculated for all mental health outcomes of interest for this review (anxiety, depression, psychological distress and stress) and will be based on data collected immediately after intervention completion. If enough studies report subsequent postintervention follow-ups, these effect sizes will be calculated to explore whether different factors contribute to sustained intervention success. Effect sizes will be used to create crisp sets to categorise studies as effective (Hedges’ g≥0.3) or not effective.86 If most interventions are effective (or not effective), a different classification system will be created to ensure adequate heterogeneity for analysis,64 for example, categorising studies as highly effective (Hedges’ g≥0.5) or not highly effective.86 Proposed cut-offs were developed based on existing meta-analyses of e-mental health interventions.87–90

    The main data table will use general effectiveness as the outcome measure, meaning the primary mental health outcome as identified in each study will be used to represent the effectiveness of that intervention. If studies include multiple outcomes of interest, but do not identify a primary outcome measure, the outcome most frequently measured in included studies will be used to evaluate intervention effectiveness. Secondary analyses may be conducted for anxiety, depression, psychological distress and stress separately, to explore whether different conditions are more important for different outcome measures. However, this is dependent on identification of an adequate number of studies for each outcome of interest. After completion of the data table, truth tables will be constructed and the software fs/QCA (V.3.1b) will be used to determine the sufficient conditions for effective e-mental health interventions.91 92

    Thematic synthesis

    Data from studies addressing implementation of e-mental health interventions for caregivers will be thematically synthesised using a deductive coding approach, to identify barriers and facilitators experienced during implementation.93 94 It will likely be necessary to integrate qualitative and quantitative data as many aspects of implementation such as acceptability, feasibility and usability, may be measured using quantitative tools.95 First, qualitative data will be thematically analysed using the Consolidated Framework for Implementation Research to guide coding.49 This framework was selected a priori as it was developed by combining multiple implementation theories into a single, comprehensive theory covering all aspects related to implementation49 and it has been used as a coding guide in other reviews on implementation.33 96 Qualitative data will be coded based on the 39 pre-defined constructs within the Consolidated Framework for Implementation Research,49 with the creation of additional codes if needed.

    Quantitative data will be narratively summarised to facilitate subsequent integration of qualitative and quantitative findings. Creating narrative summaries will involve approaches such as textually describing study findings and grouping findings based on the constructs and domains of the Consolidated Framework for Implementation Research.94 Initially, 10% of full texts included in the thematic synthesis will be coded independently by two reviewers, followed by discussion of the coding process in consultation with a third reviewer. The remaining coding will be conducted by one reviewer with regular discussions with a second reviewer, involving a third reviewer as needed. Results of the initial coding of qualitative data and narrative summaries of quantitative data will be analysed together to identify barriers and facilitators to implementation. Two reviewers will independently identify barriers and facilitators, followed by discussion involving a third reviewer as needed.93 Through this discussion, more abstract, analytical themes will be developed that go beyond the initial codes and identified barriers and facilitators.93 This process will be iterative, modifying barriers and facilitators after defining initial analytical themes, followed by further refinement of analytical themes until the analytical themes fully encompass all codes and identified barriers and facilitators.93

    Patient and public involvement

    There was no patient or public involvement in the development of this protocol. However, caregivers and healthcare professionals will be involved in the interpretation of data for the thematic synthesis.97 98 Results of the thematic synthesis will be presented to a multidisciplinary group of stakeholders (eg, caregivers, mental health service providers) to explore whether the identified themes resonate with their experiences or if they feel important aspects related to implementation have not been captured by the synthesis. Their perspectives will be incorporated into the interpretation of the results of the thematic synthesis.

    Discussion

    Despite the importance of implementation planning and need for effective e-mental health interventions for caregivers in real world practice, there have been no reviews focusing on this area. Using pragmatic trials and implementation research, this review will identify both the key characteristics of effective interventions and barriers and facilitators to implementation. A qualitative comparative analysis will be employed to identify combinations of conditions resulting in effective e-mental health interventions for caregivers, a method which, to the best of our knowledge, has not yet been used in this field. The results of the qualitative comparative analysis can be used to improve the design of future e-mental health interventions by identifying intervention components and implementation factors important to intervention effectiveness in real-world settings.

    Additionally, common barriers and facilitators to implementation of e-mental health interventions for caregivers identified in this review can be used to inform implementation planning for similar interventions designed to reduce the mental health burden experienced by caregivers. For example, results may highlight the importance of providing training to individuals delivering the intervention or involvement of management staff in implementation activities. Improving our understanding of factors associated with implementation will allow implementers to both account for and avoid common implementation challenges, thereby potentially increasing subsequent uptake and effectiveness of e-mental health programmes developed to support caregivers.

    Ethics and dissemination

    The results of this work will be disseminated in the form of a scientific publication in a peer-reviewed journal and as presentations at conferences. Plain language summaries will be prepared and provided to groups working with or supporting caregivers and healthcare organisations. Results will also be disseminated throughout the Marie Sklodowska-Curie Innovation Training Network, ENTWINE, which conducts research related to informal care and technological interventions to support caregivers.

    Acknowledgments

    We would like to thank Agnes Kotka, librarian at Uppsala University, for assisting in development of the search strategy. We are also grateful to Professor Mariët Hagedoorn, Truus van Ittersum (University Medical Centre Groningen, University of Groningen) and Dr. Nathan Davies (University College London), for providing valuable feedback as the peer-reviewers of the search strategy.

    References

    1. ↵
      1. Hoffmann F,
      2. Rodrigues R
      . Informal carers: who takes care of them? Policy brief. Vienna: : European Centre for Social Welfare Policy and Research, 2010. Available: https://www.euro.centre.org/publications/detail/387 [Accessed 8 Jul 2019].
    2. ↵
      1. National Academies of Sciences Engineering and Medicine
      . . Family caregiving roles and impacts. In: Committee on Family Caregiving for Older Adults, Board on Health Care Services, Health and Medicine Division, et al., eds. Families Caring for an Aging America. Washington, DC: The National Academies Press, 2016.
    3. ↵
      1. Girgis A,
      2. Lambert S
      . Cost of informal caregiving in cancer care. Cancer Forum 2017;41:16–22 https://search.informit.com.au/documentSummary;dn=112312308779892;res=IELHEA
      OpenUrl
    4. ↵
      1. Spasova S,
      2. Baeten R,
      3. Coster S, et al
      . Challenges in long-term care in Europe: A study of national policies. Brussels: European Comission: European Social Policy Network (ESPN), 2018.
    5. ↵
      1. Colombo F,
      2. Llena-Nozal A,
      3. Mercier J, et al
      . Help wanted? providing and paying for long-term care. OECD Publishing 2011.
    6. ↵
      1. Schulz R,
      2. Czaja SJ
      . Family caregiving: a vision for the future. Am J Geriatr Psychiatry 2018;26:358–63.doi:10.1016/j.jagp.2017.06.023pmid:http://www.ncbi.nlm.nih.gov/pubmed/28774786
      OpenUrlPubMed
    7. ↵
      1. Lambert SD,
      2. Levesque J,
      3. Girgis A
      . The impact of cancer and chronic conditions on caregivers and family members. In: Koczwara B, ed. Cancer and chronic conditions. Singapore: Springer, 2016: 159–202.
    8. ↵
      1. Li Q,
      2. Loke AY
      . The positive aspects of caregiving for cancer patients: a critical review of the literature and directions for future research. Psychooncology 2013;22:2399–407.doi:10.1002/pon.3311pmid:http://www.ncbi.nlm.nih.gov/pubmed/23712938
      OpenUrlCrossRefPubMed
    9. ↵
      1. Coumoundouros C,
      2. Ould Brahim L,
      3. Lambert SD, et al
      . The direct and indirect financial costs of informal cancer care: a scoping review. Health Soc Care Community 2019;27:e622–36.doi:10.1111/hsc.12808pmid:http://www.ncbi.nlm.nih.gov/pubmed/31293013
      OpenUrlPubMed
    10. ↵
      1. Pinquart M,
      2. Sörensen S
      . Differences between caregivers and noncaregivers in psychological health and physical health: a meta-analysis. Psychol Aging 2003;18:250–67 https://psycnet-apa-org.proxy3.library.mcgill.ca/fulltext/2003-05349-011.pdfdoi:10.1037/0882-7974.18.2.250pmid:http://www.ncbi.nlm.nih.gov/pubmed/12825775
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Loh AZ,
      2. Tan JS,
      3. Zhang MW, et al
      . The global prevalence of anxiety and depressive symptoms among caregivers of stroke survivors. J Am Med Dir Assoc 2017;18:111–6.doi:10.1016/j.jamda.2016.08.014pmid:http://www.ncbi.nlm.nih.gov/pubmed/27742585
      OpenUrlPubMed
    12. ↵
      1. Geng H-M,
      2. Chuang D-M,
      3. Yang F, et al
      . Prevalence and determinants of depression in caregivers of cancer patients: a systematic review and meta-analysis. Medicine 2018;97:e11863.doi:10.1097/MD.0000000000011863pmid:http://www.ncbi.nlm.nih.gov/pubmed/30278483
      OpenUrlPubMed
    13. ↵
      1. Birnbaum HG,
      2. Kessler RC,
      3. Kelley D, et al
      . Employer burden of mild, moderate, and severe major depressive disorder: mental health services utilization and costs, and work performance. Depress Anxiety 2010;27:78–89.doi:10.1002/da.20580pmid:http://www.ncbi.nlm.nih.gov/pubmed/19569060
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Kessler RC
      . The costs of depression. Psychiatr Clin North Am 2012;35:1–14.doi:10.1016/j.psc.2011.11.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/22370487
      OpenUrlCrossRefPubMed
    15. ↵
      1. Doran CM,
      2. Kinchin I
      . A review of the economic impact of mental illness. Aust Health Rev 2019;43:43–8.doi:10.1071/AH16115pmid:http://www.ncbi.nlm.nih.gov/pubmed/29129189
      OpenUrlPubMed
    16. ↵
      1. Hagedoorn M,
      2. Sanderman R,
      3. Bolks HN, et al
      . Distress in couples coping with cancer: a meta-analysis and critical review of role and gender effects. Psychol Bull 2008;134:1–30 https://psycnet-apa-org.proxy3.library.mcgill.ca/fulltext/2007-19419-001.pdfdoi:10.1037/0033-2909.134.1.1pmid:http://www.ncbi.nlm.nih.gov/pubmed/18193993
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Hodges LJ,
      2. Humphris GM,
      3. Macfarlane G
      . A meta-analytic investigation of the relationship between the psychological distress of cancer patients and their carers. Soc Sci Med 2005;60:1–12.doi:10.1016/j.socscimed.2004.04.018pmid:http://www.ncbi.nlm.nih.gov/pubmed/15482862
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Smith GR,
      2. Williamson GM,
      3. Miller LS, et al
      . Depression and quality of informal care: a longitudinal investigation of caregiving stressors. Psychol Aging 2011;26:584–91.doi:10.1037/a0022263pmid:http://www.ncbi.nlm.nih.gov/pubmed/21417536
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Christie J,
      2. Smith GR,
      3. Williamson GM, et al
      . Quality of informal care is multidimensional. Rehabil Psychol 2009;54:173–81.doi:10.1037/a0015705pmid:http://www.ncbi.nlm.nih.gov/pubmed/19469607
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Cooney C,
      2. Howard R,
      3. Lawlor B
      . Abuse of vulnerable people with dementia by their carers: can we identify those most at risk? Int J Geriatr Psychiatry 2006;21:564–71.doi:10.1002/gps.1525pmid:http://www.ncbi.nlm.nih.gov/pubmed/16783768
      OpenUrlCrossRefPubMed
    21. ↵
      1. Beach SR,
      2. Schulz R,
      3. Williamson GM, et al
      . Risk factors for potentially harmful informal caregiver behavior. J Am Geriatr Soc 2005;53:255–61.doi:10.1111/j.1532-5415.2005.53111.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/15673349
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Mosher CE,
      2. Given BA,
      3. Ostroff JS
      . Barriers to mental health service use among distressed family caregivers of lung cancer patients. Eur J Cancer Care 2015;24:50–9.doi:10.1111/ecc.12203pmid:http://www.ncbi.nlm.nih.gov/pubmed/24761985
      OpenUrlPubMed
    23. ↵
      1. Ploeg J,
      2. Ali MU,
      3. Markle-Reid M, et al
      . Caregiver-focused, web-based interventions: systematic review and meta-analysis (Part 2). J Med Internet Res 2018;20:e11247.doi:10.2196/11247pmid:http://www.ncbi.nlm.nih.gov/pubmed/30368439
      OpenUrlPubMed
    24. ↵
      1. Christie HL,
      2. Martin JL,
      3. Connor J, et al
      . eHealth interventions to support caregivers of people with dementia may be proven effective, but are they implementation-ready? Internet Interv 2019;18:100260.doi:10.1016/j.invent.2019.100260pmid:http://www.ncbi.nlm.nih.gov/pubmed/31890613
      OpenUrlPubMed
    25. ↵
      1. Sin J,
      2. Henderson C,
      3. Spain D, et al
      . eHealth interventions for family carers of people with long term illness: a promising approach? Clin Psychol Rev 2018;60:109–25.doi:10.1016/j.cpr.2018.01.008pmid:http://www.ncbi.nlm.nih.gov/pubmed/29429856
      OpenUrlPubMed
    26. ↵
      1. Pagliari C,
      2. Sloan D,
      3. Gregor P, et al
      . What is eHealth (4): a scoping exercise to map the field. J Med Internet Res 2005;7:e9.doi:10.2196/jmir.7.1.e9pmid:http://www.ncbi.nlm.nih.gov/pubmed/15829481
      OpenUrlCrossRefPubMed
    27. ↵
      1. Eysenbach G
      . What is e-health? J Med Internet Res 2001;3:e20.doi:10.2196/jmir.3.2.e20pmid:http://www.ncbi.nlm.nih.gov/pubmed/11720962
      OpenUrlCrossRefPubMed
    28. ↵
      1. Aldehaim AY,
      2. Alotaibi FF,
      3. Uphold CR, et al
      . The impact of technology-based interventions on informal caregivers of stroke survivors: a systematic review. Telemed J E Health 2016;22:223–31.doi:10.1089/tmj.2015.0062pmid:http://www.ncbi.nlm.nih.gov/pubmed/26274910
      OpenUrlCrossRefPubMed
    29. ↵
      1. Tang WP,
      2. Chan CW,
      3. So WK, et al
      . Web-Based interventions for caregivers of cancer patients: a review of literatures. Asia Pac J Oncol Nurs 2014;1:9–15.doi:10.4103/2347-5625.135811pmid:http://www.ncbi.nlm.nih.gov/pubmed/27981077
      OpenUrlPubMed
    30. ↵
      1. Kaltenbaugh DJ,
      2. Klem ML,
      3. Hu L, et al
      . Using web-based interventions to support caregivers of patients with cancer: a systematic review. Oncol Nurs Forum 2015;42:156–64.doi:10.1188/15.ONF.156-164pmid:http://www.ncbi.nlm.nih.gov/pubmed/25806882
      OpenUrlCrossRefPubMed
    31. ↵
      1. McKechnie V,
      2. Barker C,
      3. Stott J
      . Effectiveness of computer-mediated interventions for informal carers of people with dementia-a systematic review. Int Psychogeriatr 2014;26:1619–37.doi:10.1017/S1041610214001045pmid:http://www.ncbi.nlm.nih.gov/pubmed/24989249
      OpenUrlCrossRefPubMed
    32. ↵
      1. Wasilewski MB,
      2. Stinson JN,
      3. Cameron JI
      . Web-Based health interventions for family caregivers of elderly individuals: a scoping review. Int J Med Inform 2017;103:109–38.doi:10.1016/j.ijmedinf.2017.04.009pmid:http://www.ncbi.nlm.nih.gov/pubmed/28550996
      OpenUrlPubMed
    33. ↵
      1. Christie HL,
      2. Bartels SL,
      3. Boots LMM, et al
      . A systematic review on the implementation of eHealth interventions for informal caregivers of people with dementia. Internet Interv 2018;13:51–9.doi:10.1016/j.invent.2018.07.002pmid:http://www.ncbi.nlm.nih.gov/pubmed/30206519
      OpenUrlPubMed
    34. ↵
      1. Guay C,
      2. Auger C,
      3. Demers L, et al
      . Components and outcomes of Internet-based interventions for caregivers of older adults: systematic review. J Med Internet Res 2017;19:e313.doi:10.2196/jmir.7896pmid:http://www.ncbi.nlm.nih.gov/pubmed/28928109
      OpenUrlPubMed
    35. ↵
      1. Heynsbergh N,
      2. Heckel L,
      3. Botti M, et al
      . Feasibility, useability and acceptability of technology-based interventions for informal cancer carers: a systematic review. BMC Cancer 2018;18:244.doi:10.1186/s12885-018-4160-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/29499663
      OpenUrlPubMed
    36. ↵
      1. Hopwood J,
      2. Walker N,
      3. McDonagh L, et al
      . Internet-Based interventions aimed at supporting family caregivers of people with dementia: systematic review. J Med Internet Res 2018;20:e216.doi:10.2196/jmir.9548pmid:http://www.ncbi.nlm.nih.gov/pubmed/29895512
      OpenUrlPubMed
    37. ↵
      1. Sherifali D,
      2. Ali MU,
      3. Ploeg J, et al
      . Impact of Internet-based interventions on caregiver mental health: systematic review and meta-analysis. J Med Internet Res 2018;20:e10668.doi:10.2196/10668pmid:http://www.ncbi.nlm.nih.gov/pubmed/29970358
      OpenUrlPubMed
    38. ↵
      1. Ploeg J,
      2. Markle-Reid M,
      3. Valaitis R, et al
      . Web-Based interventions to improve mental health, general caregiving outcomes, and general health for informal caregivers of adults with chronic conditions living in the community: rapid evidence review. J Med Internet Res 2017;19:e263.doi:10.2196/jmir.7564pmid:http://www.ncbi.nlm.nih.gov/pubmed/28754652
      OpenUrlPubMed
    39. ↵
      1. Zhao Y,
      2. Feng H,
      3. Hu M, et al
      . Web-Based interventions to improve mental health in home caregivers of people with dementia: meta-analysis. J Med Internet Res 2019;21:e13415.doi:10.2196/13415pmid:http://www.ncbi.nlm.nih.gov/pubmed/31066680
      OpenUrlPubMed
    40. ↵
      1. Boots LMM,
      2. de Vugt ME,
      3. van Knippenberg RJM, et al
      . A systematic review of Internet-based supportive interventions for caregivers of patients with dementia. Int J Geriatr Psychiatry 2014;29:331–44.doi:10.1002/gps.4016pmid:http://www.ncbi.nlm.nih.gov/pubmed/23963684
      OpenUrlPubMed
    41. ↵
      1. Riper H,
      2. Andersson G,
      3. Christensen H, et al
      . Theme issue on e-mental health: a growing field in Internet research. J Med Internet Res 2010;12:e74.doi:10.2196/jmir.1713pmid:http://www.ncbi.nlm.nih.gov/pubmed/21169177
      OpenUrlCrossRefPubMed
    42. ↵
      1. Lal S
      . E-mental health: promising advancements in policy, research, and practice. Healthc Manage Forum 2019;32:56–62.doi:10.1177/0840470418818583pmid:http://www.ncbi.nlm.nih.gov/pubmed/30739487
      OpenUrlPubMed
    43. ↵
      1. Rebello TJ,
      2. Marques A,
      3. Gureje O, et al
      . Innovative strategies for closing the mental health treatment gap globally. Curr Opin Psychiatry 2014;27:308–14.doi:10.1097/YCO.0000000000000068pmid:http://www.ncbi.nlm.nih.gov/pubmed/24840160
      OpenUrlCrossRefPubMed
    44. ↵
      1. Naslund JA,
      2. Aschbrenner KA,
      3. Araya R, et al
      . Digital technology for treating and preventing mental disorders in low-income and middle-income countries: a narrative review of the literature. Lancet Psychiatry 2017;4:486–500.doi:10.1016/S2215-0366(17)30096-2pmid:http://www.ncbi.nlm.nih.gov/pubmed/28433615
      OpenUrlPubMed
    45. ↵
      1. Health Quality Ontario
      . Internet-Delivered cognitive behavioural therapy for major depression and anxiety disorders: a health technology assessment. Ont Health Technol Assess Ser 2019;19:1–199.pmid:http://www.ncbi.nlm.nih.gov/pubmed/30873251
      OpenUrlPubMed
    46. ↵
      1. Sin J,
      2. Norman I
      . Psychoeducational interventions for family members of people with schizophrenia: a mixed-method systematic review. J Clin Psychiatry 2013;74:e1145–62.doi:10.4088/JCP.12r08308pmid:http://www.ncbi.nlm.nih.gov/pubmed/24434103
      OpenUrlPubMed
    47. ↵
      1. Folker AP,
      2. Mathiasen K,
      3. Lauridsen SM, et al
      . Implementing internet-delivered cognitive behavior therapy for common mental health disorders: a comparative case study of implementation challenges perceived by therapists and managers in five European Internet services. Internet Interv 2018;11:60–70.doi:10.1016/j.invent.2018.02.001pmid:http://www.ncbi.nlm.nih.gov/pubmed/30135761
      OpenUrlPubMed
    48. ↵
      1. Gitlin LN,
      2. Marx K,
      3. Stanley IH, et al
      . Translating evidence-based dementia caregiving interventions into practice: State-of-the-science and next steps. Gerontologist 2015;55:210–26.doi:10.1093/geront/gnu123pmid:http://www.ncbi.nlm.nih.gov/pubmed/26035597
      OpenUrlCrossRefPubMed
    49. ↵
      1. Damschroder LJ,
      2. Aron DC,
      3. Keith RE, et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50.doi:10.1186/1748-5908-4-50pmid:http://www.ncbi.nlm.nih.gov/pubmed/19664226
      OpenUrlCrossRefPubMed
    50. ↵
      1. Loudon K,
      2. Treweek S,
      3. Sullivan F, et al
      . The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 2015;350:h2147.doi:10.1136/bmj.h2147pmid:http://www.ncbi.nlm.nih.gov/pubmed/25956159
      OpenUrlFREE Full Text
    51. ↵
      1. Rothwell PM
      . External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet 2005;365:82–93.doi:10.1016/S0140-6736(04)17670-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/15639683
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      1. Aves T,
      2. Allan KS,
      3. Lawson D, et al
      . The role of pragmatism in explaining heterogeneity in meta-analyses of randomised trials: a protocol for a cross-sectional methodological review. BMJ Open 2017;7:e017887.doi:10.1136/bmjopen-2017-017887pmid:http://www.ncbi.nlm.nih.gov/pubmed/28871028
      OpenUrlAbstract/FREE Full Text
    53. ↵
      1. Shamseer L,
      2. Moher D,
      3. Clarke M, et al
      . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647.doi:10.1136/bmj.g7647pmid:http://www.ncbi.nlm.nih.gov/pubmed/25555855
      OpenUrlCrossRefPubMed
    54. ↵
      1. Lizarondo L,
      2. Stern C,
      3. Carrier J, et al
      . Chapter 8: Mixed methods systematic reviews. In: Aromataris E, Munn Z, eds. Joanna Briggs Institute Reviewer’s Manual. The Joanna Briggs Institute, 2017. https://wiki.joannabriggs.org/display/MANUAL/Chapter+8%3A+Mixed+methods+systematic+reviews. (accessed 1 Jul 2019).
    55. ↵
      1. Costantino G,
      2. Montano N,
      3. Casazza G
      . When should we change our clinical practice based on the results of a clinical study? searching for evidence: PICOS and PubMed. Intern Emerg Med 2015;10:525–7.doi:10.1007/s11739-015-1225-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/25833085
      OpenUrlPubMed
    56. ↵
      1. Centre for Reviews and Dissemination
      . Systematic Reviews: CRD’s guidance for undertaking reviews in health care. 3rd ed. CRD, University of York, 2009. Available: www.york.ac.uk/inst/crd [Accessed 10 Jul 2019].
    57. ↵
      1. Institute for Health Metrics and Evaluation
      . GBD compare data visualization. Seattle, WA: : IHME, University of Washington, 2018. Available: http://www.healthdata.org/data-visualization/gbd-compare [Accessed 10 Jul 2019].
    58. ↵
      1. Sinha M
      . Portrait of caregivers, 2012, 2013. Available: https://www150.statcan.gc.ca/n1/pub/89-652-x/89-652-x2013001-eng.htm [Accessed 11 Jul 2019].
    59. ↵
      1. König H-H,
      2. Leicht H,
      3. Brettschneider C, et al
      . The costs of dementia from the societal perspective: is care provided in the community really cheaper than nursing home care? J Am Med Dir Assoc 2014;15:117–26.doi:10.1016/j.jamda.2013.10.003pmid:http://www.ncbi.nlm.nih.gov/pubmed/24321877
      OpenUrlPubMed
    60. ↵
      1. Bleijlevens MHC,
      2. Stolt M,
      3. Stephan A, et al
      . Changes in caregiver burden and health-related quality of life of informal caregivers of older people with dementia: evidence from the European RightTimePlaceCare prospective cohort study. J Adv Nurs 2015;71:1378–91.doi:10.1111/jan.12561pmid:http://www.ncbi.nlm.nih.gov/pubmed/25403434
      OpenUrlCrossRefPubMed
    61. ↵
      1. Gaugler JE,
      2. Mittelman MS,
      3. Hepburn K, et al
      . Clinically significant changes in burden and depression among dementia caregivers following nursing home admission. BMC Med 2010;8:85.doi:10.1186/1741-7015-8-85pmid:http://www.ncbi.nlm.nih.gov/pubmed/21167022
      OpenUrlPubMed
    62. ↵
      1. DuBenske LL,
      2. Wen K-Y,
      3. Gustafson DH, et al
      . Caregivers' differing needs across key experiences of the advanced cancer disease trajectory. Palliat Support Care 2008;6:265–72.doi:10.1017/S1478951508000400pmid:http://www.ncbi.nlm.nih.gov/pubmed/18662420
      OpenUrlPubMed
    63. ↵
      1. Donker T,
      2. Griffiths KM,
      3. Cuijpers P, et al
      . Psychoeducation for depression, anxiety and psychological distress: a meta-analysis. BMC Med 2009;7:79.doi:10.1186/1741-7015-7-79pmid:http://www.ncbi.nlm.nih.gov/pubmed/20015347
      OpenUrlCrossRefPubMed
    64. ↵
      1. Thomas J,
      2. O'Mara-Eves A,
      3. Brunton G
      . Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev 2014;3:67.doi:10.1186/2046-4053-3-67pmid:http://www.ncbi.nlm.nih.gov/pubmed/24950727
      OpenUrlPubMed
    65. ↵
      1. Mohr DC,
      2. Spring B,
      3. Freedland KE, et al
      . The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother Psychosom 2009;78:275–84.doi:10.1159/000228248pmid:http://www.ncbi.nlm.nih.gov/pubmed/19602916
      OpenUrlCrossRefPubMed
    66. ↵
      1. Fitzpatrick R,
      2. Davey C,
      3. Buxton MJ, et al
      . Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2:i-iv, 1-74.doi:10.3310/hta2140pmid:http://www.ncbi.nlm.nih.gov/pubmed/9812244
      OpenUrlPubMed
    67. ↵
      1. Radloff LS
      . The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas 1977;1:385–401.
      OpenUrlCrossRef
    68. ↵
      1. Zigmond AS,
      2. Snaith RP
      . The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:361–70.doi:10.1111/j.1600-0447.1983.tb09716.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/6880820
      OpenUrlCrossRefPubMedWeb of Science
    69. ↵
      1. Cohen S,
      2. Kamarck T,
      3. Mermelstein R
      . A global measure of perceived stress. J Health Soc Behav 1983;24:385–96.doi:10.2307/2136404pmid:http://www.ncbi.nlm.nih.gov/pubmed/6668417
      OpenUrlCrossRefPubMedWeb of Science
    70. ↵
      1. Vis C,
      2. Ruwaard J,
      3. Finch T, et al
      . Toward an objective assessment of implementation processes for innovations in health care: psychometric evaluation of the normalization measure development (NoMAD) questionnaire among mental health care professionals. J Med Internet Res 2019;21:e12376.doi:10.2196/12376pmid:http://www.ncbi.nlm.nih.gov/pubmed/30785402
      OpenUrlPubMed
    71. ↵
      1. Proctor E,
      2. Silmere H,
      3. Raghavan R, et al
      . Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38:65–76.doi:10.1007/s10488-010-0319-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/20957426
      OpenUrlCrossRefPubMed
    72. ↵
      1. Sajobi TT,
      2. Li G,
      3. Awosoga O, et al
      . A comparison of meta-analytic methods for synthesizing evidence from explanatory and pragmatic trials. Syst Rev 2018;7:19.doi:10.1186/s13643-017-0668-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29370830
      OpenUrlPubMed
    73. ↵
      1. McGowan J,
      2. Sampson M,
      3. Salzwedel DM, et al
      . PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol 2016;75:40–6.doi:10.1016/j.jclinepi.2016.01.021pmid:http://www.ncbi.nlm.nih.gov/pubmed/27005575
      OpenUrlCrossRefPubMed
    74. ↵
      1. Lambert SD,
      2. Duncan LR,
      3. Kapellas S, et al
      . A descriptive systematic review of physical activity interventions for caregivers: effects on caregivers' and care recipients' psychosocial outcomes, physical activity levels, and physical health. Ann Behav Med 2016;50:907–19.doi:10.1007/s12160-016-9819-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/27439530
      OpenUrlPubMed
    75. ↵
      1. Woodford J,
      2. Farrand P,
      3. Richards D, et al
      . Psychological treatments for common mental health problems experienced by informal carers of adults with chronic physical health conditions (protocol). Syst Rev 2013;2:9.doi:10.1186/2046-4053-2-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/23369319
      OpenUrlPubMed
    76. ↵
      1. Corry M,
      2. While A,
      3. Neenan K, et al
      . A systematic review of systematic reviews on interventions for caregivers of people with chronic conditions. J Adv Nurs 2015;71:718–34.doi:10.1111/jan.12523pmid:http://www.ncbi.nlm.nih.gov/pubmed/25223528
      OpenUrlPubMed
    77. ↵
      1. Vis C,
      2. Mol M,
      3. Kleiboer A, et al
      . Improving implementation of eMental health for mood disorders in routine practice: systematic review of barriers and facilitating factors. JMIR Ment Health 2018;5:e20.doi:10.2196/mental.9769pmid:http://www.ncbi.nlm.nih.gov/pubmed/29549072
      OpenUrlPubMed
    78. ↵
      1. Wozney L,
      2. McGrath PJ,
      3. Gehring ND, et al
      . eMental healthcare technologies for anxiety and depression in childhood and adolescence: systematic review of studies reporting implementation outcomes. JMIR Ment Health 2018;5:e48.doi:10.2196/mental.9655pmid:http://www.ncbi.nlm.nih.gov/pubmed/29945858
      OpenUrlPubMed
    79. ↵
      1. Farrand P,
      2. Matthews J,
      3. Dickens C, et al
      . Psychological interventions to improve psychological well-being in people with dementia or mild cognitive impairment: systematic review and meta-analysis protocol. BMJ Open 2016;6:e009713.doi:10.1136/bmjopen-2015-009713pmid:http://www.ncbi.nlm.nih.gov/pubmed/26817638
      OpenUrlAbstract/FREE Full Text
    80. ↵
      1. Bramer WM,
      2. Giustini D,
      3. de Jonge GB, et al
      . De-duplication of database search results for systematic reviews in endnote. J Med Libr Assoc 2016;104:240–3.doi:10.3163/1536-5050.104.3.014pmid:http://www.ncbi.nlm.nih.gov/pubmed/27366130
      OpenUrlCrossRefPubMed
    81. ↵
      1. Ouzzani M,
      2. Hammady H,
      3. Fedorowicz Z, et al
      . Rayyan-a web and mobile APP for systematic reviews. Syst Rev 2016;5:210.doi:10.1186/s13643-016-0384-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/27919275
      OpenUrlCrossRefPubMed
    82. ↵
      1. Higgins JP,
      2. Savovic J,
      3. Page M, et al
      . Revised Cochrane risk-of-bias tool for randomized trials (rob 2), 2019. Available: https://drive.google.com/file/d/1xBzD1BeRrXH2DHg58fnTNKinTkX5SLEt/view [Accessed 22 Jul 2019].
    83. ↵
      1. Higgins JPT,
      2. Altman DG,
      3. Gøtzsche PC, et al
      . The Cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.doi:10.1136/bmj.d5928pmid:http://www.ncbi.nlm.nih.gov/pubmed/22008217
      OpenUrlFREE Full Text
    84. ↵
      1. QSR International Pty Ltd
      . NVivo qualitative data analysis software 2014.
    85. ↵
      1. Hill LG,
      2. Cooper BR,
      3. Parker LA
      . Qualitative comparative analysis: a mixed-method tool for complex implementation questions. J Prim Prev 2019;40:69–87.doi:10.1007/s10935-019-00536-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/30671752
      OpenUrlPubMed
    86. ↵
      1. Sullivan GM,
      2. Feinn R
      . Using effect size-or why the P value is not enough. J Grad Med Educ 2012;4:279–82.doi:10.4300/JGME-D-12-00156.1pmid:http://www.ncbi.nlm.nih.gov/pubmed/23997866
      OpenUrlCrossRefPubMed
    87. ↵
      1. Davies EB,
      2. Morriss R,
      3. Glazebrook C
      . Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res 2014;16:e130.doi:10.2196/jmir.3142pmid:http://www.ncbi.nlm.nih.gov/pubmed/24836465
      OpenUrlCrossRefPubMed
    88. ↵
      1. van Beugen S,
      2. Ferwerda M,
      3. Hoeve D, et al
      . Internet-Based cognitive behavioral therapy for patients with chronic somatic conditions: a meta-analytic review. J Med Internet Res 2014;16:e88.doi:10.2196/jmir.2777pmid:http://www.ncbi.nlm.nih.gov/pubmed/24675372
      OpenUrlCrossRefPubMed
    89. ↵
      1. Lorenzo-Luaces L,
      2. Johns E,
      3. Keefe JR
      . The generalizability of randomized controlled trials of self-guided Internet-based cognitive behavioral therapy for depressive symptoms: systematic review and meta-regression analysis. J Med Internet Res 2018;20:e10113.doi:10.2196/10113pmid:http://www.ncbi.nlm.nih.gov/pubmed/30413400
      OpenUrlPubMed
    90. ↵
      1. Linde K,
      2. Sigterman K,
      3. Kriston L, et al
      . Effectiveness of psychological treatments for depressive disorders in primary care: systematic review and meta-analysis. Ann Fam Med 2015;13:56–68.doi:10.1370/afm.1719pmid:http://www.ncbi.nlm.nih.gov/pubmed/25583894
      OpenUrlAbstract/FREE Full Text
    91. ↵
      1. Ragin C,
      2. Davey S
      . Fuzzy-Set/Qualitative Comparative Analysis 3.0. Irvine, Californai: : Department of Sociology, University of California, 2016. Available: http://www.socsci.uci.edu/~cragin/fsQCA/software.shtml [Accessed 12 Jul 2019].
    92. ↵
      1. Ragin CC
      . User’s guide to fuzzy-set/qualitative comparative analysis. Irvine, California, 2017. Available: www.fsqca.com [Accessed 12 Jul 2019].
    93. ↵
      1. Thomas J,
      2. Harden A
      . Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45.doi:10.1186/1471-2288-8-45pmid:http://www.ncbi.nlm.nih.gov/pubmed/18616818
      OpenUrlCrossRefPubMed
    94. ↵
      1. Popay J,
      2. Roberts H,
      3. Sowden A, et al
      . Guidance on the conduct of narrative synthesis in systematic reviews, 2006. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.178.3100&rep=rep1&type=pdf [Accessed 31 Jul 2019].
    95. ↵
      1. Lewis CC,
      2. Fischer S,
      3. Weiner BJ, et al
      . Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci 2015;10:155.doi:10.1186/s13012-015-0342-xpmid:http://www.ncbi.nlm.nih.gov/pubmed/26537706
      OpenUrlPubMed
    96. ↵
      1. Kerins C,
      2. McSharry J,
      3. Hayes C, et al
      . Barriers and facilitators to implementation of menu labelling interventions to support healthy food choices: a mixed methods systematic review protocol. Syst Rev 2018;7:88.doi:10.1186/s13643-018-0752-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29935530
      OpenUrlPubMed
    97. ↵
      1. Pollock A,
      2. Campbell P,
      3. Struthers C, et al
      . Stakeholder involvement in systematic reviews: a scoping review. Syst Rev 2018;7:208.doi:10.1186/s13643-018-0852-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/30474560
      OpenUrlPubMed
    98. ↵
      1. Boote J,
      2. Baird W,
      3. Sutton A
      . Public involvement in the systematic review process in health and social care: a narrative review of case examples. Health Policy 2011;102:105–16.doi:10.1016/j.healthpol.2011.05.002pmid:http://www.ncbi.nlm.nih.gov/pubmed/21641075
      OpenUrlCrossRefPubMed

    Footnotes

    • Contributors CC contributed to the design of the study and wrote the manuscript. JW, who acts as the reviews guarantor, conceived the study, contributed to the study design and critically revised the manuscript draft. LvE and RS critically revised the study design and manuscript. All authors approved of the final manuscript.

    • Funding This work was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie-Sklodowska Curie grant agreement no 814 072.

    • Disclaimer Funders were not involved in the creation, development or publication of this protocol, nor will they be involved in the conduct, analysis or reporting of the resulting systematic review.

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.