Introduction Studies on various types of digital-technology-based psychotherapies (DTPs) have indicated that they are effective for post-traumatic stress disorder (PTSD) symptom relief among adults. The intervention efficacy or effectiveness hierarchy, however, is still not clear. Therefore, we propose to conduct a network meta-analysis to assess the relative effectiveness of various types of DTPs. We aim to establish the differential effectiveness of these therapies in terms of symptom reduction and provide high-quality evidence for treating PTSD.
Methods and analyses We will search Embase, CINAHL, MEDLINE, HealthSTAR, the Cochrane Library, PsycINFO, PubMed, the Chinese Biomedical Literature Database, clinical trials (eg, ClinicalTrials.gov) and other academic platforms for relevant studies, mainly in English and Chinese (as we plan to conduct a trial on PTSD patients in Wuhan, China, based on the results of this network meta-analysis), from inception to October 2020. Randomised controlled trials (RCTs) and meta-analyses investigating the effectiveness of any DTPs for PTSD patients for any controlled condition will be included. The number of intervention sessions and the research duration are unlimited; the effects for different durations will be tested via sensitivity analysis. For this project, the primary measure of outcome will be PTSD symptoms at the end of treatment using raw scores for one widely used PTSD scale, PCL-5. Secondary outcome measures will include (1) dropout rate; (2) effectiveness at longest follow-up, but not more than 12 months and (3) patients’ functional recovery ratio (such as the return-to-work ratio or percentage of sick leave). Bayesian network meta-analysis will be conducted for all relative outcome measures. We will perform subgroup analysis and sensitivity analysis to see whether the results are influenced by study characteristics. The Grading of Recommendations, Assessments, Development, and Evaluation framework will be adopted to evaluate the quality of evidence contributing to network estimates of the primary outcome.
Ethics and dissemination The researchers of the primary trials already have had ethical approval for the data used in our study. We will present the results of this meta-analysis at academic conferences and publish them in peer-reviewed journals.
PROSPERO registration number CRD42020173253.
- mental health
- depression & mood disorders
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Network meta-analysis can compare various types of digital-technology-based psychotherapies for post-traumatic stress disorder among adults by integrating indirect and direct comparisons to establish the relative effectiveness of treatments.
Potential moderators for effectiveness can be found through subgroup and sensitivity analysis.
The project will use the Cochrane risk-of-bias tool to assess the degree of bias of the studies included, and adopt the Grading of Recommendations, Assessments, Development and Evaluation framework to evaluate the quality of evidence for network estimates of the primary outcome.
Trials and meta-analyses in which patients with comorbidities might increase risk of bias will be included even though this broadens the pooled sample; this impact will be examined through relevant subgroup analysis.
The lack of an evidence base for psychotherapies delivered through smartphone apps in general limits the probability of a randomised controlled trial having been conducted on them and thereby hinders the search for relevant studies with a high quality of evidence.
The outbreak of COVID-19 starting in China has spawned a wide range of reflections on psychological distress in the face of a national and global trauma event. It has been found that public health outbreaks are likely to induce post-traumatic stress disorder (PTSD) among a wide range of people, especially in the context of nationwide quarantine.1–3 According to the American Psychological Association’s Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5), PTSD usually features four symptom groups: (1) relived experiences of the trauma (such as nightmares and flashbacks), often invasive; (2) persistent hyper-reaction (such as insomnia, difficulty concentrating and increased startle reflex); (3) active avoidance of things related to the traumatic event and (4) negative cognition and behaviours (loss of social function, absence from work, etc) that were initiated by the trauma or that worsened after it.4–6
Although the debate on which treatments work the best is ongoing,7 8 the world’s five most prestigious professional organisations have all claimed that psychotherapies are significantly effective for PTSD symptom relief.9–13 Three of them clearly stated that psychotherapies, especially exposure psychotherapies, are much more effective than drug therapies.11–13 There are many factors or barriers that may interfere with a patient’s access to necessary and appropriate psychotherapies for PTSD, such as cost (travel expenses, childcare, professional charges for face-to-face psychotherapies and time investment in travel), limited physical mobility, lack of transportation, fear of being ostracised or stigmatised for having a mental illness and the lack of qualified psychotherapists.14–20 These barriers have prompted interest in new ways of delivering effective psychotherapies and in new (digital) technologies for doing so.21
Digital-technology-based psychotherapy (DTP) usually uses internet-based platforms such as web-based services or computer or smartphone apps to deliver psychotherapies to patients, which may potentially compensate for many of the above-mentioned disadvantages of traditional face-to-face psychotherapies.22 23 Various studies have also confirmed several disadvantages, such as a higher dropout rate, increased risk of leaking of patient information, a lower level of patient participation in the programme and requiring patients to acquire technical skills.24–26 During a public health outbreak, the use of digital technology is more appropriate than face-to-face therapies for PTSD patients, who are often resistant to face-to-face contact due to fear of imagined infection, even long after the trauma event.2 27 28 For other kinds of mental illnesses, such as anxiety disorder and depression, the effectiveness of DTP has been well studied and confirmed.6 However, research on DTP for PTSD has only begun to emerge in recent years. For example, a systematic review of DTP for PTSD in veterans found that in most instances, DTP was as effective as traditional face-to-face interventions in reducing PTSD symptoms.24 29
Three weaknesses were identified in the existing systematic reviews and meta-analyses on the effectiveness of DTP for PTSD.30–36 First, the trials included are not comprehensive. Most of the meta-analyses only focused on cognitive behavioural therapy (CBT) trials.7 37 Also, in all the meta-analyses, the criteria for trauma event types did not include public health outbreaks such as SARS and Ebola. Additionally, smartphone-app-based psychotherapies were excluded in most meta-analyses. 19 38In a 2016 study, a search of the smartphone app stores (in English) turned up 28 apps essentially targeted at PTSD symptom relief or general education on PTSD.39 We conducted a search of the Chinese Android app store and found 1 mindfulness app targeting PTSD symptom relief and another 10 apps providing general mental health information that includes PTSD as a subcategory. PTSD Coach, an English-language app, has been used in 106 countries and downloaded more than 350 000 times (the most for any app targeted at PTSD symptom relief) as of March 201840; two trials evaluated its effectiveness and both found no significant effects in favour of intervention versus the control group.41 42 Indeed, like PTSD Coach, many mental health apps are not evidence based, and this would be considered as one limitation of this study.43
Second, the trials examined had very high heterogeneity or high risk of bias, so the quality of evidence is questionable.7 37 Third, most of meta-analyses focused on the effects of certain specific psychotherapies rather than exploring the comparative effectiveness of different DTPs.17 19 30–36 44 For instance, Olthuis et al’s meta-analysis confirmed that DTP was more effective when compared with effects on the waiting control group, and did not compare effectiveness among various types of DTPs.19 Moreover, the effectiveness of DTP for PTSD patients compared with traditional face-to-face psychotherapy has also been questioned. Several scholars concluded that the effectiveness of DTP cannot be confirmed due to the lack of sufficient comparative effectiveness evidence with face-to-face therapies for PTSD.34 45 Furthermore, not only has the effect-size classification for various types of DTP for PTSD been confirmed, but we also found that the recommendation hierarchy for various forms of traditional face-to-face psychotherapy for PTSD is inconsistent among the guidelines for the treatment of PTSD in adults from the five prestigious international organisations. Three of these organisations strongly recommended DTP but generally classified all kinds of DTPs into a single treatment group.9–13
As a newly developed method, network meta-analysis, through appropriate research design, can easily fill the gaps identified above. Although the assumptions of network meta-analysis are similar to those of regular meta-analysis, the key additional assumptions are transitivity (no effect-modifying factors affecting indirect comparison) and coherence (direct and indirect effect estimates are similar).6 Therefore, network meta-analysis can integrate direct evidence from comparative studies of different interventions and indirect evidence from studies of individual interventions with common control conditions, and assess the effectiveness hierarchy among various interventions.46 This method can provide meaningful evidence for clinical practice guides by comparing multiple treatments at the same time.47 By also using the Grading of Recommendations, Assessments, Development and Evaluation (GRADE) framework to rate the quality of evidence synthesised through network meta-analysis, the aim of this research is to provide high-quality clinical guidance on DTP for PTSD in adults.48
Research on traditional PTSD psychotherapy through network meta-analysis is very limited. For example, a network meta-analysis in 2019 compared the effectiveness of different psychotherapies for PTSD.49 However, the study focused on young people, not adult patients, and the psychotherapies were not based on digital technology. Network meta-analysis of traditional psychotherapies for PTSD in adults is also limited. There are no such studies published in Chinese50 and only a few articles on research conducted in China are published in English.6 38 51–53 One study concentrated on traditional face-to-face psychotherapy, with very outdated data extraction (January 2011);6 another article compared the effectiveness of different traditional psychotherapies and conducted a subgroup analysis between patients with clinical diagnosis and those without.51 Network meta-analysis research on digital-technology-based PTSD psychotherapy for adults is even more limited. Moreover, these studies also present inconsistencies with the guidelines mentioned above. For example, a network meta-analysis of DTPs indicated that the effectiveness of various psychotherapies, such as CBT, comfort counselling and eye movement desensitisation and reprocessing, does not significantly differ between them, and most of the trials included had a low quality of evidence.52 Two research protocols of network meta-analysis published in 2018 advocated for examination of the comparative effectiveness of different DTPs for PTSD in adults.38 48
Therefore, we endeavour to conduct a network meta-analysis of studies on DTPs for PTSD in adults, specifically studies that incorporate trials in a comprehensive manner and with special consideration for quality of evidence, in order to better compare relative effectiveness for different DTPs (including an effectiveness comparison with traditional face-to-face psychotherapies) and establish the differential effectiveness of these therapies for symptom reduction.
This network meta-analysis will be conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-P checklist.54
LH and YG will search Embase, CINAHL, MEDLINE, HealthSTAR, the Cochrane Library, PsycINFO, PubMed, the Chinese Biomedical Literature Database, clinical trials (eg, ClinicalTrials.gov) and other academic platforms for studies on various DTPs for PTSD in adults, mainly in English and Chinese, using the keywords and phrases detailed in table 1, from inception to October 2020. The studies included will be randomised controlled trials (RCTs) and systematic meta-analyses on DTPs for PTSD (some meta-analyses may have included RCTs we did not find otherwise). An experienced medical librarian will be consulted to improve the search strategy for each database, and any differences will be resolved through discussion; in case of disagreement, we will consult another expert.
The patients recruited in an individual RCT or in RCTs in meta-analyses are adults diagnosed with primary or secondary PTSD (according to DSM-III, IV and 5, the International Classification of Diseases and other similar standards).
Trauma events will include all types, with special attention to public health outbreaks.
The study of secondary PTSD must focus on the treatment of PTSD.
If the PTSD patients recruited in the RCT also suffer from other comorbidities, such as physical disease, they will be included in the database, and these groups of patients will be tested via sensitivity analysis.
DTP will include various technologies (eg, web-based services, PC and smartphone apps), but there must be elements of interaction between program and patients.
The selection of various types of psychotherapies is mainly based on the comprehensive analysis of PTSD therapy guidelines of the five world’s most prestigious professional societies and organisations (see figure 1), along with types of control group.
The research duration is unlimited, but the effects for different durations will be tested by subgroup analysis; the number of DTP sessions is also unlimited.
The RCT has an intervention group or control group of fewer than 10 participants (of the five organisations, National Institute for Health and Care Excellence and Department of Veterans Affairs and Department of Defense guidelines exclude RCTs with fewer than 10 participants;11 13 we think adopting this exclusion criterion is an appropriate way to increase quality of evidence).
It is a pilot study, feasibility study or crossover trial.
The recruitment criteria are for severe PTSD, which includes excessively high intent to commit suicide, high dissociative disorder, severe mania and psychosis.
If a small number of participants in an RCT meet the above exclusion criteria, we will try our best to abstract and exclude the data of those independent participants and include the trial; if we are not able to exclude these individual data but the number of these participants does not exceed 20% of the total enrolled, the trial will still be included.
Data extraction and bias analysis
RCT and meta-analysis data extraction: In order to ensure the reliability of independent data extraction, a data extraction Excel form will be designed and the following calibration exercises will be conducted among project researchers (LH, JT and YP). The data extraction form will include study characteristics (first author, year of publication, source of funding and so on) and patient and trial characteristics such as patient demographic information (age in years, gender, occupation, absence/presence of disability, type of trauma and absence/presence of related comorbidity), sample size and intervention-group and control-group characteristics. Finally, type of trauma will be regarded as a dichotomous nominal variable between health-related and non-health-related type.
Individual patient data provided by RCT and meta-analysis authors: If the complete data cannot be downloaded from the database, the authors of the selected trials or studies will be contacted and asked to share their individual patient data (including baseline measurement and other information); the corresponding author will be contacted first, and if not available, the other authors will be contacted in turn; if there is no response from the authors within 2 weeks, a second email will be sent; if there is no response within 1 month from any of the authors contacted, the RCT or meta-analysis will only be included in the analysis at the aggregate data level. Two researchers (LH and YG) will separately test the consistency of the individual patient data with the data summary published in the article. The validity of this project would be compromised if the trials with individual patient data were systematically and statistically different from those without. Therefore, we will divide data into two groups for analysis according to source origin, and compare the differences between the two groups; if there are statistical differences, we will include this result in our later assessment of quality of evidence.
Primary outcome measure
In this project, PTSD symptoms at the end of treatment using raw scores for one of the most widely used PTSD scales are extracted as the primary outcome measure. The scale we chose is the PTSD Checklist (PCL-5). PCL-5 contains 20 items designed to measure four PTSD symptom clusters according to DSM-5’s diagnostic criteria, based on answers provided by the patients; the symptom indicators are numerically coded, generating a total symptom severity score of between 0 and 80.55 If the outcome is a dichotomy variable, the authors of the RCT study or meta-analysis will be contacted and asked to provide relevant primary raw scores; if there is no response, the study will not be considered.
Secondary outcome measures
Secondary outcome measures include (1) dropout rate—the rate of patients who discontinued the trial for any reason at any time before the end of trial; (2) effectiveness at the longest follow-up period, but not more than 12 months and (3) patients’ functional recovery ratio (such as the return-to-work ratio or percentage of sick leave).
Risk of bias
The project will use the Cochrane risk-of-bias tool, V.2.0, to assess the degree of bias of the study (as being at unclear risk of bias, low risk of bias or high risk of bias) by assessing random sequence generation, allocation concealment, blinding of participants, blinding of personnel, blinding of outcome assessment, incomplete outcome data and selective outcome reporting.56
Data synthesis and analysis
Bayesian network meta-analysis will be conducted for all outcome measures.57 For the primary outcome measure, the project will calculate the weighted mean difference (WMD) and the related 95% CI. For the secondary outcome measure, if the tools for the same outcome measure are the same, we will also use the WMD and the related 95% CI;57 58 for studies using different tools for the same outcome measure, the project will convert all outcomes into a common tool according to Thorlund et al’s recommendations, and calculate the standard mean difference and the related 95% CI.58 If p value, t value, CI, range or SE are reported in the trials and meta-analysis, the project will use the method recommended by the Cochrane manual to estimate the missing SD.38
The project will use the DerSimonian-Laird random effects model to conduct standard pairwise meta-analyses (for at least two studies) for all outcomes.59 The Q-statistic and I2 will be used to evaluate the statistical heterogeneity. Each direct comparison will report study and patient characteristics, risk of bias and aggregate estimates of related outcomes.
This project will settle inconsistency by comparing direct evidence with indirect evidence of differential effectiveness of various treatments, and use the Wald test to test any statistical difference between direct and indirect estimates.60 The project will report the probability of each DTP effectiveness level. After using a rankogram to show rank probability, the surface under the cumulative ranking curve (SUCRA) value will be used to explain the comparative effectiveness of the DTP (a value of 100 is the best and 0 the worst). The software package R, V.3.4.3, will be applied for statistical analysis.
Quality of evidence assessment
The project will use GRADE to rate the quality of both direct and indirect evidence and will classify the evidence as ‘high’, ‘moderate’, ‘low’ or ‘very low’. The starting point for RCT quality of evidence is very high, yet could be downgraded due to risk of bias, imprecision, inconsistency, indirectness and publication bias according to GRADE.
Subgroup and sensitivity analysis
This project will adopt subgroup and sensitivity analysis to test seven hypotheses: (1) trials with high risk of bias, compared with those with low risk, will show a greater effect size; (2) occupational groups such as medical staff, military or police will show a smaller effect size than the civilian samples for the same DTP; (3) the longer the follow-up period, the smaller the DTP effectiveness; (4) the effectiveness of DTPs with the participation of therapists is better than that of those without therapists; (5) different trauma events may trigger different levels of symptom severity (and duration), for instance, public health emergencies may have a stronger influence than one-off events such as earthquakes; (6) the longer the duration of the treatment period, the greater the DTP’s effectiveness and (7) studies on patients with comorbidities may contain a high risk of bias compared with those on patients without.
Patient and public involvement
No patients or members of the public are involved.
We officially commenced data extraction in early September 2020 and finished it at the end of the month. We started our analysis from the beginning of October and expect it to complete within 5 months.
Ethics and dissemination
No ethics approval is needed in this protocol study. The network meta-analysis results of this project will be disseminated to organisations supporting PTSD patients and hospitals with psychiatry or psychology departments.
We will present the results of this meta-analysis at academic conferences and publish them in peer-reviewed journals. Authors who make essential contributions to the generation of the final report will be granted with authorship. Moreover, we will disseminate results to health service receivers. The results will be implemented and reported according to the Consolidated Standards of Reporting Trials statement.
Contributors LH conceived the project design and drafted the article. YG and JT assisted with design and revision. LH and JT will conduct most of the data abstraction and the risk-of-bias assessment. JT, YG and YP participated in the design of data synthesis and analysis. HP, WD, XD, XH, YG, and YP will conduct the statistical analysis. All authors have agreed to publish this protocol.
Funding This research was funded by the Youth Project of Humanities and Social Sciences (project no. 20XJC840001) of the MOE (Ministry of Education) in China.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data sharing not applicable as no datasets generated and/or analysed for this study.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.