Effectiveness of digital technology interventions to reduce loneliness in adults: a protocol for a systematic review and meta-analysis

Introduction Loneliness is an emerging public health problem that is associated with social, emotional, mental and physical health issues. The application of digital technology (DT) interventions to reduce loneliness has significantly increased in the recent years. The effectiveness of DT interventions needs to be assessed systematically. Methods and analysis Aim: To undertake a systematic review and meta-analysis on the effectiveness of DT interventions to reduce loneliness among adults. Design: Systematic review and meta-analysis. Data sources: PubMed, Medline, CINAHL, EMBASE and Web of Science. Publication period: 1 January 2010 to 31 July 2019. Inclusion criteria: Primary studies involving the application of DT interventions to reduce loneliness, involving adult participants (aged ≥18 years), follow-up period ≥3 months and published in the English language. Synthesis and meta-analysis: A narrative summary of the characteristics of included studies, findings by the type of DT intervention, and the age, gender and ethnicity of participants. A meta-analysis by the study design and duration of follow-up and determination of random effects size using the RevMan V.5 software. Quality of evidence and bias: Quality of evidence assessed the RoB V.2.0 (revised tool for Risk of Bias in randomized trials) and ROBINS-I (Risk Of Bias in Non-randomized Studies—of Interventions) tools for randomised control trials and non-randomised studies, respectively. Heterogeneity between studies will be determined by the I2 and Cochran’s Q statistics and publication bias checked with funnel plots and the Egger’s test. Ethics and dissemination Ethics approval was not required for this protocol. The findings will be disseminated through journal articles and conference presentations. PROSPERO registration number CRD42019131524

The presentation of the plans, especially in the introduction, suffer from some shortcomings, which can easily be edited out: The section Definitions in the INTRODUCTION is highly redundant. The second sentence repeats the first 8lines 24-27), but is not quite as precise as that: the first specifies that desired > real, while the second would also include desired < real, which does not make much sense, but that is not the point. At least half of the fourth sentence (lines 29-33) repeats the first two. And the last two sentences are redundant again as they both state that the health outcomes differ (lines 35-37). Same section defines social isolation "objective feeling" (line 29), which I do not think is a reasonable expression. Line 48, p. 5: an expression like "statistically significantly associated" in this context is unnecessary because are associated only if they are significantly associated. Badly constructed phrases: p. 6, line 6 ("mental health such as depression", maybe insert "condition"); p. 6, line 11 ("early/premature mortality and all-cause mortality"); Logically, any inclusion criterion can be formulated as an exclusion criterion. Providing each of them in both formulations (p. 8) is unnecessary.
You may also want to refer to previous systematic reviews on the topic, among others Chen & Schulz 2016 (JMIR)

GENERAL COMMENTS
This protocol of systematic review and meta-analysis addresses an interesting topic. The article is well written and the method is systematic and well defined. In the manuscript the authors frame their question very well using the PICO framework.
However, I have two main comments: If the evidence is derived from observational studies, a conclusion on causality cannot be provided. In addition, when different types of designs are included, the heterogeneity may be high, which would limit, according to the protocol of the authors, the performance of a meta-analysis. The authors state that they will pool data from studies that will be sufficiently homogeneous. I wonder how the studies are considered sufficiently homogeneous (based on what variables? effect size, population, type of design…). It could be a potential area of bias during the review, so this aspect needs clarification.
Minor comments: Do the authors consider that the discussion section is necessary? Why will the authors not contact the original researchers when some of the data are not available in their published articles?
Doing that could avoid excluding studies with missing values from the meta-analysis.
Regarding the timeframe of post-intervention, will it only include immediate post-intervention measures? What happens if more than one post-intervention outcome measures have been collected? Please, merge these following sentences: "If the heterogeneity was substantial (I2≥50%). We would not run the meta-analysis and therefore we will report only to narrative synthesis" (page 12, line 14 and 15). Regarding to this statement, I have two questions: 1) If heterogeneity is <50, although moderate, will the authors explain that heterogeneity? 2) Have the authors considered, when the heterogeneity is substantial (I2≥50%), run the meta-analysis and try to explain the heterogeneity through random-effect meta-regression?

Location of changes made in the manuscript
Reviewer: 1 [Peter J Schulz] the introduction, suffer from some shortcomings The section Definitions in the INTRODUCTION is highly redundant. The second sentence repeats the first 8lines 24-27), but is not quite as precise as that: the first specifies that desired > real, while the second would also include desired < real, which does not make much sense, but that is not the point. At least half of the fourth sentence (lines 29-33) repeats the first two. And the last two sentences are redundant again as they both state that the health outcomes differ (lines 35-37).
We thank to the reviewer for pointing out this issue. We have revised the Definitions subsection in the introduction. We have removed the duplication, reduced the text and rewritten the sentences.
This sub-section now reads as: In the literature, loneliness and social isolation are often reported together but they differ from each other [1] Loneliness is defined as a subjective feeling that arises when an individual perceives a descrepancy between the actual and the desired social relationships. Page 3, lines 8-14 Same section defines social isolation "objective feeling" (line 29), which I do not think is a reasonable expression.
We have taken this definition from the literature. There are many studies and reports that define social isolation as an objective feeling for example see references 1, 2, 3.

Page 3, lines 11-12
Line 48, p. 5: an expression like "statistically significantly associated" in this context is unnecessary because are associated only if they are significantly associated.
We have revised this sentence by deleting 'statistically significantly' from the sentence. It now reads as: In older adults, loneliness is associated with female gender,…..

Page 3, lines 23-24
Badly constructed phrases: p. 6, line 6 ("mental health such as depression", maybe insert "condition"); We have added the word 'conditions' in this sentence. It now reads as: Loneliness is associated with mental health conditions such as depression, low self-esteem, anxiety, perceived stress,…..

Page 4, lines 3-4
p. 6, line 11 ("early/premature mortality and all-cause mortality"); Early mortality and premature mortality are synonymously used in the literature. Therefore we wrote them together. However, we take the term 'early' out and report only 'premature' in this sentence. It reads as follows: … but also increases risk of premature mortality and allcause mortality…… Page 4, lines 6-8 Logically, any inclusion criterion can be formulated as an exclusion criterion. Providing each of them in both formulations (p. 8) is unnecessary.
Many thanks for raising this issue. We have removed the exclusion criteria, from the text and Table 1 Again, many thanks for suggesting a systematic review by Chen & Schulz 2016, which we have read and found useful but not very relevant at the protocol stage of our study.

No action
Reviewer: 2 (Walter Ricciardi) Despite this could appears more exhaustive, reporting both the inclusion and the correspondent exclusion criteria seems quite redundant. The authors could avoid to report exclusion criteria that are the exact complement of inclusion criteria.
Many thanks for raising this issue. We have removed the exclusion criteria, from the text and Table 1. Page 7, lines 27-28 and Table 1 on pages 7-8 The authors declared that in presence of heterogeneity they will not run the meta-analysis. I would suggest, in addition, to search for possible sources of heterogeneity and then, where possible, to perform stratified meta-analyses.
We agree with the reviewer's suggestion and we have included consideration of undertaking stratified metaanalyses in the case of substantial heterogeneity (I 2 >50%).

Page 13, lines 29-32
We have incorporated this suggestion in the revised text as follows: In the case of substantial heterogeneity (I 2 >50%),[49] we will consider running stratified meta-analyses and randomeffect meta-regression to ascertain whether effects size was associated with the methodological or clinical characteristics of the studies included in the metaanalysis.
[62] Furthermore, the authors should specify the statistical software/s that they intend to use for the metaanalysis.
This is a critical issue raised by this reviewer. We will use the RevMan 5 software for performing the meta-analysis. We add the following statement: We will run meta-analyses using the Review Manager (RevMan), version 5.3.5, software [59] Page 2 -abstract, lines 26-27) and page 13, lines 19-20 . If the evidence is derived from observational studies, a conclusion on causality cannot be provided.
Observational studies include case control and cohort studies (before and after studies). We will include these types of studies that use a digital technology intervention. However, we will not include cross sectional and evaluation studies. Which we have removed from the inclusion criteria.
Page 6-7 (Table 1, parameter: study types/designs) In addition, when different types of designs are included, the heterogeneity may be high, which would limit, according to the protocol of the authors, the performance of a meta-analysis.
We will consider undertaking stratified meta-analyses in the case of high heterogeneity (I 2 >50%).

Page 13, lines 29-32
The authors state that they will pool data from studies that will be sufficiently homogeneous. I wonder how the studies are considered sufficiently homogeneous (based on what variables? effect size, population, type of design…). It could be a potential area of bias during the review, so this aspect needs clarification.
We have revised this statement to avoid any confusion and facilitating meta-analysis. We have revised this sentence as follows: We will extract and analyse data at the study level.

Page 12, lines 22-23
Minor comments: Do the authors consider that the discussion section is necessary? Discussion section is not essential but we included it because some published protocol s had this section included. However, we have removed the discussion section in our revised manuscript as suggested by the reviewers.

Page 14 (lines-13-23)
Why will the authors not contact the original researchers when some of the data are not available in their published articles? Doing that could avoid excluding studies with missing values from the meta-analysis.
We had included this statement because we are not sure whether writing to authors of published articles for data would be successful, especially because a lot of academics and researchers change their jobs and employer institutions. However, we have changed our mind and we will attempt to write to the authors on included studies for additional information regarding data published in their articles. We have revised our statement as follows: If data reported in selected studies were found difficult to extract or incomplete, we will not attempt to contact the original researchers with regard to any queries regarding the data reported in their published articles.
Page 12, lines 6-8 Regarding the timeframe of postintervention, will it only include immediate post-intervention measures? What happens if more than one post-intervention outcome measures have been collected?
Thanks for raising this point, which we have clarified this issue by adding the following text at various points in the manuscript as follows: Timing and effect measures: We will extract information on measurement of loneliness at the baseline (before the introduction of the intervention) and every follow-up measurement for both the intervention groups and control group depending on the design of studies included in our systematic review. In the case of more than one follow-up measurement, we will run a series of meta-analyses as explained in the meta-analysis section below.
Data extraction process section: We will extract data reported for all measurements including the baseline measurement and all subsequent follow-up measurements.
In addition, we have included an additional column in the data extraction template (Table 3) Table 3 Please, merge these following sentences: "If the heterogeneity was substantial (I2≥50%). We would not run the meta-analysis and therefore we will report only to narrative synthesis" (page 12, line 14 and 15).
Thanks. We have merged this sentence by removing a full stop before the word 'we would'.

Page 13 , lines 29-30
Regarding to this statement, I have two questions: 1) If heterogeneity is <50, although moderate, will the authors explain that heterogeneity?
2) Have the authors considered, when the heterogeneity is substantial (I2≥50%), run the meta-analysis and try to explain the heterogeneity through random-effect metaregression?
1. Yes, we will report the observed heterogeneity levels in the meta-analyses.
2. Thanks for this suggestion. As stated above, we will run stratified meta-analyses and meta-regression to check the influence of study characteristics on the effect sizes. In this regard we have added the following statement: In the case of substantial heterogeneity (I 2 >50%),[49] we will consider running stratified meta-analyses and randomeffect meta-regression to ascertain whether effects size was associated with the methodological or clinical characteristics of the studies included in the metaanalysis. [62]