You are here

  • Home
  • Archive
  • Volume 12, Issue 4
  • Opt-out rates and reasons for non-participation in a single-arm feasibility trial (ENGAGE) of a guided internet-administered CBT-based intervention for parents of children treated for cancer: a nested cross-sectional survey

Article Text

Article menu

Download PDFPDF

Mental health
Original research
Opt-out rates and reasons for non-participation in a single-arm feasibility trial (ENGAGE) of a guided internet-administered CBT-based intervention for parents of children treated for cancer: a nested cross-sectional survey
  1. Josefin Hagström1,
  2. Joanne Woodford1,
  3. Agnes von Essen1,
  4. Päivi Lähteenmäki2,3,
  5. Louise von Essen1
  1. 1Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
  2. 2Department of Paediatrics and Adolescent Medicine, TYKS Turku University Hospital, Turku, Finland
  3. 3Pediatric Oncology and Pediatric Surgery, Department of Women’s and Children’s Health, Karolinska Institute, Stockholm, Sweden
  1. Correspondence to Professor Louise von Essen; louise-von.essen{at}kbh.uu.se

Abstract

Objectives Difficulties with recruitment into clinical trials are common. An opt-out recruitment strategy, whereby potential participants can decline further contact about a study (opt-out), and non-responders are contacted, may facilitate participation. Primary objectives examined opt-out and consent rates, mode and time point of opt-out, and sociodemographic characteristics of those who opted out versus those who chose to participate in a single-arm feasibility trial (ENGAGE) of a guided, internet-administered, cognitive–behavioural therapy-based intervention for parents of children treated for cancer. Secondary objectives examined reasons for non-participation.

Design A cross-sectional survey nested within the ENGAGE feasibility trial.

Setting The intervention was delivered from Uppsala University, with parents located throughout Sweden.

Participants Potential participants were recruited 3 months–5 years following their child ending treatment for cancer and were identified via their personal identification number (via the Swedish Childhood Cancer Registry and Swedish Tax Agency) and invited via postal invitation packs and could opt out via post, online, telephone or email. Those who did not opt out or consent, within 4 weeks, received up to five telephone calls and/or one postal reminder.

Results Of 509 invited, 164 (32.2%) opted out, 78 (47.6%) via post, 53 (32.3%) via telephone, 24 (14.6%) online, and 6 (3.7%) via email, 88 (53.7%) opted out after at least one telephone call and/or postal reminder. There was a trend for parents with lower educational levels to opt out. No need of psychological support, lack of time, and no interest in internet-administered self-help were frequently reported reasons for non-participation.

Conclusions Results emphasise the importance of using different opt-out modes and suggest future research should consider how to widen study participation for parents with lower education levels. Self-identifying a need for psychological support and the acceptability of internet-administered self-help are important factors for participation and should be considered in future research to increase recruitment.

Trial registration number ISRCTN57233429.

  • depression & mood disorders
  • statistics & research methods
  • qualitative research
  • paediatric oncology

Data availability statement

Data are available in a public, open access repository. The research data supporting the findings of this study are stored in Zenodo repository with the identifier https://doi.org/10.5281/zenodo.6136920. Access to the data stored in Zenodo is available upon written request from the corresponding author (LvE).

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/bmjopen-2021-056758

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Parents invited are a nationally representative sample of parents of children treated for cancer.

    • The use of different modes of opt-out (eg, post, online, telephone, or email) has rarely been investigated.

    • The use of closed and open questions allows for a broad understanding of reasons for non-participation which can inform strategies to increase recruitment rates of future clinical trials.

    • The study was not designed to compare recruitment rates across different strategies and conclusions concerning the effectiveness of opt-out versus opt-in strategies cannot be drawn.

    • We did not collect sociodemographic variables of non-responders and therefore it was not possible to examine whether the sample of parents who opted out, or chose to participate, is representative of the overall population in relation to key sociodemographic variables.

    Introduction

    Background

    Difficulties with recruitment into clinical trials are common,1 significantly contributing to research waste.2 Insufficient recruitment rates have been cited as the leading cause of clinical trials being terminated3 and reviews have found only 56% of publicly funded clinical trials reach their target sample size.4 Difficulties with recruitment into clinical trials may result in delays to the generation of robust evidence to inform decisions to adopt interventions into routine healthcare.5 While there is an increased focus on identifying effective clinical trial recruitment strategies,6 current evidence is limited7 with little high-quality evidence to inform decisions concerning the use of different recruitment strategies in clinical trials.6

    The ENGAGE feasibility trial8 9 is a single-arm feasibility trial of a guided, internet-administered low-intensity cognitive-behavioural therapy (LICBT)-based intervention, designed to target common mental health difficulties (eg, depression and generalised anxiety disorder) in parents of children treated for cancer.10 The ENGAGE feasibility trial examines clinical (eg, intervention drop-out, adherence, and acceptability) and procedural (eg, recruitment and retention) uncertainties. Examining recruitment uncertainties is important given that recruitment of participants into mental health intervention trials is particularly challenging.11 Barriers may include a reluctance to seek help,12 stigma,13 14 and low perceived need for support.15 Recruitment into clinical trials of e-mental health (e-MH) interventions (eg, internet-administered CBT) has been identified as particularly challenging,16 with barriers including lack of trust, poor digital health literacy, low confidence in using technology, low acceptability and preferences for face-to-face interventions.17–20

    Given challenges with recruitment into e-MH intervention trials, there is a need to examine the effectiveness of different recruitment strategies to increase recruitment rates. One strategy associated with increased recruitment rates and less biased samples is the opt-out strategy.21 Potential participants invited into a study are provided with a method of actively declining contact with the research team (opting out) if they do not wish to participate in or receive additional study information. Those who do not opt out are contacted by the research team, for example, via the telephone, to provide more study information. Contacting non-responders may result in increased recruitment rates in comparison with opt-in strategies, whereby potential participants must take action to signal willingness to participate to the research team.1 7 11 22 An opt-out strategy may be particularly effective for e-MH intervention trials.16 Mental health difficulties such as depression include lack of motivation, concentration difficulties, lack of belief in deserving intervention and negative expectations concerning recovery, which have been found to be barriers to help-seeking.16 23 24 An opt-out strategy may facilitate participation, for example, potential participants experiencing low levels of motivation or problems with self-initiating behaviour25 are contacted by the research team and provided with an opportunity to receive more information. Indeed, opt-out strategies have been found to result in the recruitment of populations who are less healthy and more functionally impaired than opt-in strategies.21 While successfully used with adult and paediatric populations,26 to the best of our knowledge, an opt-out strategy has not been used with parents of children treated for cancer.

    Furthermore, literature is scarce concerning reasons for non-participation reported by parents of children treated for cancer. Wider literature suggests common reasons include lack of perceived personal benefit, time and energy, and preconceptions about research.27 28 While qualitative research with parents of children treated for cancer has been undertaken to inform the development of recruitment strategies for clinical trials,29 to the best of our knowledge, the population’s reasons for non-participation have not been investigated. Understanding reasons for non-participation may enable the identification of modifiable barriers to participation (eg, treatment preferences, burden of trial procedures)14 and inform the development of recruitment strategies to help overcome these barriers.14

    Aims and objectives

    Primary objectives examined: (1) opt-out and consent rates; (2) mode of opt-out; (3) time point of opt-out; and (4) sociodemographic characteristics of those who opted out versus those who chose to participate in the ENGAGE feasibility trial.8 9 Secondary objectives examined reasons for non-participation.

    Methods and analysis

    Study design

    A cross-sectional survey with closed and open questions examining opt-out rates and reasons for non-participation nested within the ENGAGE feasibility trial8 9 reported, where applicable, in line with the Checklist for Reporting of Survey Studies.30

    Participants

    Eligible participants were: (1) parents of a child diagnosed with cancer during childhood (0–18 years) who had completed cancer treatment 3 months–5 years previously; (2) living in Sweden; (3) able to read and write Swedish; (4) able to access email, the internet, and a mobile telephone and BankID; and (5) self-reporting a need for psychological support related to the child’s cancer disease and treatment and a wish to work with the intervention. Parents were excluded if they: (1) had a self-reported or psychologist-assessed (based on the Mini-International Neuropsychiatric Interview31) severe and enduring mental health difficulty; and/or misuse of alcohol, street drugs, and/or prescription medication; (2) were acutely suicidal; and/or (3) attended psychological treatment.

    Recruitment

    Potential participants were recruited using two strategies: (1) postal study invitation packs; and (2) online advertisements via cancer organisations and interest groups. In this publication, recruitment procedures and data associated with postal study invitation packs are reported.

    Personal identification numbers of children who completed cancer treatment 3 months–5 years previously were obtained from the Swedish Childhood Cancer Registry and linked to the names and addresses of their parent(s) via NAVET, a population registry held by the Swedish Tax Agency. Parents were invited in blocks of 100 until the target sample size (N=50) was reached. The first block invited was preselected with parents of children treated for cancer who ended treatment near to 5 years ago. The following four blocks were randomly selected. A de-identified list of all parents provided by NAVET was prepared by a research team member. To ensure allocation concealment, a member of the U-CARE web development team, external to the research team, used a computer-generated simple randomisation procedure to allocate parents into recruitment blocks. Postal study invitation packs were sent to parents’ home addresses and included: (1) a study invitation letter, signed by the principle investigator (LvE) and a parent research partner (PRP); (2) a study information sheet and link to a secure website, the U-CARE-portal (Portal, an in-house platform which hosted the study and is designed to deliver internet-administered psychological interventions and support the collection of research data32 33); (3) a paper reply slip to register interest in participation; (4) an opt-out form and reasons for non-participation survey; and (5) a freepost envelope. The ENGAGE feasibility trial also included a Study Within A Trial investigating the effect of personalised versus non-personalised study invitation letters on recruitment and retention,9 and results have been published separately.34

    Parents could access study information, in text and video format, and provide informed consent via the Portal. Parents who wished to opt out could do so by completing an opt-out form and reasons for non-participation survey via the Portal or by paper via the post (see the Outcomes section). Parents could also opt out by telephoning or emailing the research team.

    Procedure

    Opt-out and reminder telephone calls

    Parents could sign an online form via the Portal using BankID, a secure electronic citizen authentication system used in Sweden, and were presented with a short reasons for non-participation survey (see the Outcomes section). Alternatively, parents could complete a paper-based survey, provide their name and ink signature, and return via the post. Parents who opted out via telephone or email were asked to optionally provide a reason for non-participation.

    Parents who did not respond (eg, opt out or provide consent) within 4 weeks upon the first postal study invitation pack, received up to five telephone reminder calls over 4 weeks. Telephone numbers were identified using internet search engines. A maximum of five reminder calls were made over 4 weeks, intended to confirm receipt of the postal study invitation and provide an opportunity to ask questions, or opt out from the study and further contact. Voicemails were left if the voicemail inbox had a personal recording clearly stating the parent’s name. Communication between the parent and research team was documented in a recruitment case report form (CRF). If a telephone number could not be identified, a study invitation reminder letter and study invitation pack were resent. Personal details of parents who opted out were deleted from the research database.

    Informed consent, screening and baseline

    Parents who chose to participate provided informed consent via the Portal and were contacted by the research team to schedule an eligibility interview with a licensed psychologist over the telephone. All parents were required to complete an online Portal assessment at baseline, or telephone if preferred. Full study procedures can be found in the ENGAGE feasibility trial protocol.8

    Intervention

    The intervention is an internet-administered, guided, LICBT-based intervention (the EJDeR intervention (intErnetbaserad sJälvhjälp för förälDrar till barn som avslutat en behandling mot canceR)).10 Following phase I of the Medical Research Council’s complex interventions framework,35 development of EJDeR was informed by a systematic review,36 qualitative interview studies,37 38 a single-arm trial,39 participatory action research40 and a cross-sectional online survey.41 A PRP group of two mothers and two fathers of a child treated for cancer informed further refinements.10 EJDeR is delivered on the Portal, and designed to be worked with over 12 weeks and consists of four modules: (1) Introduction and Psychoeducation; (2) Behavioural Activation; (3) Worry Management and (4) Relapse Prevention.10 Parents are guided by e-therapists, including an initial assessment session via a secure inbuilt video conferencing system or telephone call, weekly written messages via the Portal, and a ‘booster’ session mid-way through the intervention, via video conferencing or telephone call.

    Outcomes

    An online and paper-based survey, consisting of 13 items, in Swedish. The survey comprises two subsections: (1) sociodemographic characteristics (four items) and (2) reasons for non-participation (nine items) . An open question at the end of the questionnaire offered an opportunity for parents to provide any reasons for non-participation.

    Sociodemographic characteristics

    Background and sociodemographic characteristics were collected from parents who opted out via the online or paper-based survey: (1) gender (male/female/other); (2) age (years); (3a) relationship status (yes/no); (3b) cohabitation status (yes/no) and (4) highest level of education (free-text). Parents opting out via the telephone or email were not asked to provide sociodemographic characteristics.

    Reasons for non-participation

    Parents were asked a closed, multiple-choice question regarding reasons for non-participation, reasons were informed by previous research,42 43 and parents could select all applicable reasons: (1) lack of time; (2) physical health; (3) not wanting to participate in research; (4) not liking to talk about personal problems; (5) too tired; (6) not interested in internet-administered self-help; (7) no need for psychological support; (8) child’s health condition and (9) child has recently passed away. The open question at the end of the reasons for non-participation questionnaire was: ‘If you do not want to participate in the project for reasons other than those mentioned above, please write these below’.

    Whereby the parent opted out via telephone or email, a CRF was completed by the research team. The CRF included the reason(s) for non-participation listed in the survey and a free-text box in response to the open question.

    Sample size

    No formal sample size calculation was made44 however recommendations for assessing feasibility outcomes were followed45 with an aim to recruit 50 parents. It was estimated approximately 600 parents would be invited to meet the sample size.9

    Progression criteria

    A priori progression criteria46 were set, these can be found in the published protocol for the ENGAGE feasibility trial.8 A criterion that ≥9% of the total population invited into the study via the Swedish Childhood Cancer Registry chose to participate was set.

    Data analysis

    Data analysis was performed in Excel and SPSS for Windows, V.27.0 (IBM Corporation). Recruitment flow via postal study invitation packs is illustrated using an adapted Consolidated Standards of Reporting Trials flow diagram.47 Descriptive statistics were used to describe numbers and proportions who opted out; opted out via each mode (postal/online/telephone/email) and at each time point (before/after telephone and/or postal reminder/s); and responded to a multiple-choice question regarding reasons for non-participation. Study sample sociodemographic characteristics are described using descriptive statistics. Χ2 test was used to examine differences regarding categorical variables, and t-tests were calculated for continuous variables.

    An inductive qualitative content analysis48 managed using Microsoft Excel and Word was adopted to analyse responses to the open question in the reasons for non-participation survey. Free-text responses were read by two authors (JH and AvE) to gain an overall understanding of content. Next, condensed meaning units were identified (eg, text sharing a common meaning) and assigned a code by JH and AvE separately. JH reviewed initial meaning units and codes to identify any differences in coding, with differences discussed by JH, AvE and a third author (JW) if required. Codes were sorted into categories and subcategories by JH and AvE separately. As categories were identified, some content was recognised as not relating to reasons for non-participation, but included important content not to be discarded, for example, gratitude for the research. Categories were subsequently classified into two themes: (1) reasons for non-participation and (2) content not related to reasons for non-participation. JH, AvE and JW met regularly to discuss categories and subcategories and ensure content was mutually exclusive. All responses were subsequently reread by JH to check agreement with identified categories and subcategories. JH and AvE jointly prepared brief descriptions for categories and subcategories. Trustworthiness was established using peer examination of a supervising researcher (JW), two coders (JH and AvE) working independently, and record keeping.49

    Other reasons for non-participation provided by parents via telephone or email to the research team and recorded on the recruitment CRF were also categorised by JH and AvE separately, with all categorisations discussed with JW to obtain consensus.

    Patient and public involvement

    The PRP group (two mothers and two fathers) provided feedback on the design and wording of the study invitation letter, included within the study invitation pack. PRPs were sent a study invitation letter draft via email and asked to comment on wording, content, format, and design, and whether to include a PRP’s name and signature on the letter alongside that of the principle investigator (LvE). Parents did not suggest any changes and considered the letter to be concise, validating and informative, and agreed the name and signature of a PRP should be included.9 Patient and public involvement may have been improved by engaging PRPs in drafting the opt-out survey, for example, informing the options available in the multiple-choice question regarding reasons for non-participation.

    Results

    Opt-out and consent rates

    Recruitment via postal study invitation packs into the ENGAGE feasibility trial took place between 3 July 2020 and 30 November 2020. Five hundred nine parents were identified and invited. A total of 32.2% (164 of 509) opted out, and 11.8% (60 of 509) provided consent, and 11.0% were eligible and enrolled in the trial, exceeding the a priori recruitment rate progression criteria of ≥9% invited via the Swedish Childhood Cancer Registry.8 46 Eleven per cent (56 of 509) were found to be eligible for the study.

    Three parents were initially registered as opting out via the telephone by a partner who had also received a study invitation pack. Subsequently, the opt-out protocol was adapted and opting out via a partner was no longer an option. Recruitment flow via postal study invitation packs can be seen in figure 1.

    Figure 1
    Figure 1

    Recruitment flow via postal study invitation packs.

    Mode of opting out

    The most frequent mode of opt-out was via the post (78 of 164, 47.6 %), followed by telephone (53 of 164, 32.3%), the Portal (24 of 164, 14.6%) and email (6 of 164, 3.7%). Three parents (3 of 164, 1.8%) opted out via their partner over the telephone. Two who initially opted out via telephone, subsequently opted out via the post, resulting in a total of n=80 opt-outs via the post.

    Time point of opting out

    The majority of opt-outs occurred after one or more telephone reminder calls and/or postal reminders (88 of 164, 53.7%). The mean number of days between a postal study invitation pack being sent and an opt-out was 30.0 days (SD=20.6, range=2–158).

    Sociodemographic characteristics of parents who opted out versus those who chose to participate

    Sociodemographic characteristics of parents (n=105) who opted out and provided sociodemographic data and parents (n=56) who consented and were screened for eligibility are shown in table 1. No differences between groups were found, though there was a trend for parents with lower educational levels to opt out, and parents with a tertiary education to choose to participate.

    View this table:
    Table 1

    Sociodemographic characteristics of parents who opted out and who consented and were screened for eligibility

    Reasons for non-participation

    Closed question

    A total of 83.5% (137 of 164) responded to the closed question about reasons for non-participation. The most frequently reported reason was not experiencing any need for psychological support (93 of 137, 67.9%) followed by not having enough time (53 of 137, 38.7%), and not interested in using internet-administered self-help (23 of 137, 16.8%). Responses are shown in table 2 and presented separately for female and male gender where data were available.

    View this table:
    Table 2

    Reasons for non-participation in response to the closed multiple-choice question

    Open question

    Thirty parents responded to the open question regarding reasons for non-participation in the paper-based or online survey. Twenty-one provided a response via telephone or email recorded in the CRF. Content analysis of responses within the theme Reasons for non-participation resulted in six categories: Timing, Emotional well-being, Support, Perceived ineligibility, Other commitments, and Reluctance to engage in internet-administered self-help. Responses within the theme Content not related to reasons for non-participation were categorised as: Not for me, but…. and When my child was ill. Themes, categories, subcategories and quotations are shown in table 3.

    View this table:
    Table 3

    Themes, categories, subcategories and quotations

    Reasons for non-participation

    The category Timing includes the subcategories: Earlier need and Not right now. Earlier need concerns the invitation coming ‘too late’, that is, after treatment completion, with parents positing hypothetical participation had they been approached earlier. Not right now concerns not currently interested in participation, however potential interest another time.

    The category Emotional well-being includes the subcategories: Coping strategies, Family well-being, Fatigue and No need. Coping strategies concerns use of coping strategies. Parents stated wanting to forget ‘what has been’ (eg, the cancer experience) and a reluctance to ‘go back and think about these things’ or to remain involved with sickness. Conversely, parents reported focusing on the positive, accepting their situation, holding a ‘positive’ attitude towards difficult situations and post-traumatic growth. Parents could describe distancing from the cancer experience in the past and present as well as a want to ‘focus on the positive’ in the future. Family well-being concerns the entire family, for example, as their child was in good health, psychological support for the parent was not considered needed. Fatigue concerns being too tired to participate and No need concerns parents feeling well and not experiencing a need for psychological support.

    The category Prior support includes the subcategories: Professional support and Social support. Professional support concerns receipt of current or past support from a health professional, for example, a psychologist or counsellor, whereas Social support concerns support and help by family, friends and pets.

    The category Perceived ineligibility concerns parents not considering themselves eligible for the study, for example, not fulfilling the inclusion criteria, for example, insufficient knowledge of Swedish, undergoing psychological treatment as well as other reasons such as being ‘too broken’ to participate.

    The category Other commitments concerns life commitments occupying parents’ time, reference was also made to too many invitations regarding participation in research and a wish to have free time.

    The category Reluctance to engage in an internet-administered intervention included responses whereby parents reported being uncomfortable with the internet or preferring face-to-face treatment.

    Content not related to reasons for non-participation

    The category Not for me, but… includes the subcategories: Gratitude and Need among others. Gratitude concerns the importance of the study and gratitude that it was carried out. Need among others concerns a hope that other parents would participate and receive help, and that there is a need for support for parents after their child’s cancer treatment.

    The category When my child was ill includes the subcategories: Available care, Cancer illness and Family life. Available care concerns the care received during and after the child’s illness and treatment. Parents mentioned there had been little care available for their well-being after their child had completed treatment. Cancer illness concerns details about the child’s illness and treatment, for example, the duration of the treatment or the child having been ill since birth. Family life concerns parents describing experiences during their child’s illness and treatment, for example, striving for normality or difficult family events such as separating from a partner.

    Discussion

    Summarising the main findings, the opt-out rate was 32.2%, the consent rate via postal study invitation packs was 11.8%, and 11.0% were eligible and enrolled in the trial. The most frequent mode of opt-out was by paper via the post (47.6%). The majority (53.7%) opted out of further contact with the research team after one or more reminders via telephone and/or post. Although not significant, there was a trend for parents with lower educational levels to opt out, and parents with a tertiary education choosing to participate. Common reasons for non-participation in response to the closed question were not experiencing any need for psychological support, not having enough time, and not having an interest in using internet-administered self-help. Content analysis of responses to the open question agreed with results from the closed question, with reasons for non-participation including emotional well-being, receipt of psychological support, and commitments limiting time to participate. Additional reasons were suggested, including study invitation timing and parents not perceiving themselves as eligible.

    The overall consent rate exceeded the recruitment target outlined in study progression criteria.8 46 Use of an opt-out strategy has demonstrated higher recruitment rates when compared with opt-in strategies.7 50 51 For example, randomised controlled trials (RCTs) comparing opt-out versus opt-in strategies have found recruitment rates of 40%–50% for opt-out vs 26%–38% for opt-in.21 52 However, studies using opt-out strategies are highly heterogeneous, with variations in recruitment rates, populations, designs, and interventions,50 and few RCTs have compared opt-out versus opt-in strategies,7 51 limiting conclusions regarding effectiveness. The ENGAGE feasibility trial did not include an opt-in comparison arm, making it impossible to determine whether successful recruitment was a result of an opt-out strategy.

    Recruitment took place during the COVID-19 pandemic. Given physical distancing recommendations in Sweden, as in many countries globally,53 people have spent increased time at home. Research has observed increased parental stress associated with physical distancing measures and the closure of schools and childcare facilities.54 A retrospective online survey conducted in the USA found that 71.1% of parents reported increases in parenting-specific stress from before COVID-19.54 Further, a survey conducted in the UK indicated that 85% of parents of children with cancer were worried about the virus, for example, risk of infection, reduced healthcare provision, and increased risk of social isolation, and 70% no longer deemed the hospital a safe place.55 The context of a pandemic may have facilitated recruitment, for example, parents may have been experiencing elevated levels of fear and anxiety. Conversely, the pandemic may have been a barrier to participation for some, given that parents of children with cancer commonly focus on their child’s needs, at the expense of their own well-being.36–38

    The overall opt-out rate in the present study is higher than a 19% opt-out rate found in a longitudinal cohort pilot study of patients with angina21 and a 21% opt-out rate seen in a survey study regarding end-of-life care.52 However, existing literature on opt-out strategies tends to be on observational studies, as opposed to intervention or health service-orientated research50 making it difficult to compare results with the wider literature. When comparing the opt-out rate in the present study with non-participation rates in intervention studies for parents or informal caregivers using opt-in strategies, findings are also varied. In an RCT of an intervention for parents of children with cancer to reduce exposure to tobacco smoke, 24% approached about study participation within a clinic setting declined participation56 and in a pilot RCT of guided CBT self-help for informal caregivers of stroke survivors, 15% invited by post actively declined participation.57 Further, in an RCT of a psychological intervention for patients with cancer in palliative care and their family caregivers, 65.9% of potential participants actively declined participation.58

    To the best of our knowledge, this is the first study to specifically examine the rate of opt-out across four different modes of opt-out. Commonly, studies using opt-out recruitment strategies include only one mode of opt-out, for example, via the post26 59 or telephone.60 The most frequent mode of opt-out in the ENGAGE feasibility trial was paper via the post (47.6 %), followed by telephone (32.3%), with findings similar to a quantitative telephone survey with military veterans examining rural and urban differences in attitudes towards mental healthcare and influence on mental health service use, whereby opt-out via the post was more frequent (75%) than opt-out via the telephone (25%).50 Interestingly, only 14.6% of parents opted out online via the Portal in the ENGAGE feasibility trial. In comparison with the general adult population, parents of a child treated for cancer 3 months–5 years following end of treatment may be assumed to be younger, with good digital knowledge and skills.61 Therefore, it may have been anticipated that parents would prefer opt-out online. However, systematic reviews and meta-analyses have shown postal survey response to be, in general, higher than online surveys.62 63 One potential reason is rising concerns about security and privacy in the digital age.64 65 Indeed, common barriers to engaging in internet-administered interventions include security and confidentiality concerns19 Possibly, parents may have experienced opt-out via the post as implying higher levels of security and privacy than using an electronic personal identification method (BankID) to opt-out online. However, the utilisation of different modes of opt-out was easy to implement and given the variation in response rate across the four opt-out modes, future studies may wish to offer potential participants multiple modes of opt-out. Further, future studies could examine additional potential modes of opt-out, for example, via short message service.66

    The trend for parents with an upper secondary school education to opt out of the study while parents with a tertiary education chose to participate is supported by the wider literature. For example, parents of adolescents with psychosocial difficulties who have lower levels of education perceive more barriers towards accessing psychosocial care, than parents with university-level education.67 Moreover, evidence suggests online survey responders tend to be more highly educated than non-responders68 and adults with lower levels of education struggle to find health information online69 70 and require more guidance to use internet-administered interventions.70 Higher education levels are also associated with increased effectiveness and higher completion rates of guided internet-administered CBT interventions in a general adult population.71 Future research is needed to improve the acceptability and feasibility of internet-administered interventions for people with lower education levels.

    In line with previous research with depressed adult populations,72 73 the most common reason for non-participation was not perceiving a need for psychological support. Similar findings were reported in psychological intervention studies for adult patient with cancer/caregiver dyads3 and informal caregivers of stroke survivors.57 Around 14%–30% of parents of children treated for cancer report clinically relevant levels of mental health difficulties, such as depression, anxiety and post-traumatic stress.31 74 75 Therefore, at a group level, a majority of parents report levels of psychological distress within a normal range36 and it can be anticipated most do not experience a need for psychological support. However, as the 11.8% consent rate is under the prevalence rate of psychological distress in the population, some parents in need of psychological support did not consent and research to further increase participation among this group is warranted.

    Study invitation timing was the most commonly cited reason for non-participation in response to the open question, corroborating findings of a similar study conducted with parents of children with cancer examining potential barriers to recruitment.29 Parents found it difficult to identify the most appropriate time point for intervention, as periods of increased distress may differ between parents.29 Other research has found invitation timing plays a significant role in a person’s decision to participate in studies of psychosocial interventions for couples coping with cancer76 and that it is difficult to find an optimal time point to recruit patient with cancer/caregiver dyads into intervention research.77 A potential solution may be to not place any time limit from diagnosis/end of treatment.

    Finally, lack of interest in internet-administered self-help was a common reason for non-participation. Despite well-demonstrated clinical effectiveness of e-MH interventions,78 difficulties with recruitment into trials are common79 and rates of implementation are low.80 Barriers include lack of trust, poor digital health literacy, low levels of confidence in using digital technology, and preferences for face-to-face interventions.17–20 Research indicates acceptability may increase when potential users are provided with information about intervention content and/or effectiveness18 81 and future research may wish to examine the provision of additional intervention information, including, for example, screenshots or videos, on recruitment rates.

    Ethical considerations

    One concern around an opt-out strategy is the use of repeated reminders/contact attempts, and potentially coercing participants.21 While research ethics committees often require researchers to use opt-in recruitment strategies to protect participant confidentiality and (medical) privacy,82 this minimises the opportunity for researchers to speak with potential participants directly to explain the study.83 Consistent with other research using opt-out recruitment whereby few concerns have been raised,82 only one complaint concerning reminder calls was received during the study, indicating an opt-out recruitment strategy is acceptable to the population.

    Strengths and limitations

    This study has several strengths. First, parents were identified and randomly invited into the study via the Swedish Childhood Cancer Registry and a population registry held by the Swedish Tax Agency (NAVET). As such, invited parents are a nationally representative sample of parents of children treated for cancer in Sweden. Second, parents were provided with different modes of opt-out (eg, post, online, telephone or email). To the best of our knowledge, this is the first study to compare opt-out rates across different opt-out modalities. Third, by using both closed and open questions to examine reasons for non-participation, results provide us with a broad understanding of reasons for non-participation which may inform the development of future recruitment strategies. The study also has limitations. First, it was not designed to compare recruitment rates across different recruitment strategies and conclusions on the effectiveness of the opt-out strategy versus other recruitment strategies cannot be drawn. Future studies may look to investigate the effect of an opt-out recruitment strategy with an opt-in recruitment strategy on recruitment rates. Second, we did not collect sociodemographic variables of non-responders and therefore it was not possible to assess whether the sample is representative of the overall population in relation to key sociodemographic variables. Some sociodemographic data (eg, sex, age, geographical location) could have been obtained for all potential participants via the NAVET registry, which would have enabled comparisons between participants who consented, opted out, and were non-responders84 however we did not include the collection of this data in the ethics application. Third, while the open question regarding reasons for non-participation allowed for a broad understanding of reasons for non-participation, embedding semistructured interviews to explore reasons for non-participation in more depth may have facilitated a richer understanding of reasons for non-participation in the population.

    Conclusions

    This is the first study to examine the use of an opt-out strategy and different modes of opt-out within a feasibility trial examining the feasibility and acceptability of an internet-administered, guided, LICBT-based self-help intervention for parents of children treated for cancer. Though recruitment into clinical trials is challenging, projected recruitment rates were exceeded. Despite the most frequent mode of opt-out being paper via the post, given the implementation of different modes of opt-out was straightforward, coupled with the variation in response rate across the four opt-out modes, future studies should consider using multiple modes of opt-out. Developing a more comprehensive understanding of reasons for non-participation may help inform planning and decision-making concerning study design, and potentially enhance recruitment rates and reduce research waste in future trials of internet-administered interventions for both parent/caregiver and wider populations. Future research may wish to compare opt-out versus opt-in strategies within an RCT and embed qualitative interviews to explore reasons for non-participation and develop a more in-depth understanding of potential barriers to trial participation.

    Data availability statement

    Data are available in a public, open access repository. The research data supporting the findings of this study are stored in Zenodo repository with the identifier https://doi.org/10.5281/zenodo.6136920. Access to the data stored in Zenodo is available upon written request from the corresponding author (LvE).

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study involves human participants and was approved by the Regional Ethical Review Board in Uppsala, Sweden (Dnr: 2017/527), and was conducted in accordance with the Helsinki Declaration and Good Clinical Practice (GCP) guidelines. Participants gave informed consent to participate in the study before taking part.

    Acknowledgments

    The authors thank the four PRPs (Helena Börjesson, Martin Hedqvist, Anna Mautner and Erik Olsson) for their feedback on study invitation material. In addition, we wish to thank Fabian Holmberg, Ian Horne and Ylva Hägg Sylvén (Portal team members) for their assistance with developing the opt-out procedure on the Portal. We also thank research assistant, Christina Reuter, for assisting with some elements of data preparation and analysis.

    References

    1. ↵
      1. Bower P,
      2. Brueton V,
      3. Gamble C, et al
      . Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials 2014;15:399.doi:10.1186/1745-6215-15-399pmid:http://www.ncbi.nlm.nih.gov/pubmed/25322807
      OpenUrlCrossRefPubMed
    2. ↵
      1. Moher D,
      2. Glasziou P,
      3. Chalmers I, et al
      . Increasing value and reducing waste in biomedical research: who’s listening? Lancet 2016;387:1573–86.doi:10.1016/S0140-6736(15)00307-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/26423180
      OpenUrlCrossRefPubMed
    3. ↵
      1. Williams RJ,
      2. Tse T,
      3. DiPiazza K, et al
      . Terminated trials in the ClinicalTrials.gov results database: evaluation of availability of primary outcome data and reasons for termination. PLoS One 2015;10:e0127242.doi:10.1371/journal.pone.0127242pmid:http://www.ncbi.nlm.nih.gov/pubmed/26011295
      OpenUrlCrossRefPubMed
    4. ↵
      1. Walters SJ,
      2. Bonacho Dos Anjos Henriques-Cadby I,
      3. Bortolami O, et al
      . Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom health technology assessment programme. BMJ Open 2017;7:e015276.doi:10.1136/bmjopen-2016-015276pmid:http://www.ncbi.nlm.nih.gov/pubmed/28320800
      OpenUrlCrossRefPubMed
    5. ↵
      1. Richards DA,
      2. Ross S,
      3. Robens S, et al
      . The DiReCT study - improving recruitment into clinical trials: a mixed methods study investigating the ethical acceptability, feasibility and recruitment yield of the cohort multiple randomised controlled trials design. Trials 2014;15:398.doi:10.1186/1745-6215-15-398pmid:http://www.ncbi.nlm.nih.gov/pubmed/25318374
      OpenUrlCrossRefPubMed
    6. ↵
      1. Treweek S,
      2. Bevan S,
      3. Bower P, et al
      . Trial forge guidance 1: what is a study within a trial (SWAT)? Trials 2018;19:139.doi:10.1186/s13063-018-2535-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/29475444
      OpenUrlCrossRefPubMed
    7. ↵
      1. Treweek S,
      2. Pitkethly M,
      3. Cook J, et al
      . Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev 2018;2:MR000013.doi:10.1002/14651858.MR000013.pub6pmid:http://www.ncbi.nlm.nih.gov/pubmed/29468635
      OpenUrlCrossRefPubMed
    8. ↵
      1. Woodford J,
      2. Wikman A,
      3. Cernvall M, et al
      . Study protocol for a feasibility study of an internet-administered, guided, CBT-based, self-help intervention (engage) for parents of children previously treated for cancer. BMJ Open 2018;8:e023708.doi:10.1136/bmjopen-2018-023708pmid:http://www.ncbi.nlm.nih.gov/pubmed/29903802
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Woodford J,
      2. Norbäck K,
      3. Hagström J, et al
      . Study within a trial (SWAT) protocol. Investigating the effect of personalised versus non-personalised study invitations on recruitment: an embedded randomised controlled recruitment trial. Contemp Clin Trials Commun 2020;18:100572.doi:10.1016/j.conctc.2020.100572pmid:http://www.ncbi.nlm.nih.gov/pubmed/32420511
      OpenUrlPubMed
    10. ↵
      1. Woodford J,
      2. Farrand P,
      3. Hagström J, et al
      . Internet-administered cognitive behavioral therapy for common mental health difficulties in parents of children treated for cancer: intervention development and description study. JMIR Form Res 2021;5:e22709.doi:10.2196/22709pmid:http://www.ncbi.nlm.nih.gov/pubmed/34142662
      OpenUrlPubMed
    11. ↵
      1. Liu Y,
      2. Pencheon E,
      3. Hunter RM, et al
      . Recruitment and retention strategies in mental health trials - a systematic review. PLoS One 2018;13:e0203127.doi:10.1371/journal.pone.0203127pmid:http://www.ncbi.nlm.nih.gov/pubmed/30157250
      OpenUrlPubMed
    12. ↵
      1. Mackenzie CS,
      2. Erickson J,
      3. Deane FP, et al
      . Changes in attitudes toward seeking mental health services: a 40-year cross-temporal meta-analysis. Clin Psychol Rev 2014;34:99–106.doi:10.1016/j.cpr.2013.12.001pmid:http://www.ncbi.nlm.nih.gov/pubmed/24486521
      OpenUrlCrossRefPubMed
    13. ↵
      1. Clement S,
      2. Schauman O,
      3. Graham T, et al
      . What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med 2015;45:11–27.doi:10.1017/S0033291714000129pmid:http://www.ncbi.nlm.nih.gov/pubmed/24569086
      OpenUrlCrossRefPubMed
    14. ↵
      1. Hughes-Morley A,
      2. Young B,
      3. Waheed W, et al
      . Factors affecting recruitment into depression trials: systematic review, meta-synthesis and conceptual framework. J Affect Disord 2015;172:274–90.doi:10.1016/j.jad.2014.10.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/25451427
      OpenUrlCrossRefPubMed
    15. ↵
      1. Mojtabai R,
      2. Olfson M,
      3. Sampson NA, et al
      . Barriers to mental health treatment: results from the National comorbidity survey replication. Psychol Med 2011;41:1751–61.doi:10.1017/S0033291710002291pmid:http://www.ncbi.nlm.nih.gov/pubmed/21134315
      OpenUrlCrossRefPubMed
    16. ↵
      1. Woodford J,
      2. Farrand P,
      3. Bessant M, et al
      . Recruitment into a guided internet based CBT (iCBT) intervention for depression: lesson learnt from the failure of a prevalence recruitment strategy. Contemp Clin Trials 2011;32:641–8.doi:10.1016/j.cct.2011.04.013pmid:http://www.ncbi.nlm.nih.gov/pubmed/21570485
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Apolinário-Hagen J,
      2. Kemper J,
      3. Stürmer C
      . Public acceptability of e-mental health treatment services for psychological problems: a scoping review. JMIR Ment Health 2017;4:e10.doi:10.2196/mental.6186pmid:http://www.ncbi.nlm.nih.gov/pubmed/28373153
      OpenUrlPubMed
    18. ↵
      1. March S,
      2. Day J,
      3. Ritchie G, et al
      . Attitudes toward e-mental health services in a community sample of adults: online survey. J Med Internet Res 2018;20:e59.doi:10.2196/jmir.9109pmid:http://www.ncbi.nlm.nih.gov/pubmed/29459357
      OpenUrlPubMed
    19. ↵
      1. Schreiweis B,
      2. Pobiruchin M,
      3. Strotbaum V, et al
      . Barriers and facilitators to the implementation of eHealth services: systematic literature analysis. J Med Internet Res 2019;21:e14197.doi:10.2196/14197pmid:http://www.ncbi.nlm.nih.gov/pubmed/31755869
      OpenUrlPubMed
    20. ↵
      1. Vis C,
      2. Mol M,
      3. Kleiboer A, et al
      . Improving implementation of eMental health for mood disorders in routine practice: systematic review of barriers and facilitating factors. JMIR Ment Health 2018;5:e20.doi:10.2196/mental.9769pmid:http://www.ncbi.nlm.nih.gov/pubmed/29549072
      OpenUrlPubMed
    21. ↵
      1. Junghans C,
      2. Feder G,
      3. Hemingway H, et al
      . Recruiting patients to medical research: double blind randomised trial of "opt-in" versus "opt-out" strategies. BMJ 2005;331:940.doi:10.1136/bmj.38583.625613.AEpmid:http://www.ncbi.nlm.nih.gov/pubmed/16157604
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Trevena L,
      2. Irwig L,
      3. Barratt A
      . Impact of privacy legislation on the number and characteristics of people who are recruited for research: a randomised controlled trial. J Med Ethics 2006;32:473–7.doi:10.1136/jme.2004.011320pmid:http://www.ncbi.nlm.nih.gov/pubmed/16877628
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Roberts T,
      2. Miguel Esponda G,
      3. Krupchanka D, et al
      . Factors associated with health service utilisation for common mental disorders: a systematic review. BMC Psychiatry 2018;18:262.doi:10.1186/s12888-018-1837-1pmid:http://www.ncbi.nlm.nih.gov/pubmed/30134869
      OpenUrlPubMed
    24. ↵
      1. Shi W,
      2. Shen Z,
      3. Wang S, et al
      . Barriers to professional mental health help-seeking among Chinese adults: a systematic review. Front Psychiatry 2020;11:442.doi:10.3389/fpsyt.2020.00442pmid:http://www.ncbi.nlm.nih.gov/pubmed/32508688
      OpenUrlPubMed
    25. ↵
      1. Katon WJ
      . Epidemiology and treatment of depression in patients with chronic medical illness. Dialogues Clin Neurosci 2011;13:7–23.pmid:http://www.ncbi.nlm.nih.gov/pubmed/21485743
      OpenUrlPubMed
    26. ↵
      1. McCullough MB,
      2. Janicke D,
      3. Odar Stough C, et al
      . Barriers to recruitment in pediatric obesity trials: comparing opt-in and opt-out recruitment approaches. J Pediatr Psychol 2017;42:174–85.doi:10.1093/jpepsy/jsw054pmid:http://www.ncbi.nlm.nih.gov/pubmed/27328916
      OpenUrlPubMed
    27. ↵
      1. Nalubega S,
      2. Evans C
      . Participant views and experiences of participating in HIV research in sub-Saharan Africa: a qualitative systematic review. JBI Database System Rev Implement Rep 2015;13:330–420.doi:10.11124/jbisrir-2015-2051pmid:http://www.ncbi.nlm.nih.gov/pubmed/26455613
      OpenUrlCrossRefPubMed
    28. ↵
      1. Rivers D,
      2. August EM,
      3. Sehovic I, et al
      . A systematic review of the factors influencing African Americans' participation in cancer clinical trials. Contemp Clin Trials 2013;35:13–32.doi:10.1016/j.cct.2013.03.007pmid:http://www.ncbi.nlm.nih.gov/pubmed/23557729
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Canter KS,
      2. Vega G,
      3. Ramirez AP, et al
      . Strategies for successful recruitment and retention of parents in pediatric psychosocial eHealth interventions: a qualitative study in pediatric oncology. J Pediatr Psychol 2020;45:530–9.doi:10.1093/jpepsy/jsaa013pmid:http://www.ncbi.nlm.nih.gov/pubmed/32196090
      OpenUrlPubMed
    30. ↵
      1. Sharma A,
      2. Minh Duc NT
      . A consensus-based checklist for reporting of survey studies (cross). J Gen Intern Med 2021.
    31. ↵
      1. Sheehan DV,
      2. Lecrubier Y,
      3. Sheehan KH, et al
      . The Mini-International neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59:22–33.pmid:http://www.ncbi.nlm.nih.gov/pubmed/9881538
      OpenUrlCrossRefPubMed
    32. ↵
      1. Grönqvist H,
      2. Olsson EMG,
      3. Johansson B, et al
      . Fifteen challenges in establishing a multidisciplinary research program on eHealth research in a university setting: a case study. J Med Internet Res 2017;19:e173.doi:10.2196/jmir.7310pmid:http://www.ncbi.nlm.nih.gov/pubmed/28536090
      OpenUrlPubMed
    33. ↵
      1. Sjöström J,
      2. von Essen L,
      3. Grönqvist H
      . The origin and impact of ideals in eHealth research: experiences from the U-CARE research environment. JMIR Res Protoc 2014;3:e28.doi:10.2196/resprot.3202pmid:http://www.ncbi.nlm.nih.gov/pubmed/24860071
      OpenUrlPubMed
    34. ↵
      1. Thiblin E,
      2. Woodford J,
      3. Öhman M, et al
      . The effect of personalised versus non-personalised study invitations on recruitment within the engage feasibility trial: an embedded randomised controlled recruitment trial. BMC Med Res Methodol 2022;22:65.doi:10.1186/s12874-022-01553-5
    35. ↵
      1. Craig P,
      2. Dieppe P,
      3. Macintyre S, et al
      . Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337:a1655.doi:10.1136/bmj.a1655pmid:http://www.ncbi.nlm.nih.gov/pubmed/18824488
      OpenUrlFREE Full Text
    36. ↵
      1. Ljungman L,
      2. Cernvall M,
      3. Grönqvist H, et al
      . Long-term positive and negative psychological late effects for parents of childhood cancer survivors: a systematic review. PLoS One 2014;9:e103340.doi:10.1371/journal.pone.0103340pmid:http://www.ncbi.nlm.nih.gov/pubmed/25058607
      OpenUrlCrossRefPubMed
    37. ↵
      1. Carlsson T,
      2. Kukkola L,
      3. Ljungman L, et al
      . Psychological distress in parents of children treated for cancer: an explorative study. PLoS One 2019;14:e0218860.doi:10.1371/journal.pone.0218860pmid:http://www.ncbi.nlm.nih.gov/pubmed/31226159
      OpenUrlPubMed
    38. ↵
      1. Ljungman L,
      2. Boger M,
      3. Ander M, et al
      . Impressions that last: particularly negative and positive experiences reported by parents five years after the end of a child’s successful cancer treatment or death. PLoS One 2016;11:e0157076.doi:10.1371/journal.pone.0157076pmid:http://www.ncbi.nlm.nih.gov/pubmed/27272318
      OpenUrlPubMed
    39. ↵
      1. Ljungman L,
      2. Cernvall M,
      3. Ghaderi A, et al
      . An open trial of individualized face-to-face cognitive behavior therapy for psychological distress in parents of children after end of treatment for childhood cancer including a cognitive behavioral conceptualization. PeerJ 2018;6:e4570.doi:10.7717/peerj.4570pmid:http://www.ncbi.nlm.nih.gov/pubmed/29666751
      OpenUrlPubMed
    40. ↵
      1. Wikman A,
      2. Kukkola L,
      3. Börjesson H, et al
      . Development of an internet-administered cognitive behavior therapy program (ENGAGE) for parents of children previously treated for cancer: participatory action research approach. J Med Internet Res 2018;20:e133.doi:10.2196/jmir.9457pmid:http://www.ncbi.nlm.nih.gov/pubmed/29669710
      OpenUrlPubMed
    41. ↵
      1. Woodford J,
      2. Wikman A,
      3. Einhorn K, et al
      . Attitudes and preferences toward a hypothetical trial of an internet-administered psychological intervention for parents of children treated for cancer: web-based survey. JMIR Ment Health 2018;5:e10085.doi:10.2196/10085pmid:http://www.ncbi.nlm.nih.gov/pubmed/30563814
      OpenUrlPubMed
    42. ↵
      1. Barnes M,
      2. Wiles N,
      3. Morrison J, et al
      . Exploring patients' reasons for declining contact in a cognitive behavioural therapy randomised controlled trial in primary care. Br J Gen Pract 2012;62:e371–7.doi:10.3399/bjgp12X641492pmid:http://www.ncbi.nlm.nih.gov/pubmed/22546597
      OpenUrlAbstract/FREE Full Text
    43. ↵
      1. Farrand P,
      2. Woodford J,
      3. Llewellyn D, et al
      . Behavioural activation written self-help to improve mood, wellbeing and quality of life in people with dementia supported by informal carers (PROMOTE): a study protocol for a single-arm feasibility study. Pilot Feasibility Stud 2016;2:42.doi:10.1186/s40814-016-0083-xpmid:http://www.ncbi.nlm.nih.gov/pubmed/27965860
      OpenUrlPubMed
    44. ↵
      1. Sim J
      . Should treatment effects be estimated in pilot and feasibility studies? Pilot Feasibility Stud 2019;5:107.doi:10.1186/s40814-019-0493-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/31485336
      OpenUrlPubMed
    45. ↵
      1. Sim J,
      2. Lewis M
      . The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. J Clin Epidemiol 2012;65:301–8.doi:10.1016/j.jclinepi.2011.07.011pmid:http://www.ncbi.nlm.nih.gov/pubmed/22169081
      OpenUrlCrossRefPubMed
    46. ↵
      1. Mbuagbaw L,
      2. Kosa SD,
      3. Lawson DO, et al
      . The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study. Pilot Feasibility Stud 2019;5:120.doi:10.1186/s40814-019-0500-zpmid:http://www.ncbi.nlm.nih.gov/pubmed/31700654
      OpenUrlPubMed
    47. ↵
      1. Schulz KF,
      2. Altman DG,
      3. Moher D, et al
      . CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med 2010;152:726–32.doi:10.7326/0003-4819-152-11-201006010-00232pmid:http://www.ncbi.nlm.nih.gov/pubmed/20335313
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Elo S,
      2. Kyngäs H
      . The qualitative content analysis process. J Adv Nurs 2008;62:107–15.doi:10.1111/j.1365-2648.2007.04569.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/18352969
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      1. Krefting L
      . Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther 1991;45:214–22.doi:10.5014/ajot.45.3.214pmid:http://www.ncbi.nlm.nih.gov/pubmed/2031523
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      1. Miller CJ,
      2. Burgess JF,
      3. Fischer EP, et al
      . Practical application of opt-out recruitment methods in two health services research studies. BMC Med Res Methodol 2017;17:57.doi:10.1186/s12874-017-0333-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/28410585
      OpenUrlPubMed
    51. ↵
      1. Treweek S,
      2. Pitkethly M,
      3. Cook J
      . Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev 2010;4:MR000013.
    52. ↵
      1. Hunt KJ,
      2. Shlomo N,
      3. Addington-Hall J
      . Participant recruitment in sensitive surveys: a comparative trial of 'opt in' versus 'opt out' approaches. BMC Med Res Methodol 2013;13:3.doi:10.1186/1471-2288-13-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/23311340
      OpenUrlCrossRefPubMed
    53. ↵
      1. Khalifa SAM,
      2. Swilam MM,
      3. El-Wahed AAA, et al
      . Beyond the pandemic: COVID-19 pandemic changed the face of life. Int J Environ Res Public Health 2021;18:5645.doi:10.3390/ijerph18115645pmid:http://www.ncbi.nlm.nih.gov/pubmed/34070448
      OpenUrlPubMed
    54. ↵
      1. Adams EL,
      2. Smith D,
      3. Caccavale LJ, et al
      . Parents are stressed! Patterns of parent stress across COVID-19. Front Psychiatry 2021;12:626456.doi:10.3389/fpsyt.2021.626456pmid:http://www.ncbi.nlm.nih.gov/pubmed/33897489
      OpenUrlCrossRefPubMed
    55. ↵
      1. Darlington A-SE,
      2. Morgan JE,
      3. Wagland R, et al
      . COVID-19 and children with cancer: parents’ experiences, anxieties and support needs. Pediatr Blood Cancer 2021;68:e28790.doi:10.1002/pbc.28790pmid:http://www.ncbi.nlm.nih.gov/pubmed/33219739
      OpenUrlCrossRefPubMed
    56. ↵
      1. Klosky JL,
      2. Tyc VL,
      3. Lawford J, et al
      . Predictors of non-participation in a randomized intervention trial to reduce environmental tobacco smoke (ETS) exposure in pediatric cancer patients. Pediatr Blood Cancer 2009;52:644–9.doi:10.1002/pbc.21946pmid:http://www.ncbi.nlm.nih.gov/pubmed/19156856
      OpenUrlCrossRefPubMedWeb of Science
    57. ↵
      1. Farrand P,
      2. Woodford J,
      3. Coumoundouros C, et al
      . Supported cognitive-behavioural therapy self-help versus treatment-as-usual for depressed informal caregivers of stroke survivors (CEDArS): feasibility randomised controlled trial. Cogn Behav Therap 2020;13:e23.doi:10.1017/S1754470X20000239
    58. ↵
      1. Ammari ABH,
      2. Hendriksen C,
      3. Rydahl-Hansen S
      . Recruitment and reasons for non-participation in a family-coping-orientated palliative home care trial (FamCope). J Psychosoc Oncol 2015;33:655–74.doi:10.1080/07347332.2015.1082168pmid:http://www.ncbi.nlm.nih.gov/pubmed/26315857
      OpenUrlPubMed
    59. ↵
      1. Sandy LC,
      2. Glorioso TJ,
      3. Weinfurt K, et al
      . Leave me out: patients' characteristics and reasons for opting out of a pragmatic clinical trial involving medication adherence. Medicine 2021;100:e28136.doi:10.1097/MD.0000000000028136pmid:http://www.ncbi.nlm.nih.gov/pubmed/34941059
      OpenUrlPubMed
    60. ↵
      1. Vorderstrasse A,
      2. Reagan L,
      3. D'Eramo Melkus G, et al
      . Recruitment and enrollment of participants in an online diabetes self-management intervention in a virtual environment. Contemp Clin Trials 2021;105:106399.doi:10.1016/j.cct.2021.106399pmid:http://www.ncbi.nlm.nih.gov/pubmed/33857681
      OpenUrlPubMed
    61. ↵
      1. Bjertnaes O,
      2. Iversen HH,
      3. Skrivarhaug T
      . A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care. BMC Med Res Methodol 2018;18:95.doi:10.1186/s12874-018-0557-zpmid:http://www.ncbi.nlm.nih.gov/pubmed/30236067
      OpenUrlPubMed
    62. ↵
      1. Shih T-H,
      2. Fan X
      . Comparing response rates from web and mail surveys: a meta-analysis. Field Methods 2008;20:249–71.doi:10.1177/1525822X08317085
      OpenUrlCrossRefWeb of Science
    63. ↵
      1. Daikeler J,
      2. Bošnjak M,
      3. Lozar Manfreda K
      . Web versus other survey modes: an updated and extended meta-analysis comparing response rates. J Surv Stat Methodol 2020;8:513–39.doi:10.1093/jssam/smz008
      OpenUrlCrossRef
    64. ↵
      1. Acquisti A,
      2. Brandimarte L,
      3. Loewenstein G
      . Secrets and likes: the drive for privacy and the difficulty of achieving it in the digital age. J Consum Psychol 2020;30:736–58.doi:10.1002/jcpy.1191
      OpenUrl
    65. ↵
      1. Robles T,
      2. Bordel B,
      3. Alcarria R, et al
      . Enabling trustworthy personal data protection in eHealth and well-being services through privacy-by-design. Int J Distrib Sens Netw 2020;16:155014772091211.doi:10.1177/1550147720912110
    66. ↵
      1. Blatch-Jones A,
      2. Nuttall J,
      3. Bull A, et al
      . Using digital tools in the recruitment and retention in randomised controlled trials: survey of UK clinical trial units and a qualitative study. Trials 2020;21:304.doi:10.1186/s13063-020-04234-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/32245506
      OpenUrlPubMed
    67. ↵
      1. Nanninga M,
      2. Reijneveld SA,
      3. Knorth EJ, et al
      . Expectations of barriers to psychosocial care: views of parents and adolescents in the community. Eur Child Adolesc Psychiatry 2016;25:107–17.doi:10.1007/s00787-015-0717-1pmid:http://www.ncbi.nlm.nih.gov/pubmed/25969373
      OpenUrlPubMed
    68. ↵
      1. Zuidgeest M,
      2. Hendriks M,
      3. Koopman L, et al
      . A comparison of a postal survey and mixed-mode survey using a questionnaire on patients' experiences with breast care. J Med Internet Res 2011;13:e68.doi:10.2196/jmir.1241pmid:http://www.ncbi.nlm.nih.gov/pubmed/21946048
      OpenUrlCrossRefPubMed
    69. ↵
      1. Alvarez-Galvez J,
      2. Salinas-Perez JA,
      3. Montagni I, et al
      . The persistence of digital divides in the use of health information: a comparative study in 28 European countries. Int J Public Health 2020;65:325–33.doi:10.1007/s00038-020-01363-wpmid:http://www.ncbi.nlm.nih.gov/pubmed/32277246
      OpenUrlPubMed
    70. ↵
      1. van Deursen AJAM,
      2. van Dijk JAGM
      . Internet skills performance tests: are people ready for eHealth? J Med Internet Res 2011;13:e35.doi:10.2196/jmir.1581pmid:http://www.ncbi.nlm.nih.gov/pubmed/21531690
      OpenUrlCrossRefPubMed
    71. ↵
      1. Mathiasen K,
      2. Riper H,
      3. Andersen TE, et al
      . Guided internet-based cognitive behavioral therapy for adult depression and anxiety in routine secondary care: observational study. J Med Internet Res 2018;20:e10927.doi:10.2196/10927pmid:http://www.ncbi.nlm.nih.gov/pubmed/30487118
      OpenUrlPubMed
    72. ↵
      1. Brown JSL,
      2. Murphy C,
      3. Kelly J, et al
      . How can we successfully recruit depressed people? Lessons learned in recruiting depressed participants to a multi-site trial of a brief depression intervention (the 'CLASSIC' trial). Trials 2019;20:131.doi:10.1186/s13063-018-3033-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/30760305
      OpenUrlCrossRefPubMed
    73. ↵
      1. Hughes-Morley A,
      2. Young B,
      3. Hempel RJ, et al
      . What can we learn from trial decliners about improving recruitment? Qualitative study. Trials 2016;17:494.doi:10.1186/s13063-016-1626-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/27733181
      OpenUrlPubMed
    74. ↵
      1. van Warmerdam J,
      2. Zabih V,
      3. Kurdyak P, et al
      . Prevalence of anxiety, depression, and posttraumatic stress disorder in parents of children with cancer: a meta-analysis. Pediatr Blood Cancer 2019;66:e27677.doi:10.1002/pbc.27677pmid:http://www.ncbi.nlm.nih.gov/pubmed/30816008
      OpenUrlPubMed
    75. ↵
      1. Wikman A,
      2. Mattsson E,
      3. von Essen L, et al
      . Prevalence and predictors of symptoms of anxiety and depression, and comorbid symptoms of distress in parents of childhood cancer survivors and bereaved parents five years after end of treatment or a child’s death. Acta Oncol 2018;57:950–7.doi:10.1080/0284186X.2018.1445286pmid:http://www.ncbi.nlm.nih.gov/pubmed/29498559
      OpenUrlCrossRefPubMed
    76. ↵
      1. Badr H,
      2. Krebs P
      . A systematic review and meta-analysis of psychosocial interventions for couples coping with cancer. Psychooncology 2013;22:1688–704.doi:10.1002/pon.3200pmid:http://www.ncbi.nlm.nih.gov/pubmed/23045191
      OpenUrlCrossRefPubMed
    77. ↵
      1. Heckel L,
      2. Gunn KM,
      3. Livingston PM
      . The challenges of recruiting cancer patient/caregiver dyads: informing randomized controlled trials. BMC Med Res Methodol 2018;18:146.doi:10.1186/s12874-018-0614-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/30463520
      OpenUrlPubMed
    78. ↵
      1. Cuijpers P,
      2. Quero S,
      3. Noma H, et al
      . Psychotherapies for depression: a network meta-analysis covering efficacy, acceptability and long-term outcomes of all main treatment types. World Psychiatry 2021;20:283–93.doi:10.1002/wps.20860pmid:http://www.ncbi.nlm.nih.gov/pubmed/34002502
      OpenUrlCrossRefPubMed
    79. ↵
      1. Robinson H,
      2. Appelbe D,
      3. Dodd S, et al
      . Methodological challenges in web-based trials: update and insights from the relatives education and coping toolkit trial. JMIR Ment Health 2020;7:e15878.doi:10.2196/15878pmid:http://www.ncbi.nlm.nih.gov/pubmed/32497018
      OpenUrlPubMed
    80. ↵
      1. Brantnell A,
      2. Woodford J,
      3. Baraldi E, et al
      . Views of implementers and nonimplementers of internet-administered cognitive behavioral therapy for depression and anxiety: survey of primary care decision makers in Sweden. J Med Internet Res 2020;22:e18033.doi:10.2196/18033pmid:http://www.ncbi.nlm.nih.gov/pubmed/32784186
      OpenUrlPubMed
    81. ↵
      1. Gulliver A,
      2. Calear AL,
      3. Sunderland M, et al
      . Consumer-guided development of an engagement-facilitation intervention for increasing uptake and adherence for self-guided web-based mental health programs: focus groups and online evaluation survey. JMIR Form Res 2020;4:e22528.doi:10.2196/22528pmid:http://www.ncbi.nlm.nih.gov/pubmed/33118939
      OpenUrlPubMed
    82. ↵
      1. Nattinger AB,
      2. Pezzin LE,
      3. Sparapani RA, et al
      . Heightened attention to medical privacy: challenges for unbiased sample recruitment and a possible solution. Am J Epidemiol 2010;172:637–44.doi:10.1093/aje/kwq220pmid:http://www.ncbi.nlm.nih.gov/pubmed/20660123
      OpenUrlCrossRefPubMedWeb of Science
    83. ↵
      1. Ward HJT,
      2. Cousens SN,
      3. Smith-Bathgate B, et al
      . Obstacles to conducting epidemiological research in the UK general population. BMJ 2004;329:277–9.doi:10.1136/bmj.329.7460.277pmid:http://www.ncbi.nlm.nih.gov/pubmed/15284154
      OpenUrlFREE Full Text
    84. ↵
      1. Halbesleben JRB,
      2. Whitman MV
      . Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias. Health Serv Res 2013;48:913–30.doi:10.1111/1475-6773.12002pmid:http://www.ncbi.nlm.nih.gov/pubmed/23046097
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • JH and JW are joint first authors.

    • Contributors Author contributions are written in accordance with the CRediT statement: JH—formal analysis, investigation, data curation, writing (original draft), visualisation. JW—Methodology, formal analysis, writing (original draft), supervision, project administration. AvE—validation, investigation, data curation, formal analysis, writing (review and editing). PL—resources, writing (review and editing). LvE—guarantor, conceptualisation, methodology, resources, writing (review and editing), supervision, project administration, funding acquisition.

    • Funding This work is supported by the Swedish Research Council (grant number K2015-99X-20836-08-4 and 2018-02578), the Swedish Cancer Society (grant number 150673 and 180589), the Swedish Childhood Cancer Foundation (grant number PR2017-0005), and funding via the Swedish Research Council to U-CARE, a Strategic Research environment (Dnr 2009-1093).

    • Disclaimer The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

    • Provenance and peer review Not commissioned; externally peer reviewed.