Objective To compare online and paper-based screening for depression and anxiety in adults with cystic fibrosis (CF).
Design and setting Cross-sectional study in CF clinics in Ireland and through the Cystic Fibrosis Ireland online community.
Participants 160 adult patients aged 18 or above were recruited. Of these, 147 were included in the analysis; 83 online and 64 paper-based. The remaining 13 were excluded because of incomplete data.
Measures Depression and anxiety were measured using the Hospital Anxiety and Depression Scale (HADS). Data on pulmonary function (forced expiratory volume in 1 s %) and body mass index were self-reported based on clinical assessments. Sociodemographic data were collected.
Results Compared with the paper-based participants, the online participants were more likely to be female (61.7% vs 48.4%), older (mean 32.2 vs 28.2 years) and were more likely to be married (32.5% vs 15.6%), living with their spouse or partner (42.5% vs 22.6%) and working either full time (33.7% vs 15.9%) or part time (30.1%vs 17.5%). The prevalence rates of elevated anxiety and depression were not significantly different (P=0.71 and P=0.56). HADS anxiety and depression scores were not statistically different between online (P=0.83) and paper-based (P=0.92) participants based on Mann-Whitney U test. A significant negative correlation was found between depression and pulmonary function (r=−0.39, P=0.01) and anxiety and pulmonary function (r=−0.36, P=0.02). Based on Cronbach’s alpha, there were no statistically significant differences between the online and paper-based participants on the internal consistency of the HADS anxiety (P=0.073) and depression (P=0.378) scales.
Conclusions Our findings suggest that online and paper-based screening for depression and anxiety in adult patients with CF yield comparable findings on prevalence rates and scores, associations with health and internal consistency of subscales. This study highlights that online screening offers an alternative method to paper-based screening. Further research with a larger sample and assessment of measurement equivalence between online and paper based screening is needed to confirm our results.
- cystic fibrosis
- online versus paper based screening
- pulmonary function
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study to explore online compared with paper-based screening of depression and anxiety in an adult population of patients with cystic fibrosis (CF).
Given the recent publication on international guidelines for depression and anxiety screening in the CF population, our study is timely because it draws attention to online screening as an alternative method to paper-based screening.
The small sample size and self-selection bias in this study are threats to its internal and external validity.
A comparison of psychometric properties between the online and paper-based administration of the Hospital Anxiety and Depression Scale was limited to internal consistency; further research is needed to assess measurement equivalence, noting that the recent international guidelines support the use of the Patient Health Questionnaire-9 for depression and the General Anxiety Disorder 7-item scale for anxiety in the CF population.
Cystic fibrosis (CF) is chronic, progressive and life-shortening, although increased survival rates into middle adulthood are now expected because of treatment and management advances.1 The disease is characterised in part by increased susceptibility to recurrent chest infections due to excessive production of mucus in the small respiratory airways. In addition, obstruction of pancreatic enzyme secretion due to increased production of mucus in the gastrointestinal tract results in poor growth and weight loss.2
To date, most research efforts have focused on understanding the pathophysiology of CF and the most promising treatment strategies.3 However, in recent years, there has been a growing body of research on the psychological health of individuals with CF, particularly on the prevalence of depression and anxiety, and their associations with physical health. The International Depression/Anxiety Study (TIDES) is a landmark collaboration across several European countries and the USA which screened for depression and anxiety in the CF adult and adolescent populations, and parent caregivers.4–9 In adults with CF, elevated depression and anxiety scores were associated with poorer lung function and quality of life.4 5 One of the main conclusions of TIDES was to recommend annual screening of patients with CF for symptoms of depression and anxiety, so that those affected received timely further assessment and treatment.4 This was endorsed by the European Cystic Fibrosis Society’s Standards of Care published in 2014.10 In 2016, guidelines for screening and treating depression and anxiety from the International Committee on Mental Health in Cystic Fibrosis (ICMH-CF) were published recommending that annual screening be conducted by healthcare professionals, preferably mental health specialists.11
Studies on screening for depression and anxiety in patients with CF and their parent caregivers have collected data during clinic visits with high response rates.5 7 12–15 However, high response rates seen in research may not be feasible within the context of routine clinic visits or annual review assessments. CF teams may face significant challenges in implementing screening for depression and anxiety. In a large-scale study on CF mental health delivery,16 limited staff time and limited personnel were ranked as the two highest barriers to implementing a mental health screening programme. Access to electronic tools for screening administration and scoring was ranked by 40% of respondents as among the top three areas that would be helpful in implementing mental health screening.16 Yet, to date, little consideration has been given to the possibility of online screening of patients with CF for depression and anxiety.
Computerised screening for depression has been available since the 1990s,17 with the potential for internet-based screening of large populations.18 Online screening for depression and/or anxiety has been used in clinical populations such as patients with lung and breast cancer,19 and women with postpartum depression.20 Advantages of online screening include reduced costs,20 an ability to assess large populations,18 21 feasibility and acceptability for patients to use,20 and a practical approach to screening for clinical teams.22 Reported disadvantages of online screening are the possibility that older adults may be less likely to participate,18 the need for computer literacy23 and that response or retention rates may be low.21
There has been no research to date comparing the results of online and paper-based assessment of mental health among patients with CF. Comparisons have been made in patients with tinnitus24 using the Hospital Anxiety and Depression Scale (HADS). Compared with the clinic paper-based sample, the online sample had higher prevalence rates of depression (17% vs 15%) and anxiety (25% vs 15%), and had significantly higher scores for both which the researchers speculated may have been due to anonymised self-recruitment offering an internet intervention on psychological support. Other researchers compared psychometric properties between online and paper versions of depression instruments administered to primary care and psychiatric care patients.22 The findings indicated equivalence and no clinically relevant differences between method of administration with high correlations found between both scores. It is not possible to draw conclusions from existing research for the CF population because of the different clinical populations.
The aim of this study was to compare online versus clinic paper-based assessment of depression and anxiety in a CF adult population in Ireland in relation to sample characteristics, prevalence data and associations with physical health (pulmonary function and body mass index; BMI). In addition, the study aimed to compare online and paper versions for internal consistency of the depression and anxiety measure, which for this study was the HADS.
Study design and setting
We conducted this exploratory study as part of a larger cross-sectional study in Ireland on the national prevalence of depression and anxiety in the CF population and associations with physical health and quality of life. The larger study included adolescents, their caregivers and adult patients. This paper reports on the adult data. We aimed to collect data from adults in all six adult CF centres in Ireland. At the outset, we intended to collect data during scheduled clinic visits similar to the studies involved in TIDES,4 and ethical approval was obtained for this. However, due to problems with recruitment data were also collected online.
Participants and recruitment
The study aimed to recruit adults with CF aged 18 years or over. The only exclusion criteria were adults who had received lung or heart–lung transplantation. Data available from the CF Registry of Ireland at the outset of the study indicated that the total adult population with CF aged 18 years and over was 595.25 In order to estimate the true proportion of depression and anxiety within a CI of ±3% for this population size, it was calculated that we needed to recruit 382 adults.
Our initial plan was to recruit adults during scheduled clinic visits at the CF centres. We sought access to the study sample through the CF teams by writing to the consultant respirologists informing them about the study. We offered to meet the CF teams to discuss the study further, and a meeting was held in three of the six centres.
For three of the CF centres, access was generally unproblematic and was supported by the CF healthcare team. Access was not achieved in one centre because of no response to our efforts to contact the relevant respiratory consultants. In two centres, staffing resource constraints made data collection at clinics impossible. Research access to clinical sites can be challenging and is reliant on key gatekeepers in the services.26 27 Even in the remaining three clinics where we did have access, data collection was slow because of time constraints for adults in completing questionnaires or busy workloads of clinic staff making it difficult to administer questionnaires during clinic appointments. In view of the challenges faced in collecting data, we reapplied for ethical approval to recruit adults online through the Cystic Fibrosis Ireland (CFI) community network.
Data collection procedures
Data collection occurred between July 2014 and July 2015. At clinics, patients were approached by a member of the CF multidisciplinary team and asked to participate in the study. Patients were informed that if they scored above the clinical cut-off for depression and/or anxiety symptoms, their CF consultant would be informed to ensure appropriate follow-up and psychological support. For this reason, data collection was not anonymous. After obtaining informed consent, patients were given the questionnaire pack which could be completed on site or at home and returned to the research team by stamped addressed envelope. Data collection was slow such that over a period of 12 months, paper-based data collection was completed for just 64 adults across three CF centres. Of these, 39 completed the questionnaire on site during a clinic visit and 25 completed the questionnaire at home. The questionnaire took approximately 15–20 minutes to complete. The response rates in the three centres were 53%, 52% and 29%. The overall response rate for the clinic sample was 51%. Patients were not specifically asked for reasons for non-participation but some volunteered that they were ‘not interested’ or already involved in other research projects.
The online version of the questionnaire was developed and administered using the online survey development software, Survey Monkey. The link to the questionnaire, information sheet and consent form was emailed by the administrator of the CFI community network to 345 adults with CF with one follow-up reminder after 7 days. After a period of 2 weeks, 99 responses were collected which was a response rate of 29%; reasons for non-participation could not be gleaned. The invitation email clearly instructed that if the questionnaire was already completed during a clinic visit, there was no need to complete the online survey. We cross-checked all returned online questionnaires with paper-based questionnaires for duplication and removed three duplicates.
In total, 160 adult patients aged 18 or over were recruited from CF clinics or online. Of these, 147 were included in the analysis of whom 83 were the online sample and 64 were the paper-based sample. The remaining 13 respondents were excluded because of incomplete data. With 595 adults entered in the CF Registry of Ireland,25 the sample size of 147 represented 27.6% of the adult CF population in Ireland.
Study packs contained: an information leaflet and consent form; a background information questionnaire and the HADS.28 The background information questionnaire gathered sociodemographic data such as age, gender, marital status, living arrangements and educational and employment information as well as a section on physical and mental health information. In this section, participants were asked to provide self-reported data, based on their most recent clinic visit, on their pulmonary function (forced expiratory volume in 1 s %; FEV1%) and height (cm) and weight (kg) which were used to calculate BMI.
The HADS is a brief 14-item instrument used for screening depression (seven items) and anxiety symptoms (seven items). Each item requires the participant to select one statement from four options on a Likert scale. Scores range from a minimum of 0 to a maximum of 21 for both anxiety and depression. The HADS has well established clinical cut-off scores with a score higher than 7 indicating elevated symptoms of anxiety or depression and a score higher than 10 indicating anxiety or depression in the clinically significant range. The HADS has good psychometric properties as indicated by analysis of internal consistency, discriminant validity and factor structure.29 It was also the instrument used in TIDES to assess anxiety and depression symptoms in patients with CF,4 although since then the Patient Health Questionnaire-9 (PHQ-9) for depression and General Anxiety Disorder 7-item scale (GAD-7) for anxiety have been recommended through international consensus.11 A referral process for follow-up psychological support was in place for participants with elevated depression and anxiety scores.
Data were analysed using SPSS V.22.0 for Windows. To determine if there were statistically significant differences between the online and paper-based samples on demographic variables, independent samples t-tests and χ2 tests were run. In cases where 80% of expected cell count were not greater than five, or all expected cell counts were not greater than one, Fisher’s exact test was used. As the HADS data did not meet requirements of the normal distribution, the median and IQR are reported, and the Mann-Whitney U test was run to compare the online and paper-based samples. Prevalence of elevated and clinically significant anxiety and depression in online and paper-based samples were compared using χ2 tests. The mean and SD of physical health outcome variables, pulmonary function and BMI are reported, and the online and paper-based samples were compared using independent samples t-tests. To examine the relationship between HADS scores and physical outcome variables, Spearman’s correlation coefficients were calculated. Cronbach’s alpha was used to examine the internal consistency of the HADS anxiety and depression scales for both the online and paper-based samples, and Fisher-Bonett test was used to examine statistically significant differences in Cronbach’s alpha between the two samples. Missing data were excluded from the analysis.
Demographic characteristics of online and clinic samples
A total of 147 adults with CF participated in the study. Of these, 56.5% (n=83) completed online and 43.5% (n=64) completed paper-based questionnaires. The demographic characteristics of the online and paper-based samples are presented in table 1. Significant differences were detected between the two groups in the demographic variables of age (P=0.008), marital status (P=0.044), general living arrangements (P=0.024) and employment status (P=0.005). There were no clear differences between the two groups for level of education.
Internal consistency of the HADS scale in online and paper-based samples
The internal consistency of both online and paper-based HADS measure were similar. For the anxiety subscale, the Cronbach’s alpha level was 0.84 for the online sample and 0.89 for the paper-based sample. For the depression subscale, the alpha level was 0.87 for the online sample and 0.88 for the paper-based sample. There were no statistically significant differences between the online and paper-based samples on Cronbach’s alpha for the anxiety subscale (P=0.073) or depression subscale (P=0.378).
Prevalence of anxiety and depression
In the online sample, 25% (n=20) had HADS scores greater than 7 indicating elevated anxiety, and 11.3% (n=9) had anxiety scores in the clinically significant range with scores higher than 10. In the paper-based sample, 28.1% (n=18) had elevated anxiety scores, and 14% (n=9) of the sample had anxiety scores in the clinically significant range. For depression, 7.5% (n=6) of the online sample had elevated scores higher than 7 and 3.8% (n=3) of the sample scored within the clinically significant range higher than 10. In the paper-based sample, 10.9% (n=7) had elevated depression scores and 4.7% (n=3) had depression scores in the clinically significant range. There were no significant differences in prevalence between the online and paper-based samples who had elevated anxiety (P=0.71, 95% CI −0.12 to 0.19) or depression (P=0.56, 95% CI −0.07 to 0.15) scores. Likewise, there was no significant difference in prevalence between those with clinically significant anxiety (P=0.61, 95% CI −0.0 to 0.16) or depression (P=0.78, 95% CI −0.07 to 0.11) scores.
HADS scores and physical health outcome variables
We compared the online and paper-based samples on their HADS scores and physical health outcome variables. As illustrated in table 2, there were no significant differences between the online and paper-based samples on their HADS anxiety (P=0.83) or depression scores (P=0.92). Similarly, there were no significant differences between the two groups on pulmonary function (P=0.39) or BMI values (P=0.60).
Associations between HADS scores and physical health variables
Table 3 illustrates the associations between HADS anxiety and depression scores with the physical health variables of pulmonary function and BMI for both the online and paper-based samples. The results of Spearman’s correlation coefficient indicate that there was a significant negative correlation between HADS depression scores and pulmonary function for both the online (r=−0.39, P=0.002) and the paper-based (r=−0.36, P=0.016) samples. There were no significant associations between HADS anxiety scores and pulmonary function, or between HADS depression/anxiety scores and BMI for either the online or paper-based samples.
In this cross-sectional study, we compared online and paper-based screening for depression and anxiety in adults with CF. Comparable results were found in both groups for prevalence rates and mean scores of depression and anxiety, with no statistical differences evident. Likewise, online and paper-based groups yielded similar results for associations between the mental health variables and physical health variables—pulmonary function (FEV1%) and BMI. In both groups, a significant negative association was found between depression and pulmonary function; the strength of this association was comparable. Neither group was found to have significant associations between anxiety and pulmonary function or between depression/anxiety and BMI.
In contrast to our finding, a previous study that used HADS in patients with tinnitus found significantly higher rates of depression in the online group.24 Self-recruitment and the option to remain anonymous were offered as a possible explanation for this finding, indicating less inhibition in reporting mental health problems.24 In our study, although online participants were self-recruiting, they provided their names and contact details with the knowledge that HADS scores above clinical cut-off point would involve disclosing this information to their CF physicians to ensure appropriate follow-up care. Given the small samples in both groups in our study, there is a need for further research in larger samples to compare prevalence rates and scores of depression and anxiety using online versus paper-based assessment.
To the best of our knowledge, this is the first published study comparing online versus paper-based screening of depression and anxiety in the adult CF population. Previous research comparing online versus paper-based assessment of mental health has been conducted in patients recruited from primary care or psychiatric care,22 patients with tinnitus24 and in psychology students.30 In these studies, regardless of the measurement used to assess anxiety and/or depression, the internal consistency for online and paper-based versions was similar. Our findings concur. In both the online and paper-based groups, the internal consistency of the HADS was comparably high and not significantly different. These findings suggest that online screening of mental health in adults with CF offers a reliable alternative to paper-based screening in clinical settings.
Our study demonstrated the potential of online mental health screening to yield higher participation rates within a shorter time frame compared with paper-based administration during clinics (or later at home). Over a period of 2 weeks, 99 HADS online questionnaires were returned whereas it took 12 months to collect 64 paper-based questionnaires. There is some evidence to suggest that paper-based screening within clinical settings is a challenge. In a feasibility study31 on implementing the ICMH-CF guidance11 on screening for depression and anxiety symptoms during CF clinics, concerns were expressed by staff that lengthy clinic visits and a requirement to complete ‘another test’ were barriers to paper-based screening. Giving an option to take home the screening measure for completion was also seen as problematic because it risked non-return and the possibility of missing symptoms of depression or anxiety.31 A conclusion from that study was that there is a need to consider online screening outside the clinical setting to allow for faster administration and scoring.31 Given the time and resource constraints of conducting mental health screening during CF clinic visits,16 31 the participation rates within associated time frames identified in our study indicate that online screening may offer a practical alternative to paper-based screening and may contribute to increased participation rates. We acknowledge that our online response rate was low at 29%. A mitigating factor precluding us from extending the online survey beyond 2 weeks related to timelines for study completion set by the funding bodies. It is likely that the response rate would have been higher if more time was allowed, a finding evident in a feasibility study on web-based screening of physical symptoms in patients with CF which had a response rate of 80% over a 9-month period that included repeated reminders.32
Our online and paper-based samples differed on a number of demographic variables, but they did not differ on lung function or BMI, and for both depression was associated with lower lung function. The online sample was older, more were married, living with a spouse/partner and in full-time or part-time employment. It could be assumed from this finding that the online sample had busier life circumstances in which case online screening may be more convenient over paper-based screening. However, caution is needed with this assumption. Further research is required to determine the factors motivating the completion of online versus paper-based mental health screening among adult patients with CF. The acceptability of online screening among adults also needs consideration.
Mental health screening for depression and anxiety is now set to become integral to the health assessment of patients with CF based on international guidelines.11 While our study demonstrates the potential of online mental health screening as a practical, efficient and reliable approach, there are issues to be considered regarding implementation into CF services. An information technology infrastructure supporting the use of electronic health records is needed to ensure a seamless, paperless system. Electronic health records could be used to administer, score, track and provide results to healthcare professionals compliant with data protection legislation. Peckham et al found that the implementation of electronic records incorporating a coding structure for CF care into CF centres of three hospitals led to greater efficiencies such that completion of annual assessments regarding physical health more than doubled from 43% to 92%.33 Furthermore, information exchange between multidisciplinary team members improved.33 A recent report on the e-Health status of European Union countries showed that almost half of the member states do not have national electronic health record systems with funding being a major barrier to their implementation.34 The impact of implementing electronic records that include mental health assessment and care of adults with CF remains unknown to date.
A further consideration for clinical practice regarding online screening relates to patient feedback and follow-up care. In contrast to the immediate access that healthcare professionals have to patients during clinic visits, online screening is more removed and therefore requires a planned approach to providing feedback and follow-up care. There is evidence from an Australian study on the general population that tailored feedback following online screening may not promote the use of professional services to deal with mental health problems.35 In that study, mental health screening was not embedded within an existing health service that participants were using. While it could be expected that mental health screening of adults with CF that complies with the recent international guidelines11 will be embedded in CF services regardless of being online or paper-based, inadequate numbers of psychosocial professionals and staff trained in mental healthcare within CF teams is a concern.16 31
The findings of our study must be interpreted with some caution in light of limitations. The setting in which questionnaires were completed may have acted as a confounding variable as paper-based questionnaires were completed in clinics or in the participant’s home. Sample size was small in both online and paper-based groups, therefore undermining the internal and external validity of the findings. Self-selection bias exists which also threatens the internal and external validity of the findings. The comparable findings on prevalence rates of depression and anxiety between online and paper-based groups are not based on analyses that adjusted for baseline differences in demographic data, for example, age, sex, education status and living arrangements. We did not adjust for baseline differences because of the small sample size. The cross-sectional design of the study did not allow for assessment of changes in depression and anxiety over time and what circumstances might influence these changes. Longitudinal data would help address this limitation.
A further limitation relates to the assessment of psychometric properties in that Cronbach’s alpha only was used to test if method of administration differed for internal consistency. During the time of conducting our study, the reliability of the HADS as a clinical screening tool was critically questioned noting that although the HADS was used in TIDES, different measures were recommended from that international study.36 These measures were the PHQ-9 for depression and the GAD-7 for anxiety.4 36 Both these measures are recommended in the international guidelines for mental health screening of patients with CF,11 which are since being used in CF clinical settings.15 31 Because of the current shift away from using HADS as a screening tool for depression and anxiety in CF, and because of the small sample sizes in each group (online and paper-based), we did not test measurement invariance to determine if the online version of HADS is equivalent to the paper based version. Although scant, there is some evidence that depression scores between online and paper administration of the PHQ-9 are not significantly different.37 An online version of GAD-7 has been reported as reliable,38 but we found no evidence of comparisons with paper administration. Future research needs to conduct confirmatory factor analysis to assess the psychometric invariance between online and paper-based administration of these screening tools in the CF population, including the settings in which they are administered.
A strength of our exploratory cross-sectional analysis is that it is the first to compare online versus paper-based assessment of depression and anxiety in adults with CF. In the TIDES international study, all data collection was paper-based.4 Therefore, our study draws attention to the potential of online screening for depression and anxiety in adult patients with CF. The similarities in findings between both methods of administration is encouraging regarding prevalence rates of depression and anxiety, associations with pulmonary function and BMI. These findings demonstrate that online screening could be an alternative method to paper-based screening for those who prefer this option or miss clinic appointments, for example, due to family or work responsibilities; or wish to avoid spending extra time at clinics or annual review completing ‘another test’ as shown in previous research.31
In conclusion, some adult patients with CF experience symptoms of depression and anxiety. Both depression and anxiety can negatively impact on pulmonary function. The international recommendation for annual screening of depression and anxiety11 therefore applies to patients with CF living in Ireland. This study has highlighted that online screening offers an alternative method to paper-based screening. The feasibility, acceptability, reliability and validity of online screening compared with paper-based screening need further research.
We are grateful to all adult patients with cystic fibrosis (CF) who participated in the study, and to all CF teams who supported access to clinics and data collection. We are also grateful to Cystic Fibrosis Ireland who supported administration of the online survey.
Contributors ES, IJP, AJD, KAR, APF, MNC contributed to design and application for funding. ES and JC prepared for ethical approval and negotiated access to CF centres in Ireland. JC and BH collected data. JC conducted data analysis supported by APF and EL. AH and AJD offered critical insights into mental health screening. All authors contributed to the interpretation of results. JC and ES drafted the manuscript. All authors have read, edited and approved the manuscript.
Funding This work was supported by a research grant jointly funded by the Health Research Board, Ireland and Cystic Fibrosis Ireland (grant file reference: MRCG/2011/9).
Competing interests None declared.
Patient consent Not required.
Ethics approval Clinical Research Ethics Committee, University College Cork (Ref no: ECM4(ii)04/12/12, and amended for online data collection-(Ref no: ECM4(ii)03/03/15).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.