Article Text
PDF
Abstract
Objectives The Brief Health Check (BHC) is a health screener used by the Get Healthy at Work programme, which identifies workers with chronic disease risk and provides them with advice and referrals to support services. The BHC was revised to include mental health to provide a holistic approach to workplace health. This study aimed to evaluate the acceptability and appropriateness of the revised BHC by comparing the results around psychological distress and future risk with previous research, and a participant feedback survey.
Method Data collection took place between October 2018 and May 2019. The study used data that were collected as part of programme delivery, as well as a participant feedback survey that was administered after the health check was completed.
Results BHCs were completed by n=912 workers, out of which, n=238 completed the feedback survey. The mean Distress Questionnaire 5 score was 10.5, and 10% of participants met the threshold for ‘high’ future risk. The feedback survey revealed that the majority of participants found the mental health advice to be useful (76%), agreed with their mental health distress and risk ratings (92%–94%) and most intended on using the referred services (62%–68%).
Conclusion The findings around mental health risk were comparable to previous findings in employed samples. The inclusion of mental health assessments, advice and referral pathways into the BHC was found to be acceptable and the subsequent referrals were appropriate, indicating that this approach could be scaled up and implemented to help address worker’s mental ill health
- public health
- mental health
- psychiatry
Data availability statement
Data are available on reasonable request. Data are available on reasonable request to the authors.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The study is the first to use the Distress Questionnaire 5, a general population health screener tool, in a working population.
This is the first health screener in Australia to include both current psychological distress and future mental health risk.
The study did not employ a longitudinal design, and future research could follow-up with employees to assess the impact of the health check.
The study did not ask about demographics in the participant feedback survey, so it is uncertain how the participant feedback survey sample compared with the larger sample which completed the Brief Health Check.
Introduction
Mental health issues are very prevalent in the Australian population, with one in five adults (aged 18–85) having experienced mental disorders within the last 12 months, and 45.5% of the total population having experienced a mental disorder at some point in their lifetime.1 Mental health issues in the working population can be costly to employers in terms of lost productivity and turnover, as well as to society at large in the form of health service use, where US$9.9 billion was spent on mental health related services in Australia from 2017 to 2018.2–5 For individual workers, mental health issues can impact negatively on workplace engagement as well as overall quality of life.6 7 In recent years, governing bodies in Australia have implemented strategies to facilitate the promotion of mental well-being in the workplace.8 9 One of the channels through which this strategy is implemented is through existing workplace health programmes, which have significant reach in the working population and present opportunities for promoting mental well-being (eg, the Mentally Healthy Workplaces programme from SafeWork New South Wales (NSW)).10
Workplace health programmes are health promotion and protection strategies implemented in the workplace,11 with the goal of establishing organisational cultures that promote and provide healthy lifestyle choices. Systematic reviews of such programmes have found positive impacts on the health and well-being of workers as well as the productivity of the organisation.12–14 In NSW, the Get Healthy at Work programme was launched in 2014, along with a Brief Health Check (BHC) with the aim of reducing type 2 (T2) diabetes and cardiovascular disease risk among workers. The supports workplaces to create health promoting structures and processes, along with a BHC designed to help individual workers to reduce their lifestyle risk factors (ie, waist circumference, diet, physical activity and smoking). The BHC identifies workers with high T2 diabetes and cardiovascular risk, refers them to external support services, and offers personalised advice.15
In late 2018, the Get Healthy at Work programme sought to include mental health into the BHC to provide a holistic assessment for employee health and well-being. The BHC was expanded to include mental health assessments, referral pathways to mental health support services, as well as personalised mental health advice. Similar to the development of the original BHC,16 the development of the mental health items followed a translational formative evaluation process,17 which began with synthesising the evidence, consulting with practitioners/academics, as well as stakeholders. From this process, programme managers decided to use the Distress Questionnaire 5 (DQ5), which is a short assessment of current psychological distress.18
The BHC also sought to prevent future incidence of psychological distress in participating workers. Therefore, the revised BHC includes a risk algorithm developed by Morris and Glozier (an unpublished internal report) to identify participants who are at risk of experiencing mental health issues within the next 12 months.
Based on advice from the clinical advisory panel, the revised BHC refers participants with high current distress (according to the DQ5 score) to the MindSpot free online supported mental health clinic,19 as well as to a general practitioner. Those found to have moderate current distress are referred to myCompass,20 an online mental health programme that is self-guided. Both myCompass and MindSpot21–23 have demonstrated efficacy in improving mental health outcomes. Participants with high future risk scores are given advice to help manage their mental well-being. Further, because of the importance of positive lifestyle modification in promoting mental well-being,24–26 the BHC offers personalised advice around how individuals could improve their mental well-being by modifying their lifestyle through improved diet and physical activity.
Following the translational formative evaluation process,17 the current study aims to evaluate the revised BHC within workplaces to assess whether it can be scaled up for state-wide delivery and identify ways in which the tool can be improved. The key implementation research questions to be examined were: (1) Comparability: How do the findings around current psychological distress and high mental health risk in the applied setting compare with previous research? (2) Acceptability: Do workers find the new mental health questions easy to understand? Do participants agree with the results they received? Is there any potential harm in using these assessments? Do participants agree with the risk ratings they received? (3) Uptake and engagement: What is the uptake of referrals made? Do participants intend on using the services to which they were referred? Do participants find the personalised mental health advice useful?
Method
The study used BHC cross sectional survey data that was collected as part of regular programme delivery to determine the comparability of results and uptake of referral pathways. A cross-sectional feedback survey was administered after completing the BHC. The feedback survey was included to help answer the research questions around acceptability, uptake of referrals and engagement with advice.
Sample
The revised BHC was first administered within two NSW government organisations that consented to using the revised BHC: the Department of Education, and a government insurance and workers compensation unit. Data collection for the current study ran from October 2017 to May 2018. The worksites for both organisations were in metro and regional/rural areas. Each participating organisation promoted the BHC at each worksite, and participants who completed the BHC were asked to complete the feedback survey immediately after completing the BHC. The participant feedback survey was administered at worksites that allowed the participant feedback survey to be administered (ie, 13 of the 35 worksites that were involved in the pilot). The study made use of all BHC data that was collected during the study period, as well as all participants who consented to provide feedback via the survey. The BHC sample was large enough to detect small effect sizes (Cohens’s d=0.2 at 80% power) when comparing samples on the DQ5.
Participant involvement
Participants provided data for the study and were not involved in the design, reporting or dissemination for this project.
Measures
Distress Questionnaire 5
The DQ5 has greater sensitivity than other widely used measures (ie, Kessler 6 and 10) for identifying individuals currently at risk for specific anxiety disorders. The development of the DQ was described in detail in the paper by Batterham et al.18 The BHC uses the cut-points defined by Batterham et al 18 to classify participants into different levels of current distress. That is, participants with DQ5 scores equal to or greater than 11 were identified as having ‘moderate’ current distress, where a participant is likely to meet the criteria for a wide range of disorders, and those with DQ5 scores equal to or greater than 14 were identified as having ‘high’ current distress, where a participant is likely to meet the criteria for specific disorders with a lower rate of false positives compared with participants who are classified as having ‘moderate’ distress.
Future risk tool
The future risk tool used in the BHC was adapted from Fernandez et al 27 by Morris and Glozier, which is the first mental health risk algorithm to be created for the working population in Australia. Morris and Glozier updated the algorithm using 2015 and 2016 data from the Household and Income Labour Dynamics in Australia survey and obtained a comparable C-index (0.71) and positive predictive value (0.28) in validation (The formula for the future risk algorithm is: Yi = −1.288 + (0.03) age: 35–39 + (−0.167) age: 40–44 + (−0.04) age: 45–54 + (−0.167) age: 55–54 + (−0.207) age: 65 and over + (0.104) country of origin: Asia + (−0.011) country of origin: Middle East/N. Africa + (−0.080) country of origin: other + (0.032) Aboriginal or Torres St. Islander + (−0.085) sex:male + (0.672) recent mental illness + (0.281) bullied + (−0.068) health satisfaction + (0.151) loneliness + (0.047) binge drink + (0.158)Smoker + (0.056) physically inactive). The coefficients for the future risk algorithm are presented in table 1. For future risk scores, the revised BHC uses thresholds defined by Morris and Glozier, in which participants who exceed the algorithm’s threshold for high risk are expected to have a 28% chance of experiencing psychological distress in the next 12 months. Participants who exceed the threshold for moderate risk are expected to have a 22% chance of experiencing psychological distress in the next 12 months.
Future risk model
Analysis
Comparability: The results around current distress and prevalence of future risk categories in the BHC were compared with previous research. The BHC sample was weighted for age and gender before the results were compared with previous data, which examined findings at the population level. The weight values were based on the 2016 Australian Census filtered for individuals who were employed.28 A two-sample t-test was used to compare the mean DQ5 score from the current study with the results from Batterham et al,18 and the prevalence of future risk was compared with the models that informed the development of the future risk tool by descriptive statistics.
Acceptability: The feedback survey asked participants whether the questions were difficult to understand, and whether participants felt uncomfortable about answering any of the mental health questions. Both were examined using ‘Yes/No’ questions followed by open-ended questions to identify the items that were difficult or made participants feel uncomfortable. These questions aimed to assess any potential issues with comprehension and harm associated with the revised BHC.
Uptake and engagement: The uptake of the referred services was recorded in the BHC questionnaire, where participants have ‘accepted’ referrals if they agreed to be referred during the BHC session by the health professional, or indicated that they will register for the service after the BHC. Referral outcomes were stratified by current help seeking behaviour (ie, whether participants are currently seeing a mental health professional), as well as demographic characteristics to assess the rate of uptake in those who are not receiving help, and a range of population groups. Uptake of the referred services was also examined through the participant feedback survey, which asked participants whether they intend on using the service to which they were referred in the BHC (examined using multiple choice ‘Yes/No/Intend to use at a later time’). The feedback survey also asked participants whether they found the mental health advice useful on a five-point scale. The authors do not have visibility of the number of participants who access their referred service after the BHCs were conducted.
Participants who did not complete the DQ5 or future risk questionnaires were excluded from the analyses. Participants who did not answer a question in the feedback survey were removed from the analysis of that question.
Brief Health Check
The revised BHC was administered face to face within participating worksites by trained health professionals, such as accredited dietitians or exercise physiologists. Participants completed a questionnaire related to diet, physical activity, demographic characteristics, and physical and mental health risk profiles, distress (DQ5) and health-related behaviours. The BHC questionnaires were completed on paper (n=198) or equivalent digital forms (n=714). Once the risk scores were calculated, the health practitioners provided feedback about the risk scores, and provided appropriate referrals and advice depending on the risk profile of the participant. Health professionals then recorded whether participants accepted referrals. A single BHC session took around 20 min to complete.
Participant feedback survey
Between 1 and 3 interviewers were present at each of the 13 worksites participating in the feedback survey. Once participants completed the BHC, they were asked to participate in a survey administered by the trained interviewers, who were blinded to the results from the BHC. The surveys included seven questions that were administered verbally and responses were collected on paper forms. The survey took no longer than 10 min to complete. The questions were a combination of closed and open-ended responses that were developed for this study. The open-ended responses about difficulties understanding questions and feeling uncomfortable about answering questions were analysed by two coauthors (JX and VM) using closed-coding to identify the specific BHC questions referenced in participant feedback. Open-ended responses around participant feedback, agreement with current distress and future risk ratings, and intention to use services were analysed using open coding. The authors coded the responses independently, and then met to reach a consensus on the assignment of the codes.
Results
Profile of worksites
A total of 35 worksites participated in the study and 13 worksites allowed participant feedback surveys to be administered. The total number of completed BHCs was n=912, and a total of n=238 participants completed the feedback survey. The authors did not have visibility of the number of employees within each organisation that were invited to complete the BHC. Based on an estimate of the number of employees across the worksites (n=7200), and the assumption that all employees at each worksite were invited, a conservative estimate of the response rate for the BHC (with n=912 completes) is 12.7%. A breakdown of completed BHCs, number of surveys within organisations and the location of the worksite is presented in table 2. The urban/rural/remoteness of the worksite was based on postcode, using the Accessibility and Remoteness Index of Australia.29
Breakdown of Brief Health Checks and surveys by organisation
Comparability
The characteristics of participants, including the current distress results and prevalence of future risk categories, are presented in table 3. Two participants did not complete the DQ5 and were excluded from the analyses. Future risk scores were only calculated for participants who did not have a high level of current distress (ie, those with DQ5 scores<14). In this study, the weighted mean DQ5 score was 10.5 (SD=4.2). This was significantly higher than the weighted mean scores from the study by Batterham et al 18 (mean DQ5 score=9.28, SD=4.08), via an independent samples t-test: t (4083)=7.8, p<0.001, and the difference was small-medium in terms of effect size (Cohen’s d=0.29). For the prevalence of future risk in the weighted sample, 9.6% of participants met the threshold to be in the ‘high’ future risk category, in which 28% of participants are expected to experience psychological distress within 12 months. This is consistent with the population proportion that was expected to meet this threshold according to the future risk algorithm (10% or 90th percentile).
Characteristics of participants who completed Brief Health Checks (BHC) (n=912)
Acceptability
Mental health questions
The participant feedback survey revealed that 17.2% (n=41) of respondents found the mental health questions difficult to understand. Participants reported that they found one (13.4%, n=32) or two (2.5%, n=6) questions difficult, and the remainder reported that their difficulties were due to general comprehension or recall (1.2%, n=3). The responses were back-coded to identify the specific questions that were difficult to understand, which showed that 10.9% (n=26) of participants found the future risk questions to be difficult, and 6.7% (n=16) of participants found the DQ5 questions to be difficult. Of the participants who found the future risk questions to be difficult, themes emerged regarding whether the question around ‘satisfaction with your health’ referred to mental or physical health, and whether the question ‘Have you had mental health problems in the past 2 years’ referred to mental health issues that were diagnosed or included all mental health problems. Most of the participants who had trouble understanding the DQ5, linked their difficulties to the question ‘I found social settings upsetting’ and whether ‘social settings’ referred to all social settings or just those in the workplace. Many participants who had trouble with DQ5 or the future risk questions also reported that the health professionals conducting the BHC offered useful prompts which helped them answer these questions. A small proportion (7.6%, n=18) reported that they felt uncomfortable about answering one or more of the mental health questions. When probed further about the specific questions they had concerns about, most of these participants indicated that they felt uncomfortable about talking about mental health in general (n=12, 5%), while 1.3% (n=3) linked their response to the DQ5, and 2.1% (n=5) linked their response to the future risk questions.
Agreement with risk ratings
From the participant feedback surveys, only 5.9% (n=14) of participants disagreed with their current distress scores, and 8.0% (n=19) disagreed with their future risk scores. Of the participants who disagreed with their either their current or future mental health risk, there was a mix of those who expected their scores to be higher (current: n=1, 0.4%; future: n=3, 1.3%) or lower (current: n=4, 1.7%; future: n=7, n=2.9%) than what they received. For those who disagreed with their current distress or future risk scores, some participants did not disagree with the rating per se but expressed scepticism that the questions could provide an accurate assessment of their mental health state or predict their future risk: ‘Assessing risk for the future seems unrealistic—impossible to know what will happen in the future. Not sure how the assessment/questions work’.
Uptake and engagement
Uptake of referrals
The breakdown of participants who accepted referrals during the BHC session are presented in table 4. Questions around whether participants were currently seeing a mental health professional were introduced later in the pilot, and so the sample size for table 4 is smaller than the total number of completed BHCs. Of participants who had high current distress and were not currently receiving support from a mental health professional (n=139), the majority (n=95, 68.3%) accepted referrals to MindSpot, and most participants accepted referrals to their GP for mental health support (n=86, 61.9%). Referral outcomes were further examined by age, gender and cultural background to assess whether referral rates differ across population groups. For participants with high current distress, there were no significant differences between any demographic groups in accepting referrals to MindSpot or their GP (using χ2 tests; p’s>0.05). Females (n=97, 65.1%) were significantly more likely than males (n=33, 49.3%) to accept a referral to myCompass (χ2=4.2, p=0.04).
Brief Health Checks referral outcomes for mental health
Based on the participant feedback surveys, the majority of participants indicated that they intended to access the mental health services to which they were referred (myCompass n=62/81, 76.5%; MindSpot n=31/43, 72.1%; n=21/29, GP 72%). Some participants who indicated that they did not plan on accessing MindSpot or myCompass suggested that they would prefer face-to-face mental health support: ‘No, not likely to go online…I would rather see someone face-to-face’. However, a number of participants suggested that they might use these services in the future: ‘I don’t think I need [MindSpot] right now, but it is good to know about it if I need to access it later’.
Advice
Out of the participants who received mental health advice during the BHC, most reported that the advice they received was useful (n=89, 76.1% reported that the advice was ‘Very useful’/‘Fairly useful’; n=26, 22.2% reported that the advice was ‘A little useful’/‘Not useful at all’; and n=2, 1.7% indicated that they ‘Don’t know’). When asked to provide further feedback about the advice they received, some participants suggested that the advice helped them learn more about their mental well-being: ‘I knew much of the information on physical health, but mental health was all new to me. Surprised about the links between physical health and mental health…I didn't previously ever even consider my mental health’. Participants who suggested that the advice confirmed what they already know, saw this as a useful instance of reinforcing their understanding of healthy lifestyle behaviours: ‘[I] already know about own mental and physical states, but was good to get confirmation and reminder’. Participants who felt that the advice was ‘A little useful’/ ‘Not useful’ indicated that the advice was not specific enough: ‘I am [already] conscious of my physical and mental health, the check-up was very broad’.
Discussion
The findings from his study suggest that the revised BHC is appropriate for assessing both current and future mental health risk in the workplace context. The mean DQ5 score from the current study is higher than that from Batterham et al, which is consistent with previous research. Specifically, Jarman et al 30 compared the psychological distress from a general population with the findings from an employee well-being survey among public servants in Tasmania. The authors found that the mean psychological distress (using the Kessler 10) scores from public service workers was higher than the general population, and suggested that the differences could be attributed to workplace specific stressors such as the rationalisation of the workforce, job insecurity and effort–reward imbalance.31 32 The lower levels of well-being among public sector employees has also been found in other jurisdictions.33 A recent study of secondary school teachers in NSW by Parker et al 34 found a mean DQ5 value (ie, mean=11.25, SD=3.8), a similar result to the current study, although this was from a small sample. The prevalence of high future mental health risk is comparable with the models that informed the development of the future risk tool, which used the same measures in a statewide sample across many different occupational groups. While different occupational groups commonly report very different levels of mental ill health,35 the similarity in risk prevalence between this study and earlier work suggests that there are common drivers of mental ill health risk across industries (eg, prior ill health, discrimination).
The majority of those who were not receiving mental health support at the time of the BHC accepted referrals to mental health support services (ie, MindSpot, mental health GP referrals and myCompass) based on their risk scores. There were no differences in the demographic characteristics of participants with high current distress that accepted referrals compared with those who did not accept referrals. The findings from the participant survey suggest that only a small number of participants felt uncomfortable about answering the mental health questions, and most participants agreed with their mental health risk scores. Participants mostly reported that the advice that was offered as part of the BHC was useful and that they intended on using the mental health services to which they were referred. Overall, these results suggest that the revised BHC is suitable for use among workers.
The participant feedback survey revealed that around one out of five participants found the mental health questions difficult to understand, which would require the BHC to be refined to facilitate understanding. The findings from the survey also highlighted ways in which comprehension could be improved. Specifically, confusion around the DQ5 question ‘I found social settings upsetting’, and around whether the future risk question for whether participants have had ‘mental health problems in the past 2 years’ could be addressed by providing participants with suitable prompts. For the future risk question around ‘satisfaction with health’, prompts could be offered to clarify that health refers to both mental and physical health, or reorder the question to a location where the participant would not be biased towards interpreting the question as referring to either physical or mental health. In terms of next steps, it is recommended that the prompts for the DQ5 and future risk tool are added to the revised BHC before it is implemented on a wider scale. The prompts will only be provided by the health professional if a worker has trouble with the instrument and are not expected to impact on the validity of those instruments.
For participants who reported that they prefer a more comprehensive health check or were sceptical that their future mental health risk can be accurately determined from a small number of questions, their experience could be improved by setting more realistic expectations about the programme. That is, the BHC should be introduced as a concise screener tool used to identify participants who are ‘at risk’ and refer them to clinical support services, as opposed to a definitive diagnostic test, consistent with the messaging from other online assessment tools such as the Black Dog Institute’s Online Clinic assessment.36 The information about how future risk is calculated (ie, a combination of physical and mental health questions, modifiable and non-modifiable factors) as well as noting that the future risk score is based on existing research, will help assure participants who are sceptical about the validity of the assessments. To improve workers’ experience with the tool, it is recommended that these adjustments are incorporated into the standard BHC protocol.
As an adaptation, the BHC could be implemented as an online assessment (eg, with automated scoring, advice and referrals), which presents an opportunity to scale up the programme and extend the reach to a larger number of organisations and remote locations. Future research could explore whether participants would find an online BHC to be as useful as a face-to-face version, given that the participants have responded positively to the personalised advice delivered by health professionals. The ease of administering the revised BHC as an online tool presents opportunities for a mental health screener to be deployed at scale in the workplace, while offering relevant advice and referral pathways. The introduction of an accessible health screening tool aligns with the recommendation from public and mental health professionals to improve the mental health of workers.37 38 However, the BHC with feedback and advice might not be sufficient in isolation, as studies have suggested that improvements to some health outcomes are better achieved through a combination of health assessments and other health promotion activities (eg, health education, policy and environmental change),39 which highlights the importance of implementing other workplace health initiatives prescribed by the Get Healthy at Work programme alongside the BHC.
A limitation of the this study is that the current study did not collect demographic information in the participant feedback survey, so the sample from the feedback survey cannot be compared with the BHC sample. Additionally, the current study does not provide any insight into the long-term benefits of the programme. Future research can also use the BHC to track the health of workers longitudinally and examine the relative impacts of the workplace health programme on the health outcomes of workers. The predictive accuracy of the future risk tool may also be a limitation of the current study. Although the tool has modest predictive accuracy, there are no established risk prediction tools that perform better in identifying the risk of future mental ill health. Predictive validity of such tools will be limited by a multitude of risk factors that influence distress and the relatively low base rate of distress in general population settings.
The current research suggests that the revised BHC with mental health assessments, referral pathways and advice are acceptable and suitable for the workplace setting, but also highlights ways in which the revised BHC could be improved. To our knowledge, this is first study to assess the acceptability and appropriateness of the DQ5, a population health screener, in a workplace setting. Additionally, the revised BHC is the first mental health assessment that tests for both current and future mental health risk in the workplace.
Data availability statement
Data are available on reasonable request. Data are available on reasonable request to the authors.
Ethics statements
Patient consent for publication
Ethics approval
Ethical approval for the analysis of routine programme data and participant feedback was obtained from South Western Sydney Local Health District Human Ethics Committee (Ref: ETH12061). The ethics approval covered the routine analysis of programme data (BHCs) and the participant feedback survey, for which verbal consent was obtained from participants.
Acknowledgments
The authors would like to thank Ms Jillian Green (SafeWork NSW) and Prof Samuel Harvey (University of New South Wales) for their support and advice.
References
Footnotes
Contributors JX drafted the manuscript and conducted data analyses. JX and VM conducted qualitative analyses on the survey responses. All other authors (AW, VL, NG, PJB, RWM and CR) contributed to revising the manuscript. JX is the guarantor for this work.
Funding This work was funded by the Centre for Population Health, NSW Ministry of Health.
Disclaimer The contents of this paper are solely the responsibility of the individual authors and do not reflect the view of NSW Ministry of Health.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.