Article Text
PDF
Abstract
Objective To conduct an overview of systematic reviews to examine the effectiveness of vocational interventions to help adults with long-term health conditions or disability gain and maintain new paid work and to analyse the spread and quality of evidence in this area.
Methods We pre-published our protocol in PROSPERO (CRD42019132448). We searched Cochrane Database of Systematic Reviews, MEDLINE, EMBASE, PsycINFO, AMED, CINAHL, Proquest Dissertations and Theses database, NICE and Business Source Complete from inception to 21 August 2020. We included any systematic reviews of clinical trials on vocational interventions for adults with long-term health conditions or disability who were not in work or had recently gained work. We excluded reviews of vocational interventions for employed people on sick leave. Two researchers identified, critically appraised,using A MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2, and extracted data from included reviews. We used Grading of Recommendations Assessment, Development and Evaluation to evaluate strength of evidence underpinning overview findings.
Results We identified 26 reviews (5 high-quality and 21 critically low to moderate quality) that focused on vocational interventions for acquired brain injury, autism, intellectual disability, multiple sclerosis, mental health conditions, spinal cord injury and general disability populations. We identified moderate quality evidence that people with moderate to severe mental health conditions who participate in supported employment, particularly individual placement and support, are more likely to gain competitive employment compared with people who receive traditional vocational services (risk ratio 2.07; 95% CI 1.82 to 2.35; 27 studies, 6651 participants). We found only very low-quality to low-quality evidence on vocational intervention for people with any other health condition. We found little to no data on employer or employee satisfaction with work outcomes or the cost effectiveness of interventions.
Conclusion Given the importance placed on work opportunities for people with long-term health conditions or disability, there is urgent need for more high-quality research on vocational interventions for this population.
Prospero registration number CRD42019132448.
- occupational & industrial medicine
- mental health
- rehabilitation medicine
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. All data relevant to the study are included in the article or uploaded as supplementary information. The pre-published protocol for this overview is published here: http://hdl.handle.net/10292/12440.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The use of a detailed, prepublished protocol minimised the risk of bias arising from subjective decision-making during critical appraisal of included reviews, data extraction, data analysis and write up of this overview.
The comprehensive search strategy used in this overview minimised the risk of bias arising from accidental omission of systematic reviews relevant to the overview question.
This overview is limited to people with long-term health conditions or disability who are not in paid work or who recently gained paid work and excludes findings from research on helping people already in paid word return to their jobs after acquiring injuries, disabilities or long-term health conditions.
The quantity and quality of findings arising from this overview are limited to the available systematic reviews relevant to our overview question.
Aspects of Grading of Recommendations Assessment, Development and Evaluation (GRADE), the tool used for evaluation of strength of evidence, are open to some subjective interpretation, meaning that a different approach to the application of GRADE may have produced slightly different statements regarding the strength of evidence for some of the findings reported in this overview.
Introduction
People who experience long-term health conditions or disability often also have difficulty accessing paid work due to several complex factors. These factors are not limited just to impairments arising from injuries or illnesses, but are often associated with broader societal issues, for example, stigma, poorly informed judgements about what makes somebody work ready, and challenges around negotiating alternative work arrangements.1–5 Researchers and vocational service providers have developed a number of interventions designed to improve work outcomes for people experiencing long-term health conditions or disability.6 7 Some aspects of what is effective and relevant may be the same for people with different health conditions. However, particular conditions, cultural contexts or circumstances can affect what is most relevant to address for an individual.8–10
There already exists a number of systematic reviews of research investigating the effectiveness of strategies to support people living with a long-term health condition or disability gain and maintain paid work.11–14 What is not known is the quantity, quality and spread of these reviews. Also unknown are the similarities and differences in the conclusions that these reviews reach, and what might be possible to learn by examining evidence across different reviews. Therefore, we conducted an overview to synthesise systematic reviews of vocational interventions designed to help people living with long-term health conditions or disability gain and maintain paid work. Our overview questions were as follows:
What is the effectiveness of vocational interventions to help adults with long-term health conditions or disability gain and maintain paid work, in what contexts?
What is the quality of evidence on vocational interventions to help adults with long-term health conditions or disability gain and maintain paid work?
What evidence gaps exist in current systematic reviews on vocational interventions to help people with long-term health conditions or disability gain and maintain paid work?
Methods
We registered an overview protocol with PROSPERO (Reference number: CRD42019132448) and pre-published our methods.15 While methods for overviews of systematic reviews are still under development,16 we based our approach on recommendations from authors of overviews published in the Cochrane library,16 17 We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses18 to guide our reporting of this overview.
Patient and public involvement
At three key stages in the process—protocol development, considering transferability and external validity of the evidence profile and dissemination of findings—we consulted with four stakeholder reference groups. Three condition-specific stakeholder reference groups were made up of people with lived experience of varied long-term conditions (mental health conditions, amputation, multiple sclerosis) and people involved in related services and advocacy. We also consulted a specific Māori stakeholder reference group (Indigenous culture of Aotearoa New Zealand in which the review was undertaken) that included people with professional and personal experience across a variety of long-term conditions. The stakeholder reference groups also provided input into contextualising findings for the local context and peoples, and assisted with dissemination in the community including meetings, discussions, webinars and user-friendly summaries.
Selection criteria for included reviews
We included any systematic reviews on vocational interventions to assist adults (16 years or older) with long-term health conditions or disabilities, who were not in work or who had only recently been employed (eg, within 3 months of a new job placement), gain and maintain paid work. We only included reviews that reported on randomised controlled trials (RCTs), pseudo-RCTs, cross-over RCTs and non-RCTs (NRCTs). Broad categories of long-term conditions or disabilities considered in scope for this overview included: musculoskeletal, neurological, cardiac, respiratory, congenital and paediatric, sensory, endocrine and metabolic, skin and subcutaneous, or mental health conditions. Vocational interventions included: training or education to prepare people to apply for paid work; vocational counselling; support for people applying for jobs or actively seeking work; identification of barriers to employment and supports to address these barriers; job development; job placement schemes; support for people newly in paid work; job coaching and community development specifically focused on creating employment opportunities for people living with long-term conditions or disabilities. We excluded: training that was not intended to end in paid work; general education not specifically designed for people with long-term health conditions or disabilities; and return to work interventions for employed people on sick leave or with newly acquired health conditions or disabilities. We excluded reviews that did not report an explicit review objective or question, search strategy, inclusion criteria, quality assessment methods and data extraction plan. We excluded reviews that were not published in English as we were not resourced to translate reviews in other languages. (See online supplemental file, appendix A for more detail regarding the inclusion/exclusion criteria).
Supplemental material
Outcomes of interest
The primary outcomes of interest were (1) work placement in either full-time or part-time paid work, and (2) successful maintenance of new work. We considered paid work to be successfully maintained if paid work had been continuously sustained over a period of time (eg, 6 months) regardless of whether or not that work was in the same job. We considered any gap greater than 4 weeks between jobs to be non-continuous (ie, paid work was not successfully maintained. Secondary outcomes for this review were: (1) proportion in full-time paid work, (2) satisfaction of employers with employment outcomes or work performance, (3) satisfaction of participants with employment outcomes, (4) pay rates, (5) cost of vocational interventions, (6) cost of ongoing work support and (7) quality of life. We were interested in data on outcomes at three time points: the end of intervention, up to 1 year after the end of an intervention, and more than 1 year after end of an intervention.
Search strategy
We identified relevant reviews by searching: The Cochrane Database of Systematic Reviews (Wiley), MEDLINE (OvidSP), EMBASE (OvidSP), PsycINFO (OvidSP), AMED (OvidSP), CINAHL (EBSCOhost), Proquest Dissertations and Theses database, Evidence Search (NICE) and Business Source Complete. We search all databases from inception to 21 August 2020. (See online supplemental files, appendix B for full search strategies). We searched Cochrane Database of Systematic Reviews (Wiley) and Prospero to identify any ongoing or recently completed reviews. We contacted review authors for advice as to other relevant reviews and to seek full review reports of any potentially eligible reviews that were published as abstracts or conference proceedings only.
Selection of reviews
Two researchers independently considered the title and abstracts from the identified reviews for inclusion. Disagreements were resolved through discussion of full review reports, involving a third researcher where necessary. We accessed relevant prepublication information (eg, published protocols) where possible.
Data extraction
Two researchers independently extracted data. We resolved disagreements by discussion, with assistance from a third researcher where necessary. We used an electronic, online data collection form specifically designed and piloted by the overview team. We extracted data on review methodology and characteristics, and data on review findings. For review methodological and content characteristics, we extracted data on: publication details (eg, journal, year of publication, country of origin, authors), methodological characteristics (eg, methods used for study identification, data extraction and data analysis; existence of a prepublished protocol; date of last search), content characteristics (type of studies included; review question components—population, intervention(s), comparison(s), primary and secondary outcomes; setting; number of included studies; type and number of analyses conducted; methods used by review authors for grading risk of bias and evidence).
For review findings, we extracted data only on specific outcomes relevant to the objectives of this review, but where relevant we collected data on the number of studies and participants contributing to each outcome presented, the intervention effects as presented by the review authors (risk ratios (RRs), ORs or mean differences (MDs), or standardised MDs; with 95% CIs), measures of heterogeneity (eg, I2 statistics)) or narrative text of results if no statistical results were presented. We also collected data on the population behind each outcome (eg, age, severity, time postinjury), the intervention behind the outcome as stated by the review authors, the comparison condition, and any reasons the review authors gave for grading the risk of bias or quality of evidence underpinning their findings.
Evaluation of the quality of included reviews
Two researchers independently assessed the methodological quality of each review using AMSTAR-2.19 Disagreements were resolved through discussion or involvement of a third researcher. We did not reassess the quality of studies included within reviews but instead report study quality according to review authors’ assessments.
Data synthesis
Using a spreadsheet, we collated information on the studies included within all identified reviews to explore where reviews cover the same studies. When overlap between reviews were identified, all overview authors discussed the overlap with consideration of each review question and comparisons explored, the date of the last search and key aspects of methodological quality (eg, types of studies included, risk of bias assessment). We used these details to reach an agreement regarding which data from which review comparisons should be included within the overview.
We reported findings from the overview descriptively. We did not reanalyse data from primary studies. However, for the purposes of this paper, where possible we convert ORs and risk differences into RRs, so we could report findings across reviews on a common metric. We analysed publication trends, the range of review topics (ie, types of populations, types of interventions, and their contexts), the range of outcomes reported, and the quality of included reviews. We synthesised information on common problems with quality of reporting, quality of methodology and quality of evidence.
We scored the quality of the evidence for all quantitatively reported intervention effects using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach,20 which considers risk of bias, imprecision, inconsistency, indirectness, and other factors (such as publication bias). To improve objectivity with GRADE scores, we used criteria for grading evidence adapted from that used by Pollock et al21
We organised review findings by type of health condition. We collated a summary of intervention effectiveness according to a framework adapted from Shepherd et al22 and Farquhar and Marjoribanks,23 reporting on: effective interventions (high-quality evidence of effectiveness for an intervention), promising interventions (moderate-quality evidence of effectiveness for an intervention, but more evidence is needed), ineffective interventions (quality evidence of lack of effectiveness for an intervention), probably ineffective interventions (quality evidence suggesting lack of effectiveness for an intervention, but more evidence is needed), and where no conclusions were possible (low-quality or very low-quality evidence, or insufficient evidence to comment on the effectiveness of an intervention). We based our categorisation on our GRADE assessments for quality of evidence. We examined the spread of evidence across the overview to identify any large evidence gaps that exist for particular populations, types of interventions or types of outcomes relevant to this review.
Results
Results of the search
Our search identified 3448 titles after removing duplicates. We excluded 3350 on screening of titles and abstracts, and a further 72 on full-text screening, leaving 26 reviews for data extraction (see figure 1). Reasons for excluding each article based on full text screening are given in online supplemental file, appendix C.
PRISMA flow diagram for review selection. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Description of included reviews
Of the 26 reviews, 7 included just RCTs24–30 and 19 included RCTs plus other type of studies.11 14 31–47 Four of the reviews had been published by Cochrane Review groups.26 28 30 36 Eleven reviews focused on severe mental illness24–26 28–32 34 38 40—with 1 of these 11 also including studies on moderate mental illness.24 Three focused on autism spectrum disorder,11 45 47 two on acquired brain injury,27 39 two on multiple sclerosis,36 44 one on spinal cord injury,43 one on intellectual disability41 and six focused on studies of vocational interventions for any kind of health condition or disability.14 33 35 37 42 46
The majority of reviews (19/26) assessed the effectiveness of a number of different types of intervention to facilitate employment, for example, sheltered workshops; interventions to target job skills; work placement interventions; and supported employment programmes.11 14 25–27 30 33 35–37 39–47 Six reviews focused solely on supported employment programmes.24 28 29 32 34 38 One review focused on the effectiveness of computer-assisted cognitive remediation for people with mental health conditions as an adjunct to using vocational services.31
The definition of different types of vocational intervention varied. Supported employment, the most commonly described intervention in included reviews, can be broadly defined as programmes that help people ‘obtain competitive employment quickly… provid[ing] them with ongoing support to maintain employment’.30 For people living with mental health conditions, one specific approach to this is called individual placement and support (IPS). IPS is a well defined, principle-based approach that involves: (1) low caseloads for IPS providers (20 or fewer); (2) no exclusion criteria—working with any client who wants a job; (3) integration with mental health treatment; (4) a focus on gaining competitive employment, (5) integration with benefits counselling and other work incentives planning, (6) personalising job searches to individual interests and skills; (7) rapid job search for competitive employment; (8) engagement of employment specialists in job development on behalf of clients and (9) personalised, follow-along support for as long as required.24 29 30 32 High fidelity IPS is IPS programmes with measurable, reported compliance with these principles. Sometimes IPS is combined with other interventions, such as cognitive training, in which case it is called augmented IPS.30
The reviews included an average of 25.5 studies each (SD 23.3 studies; range 2–89 studies), but only an average of 10.0 studies per review (SD 11.6 studies; range 1–48 studies) were relevant to the objectives of this overview. Insufficient information was provided in four reviews to be clear about how many of the identified studies were relevant to this overview.14 33 42 44 Irrelevant studies included those that focused on employed workers returning to work or maintaining employment (ie, not newly gained employment), or involved methods that were not RCTs, cluster RCTs, pseudo-RCTs or NRCTs.
Of the reviews that included RCTs, each review identified an average of 11.8 RCTs (SD 12.6 RCTs; range 1–48). Six reviews in the field of mental health reported on at least one meta-analysis of findings from included studies. One additional review involving people with any type of health condition included meta-analyses on work outcomes but insufficient information was provided to judge whether all studies included in the meta-analyses were relevant to the objectives of this overview.33 Meta-analyses were not possible in the other reviews due to insufficient numbers of studies or fundamental differences in research aims or designs. More details on the characteristics of included reviews are reported in online supplemental file, appendix D.
Methodological quality of included reviews
Key elements (preselected in the overview protocol) of the AMSTAR ratings, plus summary judgements of the overall confidence in each review, are presented in table 1. Full AMSTAR ratings are provided in online supplemental file, appendix E. Overall, we had high confidence in three reviews that focused on people with mental health conditions,26 28 30 one review about people with traumatic brain injury,27 and one review about people with any long-term health condition living in low-income to moderate-income countries,46 but critically low to moderate confidence in the remainder of reviews. Common problems that reduced confidence in reviews included the lack of prepublished protocols, lack of transparency around how studies were selected and decisions to exclude studies, limitations in how risk of bias was evaluated and how this information was incorporated in the interpretation of review findings.
Overview authors’ judgements about preselected AMSTAR-2 review quality items, with reviews grouped by health condition
Effect of interventions for specific populations
We report the main results below, organising these results by health condition and strength of evidence. A summary of the main findings from this overview is presented in table 2. An extended version of table 2, which includes the rationale for GRADE scores, is included in online supplemental file, appendix F.
Summary of findings on employment interventions for various health conditions
Acquired brain injury
Of the two reviews that focused on people with acquired brain injury, one was both the most recent (updated November 2015) and of high methodological quality.27
No conclusions possible: very low-quality evidence
Very low-quality evidence from three small RCTs suggested no clear differences in gaining paid work for people with acquired brain injury participating in (1) artificially intelligent virtual reality-based training in comparison to psycho-educational training (RR 2.10; 95% CI 0.75 to 5.59; 1 study, 50 participants), (2) supported employment with cognitive symptom management in comparison to supported employment alone (RR 1.00; 95% CI 0.62 to 1.61; 1 study, 50 participants) and (3) inpatient interdisciplinary neurorehabilitation and vocational interventions in comparison to home-based rehabilitation (RR 0.95; 95% CI 0.85 to 1.05, 1 study; 120 participants).27
Autism spectrum disorder
Of the three reviews that focused on people with autism spectrum disorder, two were of moderate quality but the least current (updated in December 201145 and December 2008.47 The most recent review (updated October 2015) was of critically low quality.11
No conclusions possible: low-quality evidence
Low-quality evidence from one small NRCT suggested that people with autism may be more successful gaining paid work if they participate in supported employment compared with people with autism who do not (RR 2.53; 95% CI 1.13 to 5.67; 1 study, 50 participants), and that they may be more likely to earn higher wages per hour and work for longer periods of time, but not work more hours per week.45 47 Very low-quality evidence from one other NRCT suggested that people with autism who participate in supported employment may have higher quality of life compared with people who participate in sheltered work.45 47 However, the size or significance of this effect on quality of life was not reported.
Intellectual disability
One review of critically low quality, last updated in February 2019, focused on people with intellectual disabilities.41
No conclusions possible: very low-quality evidence
Very low-quality evidence from one small RCT suggested no difference in gaining paid work for people with intellectual disability participating in supported employment compared with those receiving no support (RR 11.44; 95% CI 0.67 to 196.30; 1 study, 49 participants).41
Multiple sclerosis
Of the two reviews that focused on people with multiple sclerosis, one was both the most recent (updated February 2011) and of higher methodological quality.36 Nonetheless, methodological limitations in this review reduced our confidence in its findings to low.
No conclusions possible: very low-quality evidence
Very low-quality evidence from one small NRCT suggested no clear difference in gaining paid work for people with multiple sclerosis participating in a structured vocational intervention programme compared with minimal telephone contact plus written information (RR 1.07; 95% CI 0.38 to 2.99; 1 study, 47 participants).36
Severe mental illness
Of the 11 reviews on vocational interventions for people with severe mental illness, the second most recently updated review (updated November 2016) was the review of highest quality.30 This review involved a network meta-analysis that compared a range of different types of vocational interventions for people with severe mental illness across multiple studies. The most recent review (updated September 2019) was of moderate quality, included studies involving people with moderate as well as severe mental health conditions, but focused on one type of vocational intervention—IPS programmes with demonstrated moderate to high programme fidelity.24 One additional review (low quality; last updated September 2014) investigated the effectiveness of computer-assisted cognitive remediation as an augmentation for people with schizophrenia who were participating in any type of vocational intervention.31
Promising interventions: more evidence needed
Moderate-quality evidence suggests that people with moderate to severe mental health conditions who participate in IPS are more likely to gain paid work (specifically in competitive employment) compared with people who receive traditional vocational services or treatment as usual (RR 2.07; 95% CI 1.82 to 2.35; 27 studies, 6651 participants).24 Subgroup analysis of the studies contributing to this finding indicated that the effectiveness of IPS remains stable regardless of country-specific work or employment policies, practices or conditions including regulation of temporary employment, types of disability benefits, national economic wealth, unemployment rates or employment rate of people with low education. The only factor identified to moderate the effect of IPS was the presence of strong legal protection against dismissal of employees, which appeared to have a modest negative impact on the effectiveness of IPS.24
Moderate quality evidence also suggested that people with severe mental health conditions who participate in augmented supported employment are more likely to gain paid work compared with people who receive standard psychiatric care (RR 3.81; 95% CI 1.99 to 7.31; 1 study, 256 participants) and that people with severe mental health conditions who participated in supported employment were more likely to gain paid work in competitive employment compared with people who participate in transitional employment—a practice that involves segregated employment of people with disabilities who are deemed not yet ready to enter competitive employment (RR 2.71; 95% CI 1.80 to 4.06; 4 studies, 87 participants).
No conclusions possible: low-quality to very low-quality evidence
Low-quality evidence suggested that augmented supported employment is better than transitional employment (RR 3.79; 95% CI 2.34 to 6.14; 2 studies, 212 participants) and better than prevocational training (RR 3.02; 95% CI 1.88 to 4.87; 2 studies, 193 participants) for assisting people with severe mental health conditions gain employment.30 Low-quality to very low-quality evidence also suggested that supported employment (not augmented) is better than psychiatric care (RR 2.72; 95% CI 1.55 to 4.76; 1 study, 2238 participants) and better than prevocational training (RR 2.16, 95% CI 1.59 to 2.93; 9 studies, 1569 participants) for assisting people with severe mental health conditions gain employment.30
Low-quality evidence suggested that there is no benefit of providing augmented support employment over non-augmented support employment in terms of helping people with severe mental health conditions gain paid work (RR 1.40; 95% CI 0.92 to 2.14; 3 studies, 205 participants).30 However, low-quality evidence also suggested that cognitive remediation interventions as a supplement to any type of vocational intervention may help people with severe mental health conditions gain paid work (RR 1.66; 95% CI 1.00 to 2.74; 6 studies, 417 participants).31
Spinal cord injury
One review of critically low quality, last updated in January 2014, focused on people with spinal cord injury.43
No conclusions possible: low-quality evidence
Low-quality evidence from one RCT suggested that people with spinal cord injury who participate in supported employment programmes may be more likely to gain paid work compared with people who receive treatment as usual from spinal cord injury rehabilitation services (RR 2.46; 95% CI 1.16 to 5.22; 1 study, 157 participants)
Additional subgroup analyses
Of the six reviews that involved people with any kind of health condition,14 33 35 37 42 46 three focused on the effectiveness of vocational interventions for specific subgroups: (1) young adults with disabilities,35 (2) adults with childhood onset disabilities37 and (3) people with disabilities in low-income and middle-income countries.46 No meta-analyses were conducted on employment outcomes for any of these subgroups. One moderate-quality review reported that young people with any type of disability who participate in a ‘youth transition demonstration enhanced employment service’ were more likely to report employment in any job and have significantly greater income over 3 years than young people with disabilities who did not (1 RCT, 799 participants; RR not reported).35 Another high-quality review reported no difference in employment outcomes for people with any type of disability in a low income country participating in a multicomponent vocational intervention compared with those who did not receive this intervention (1 NRCT, 48 participants; RR not reported).46 No further quantitative findings were described in these six reviews that altered the conclusions regarding the effectiveness of vocational intervention by health condition type reported above.
Discussion
This overview provides the most comprehensive summary to date of the best available evidence regarding the effectiveness of vocational interventions to help people with long-term health conditions or disability gain and maintain paid employment. Across all health conditions and disabilities, the most robust evidence is around the use of supported employment, particularly IPS, to help people with severe mental health conditions gain employment. IPS can double the likelihood of people with severe mental illness gaining paid work, with this effect being demonstrated across a wide range of different countries and economies. In comparison however, very little high-quality research on vocational interventions has been conducted involving any other type of disability or long-term health condition. This is surprising given that governments internationally have been investing in these programmes for over 100 years,48 and given the current focus on vocational rehabilitation and employment as central to disability policy.49–52
It is possible that this overview failed to identify important evidence because the relevant systematic reviews have not yet been conducted. In order to test this assumption, and to fill in some of these possible gaps in the evidence subsequent to this overview, we undertook three additional systematic reviews on vocational interventions for people living with: (1) mild to moderate mental health conditions, (2) amputation and (3) progressive neurological conditions, and a fourth systematic review on vocational intervention for Indigenous people with long-term conditions living in a postcolonial context.53 However, this revealed no further evidence relevant to this overview, except in the case of research involving people with mild to moderate mental health conditions, where we found low-quality evidence supporting the use of modified IPS approaches to help people gain paid work (RR 1.70; 95% CI 1.23 to 2.34).54 There is therefore urgent need for further well-conduct experimental trials of vocational interventions for people living with long-term health conditions or disability.
Future research studies need to address the common methodological limitations identified in the systematic reviews that contributed to this overview. Furthermore, beyond outcomes related to rates of paid work, future studies also ought to report on successful maintenance of new work over periods of time (ie, how long participants stay in any work following initial placement), satisfaction of employers and employees with employment outcomes and work performance, pay rates and the cost-effectiveness of vocational interventions and ongoing work support.
This overview also provides guidance for authors of future systematic reviews in this area. We strongly recommend that any future review begins with a prepublished protocol and clear review question; includes explicit information on search strategies used; reports transparently on how decisions were made to include or exclude studies; incorporate critical appraisal of evidence into statements regarding certainty in the review findings; uses GRADE to evaluate strength of evidence20; and reports on all review findings using standardised methods (eg, MDs, RRs or ORs with 95% CIs).55 We also recommend that when undertaking systematic reviews and studies of vocational intervention that a clear distinction is made between studies involving people who are returning to prior employment and those involving people who are unemployed and seeking new employment, since these two situations comprise quite different needs and challenges.56
Regarding the quality of this overview, we conducted systematic searches based on a comprehensive search strategy. We have a high degree of confidence that it is unlikely that we missed important systematic reviews that may have substantively changed the conclusions of this overview. We also have used standardised, reportable processes for review selection, appraisal and data extraction. However, a slightly different interpretation of the overview findings could have resulted from a different approach to addressing overlap between included reviews or from application of GRADE to the overview findings. For instance, some aspects of GRADE, such as deciding when methodological elements are sufficiently concerning to downgrade a finding’s GRADE score, are open to a degree of subjective interpretation. However, to address this we have endeavoured to report as much as possible the criteria we used to upgrade or downgrade evidence for each finding in this overview (eg, see the comments’ section of the table in the online supplementary file, appendix F).
Conclusion
Moderate quality evidence supports the use of IPS to help people with moderate to severe mental health conditions gain new work. There is a paucity of evidence, however, regarding the effectiveness of vocational interventions for any other population of people with long-term health conditions or disability who are out of work. There is, therefore, an urgent need for further research in this area to support evidence-based policy, practice and investment in vocational intervention for this population.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. All data relevant to the study are included in the article or uploaded as supplementary information. The pre-published protocol for this overview is published here: http://hdl.handle.net/10292/12440.
Ethics statements
Patient consent for publication
Acknowledgments
Thank you to the leads of the stakeholder reference groups who contributed to this work: Dr Helen Lockett, Sean Gray, Dr Matire Harwood, Neil Woodhams, and to the contributors in the stakeholder reference group meetings.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors WMML and JKF contributed equally to review methods design, search strategy, study selection, data extraction, data analysis, and write up. David Anstiss contributed as a third reviewer for study selection, critical appraisal and data extraction. WMML wrote the first draft of this manuscript and is the guarantor of this work.
Funding WMML and JKF report receiving a grant from New Zealand Ministry of Social Development and Health Research Council (partnership grant number 18/804) in order to conduct of this overview.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.