Table 2

Quality assessment and risk of bias

Uncontrolled before and after evaluations
Study	Quality criteria	Risk of bias	Notes
Grant 2000 RCT	Sequence generation	Low	Sealed opaque envelopes prepared by research team. Stratification by practice and blocks of six used (three intervention/three control).
	Allocation concealment	Unclear	Sequentially numbered envelopes opened. In two practices, there was evidence that the randomisation process was initially misunderstood: six patients excluded.
	Blinding of participants and personal	Not possible
	Blinding of outcome assessment	Unclear
	Incomplete outcome data	High	32% loss to follow-up at 4 months
	Selective outcome reporting	Unclear
	Other potential threats to validity	Unclear	Numbers potentially eligible but not recruited unknown Recruited general practices were not a random sample: participating doctors were likely to be more interested in the research question and may have managed psychosocial problems more actively, which could have diminished reported estimates of effects
Maughan 2016 CBA	Is there a suitable comparison group?	Yes	One intervention and one control group, drawn from the same general practice with similar patient characteristics. Models environmental costs (in terms of carbon footprint) Data were retrospectively collected from GP health records for a 2-year period. Two participants in intervention group excluded from analysis Financial and environmental impacts calculated for each outcome using national averages or accepted conversion factors
	Do the authors use theory to underpin the project/evaluation?	No
	Were appropriate methods used for data collection and analysis?	Yes
	Were efforts made to assess patient experience?	No
Uncontrolled before and after evaluations
Study	Quality criteria	Risk of bias	Notes
Dayson 2014	Was the study question or objective clearly stated?	Yes	Small sample of those referred (N=1607) participated in evaluation—HES data at 6 months N=451, at 12 months N=108; well-being data at 3–4 months 280/819 Methods of qualitative analysis of patient experience unclear
	Were eligibility/selection criteria for the study population prespecified and clearly described?	Not reported
	Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest?	Yes
	Were all eligible participants that met the prespecified entry criteria enrolled?	Not reported
	Was the sample size sufficiently large to provide confidence in the findings?	No
	Was the test/service/intervention clearly described and delivered consistently across the study population?	Not reported
	Were the outcome measures prespecified, clearly defined, valid, reliable and assessed consistently across all study participants?	Yes
	Were the people assessing the outcomes blinded to the participants’ exposures/interventions?	Not reported
	Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis?	No
	Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p values for the pre-to-post changes?	Yes
	Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (ie, did they use an interrupted time-series design)?	No
	If the intervention was conducted at a group level (eg, a whole hospital, a community, etc), did the statistical analysis take into account the use of individual-level data to determine effects at the group level?	Not applicable
Friedli 2012	Was the study question or objective clearly stated?	Yes	Details of preintervention and postintervention outcomes not reported Small sample size Timing of post intervention assessment not reported Methods of qualitative analysis of patient and provider/referrer experience unclear
	Were eligibility/selection criteria for the study population prespecified and clearly described?	No
	Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest?	Yes
	Were all eligible participants that met the prespecified entry criteria enrolled?	Not applicable
	Was the sample size sufficiently large to provide confidence in the findings?	No
	Was the test/service/intervention clearly described and delivered consistently across the study population?	Not reported
	Were the outcome measures prespecified, clearly defined, valid, reliable and assessed consistently across all study participants?	No
	Were the people assessing the outcomes blinded to the participants’ exposures/interventions?	No
	Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis?	No
	Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p values for the pre-to-post changes?	No
	Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)?	No
	If the intervention was conducted at a group level (eg, a whole hospital, a community, etc), did the statistical analysis take into account the use of individual-level data to determine effects at the group level?	Not applicable
Grayer 2008	Was the study question or objective clearly stated?	Yes	GP practices volunteered and may not be representative of practices overall Patients who consented to participate in evaluation were more likely to speak English as a first language than those who did not consent No significant differences at baseline between those successfully followed up and those lost to follow-up 95% CIs (no p values) reported for changes in GHQ-12, CORE-OM and WSAS scores
	Were eligibility/selection criteria for the study population prespecified and clearly described?	Yes
	Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest?	Yes
	Were all eligible participants that met the prespecified entry criteria enrolled?	No
	Was the sample size sufficiently large to provide confidence in the findings?	No
	Was the test/service/intervention clearly described and delivered consistently across the study population?	Yes
	Were the outcome measures prespecified, clearly defined, valid, reliable and assessed consistently across all study participants?	Yes
	Were the people assessing the outcomes blinded to the participants’ exposures/interventions?	Not reported
	Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis?	No
	Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p values for the pre-to-post changes?	Yes
	Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)?	No
	If the intervention was conducted at a group level (eg. a whole hospital, a community, etc), did the statistical analysis take into account the use of individual-level data to determine effects at the group level?	Not applicable
Kimberlee 2014	Was the study question or objective clearly stated?	Yes	SROI analysis presents data for all baseline completers and the smaller percentage who were followed up; possible bias towards positive finding for intervention Unclear whether calculations of mean differences in scale scores used all baseline data or baseline data for follow-up completers only p values reported for change from baseline at 3 months in PHQ-9 depression scores
	Were eligibility/selection criteria for the study population prespecified and clearly described?	No
	Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest?	Yes
	Were all eligible participants that met the prespecified entry criteria enrolled?	Not applicable
	Was the sample size sufficiently large to provide confidence in the findings?	No
	Was the test/service/intervention clearly described and delivered consistently across the study population?	Not reported
	Were the outcome measures prespecified, clearly defined, valid, reliable and assessed consistently across all study participants?	Yes
	Were the people assessing the outcomes blinded to the participants’ exposures/interventions?	Not reported
	Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis?	No
	Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p values for the pre-to-post changes?	Yes
	Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)?	No
	If the intervention was conducted at a group level (eg. a whole hospital, a community, etc), did the statistical analysis take into account the use of individual-level data to determine effects at the group level?	Not applicable

CBA, Controlled Before and After study; CORE-OM, Clinical Outcomes in Routine Evaluation-Outcome Measure; GP, general practice; GHQ, General Health Questionnaire; HES, hospital episode statistics; PHQ, Patient Health Questionnaire; RCT, randomised controlled trial; SROI, Social Return on Investment; WSAS, Work and Social Adjustment Scale.