Article Text
PDF
Abstract
Objectives To assess the feasibility of conducting a cost-effectiveness study of using care farms (CFs) to improve quality of life and reduce reoffending among offenders undertaking community orders (COs). To pilot questionnaires to assess quality of life, connection to nature, lifestyle behaviours, health and social-care use. To assess recruitment and retention at 6 months and feasibility of data linkage to Police National Computer (PNC) reconvictions data and data held by probation services.
Design Pilot study using questionnaires to assess quality of life, individually linked to police and probation data.
Setting The pilot study was conducted in three probation service regions in England. Each site included a CF and at least one comparator CO project. CFs are working farms used with a range of clients, including offenders, for therapeutic purposes. The three CFs included one aquaponics and horticulture social enterprise, a religious charity focusing on horticulture and a family-run cattle farm. Comparator projects included sorting secondhand clothes and activities to address alcohol misuse and anger management.
Participants We recruited 134 adults (over 18) serving COs in England, 29% female.
Results 52% of participants completed follow-up questionnaires. Privatisation of UK probation trusts in 2014 negatively impacted on recruitment and retention. Linkage to PNC data was a more successful means of follow-up, with 90% consenting to access their probation and PNC data. Collection of health and social-care costs and quality-adjusted life year derivation were feasible. Propensity score adjustment provided a viable comparison method despite differences between comparators. We found worse health and higher reoffending risk among CF participants due to allocation of challenging offenders to CFs, making risk of reoffending a confounder.
Conclusions Recruitment would be feasible in a more stable probation environment. Follow-up was challenging; however, assessing reconvictions from PNC data is feasible and a potential primary outcome for future studies.
- mental health
- substance misuse
- public health
- social medicine
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
A strength of the study was being able to link to reconvictions data from the Police National Computer for 90% of participants.
A further strength is the use of propensity weights which provide a robust way to deal with the differences between care farm (CF) and comparator sites.
A limitation was that, due to significant changes within probation services in England, we were only able to complete follow-up questionnaires at 6 months for 52% of our participants.
The study has costed the use of CFs by probation services providing valuable new knowledge. A limitation, again due to changes within probation services, was our inability to collect cost data from more than one study region.
Our study was designed to test feasibility and pilot methods, not to test the effectiveness of CFs in improving quality of life and reducing offending. A fully powered natural experiment is recommended.
Introduction
Care farming (also known as social farming) is the therapeutic use of farming practices.1 Care farming is a truly complex intervention.2 Care farms (CFs) differ in the type of farming activities (eg, horticulture and livestock farming), other activities (eg, gardening, conservation and woodwork), the level of support provided (eg, health promotion, counselling, skills qualifications) and the range of service-user groups. The number of CFs has been growing across Europe3 4 and there are now over 250 CFs in the UK.5 The evidence base for the effectiveness of care farming is relatively recent and a mixed-methods systematic review6 found no studies designed and powered to detect effectiveness.
Offenders serving community orders (COs) are an important user group for CFs in the UK; 27% of CFs in England were working with probation in 2012.5 In England, there is a policy emphasis on the use of COs which serve to avoid prison sentences, offer rehabilitation where needed and provide a means of ‘paying back’ the local community. COs may be spent on a CF or other location such as picking litter, furniture restoration or helping in a charity shop. A randomised controlled trial (RCT) design with service users randomly allocated to CF or comparator would not be acceptable within probation services as CO allocation is often on a needs basis. The challenges of using an RCT design to assess CF effectiveness are also identified by Hine et al.7 Instead, we piloted a natural experiment to identify cost-effectiveness of CFs compared with other CO sites in improving quality of life and reducing recidivism.8 Our study aimed to (1) explore the use of CFs by probation services, (2) assess feasibility of recruitment and retention following 6 months on CO, (3) pilot questionnaires and (4) assess feasibility of participants consenting to, and then, linking routinely collected individual reconviction data with questionnaire and probation data.
Methods
Setting
The study was conducted in three sites. Each site was a probation service region in England and included a CF and at least one comparator CO project. Our participants were adult probationers (18 years and over) serving a CO. The three sites demonstrated the range in types of CFs with one social enterprise specialising in aquaponics, horticulture and skills building (CF1); a religious charity with emphasis on horticulture (CF2) and one family-run cattle farm focusing on rehabilitation (CF3). Probation services used CF 1 and CF 2 as unpaid hours COs (a form of payback to the community) and in site 3, care farming was used as a rehabilitative order for those with social and employment needs.
Identifying suitable comparator sites was challenging. Comparator users in site 1 were allocated to a charity warehouse sorting secondhand clothes and in site 3 comparator users attended a range of specified activity requirements, including addressing alcohol misuse, domestic violence, anger management and drink-driving. Unlike the other sites, users at site 2 were allocated to multiple comparator projects during their CO. These included maintenance of local authority parks, painting railings, litter picking, decorating a children’s centre. Projects at site 2 would change depending on the completion and cyclical nature of the work so that some were one-off projects and others rolling.
Outcomes and measurements
The primary outcome was quality of life measured by the Clinical Outcome in Routine Evaluation—Outcome Measure (CORE-OM).9 Secondary outcomes included mental well-being measured using the Warwick-Edinburgh Mental Well-being Scale (WEMWBS)10; smoking, alcohol, drug use, diet and physical activity adapted from General Lifestyle Survey and Health Survey of England11; relationship with nature through two measures12 13 and tailor-made social-care and health-resource use. Questionnaires were delivered face to face wherever possible by a researcher at the beginning of users’ CO and at 6 months. Where we could not conduct face-to-face follow-up (because offender attendance was unpredictable), several methods were used to encourage questionnaire completion by post including short messaging service messages and an incentive of a £10 food-shopping voucher.
Participants were also asked to consent to our accessing their probation service data and police records providing Offender Group Reconviction Scale (OGRS) scores—a predictor of reoffending combining age, gender and criminal history14 and reconviction data, respectively. Offences and reconvictions at 6 months or longer (up to 18 months) following the start of their CO were collected.
Statistical analysis
Differences in participants’ characteristics at baseline between CO allocation and completion of follow-up questionnaire were summarised and tested using regression models, depending on the characteristic (median with IQR and median regression for continuous non-normal characteristics; numbers with percentages and Fisher’s exact test for categorical characteristics; and median with IQR and negative binomial regression for counts). To impute missing data, we used switching regression, an iterative multivariable regression technique which retains an element of random variation in the estimates.
The differences in reconviction at 18 months by CO allocation were explored using Cox proportional hazard regression models that account for variable length of follow-up. The estimates were shown unadjusted and adjusted for differences in characteristics between the two CO allocation groups that might also account for any differences in the risk of reconviction. Adjustment for ‘confounding by indication’ was carried out by applying stabilised inverse-probability treatment weights (IPTWs).15 16
The potential confounders to include in the model to calculate the IPTW were identified from Cox proportional hazard regression and logistic regression models, where confounders were characteristics associated with both reconviction and CO allocation or reconviction only.
Economic evaluation
We evaluated the feasibility of an economic evaluation of CFs from health and social-care perspectives. This included evaluating costs incurred in intervention provision, as well as healthcare resource and social services use as reported by users during follow-up (number of health services visits, number of social services visits, use of hospital services and medication use).
CORE-OM can be used to measure a utility score17 and derive quality-adjusted life years (QALYs), which is the preferred outcome for a cost-effectiveness analysis.18
The study also included a qualitative study exploring the perceptions of probation services, care farmers and offenders on the role of CFs in probation and a systematic review of the effectiveness and possible change mechanisms for any client group at a CF.6
Results
Feasibility of recruitment
We recruited 134 participants over a 12-month period (March 2013 to March 2014). Only 13% (21) of the probation service users approached declined to participate (see figure 1). Of the 134 participants 37% were allocated to CFs and the remainder to comparator locations.
Recruitment and retention within the study. CF, care farm; CO, community order.
Two major events impacted on recruitment: the privatisation of probation services in England and the closure of one farm site. In June 2014, probation trusts were disbanded and services were split between regional privately owned Community Rehabilitation Companies (CRCs) and a publicly owned National Probation Service (NPS). Decisions regarding where an offender is managed (either CRC or NPS) and the types of activities undertaken will depend on their risk of harm and offending history.
Operational differences in each probation service necessitated bespoke recruitment strategies. Factors which aided recruitment and data collection included: having a research assistant seconded from probation services; one of the study coinvestigators working at a senior level within a probation service; incentivising users by allowing time spent with the researcher to count towards their unpaid hours; including users with multiple requirement orders; recruiting at weekends and during the week.
Sample characteristics
The characteristics of the participants at CFs and comparators are presented in table 1.
Baseline characteristics by CO allocation
At baseline, we found significant differences between users allocated to CFs and comparators in terms of gender (4% CF users were female compared with 44% at comparator sites); risk of reoffending scores (OGRS) were 26 points higher (95% CI 6.86 to 45.14) among CF users; substance use and smoking were 47% and 78%, respectively, among CF users, 24% and 57% among comparators; comparators found healthy foods preferable. CF users had 139% (95% CI 21% to 370%) more missing CORE-OM questions.
Follow-up using questionnaires
We were able to follow up 52% of participants. This relatively low retention reflects the challenges of conducting face-to-face questionnaires at the end of participants’ COs and the loss of participants following the unexpected closure of CF1. Ultimately, 14 out of the 51 ‘lost’ participants (27%) returned their questionnaires in the post following the end of their CO.
We found significant differences between those followed up and lost. Those followed up were more likely to be NPS rather than CRC, older, non-smokers, used fewer substances and less health services than those not followed up (see table 2).
Baseline characteristics by follow-up status
Feasibility of data collection using questionnaires
Questionnaires worked well, the WEMWBS was the most easily understood with 10% of participants requiring assistance with questions, compared with CORE-OM, nature connectedness and relatedness scales (14%) and health and social-care use (15%).
Feasibility of linkage to routine Police National Computer reconvictions data
Participants consented to, and we were able to access and link, probation service and reconviction data for 90% of respondents. Police National Computer (PNC) numbers for each participant were used to request reconviction data from the Ministry of Justice (MoJ). Ten per cent of our participants could not be found on the MoJ system despite all unique PNC identifiers being checked. Gaining access to reconvictions data required persistence. The time taken to develop the agreements for all three sites was substantial and should not be underestimated in any future study.
Economic evaluation
Deriving QALYs
We transformed the CORE-OM scores into six-dimensional Clinical Outcome in Routine Evaluation (CORE-6D), a utility measure.16 Utility represents users’ overall quality of life and was multiplied by the time spent in each state to generate QALYs. One QALY is the equivalent of 1 year of full health. As expected in a feasibility study with a small number of complete cases (n=18 in CFs and n=35 in comparators), no significant difference was found in the mean CORE-6D index score at baseline and 6-month follow-up between the CF (mean 0.835 (SD 0.118)) and comparator arms 0.849 (SD 0.122), difference p value of t-test 0.679 (see table 3).
Mean CORE-6D index score at baseline and 6-month follow-up by study arm
Measurement of costs
The total cost of CF and comparator site was calculated combining the reported resource usage at follow-up and unit cost data along with the intervention cost.
Collecting cost data from all three study sites was particularly challenging due to the disbanding of probation services and subsequent impact on contracts with CFs. Data were collected from only one centre (centre 1) in the study as a pragmatic solution. The intervention cost was based on interviews with the Business Manager with responsibility for COs and neither of the sites (CFs or comparator) were contacted to get further details of the activities and their running costs. We estimated the costs incurred in the provision of the COs. We collected from sites the number of hours that probation service users were supervised. The annual number of supervised hours was then multiplied by the estimated hourly rate; in CFs the hourly rate was reported by the probation services to be £3.10 while it was £3.17 in the comparator settings.
We also collected travel expenses, while offenders have to meet the first £2.20 of their travel costs, the Business Manager informed us that the probation trusts reimburse the difference on production of a bus ticket via petty cash and this was estimated to equal an average of £1 per reporting occasion. The CF site reported 2820 occasions while the comparator site reported 2260 occasions. The total annual cost per site was therefore £65 466 in the comparator site and £51 234 in the CF; the comparator CO was therefore £14 232 more expensive than the CF. Reported health and social-care usage were converted into costs using unit cost figures from Personal Social Service Research Unit’s Unit Costs of Health and Social Care 201519 and the British National Formulary.20 The total health and social services resource use costs in the past month averaged over the two collection points were £93 for CO users and £33.5 for CF users and were significantly different at the 5% level. Total medication costs were marginally higher in the CF sample (£5.5 vs £3), however, the difference was not significant. When including the intervention cost in the mean total cost of healthcare resource use, total costs over the last month were marginally higher in the comparator (£95.7 vs £67.2), see table 4.
Average costs of resources used within the past months by study arm
Identifying confounders
To inform future studies we identified factors associated with allocation to a CF and also associated with the outcome. Given the more complete follow-up data for reconvictions, this was used to identify confounders and inclusion in the IPTW (see table 5).
Potential variables to be included in IPTW
The following five variables were found to be significantly associated with both allocation to a CF and with reconvictions within 18 months: (1) probation type (ie, NPS or CRC) at baseline; (2) substance use in the 4 weeks before baseline; (3) smoker at baseline; (4) risk of reoffending (OGRS) at baseline and (5) agreement with ‘healthy foods are enjoyable’ statement (see table 5). The OGRS proved to be a valuable composite measure for adjusting for the differences between those allocated to CFs and comparators. Probation type was found to be only associated with reconviction within 18 months and not allocation to CFs, and so identified for inclusion in the IPTW.
The unadjusted risk of reconviction within 18 months is over three times higher (95% CI 1.58 to 5.96) in those allocated to CFs compared with those allocated to a comparator. On adjustment for differences in the CO allocation groups, there is no significant difference in the risk of reconviction between the CF and comparator groups (HR 1.50, 95% CI 0.64 to 3.53). This would indicate that any difference in reconviction between the two groups is as a result of differences in the characteristics of those allocated to a CF and those allocated to other COs.
Discussion
Main findings of the study
Despite the challenges due to changes in probation services, we were able to recruit sufficient numbers of participants to assess feasibility of recruitment, follow-up and linkage with PNC data. With only 14% declining to participate and a low level of missing data on questionnaire outcomes, our results show that offenders serving COs are prepared to participate and provide data in this type of study, particularly where their time counts towards a reduction in their ‘unpaid hours’. We found significantly worse health and risk of reoffending among those at CFs, reflecting use of CFs by probation to manage challenging offenders.
Only half of our participant groups were followed up. In future studies, this may introduce bias as healthy participants are more likely to complete their follow-up questionnaire. However, using individual reconviction data enabled us to assess reconviction outcome data for 90% of participants. Reconviction is the main performance indicator of new CRCs,21 so this would be a meaningful primary outcome for future studies. Evidence shows a strong association between reducing reoffending and improving health,22 indicating that this outcome measure is also of great relevance to public health commissioners.
Identifying ‘comparable’ users was challenging, with the comparator in site 3 taking users with very different specified requirements (eg, to counteract drink-driving) compared with those at the CF. Given continued and even increased diversity of approaches within the new CRCs,21 23 these differences are unlikely to lessen. While these issues of comparability clearly undermine a standard controlled-study design, our statistical methods, particularly use of propensity weights, present a robust way to deal with these differences, allowing comparison between sites with participants with different characteristics.
The detailed exploration of the activities and approaches used on each of the CFs highlighted the significant differences. Our recent review of CFs for different groups6 highlights how some activities and approaches may be more beneficial for some types of clients than others. With this in mind it is important that any future study of the effectiveness of CFs considers the different components of the CF intervention. Such points of difference include whether users work with animals, horticulture or other forms of husbandry such as aquaponics; whether vocational training is offered; the extent of team working and social interaction and the role of the care farmer and staff. These factors need to be clearly documented and their potentially differing impacts explored in future studies.
The weights used in the adjustment were calculated based on relatively few variables (with OGRS making the largest contribution), further supporting feasibility of data collection to adjust appropriately for differences between CO allocations. Furthermore, these adjustment methods can also be employed to adjust for lost to follow-up and used in future studies assessing outcomes measured by self-report questionnaire.
Taking into consideration the statistical methods employed, this analysis illustrates the feasibility of comparing outcomes between CO allocation groups with robust and valid adjustment for factors that drive the decision of CO allocation. The use of propensity weight adjustment provides a viable method for comparison despite differences in participants at CFs and comparator sites. These methods are becoming more common, particularly in evaluation of complex interventions where it is not possible or appropriate to randomise.
From the economic evaluation, we collected healthcare resource use in self-reported questionnaires and generate QALYs. This demonstrated that all key variables could be measured to undertake a full cost–utility analysis in a larger study. However, we were limited in costing interventions as we only collected data information from one Probation Trust. In a larger study, it would be advisable to collect full details of all CO activities and the cost for sites to run them.
A major challenge to conducting a future definitive study is the limited number of CFs that are currently contracted to receive probation service users on COs. A recent parliamentary assessment of the MoJ’s ‘Transforming Rehabilitation’ programme emphasises the need for MoJ to overcome barriers to innovative practice in rehabilitating offenders and to realise the ‘full potential of the third sector’ (p6–7).23 These recommendations bode well for CFs. As third sector organisations they are able to provide innovative solutions with potential to reduce recidivism. It is also worth noting concerns that the third sector is under considerable pressure to take on many services previously provided by the government sector.24 An increase in use of CFs by CRCs would facilitate the recruitment of sufficient participants for a fully powered study.
What is already known?
Our systematic review of published and grey literature6 found only one study of 10 participants25 assessing the impact of CFs for offenders for any outcome including quality of life or reconvictions. Our study is also the first to test the feasibility of linkage to PNC data on reconvictions. Observational and qualitative studies among those with mental ill health, learning disabilities and disaffected youth indicate that CFs may improve self-efficacy, confidence, coping skills, personal identity, social interaction, vocational skills and well-being and reduce negative behaviours and medication use.3 25–29
What this study adds?
This study is one of the few exploring the health and social impact of CFs among offenders. We have shown that, given sufficient CFs receiving offenders serving COs, a natural experiment would be feasible, with participants willing to provide self-report data on quality of life, mental well-being, lifestyle behaviours, relationship with nature and resource use. We have demonstrated the feasibility and success in linkage to probation and reconviction data. This is a viable and valuable alternative to measuring outcomes through self-report questionnaire with this challenging group. Our study has identified confounders associated with both CF allocation and outcome measures, particularly the OGRS risk of reoffending score; this is vital information for design of future studies.
Limitations of the study
The key limitation of our study was the limited recruitment and follow-up due, in large part, to the changes in probation. This limited the extent to which we could explore different aspects of feasibility and may have painted an overly challenging picture of conducting natural experiments of the impact of CFs.
Conclusions
Recruitment would be feasible in a more stable probation environment. Follow-up with offenders remains challenging. Using reconvictions as a main outcome measure, using existing MoJ data rather than follow-up questionnaires is one solution to retention challenges. We found significantly worse health and risk of reoffending among those allocated to CFs, reflecting use of CFs by probation to manage challenging offenders. Propensity weight adjustment provides a viable method for comparison despite differences in offenders at CFs and comparator sites. While randomisation is not possible within probation, a sufficiently powered natural experiment is feasible and would be of value to commissioners.
References
Footnotes
Contributors HFE, TF, CB, DS, ST, JC, ME, RG, RB designed the feasibility study. JM, ZR, NW conducted all data collection. HFE and JM oversaw the implementation and management of the study. TF conducted the statistical analysis and ST and NW conducted the health economics analysis. HFE drafted the manuscript and all authors commented and agreed to the final version.
Funding National Institute for Health Research Public Health Research Programme (project number 11/3050/08).
Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests RB has taken up a post with Care Farming UK following her involvement in this study.
Patient consent Not required.
Ethics approval University of Leeds School of Medicine Research Ethics Committee. The study also received approvals from the Ministry of Justice and the probation services involved (SoMREC/13/014 in December 2013 and NOMS approval in February 2014 (NOMS 2013-257)).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement While we are unable to publicly share our data on reconvictions, we would welcome data-sharing requests from researchers regarding other aspects of our quantitative and qualitative data. Please contact HFE (h.elsey@leeds.ac.uk) directly to explore further.