Article Text
PDF
Abstract
Objective The Identification and Referral to Improve Safety (IRIS) cluster randomised controlled trial tested the effectiveness of a training and support intervention to improve the response of primary care to women experiencing domestic violence (DV). The aim of this study is to estimate the cost-effectiveness of this intervention.
Design Markov model-based cost-effectiveness analysis.
Setting General practices in two urban areas in the UK.
Participants Simulated female individuals from the general UK population who were registered at general practices, aged 16 years and older.
Intervention General practices received staff training, prompts to ask women about DV embedded in the electronic medical record, a care pathway including referral to a specialist DV agency and continuing contact from that agency. The trial compared the rate of referrals of women with specialist DV agencies from 24 general practices that received the IRIS programme with 24 general practices not receiving the programme. The trial did not measure outcomes for women beyond the intermediate outcome of referral to specialist agencies. The Markov model extrapolated the trial results to estimate the long-term healthcare and societal costs and benefits using data from other trials and epidemiological studies.
Results The intervention would produce societal cost savings per woman registered in the general practice of UK£37 (95% CI £178 saved to a cost of £136) over 1 year. The incremental quality-adjusted life-year was estimated to be 0.0010 (95% CI −0.0157 to 0.0101) per woman. Probabilistic sensitivity analysis found 78% of model replications under a willingness to pay threshold of £20 000 per quality-adjusted life-year.
Conclusions The IRIS programme is likely to be cost-effective and possibly cost saving from a societal perspective. Better data on the trajectory of abuse and the effect of advocacy are needed for a more robust model.
Trial registration Current Controlled Trials, ISRCTN74012786.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
Article summary
Article focus
The aim of this study was to assess the cost-effectiveness of the IRIS training and support intervention for primary care clinicians from the UK societal and NHS perspectives.
Key messages
The intervention is likely to be cost saving from a societal perspective with a high likelihood of being under a £20 000 per quality-adjusted life-year willingness to pay threshold.
Strengths and limitations of this study
We have minimised bias in estimating the effect size of the IRIS programme by basing it on a randomised controlled trial.
By using epidemiological and cost data external to the trial, we were able to extrapolate from directly measured trial outcomes (DV disclosure and referral rates) to quality of life, health and economic outcomes.
The uncertainty of the transition probabilities based on assumptions was addressed by probabilistic sensitivity analysis, contributing to the robustness of the model.
Important limitations of that data are the paucity of longitudinal studies measuring the trajectory of abuse and uncertainty about the effect of DV advocacy for women not living in a refuge or shelter.
Introduction
Domestic violence (DV) is threatening behaviour, violence or abuse (psychological, physical, sexual, financial or emotional) between adults who are in the same family or who are or have been intimate partners. The lifetime population prevalence of physical and sexual violence against women varies internationally from 15% to 71%.1 This large variation reflects different measures of abuse, as well as actual differences in national, regional and local prevalence. Survivors suffer chronic health problems including gynaecological problems,2 gastrointestinal disorders,2 neurological symptoms,2 chronic pain3 and cardiovascular conditions.4 The impact on mental health, including post-traumatic stress disorder, depression, anxiety and substance abuse,5 ,6 can persist long after the violence has ceased. In 2008, the aggregate UK cost estimate for DV, including medical and social services, lost economic output and emotional costs, was £15.7 billion ($29.1 billion).7 While estimates of the cost burden can help highlight the importance of the issue in policy terms, we also need to link the size of the economic impact to estimates of how this can be reduced through interventions in order to make an economic case for investment in DV programmes. To do this, we need to combine evidence of the costs experienced by groups with and without a DV intervention with evidence of the outcomes experienced by those two groups in an economic evaluation.8 Here, we report an economic evaluation of DV interventions based on a randomised controlled trial, the least biased method for measuring the effect of an intervention.9
Healthcare may be a survivor's first or only point of contact with professionals, and abused women are more likely to contact health services than any other agency.10 The magnitude of the health consequences of DV contrasts starkly with its virtual invisibility within primary healthcare; in one UK general practice-based questionnaire study, only 15% of women with a history of DV had any reference to violence in their medical record.11 If women disclose DV to a clinician, whether in a primary care or specialist setting, there is evidence of an inappropriate poor-quality response.12 Doctors and nurses are largely unaware of appropriate interventions and have seldom received effective or, in the UK, any training11; yet abused women identify doctors as the professionals from whom they would most like to seek support.12
The Identification and Referral to Improve Safety (IRIS) trial13 tested the effectiveness of a training and support intervention for general practice teams, including training within the practice, a prompt to ask about DV embedded in the electronic medical record, a care pathway including referral to a specialist DV agency and continuing contact from that agency. Recorded referral in the medical record was 22 times greater in the intervention practices compared with the control practices and referral received by DV agencies was seven times greater. These agencies provided specialist advocacy to women experiencing DV: access to refuges/shelters, emergency housing, psychological services and ongoing support, as well as provision of information on legal, housing and financial options. In this paper, we present an economic evaluation of the IRIS programme (box 1), refining a previous model based on a pilot study of the intervention.14
Description of the Identification and Referral to Improve Safety (IRIS) programme
The primary intervention consisted of two 2 h multidisciplinary training sessions, targeted at the clinical team. The training sessions were designed to address the expressed and tacit barriers to improving the response of clinicians to women experiencing abuse through improved identification, support and referral to specialist agencies. These sessions incorporated case studies and practice in asking about violence and responding appropriately. They were delivered by an advocate educator based in one of the two collaborating specialist agencies, and either a clinical psychologist specialising in domestic violence or an academic general practitioner. The advocate educator was central to the intervention, combining a training and support role to the practices with provision of advocacy to women referred from the practices. The training sessions were followed by periodic contact with the practice in clinical meetings, feeding back anonymised practice data on disclosure and referral to the advocacy service, and reinforcing guidance on good practice with regard to domestic violence, as well as ad hoc telephone conversations and email exchanges with clinicians about referrals or advice. One-hour training sessions with administrative staff focused on issues of confidentiality and safety for patients experiencing abuse and introduced the IRIS information materials signposting domestic violence agencies. Ongoing support to clinicians and reception staff in the practices was provided by the named domestic violence advocate educator, with the aim of consolidating the initial training. Intervention practices also were asked to identify a ‘champion’ for the project; with the agreement of the practice, a member of staff from any of the clinical disciplines was invited to attend an additional 2.5 days of training about domestic violence and to integrate this into the work of the practice. Other components of the intervention included a template in the electronic medical record linked to diagnoses (such as depression, anxiety, irritable bowel syndrome, pelvic pain and assault), an explicit referral pathway to a named domestic violence advocate and publicity materials about domestic violence visible in the practices. Clinicians were trained to have a low threshold for asking about domestic violence as a ‘clinical enquiry’, not screening.
Methods
We conducted a cost-effectiveness analysis based on a comparison of the 24 intervention with 24 control practices in the IRIS trial (143 868 eligible women patients). We constructed a Markov model to estimate lifetime quality-adjusted life-years (QALYs) and costs from a UK national health service and a societal perspective. Markov modelling is a technique for estimating the costs and outcomes in a hypothetical cohort of women over time. We simulated a cohort of 10 000 UK representative women with and without the IRIS programme and used the differences between the two simulations to calculate the incremental costs and outcomes associated with IRIS. We report our findings in terms of costs, QALYs and incremental costs per QALY gained. In the UK, an incremental cost-effectiveness ratio, which is the ratio of differences between the costs of two alternatives and the difference between their effectiveness, of less than UK£20 000–30 000 (US$37 122–US$55 683) is generally thought to be cost-effective by policy makers.15 To construct the Markov model, we defined a set of mutually exclusive and exhaustive states experienced by women in relation to DV. The model then simulated the hypothetical cohort of women moving between the states, using a matrix of transition probabilities reflecting the likelihood of moving from each state to every other state within each discrete time period (in this case, 6 months, as this is the average amount of time a woman stays in contact with advocacy services).16 In order to aid comparison with the model for the pilot intervention,14 a 10-year time horizon was used, and future costs and outcomes were discounted at 3.5%.17
Model structure
We drew upon the model that was developed to estimate the long-term cost-effectiveness of the pilot intervention, Prevention of Domestic Violence (PreDoVe).14 The IRIS model was based on fewer assumptions about women's trajectories into and out of abuse and we used trial data instead of pilot data. In the IRIS Markov model (figure 1), the possible states of individual women were as follows:
No abuse: women not experiencing DV
Abuse unidentified: women who are experiencing abuse but have not been identified by a healthcare professional
Advocacy: women who are experiencing abuse and have seen or are seeing a DV advocate
Identified existing victim: women who are experiencing abuse and have been recognised as such by a healthcare professional but are not seeing an advocate
Death: women who have died from DV or other causes
Markov model diagram.
We excluded states that could not be informed by contemporary reliable data. For example, PreDoVe included a state to represent the medium-term improvement of women whose condition had improved but had not yet reached the point where they were completely free of abuse.14 This state was intended to reflect a stage on the trajectory to living without abuse, and we hoped that in the IRIS model the costs and quality of life (QoL) associated with this state would be informed by new studies. No relevant epidemiological studies have been reported, so we omitted this intermediate state and incorporated the effects of the intervention in the remaining states in order to reduce the number of transition probabilities based on assumptions.
The states are simplified in relation to available data. For example, the advocacy state includes identification and referral of women to an advocate. There are some transitions between states that we know might exist in the long term, such as between identified existing victim and advocacy (denoting self-referral of women to advocacy after they have disclosed to a healthcare professional), but for which we have no data. We envisage that this transition rate would be very small and so we do not expect any material changes in the model results if this transition was included. In other key areas, however, the IRIS model was able to distinguish more clearly the movement of women over time, as a result of improved data collection within the trial. For example, IRIS data distinguished between a referral recorded in the clinical record, a referral received by the agency and actual attendance by the patient, whereas the PreDoVe model was not able to distinguish between these outcomes.
This model structure is informed by the epidemiology of DV, including a cross-sectional study11 of prevalence in general practice populations similar to the IRIS sample, the trajectory of abuse18 and estimates of the effect of DV advocacy on cessation of abuse and improvements in QoL and psychological outcomes.19 Research by Bybee and Sullivan has contributed both to the small group of longitudinal studies of women experiencing DV20 and provided the most robust estimates of the effects of advocacy.21
Transition probabilities
In the IRIS model, the advocacy state only included those women who had contact with the advocate, a more accurate estimate of exposure to advocacy than the primary outcome of the trial, which was referral to advocacy noted in practice records but did not necessarily indicate that the woman saw an advocate. The advocacy state also included women who self-referred to advocacy as a result of a poster or card in the practice drawing attention to the service. The identified existing victim state consisted of women identified as experiencing abuse but not having contact with the advocate.
The number of women being identified (ie, the transition probability from abuse unidentified to identified existing victim) and seen by an advocate (abuse unidentified to advocacy) was significantly different for women in the intervention and control practices (table 1). The transition probabilities for recovery from abuse were drawn from a systematic review19 of the rate of recovery from physical abuse for women in intensive advocacy, which was largely based on a trial by Sullivan and colleagues recruiting IPV survivors in shelters (refuges) and following them up in the community for 3 years.24 In our model, we reduced the probability of transition to the non-abused state by 75% to account for the lower intensity of the IRIS intervention compared with those in the advocacy trials.
Model assumptions and sources for the six-monthly transition probabilities
All-cause mortality for women in the UK aged 16 years and older in 2008 was 0.011646 (264 031/22 671 300).22 For women experiencing DV, mortality resulting from DV (MDV) was 0.00019 (102/529 000).23 The mortality rate from DV for all women was calculated (102/22 671 300) and subtracted from the death rate for all women to get 0.011642, the annual mortality rate for women not experiencing DV (MNA). The annual mortality rate for women experiencing abuse (MA) was estimated to be 0.011834 (MNA + MDV). The six-monthly probabilities used in our model are shown in table 1.
The IRIS model captured the movements of all women registered at the practice aged 16 years and older, with a death rate to match this age bracket, more accurately reflecting the exposure of older and younger women to the intervention than the PreDoVe model that only used a death rate for women aged 16–45 years, while still estimating the total population costs and benefits of the intervention. The transitions are provided in table 1.
Intervention costs
Intervention costs were directly measured in the trial (table 2). The total six-monthly intervention costs were divided by the number of registered women in the intervention practices.
Model assumptions and sources for the costs and utilities
Advocate educator
The advocate educator was central to the IRIS intervention, combining a role in training and support to the practices with provision of advocacy to women referred from the practices.13 ,27 The advocate educator conducted training sessions, gave ongoing support to the practice and initial management of women referred by the practices. The advocate educator costs included salary, on costs and management costs, and the inclusive hourly cost was estimated to be £34. We deducted the cost of time allocated to research support. We did not adjust costs by practice size or by volume of referral because (1) the advocate educator salary costs were fixed at the start and (2) the practices that referred disproportionately fewer women, with a smaller impact on advocate caseload required greater training and support input to encourage increased engagement with the programme, increasing the workload of the advocate.
Other training time and costs
A clinical psychologist attended one of the two training sessions per practice and an academic general practitioner (GP) attended the other. We costed the allocated time rather than the actual training time, as the latter varied, but the prospective time commitment was fixed. We made the conservative assumption that training would need to be repeated annually with the same (and new) staff in attendance. We used the cost of practice reimbursement for trial participation to offset the costs of staff time spent in the training sessions.
Costs of onward referral from advocates
Within the IRIS trial, 57% of women in contact with an advocate were given an onward referral within the agency or to an external agency. Of those given an onward referral, 63% accepted it. The amount of time per onward referral varies greatly. We assumed an average of 57 h per onward referral to other advocacy services based on the main Sullivan trial that informed our estimate of the effect of advocacy24 ,28 less the time spent by advocate educators within the IRIS intervention. In this way, we were able to link the costs associated with advocacy with the transition rates out of the identified existing victim and advocacy states.19 ,28 The cost per hour of an onward referral was drawn from the advocate educator unit costs.
Costs and outcomes of abuse
For costs associated with events beyond the measured trial outcomes (identification and referral to DV advocacy), estimations were drawn from Walby's analysis of the societal and personal costs incurred by women experiencing abuse (table 3).23 While these figures were updated in 2009, the report noted that DV has decreased while services for victims have increased, resulting in similar overall cost figures to those in the original report.7 Healthcare costs for major medical events and additional primary care visits related to abuse were included. We assumed that the cost of abuse would be the same, regardless of whether women had been identified or referred to an advocate.
Average six-monthly costs for women from Walby23 (2008 UK pounds)
QoL for the states involving abuse (abuse unidentified, identified existing violence and advocacy) was taken from survey data by Wittenberg and colleagues and can be found in table 226 ,29; QoL for women in no abuse was taken from the female subset of a UK general population survey.25
Model calibration
As there was considerable uncertainty surrounding a number of the transition probabilities, we used known prevalence figures11 to calibrate the model. The model was run until steady states for the population in each group were achieved, allowing comparison with these known prevalence figures. If these figures were very dissimilar, the transition rates that had the most uncertainty around them were adjusted accordingly. The rates with the most uncertainty were the rate from no abuse to abuse unidentified and the rate from abuse unidentified to no abuse as there are no empirical studies to inform the assumptions. Steady states determined the initial distribution of women in the model.
The model calibration demonstrated that the rates from no abuse to abuse unidentified and the rate from abuse unidentified to no abuse needed to be adjusted. The rate of spontaneous recovery from abuse (abuse unidentified to no abuse) was increased from 0.005 (PreDoVe assumption) to 0.025. To compensate for this increase, the rate from no abuse to abuse unidentified was raised from 0.0027 to 0.005. With these adjustments, the model better reflected the population prevalence of abuse at 19.3%.11 Starting populations for each state were 80.7% in no abuse, 16.8% in abuse unidentified, 0.3% in advocacy and 2.2% in identified existing victim.
Cost-effectiveness analysis
The costs and outcomes along with the intervention costs described above were applied to the model states as appropriate. All costs and benefits were reported in 2008 British pounds to reflect the trial setting; costs taken from other years were adjusted as appropriate.30
We determined the mean incremental costs and incremental QALYs of implementing the IRIS approach relative to usual care delivered in the control practices and reported incremental cost-effectiveness ratios. Non-parametric bootstrapping with 1000 iterations was used to calculate 95% CIs for the incremental costs and incremental QALYs. As this is a public health intervention, the initial analysis was from a societal perspective with a secondary analysis focusing on costs and cost-effectiveness from a healthcare provider perspective. The secondary analysis only included costs of medical service and mental health consequences for women.
Sensitivity analysis
To assess the robustness of our result, we performed a probabilistic sensitivity analysis (PSA) on the model using appropriate distributions for each variable. Generally, distributions reflected the certainty of the data source with SDs of 10% of the mean for values from the trial and 75% for assumptions and values from the literature (tables 1 and 2). To calculate the parameters for the β distributions used for the QoL data, we made an assumption about the linearity of the sample variances for each state so that the sample variance decreased as the QoL value increased. To ascribe potential cost benefits for women who have received help from an advocate, we made the assumption that the SD for the advocacy state was 0.75. Given that identification alone can confer some benefit, we assumed that the SD for the identified existing victim state was 0.50.
Results
The intervention arm demonstrated a societal cost savings per woman registered of UK£37 (US$67) (95% CI UK£178 saved to a cost of UK£136 (US$330 to US$253)) per year. The incremental QALY was estimated to be 0.0010 (95% CI −0.0157 to 0.0101) gained per woman (table 4). Since the intervention programme demonstrated lower costs and higher effectiveness, it dominated current practice. When only NHS costs for medical attention and mental health were considered, the result was a cost savings of UK£1.07 (US$1.99) per woman per year, which is equivalent to UK£3155 (US$5851) per practice per year.
Results per registered woman over 1 year
These projected savings are calculated per woman registered in a practice and not all women will enter into an abusive relationship. The small NHS cost savings per woman per year are substantial when aggregated at a practice level. Costs decrease because more women leave the abuse unidentified state with more entering into the no abuse state. In the broader societal perspective, the annual projected savings from the IRIS intervention are striking.
Sensitivity analysis
The PSA also had 78% of model replications under a willingness to pay threshold of UK£20 000 (US$37 122) per QALY (figure 2). The key drivers of uncertainty were the transition rate from identified existing victim to no abuse, the QoL associated with the abuse unidentified state and employment loss of output costs due to abuse. The ICER was not sensitive to changing the time horizon to 20 years as the costs and outcomes associated with the intervention were almost uniformly distributed over time.
Cost-effectiveness acceptability curve. Probability that the Identification and Referral to Improve Safety (IRIS) intervention is cost-effective after 10 years plotted as a function of societal willingness to pay per quality-adjusted life-year.
Discussion
This is the first economic evaluation of a DV intervention in the context of a randomised controlled trial. We found that the IRIS programme is likely to be cost-effective and possibly cost saving from a societal perspective and a healthcare perspective in the UK. The relatively modest costs of the primary care-based intervention and its projected long-term cost benefits means this is likely to be the case in other developed healthcare economies.
We have minimised bias in estimating the effect size of the IRIS programme by basing it on a randomised controlled trial that evaluated a crucial link between healthcare and access of women to specialist DV services. By using epidemiological and cost data external to the trial, we were able to extrapolate from directly measured trial outcomes (DV disclosure and referral rates) to QoL, health and economic outcomes. The uncertainty of the transition probabilities based on assumptions was addressed by PSA, contributing to the robustness of the model. By reporting societal as well as healthcare cost-effectiveness, our findings can inform societal prioritisation of investment in relation to DV as well as specific decisions at a healthcare commissioning level.9
This model was an improvement on the previous version developed in the pilot study, which preceded the Cochrane review of advocacy trials to inform some of the key transition rates. This pilot study only involved four practices and was based on a non-randomised comparison, and the model did not use a PSA to characterise the uncertainty of the cost-effectiveness estimate.
The DV prevalence of 17% came from women attending general practice rather than all women registered at the practice. As not all women see their GP and as women experiencing abuse are likely to see their GP more often than the general population, this figure is potentially higher than it should be for the population of women in our model. This may have overestimated the cost-effectiveness of the intervention. Research by Fishman and colleagues31 in the USA indicates that we may be underestimating the higher healthcare costs in women in the years following cessation of abuse, but no research has quantified these costs in a UK setting. Our model does not include any benefits of the intervention for children who are exposed to DV,32 underestimating the potential cost-effectiveness. The downstream benefit of the intervention on women is crucially dependent on trials from one centre in the Cochrane systematic review of the effectiveness of advocacy, which evaluates women receiving intensive (60 h) DV advocacy in the context of a refuge (shelter). To guard against a potential overestimate of benefit in our model, we used this effect reduced by 75% to extrapolate the Cochrane review findings to community-dwelling women referred to less intensive advocacy from primary care. Our projections of longer term benefit of DV advocacy were problematic because trials in this field measure relatively short-term outcomes. Moreover, there is a dearth of longitudinal studies measuring the medium- to long-term trajectory of DV events in terms of continuing or repeated abuse and no prospective medium- or long-term data on QoL and health status following abuse.
As Gold and colleagues9 argue in their methodological paper on evaluating the cost-effectiveness of healthcare programmes for partner violence, the resources required to provide effective interventions that reduce violence and improve outcomes for women could be used for other healthcare interventions or social care programmes. Despite the growth in the number of randomised controlled trials, the analysis of cost-effectiveness within the field of DV research is still embryonic. Given the long-term benefits of cessation of violence and the necessarily limited follow-up in trials, as well as the particular challenges of following up participants who have experienced DV,33 models such as the one we have reported here are essential in evaluating the cost-effectiveness of DV programmes in healthcare and other settings. The robustness of these models will be improved by longitudinal studies characterising the different trajectories of abuse and their sequelae. Notwithstanding the need for better longitudinal data to populate these models, the analysis that we have reported in this paper is evidence of cost-effectiveness that can inform the commissioning of the IRIS programme in the context of primary healthcare services.
Acknowledgments
We thank Clare Rutterford, statistician for the IRIS trial, for providing all the trial data used in this paper. We are grateful to all the other members of the IRIS trial team (Roxane Agnew Davies, Kathleen Baird, Danielle Dunne, Chris Griffiths, Alison Gregory, Annie Howell, Medina Johnson, Jean Ramsay, Clare Rutterford, Debbie Sharp) and the IRIS steering group (Jackie Beavington, Jan Buss, Kim Howell, Carol Metters, Lesley Welch). We thank the general practice teams that participated in the trial and our collaborating specialist DV agencies, Next Link and the nia project, in the provision of training and support to practices, and in their advocacy work with the women referred during the trial.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online table 1
Footnotes
To cite: Devine A, Spencer A, Eldridge S, et al. Cost-effectiveness of identification and referral to improve safety (IRIS), a domestic violence training and support programme for primary care: a modelling study based on a randomised controlled trial. BMJ Open 2012;2:e001008. doi:10.1136/bmjopen-2012-001008
Contributors All authors designed the analysis. AD constructed and ran the model, supervised by AS. GF contributed to specification of the transition probabilities and underlying data assumptions. SE and RN contributed to modifications of the model. AD wrote the first draft of the paper and all authors contributed to its revision.
Funding This work was supported by the Health Foundation grant number 2780/4601.
Competing interests None.
Ethics approval The Identification and Referral to Improve Safety (IRIS) programme had ethics approval from the South East Research Ethics Committee (REC Reference: 07/MRE01/65).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data set available from the principal investigator at gene.feder{at}bristol.ac.uk. Further details on the Markov model are available from the corresponding author.