You are here

  • Home
  • Archive
  • Volume 6, Issue 9
  • Prescriber preferences for behavioural economics interventions to improve treatment of acute respiratory infections: a discrete choice experiment

Article Text

Article menu

Download PDFPDF

XML
Health economics
Research
Prescriber preferences for behavioural economics interventions to improve treatment of acute respiratory infections: a discrete choice experiment
  1. Cynthia L Gong1,
  2. Joel W Hay1,
  3. Daniella Meeker1,2,
  4. Jason N Doctor1
  1. 1University of Southern California Leonard D. Schaeffer Center for Health Policy & Economics, Los Angeles, California, USA
  2. 2University of Southern California Keck School of Medicine, Los Angeles, California, USA
  1. Correspondence to Dr Cynthia L Gong; gongc{at}usc.edu

Abstract

Objective To elicit prescribers' preferences for behavioural economics interventions designed to reduce inappropriate antibiotic prescribing, and compare these to actual behaviour.

Design Discrete choice experiment (DCE).

Setting 47 primary care centres in Boston and Los Angeles.

Participants 234 primary care providers, with an average 20 years of practice.

Main outcomes and measures Results of a behavioural economic intervention trial were compared to prescribers' stated preferences for the same interventions relative to monetary and time rewards for improved prescribing outcomes. In the randomised controlled trial (RCT) component, the 3 computerised prescription order entry-triggered interventions studied included: Suggested Alternatives (SA), an alert that populated non-antibiotic treatment options if an inappropriate antibiotic was prescribed; Accountable Justifications (JA), which prompted the prescriber to enter a justification for an inappropriately prescribed antibiotic that would then be documented in the patient's chart; and Peer Comparison (PC), an email periodically sent to each prescriber comparing his/her antibiotic prescribing rate with those who had the lowest rates of inappropriate antibiotic prescribing. A DCE study component was administered to determine whether prescribers felt SA, JA, PC, pay-for-performance or additional clinic time would most effectively reduce their inappropriate antibiotic prescribing. Willingness-to-pay (WTP) was calculated for each intervention.

Results In the RCT, PC and JA were found to be the most effective interventions to reduce inappropriate antibiotic prescribing, whereas SA was not significantly different from controls. In the DCE however, regardless of treatment intervention received during the RCT, prescribers overwhelmingly preferred SA, followed by PC, then JA. WTP estimates indicated that each intervention would be significantly cheaper to implement than pay-for-performance incentives of $200/month.

Conclusions Prescribing behaviour and stated preferences are not concordant, suggesting that relying on stated preferences alone to inform intervention design may eliminate effective interventions.

Trial registration number NCT01454947; Results.

  • discrete choice
  • conjoint analysis
  • stated preference
  • revealed preference
  • antibiotic prescribing

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2016-012739

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • In this discrete choice experiment, prescribers were asked about which interventions they felt would be most effective in reducing their rates of inappropriate antibiotic prescriptions for acute respiratory infections; such information can be valuable when developing clinical quality improvement interventions.

    • This is one of few studies in the healthcare literature that not only elicits prescriber preferences using a discrete choice experiment but also compares these stated preferences to actual prescribing behaviour as observed in a randomised controlled trial.

    • Results indicate that stated and revealed preferences are not concordant, suggesting that clinical quality improvement programmes should not rely solely on clinician input, but instead combine expert-driven and clinician-driven approaches.

    • The discrete choice experiment may not necessarily capture true preference, but instead be a reflection of convenience and ease of use of a specific clinical quality improvement intervention.

    • Stated preferences for this group of healthcare providers may not necessarily reflect those of a national sample of providers.

    Introduction

    According to the Centers for Disease Control and Prevention, up to 50% of antibiotics are not optimally prescribed, leading to an estimated 2 million illnesses and 23 000 deaths due to antibiotic resistance alone.1 Most antibiotics in the USA are prescribed for acute respiratory tract infections (ARIs), and about half of these prescriptions are issued to patients with non-bacterial diagnoses.2 ,3 Despite attempts to curb inappropriate antibiotic prescribing through interventions such as physician and patient education, electronic clinical decision support and financial incentives, these have only resulted in modest reductions in antibiotic prescribing rates for non-bacterial ARIs.4

    Clinical quality improvement interventions frequently rely upon changing clinicians' practice behaviours, such as reducing orders for inappropriate treatments or diagnostic tests. The consensus-recommended best practices in design and implementation of quality improvement interventions include some component of ‘local participatory’ approaches, that is, working with frontline staff, including the target population.5–7 This engagement may directly or indirectly influence the intervention design, but it is unclear if this practice yields the optimal design. For example, theory and survey results suggest that physicians are likely to indicate preferences for direct financial incentives (bonuses).8–10 However, studies of effectiveness have shown mixed results of direct incentives in practice.10 ,11 This raises questions regarding whether stated preferences for intervention features elicited in a participatory process should be incorporated into design.

    One approach to changing prescribing behaviour applies ideas from the behavioural sciences, using social cues and subtle changes in the clinic environment to influence clinical decision-making.12 ,13 In fact, the UK government implements behavioural economics ‘nudges’ into its health services, and a recent study in the UK found that social norm feedback from England's Chief Medical Officer was highly effective in reducing inappropriate antibiotic prescribing at a low cost.14–16 In the USA, the ‘Use of behavioral economics and social psychology to improve treatment of acute respiratory infections (BEARI)’ study, a multisite cluster randomised controlled trial (RCT), applied behavioural techniques and assessed the impact of various behavioural interventions on the rates of inappropriate antibiotic prescribing in various practices in Illinois, Massachusetts and Southern California.17 ,18 As part of the BEARI study, a discrete choice experiment (DCE) was conducted to elicit prescribers' stated preferences to evaluate prescriber preferences for one intervention compared to another.

    The use of discrete methods to elicit preferences for various programmes and interventions in healthcare has significantly increased in recent years.19–22 These methods have been used in the context of health system reforms or quality improvement programmes, and healthcare policies, programmes, services, incentives and interventions.9 ,20 While they provide valuable feedback regarding factors that should be considered for a given programme or intervention, few studies in healthcare have evaluated how stated preferences compare to real-life behaviour. Those who have assessed external validity have generally found that stated preferences are consistent with actual decision-making behaviour on an aggregate level, while individual-level concordance is limited.23 ,24 Previous studies evaluating an individual's decisions for vaccination or disease screening indicated a positive predictive value for DCEs of 85%, but that the negative predictive value was only 26%; however, the authors noted that the majority of people opt for preventive healthcare in these situations, yielding the overall predictive values of the DCE.21 ,25 ,26 Furthermore, only one study has specifically considered physician decision-making—the ultimate driver of quality and cost of care.27

    Based on these studies, DCE-elicited responses may generally reflect real-life behaviours. However, few studies have tested behaviour in a RCT and compared the results to DCE-elicited preferences for the same group of individuals (particularly physicians) to validate the DCE responses.28 Thus, the objective of this study was to elicit prescriber preferences for different behavioural economics interventions to reduce antibiotic prescribing, and compare these to actual prescribing behaviour as revealed in the BEARI study.

    Methods

    Design

    Details for the BEARI study have been described elsewhere.17 Briefly, this was a multicentre trial conducted in Illinois, Massachusetts and Southern California to determine whether behavioural economic interventions could influence prescriber behaviour. The interventions implemented in the BEARI study included: (1) Suggested Alternatives (SA), which used computerised clinical decision support to suggest over-the-counter and prescription non-antibiotic treatment choices to clinicians prescribing an antibiotic for an ARI; (2) Accountable Justifications (JA), which prompted clinicians to enter an explicit justification when prescribing an antibiotic for an ARI that would then appear in the patient's electronic health record; and (3) Peer Comparison (PC), which was an email sent periodically to each prescriber that compared his/her rate of inappropriate antibiotic prescribing relative to top-performing peers.17 All clinicians received an education module reviewing ARI diagnosis and treatment guidelines, and each was assigned to 0, 1, 2 or all 3 interventions. Interventions were in place for 18 months at each practice site.

    Upon completion of the study, all prescribers were asked to complete a computerised exit survey, which assessed prescribers' level of satisfaction with each of the interventions, and included the DCE. A DCE is used to determine an individual's preferences for specific alternatives given a specific scenario, rather than observing an individual's behaviour in real markets to determine his/her revealed preferences.29 The individual is presented with a pair of choices with varying levels of attributes and is asked to choose the preferred alternative based on the attributes presented (refer to Supplementary materials for details).

    In the DCE, participants were presented with the following scenario, followed by a set of 10 choice pairs from which prescribers had to choose a preferred alternative that they felt would most likely reduce their ARI antibiotic prescribing:Suppose that your Health Care Organization implements various measures to reduce overprescribing of antibiotics for acute respiratory infections (e.g. Acute bronchitis, Acute Pharyngitis, Sinusitis, etc.). Assume implementation with an electronic health record system with electronic prescribing options and physician monitoring that can be automatically enabled or disabled. You will be provided with a short patient handout that discusses antibiotic overprescribing for acute respiratory infections.

    The five attributes included for each alternative were JA, SA, PC, pay-for-performance ($100, $200, or $0) and additional time spent with the patient (5 or 0 min) (table 1).

    View this table:
    Table 1

    Attributes and descriptions

    These attributes and their levels were chosen based on interviews with prescriber focus groups in Los Angeles. Each attribute was toggled on or off in the choice pairs presented to respondents (figure 1). Respondents were asked to evaluate a total of 10 choice pairs which were assigned using a fractional factorial design with orthogonality of main effects.30

    Figure 1
    Figure 1

    DCE treatment scenarios. DCE, discrete choice experiment.

    A pilot survey was administered to a small number of prescribers in Los Angeles and Boston who were not involved in the BEARI study and thus not exposed to any of the interventions, and to a subset of prescribers in Chicago who did receive exposure to the BEARI interventions. The DCE was then administered as part of the exit survey for all prescribers involved in the BEARI study.

    Data analysis

    Baseline provider characteristics were collected at the start of the study, and information about the clinic environment and patient population mix were collected during the exit survey. Descriptive statistics were performed on these data.

    Preference data from the DCE were analysed using multinomial logit and mixed logit models drawn 500 times to obtain a robust output. In addition, the models were run on the subset of responders who completed 6 or more of the 10 choice tasks, and the results compared to those of the entire sample. The impact of intervention assignment on stated preference was evaluated by including intervention assignment as an explanatory variable in the mixed logit model, and interacting intervention assignment on each programme. Willingness-to-pay (WTP) for each intervention type was calculated by taking the ratio of the estimated coefficient for a given attribute to the cost coefficient, for example: Embedded Image. This calculation was performed for each intervention to quantify the monetary value of each attribute included in the DCE.31

    All data were analysed in Stata V.12.0 (StataCorp. Stata Statistical Software: Release 12. StataCorp LP: College Station, Texas, 2011).

    Results

    Details for the outcomes of the BEARI study have been described elsewhere.18 Briefly, the BEARI study found that the most significant reduction in the rates of inappropriate antibiotic prescribing was achieved with JA and PC, when compared to the education control. SA yielded a non-significant reduction in inappropriate antibiotic prescribing rates.18 These results suggest that PC and JA may be the most effective interventions to influence prescribing.

    Of the 253 prescribers recruited in the BEARI RCT study, n=234 responses were received for the DCE survey, although only 157 provided responses to all 10 treatment scenarios. Of the responders, 26 were exposed to all three BEARI interventions, 35 to SA and JA, 32 to SA and PC, and 23 to JA and PC in the BEARI RCT. Prescribers were primarily medical doctors with an average of 20 years of clinical practice, with more men in the control group (table 2).

    View this table:
    Table 2

    Baseline demographics

    Results of the mixed logit model are presented in table 3.

    View this table:
    Table 3

    BEARI study mixed logit regression results

    χ2 statistic indicates that coefficients were significantly different from one another (p<0.001). Overall, prescribers tended to prefer any of the presented alternatives to prescriber education and guideline review, regardless of exposure group. Prescribers did not prefer PC as much as other interventions for reducing inappropriate antibiotic prescriptions. Instead, prescribers consistently preferred SA regardless of which intervention they were actually exposed to, including those in the control group. These results match those found in the pilot study; those who were not exposed to any of the BEARI interventions also strongly preferred SA (OR=2.75, p=0.003). Results were similar in the pilot group exposed to the interventions, though slightly less pronounced (OR=2.07, p=0.018) (see online supplementary table S1). These trends remained consistent whether prescribers were exposed to just one, two or all of the interventions, although the magnitude and significance of preference varied compared to the results by individual intervention (see online supplementary table S2).

    In contrast, PC was not strongly preferred as an intervention most likely to reduce ARI antibiotic prescribing, though still statistically significant among controls (OR=1.56, p=0.045), and those exposed to SA (OR=1.55, p=0.002) and JA (OR=1.43, p=0.008). In fact, those who were exposed to PC had a non-significant preference for this intervention (OR=1.285, p=0.167), even though it was highly effective in reducing inappropriate antibiotic prescribing rates in the BEARI experiment. Overall, prescribers were either indifferent to or did not prefer JA (although this was non-significant), unless they were exposed to that intervention in the study, in which case there was a significant preference for the intervention (OR=1.45, p=0.02). In contrast, the pilot study showed that JA was non-preferred among non-exposed prescribers (OR=0.68, p=0.049), though preferred among those exposed (OR=1.79, p=0.247).

    Pay-for-performance was marginally preferred among prescribers across all groups (control and treatment) with an OR=1.007, implying that each additional $100 in financial incentives would increase the probability of preferring pay-for-performance by 70%. This suggests that clinicians feel that every $100 increase in compensation would improve their inappropriate antibiotic prescribing rates by 70% relative to the control group. These results also generally held true in the pilot study. Additional office time was more strongly preferred than pay-for-performance, although this preference was non-significant in the control group (OR=1.085, p=0.082). Among those exposed to SA, JA and PC, additional office time was highly significantly preferred, with OR=1.19, 1.17 and 1.2, respectively (p<0.05). These results imply that overall, clinicians felt that additional time spent with a patient to form an appropriate diagnosis and treatment plan would be more effective than financial incentives to reduce inappropriate antibiotic prescribing.

    When adjusted for intervention in the interacted model, trends in stated preferences remained stable (see online supplementary table S3). Those exposed to JA did not favour SA as strongly as those exposed to SA or PC (OR=1.889, p<0.001, vs OR=2.085, p<0.001 and OR=2.406, p<0.001, respectively), as in the base case model. In addition, JA remained non-preferred even when stratified by exposure group, except for those exposed to JA in the BEARI trial.

    WTP estimates indicate the value of each intervention to prescribers (table 4).

    View this table:
    Table 4

    WTP estimates

    SA was equivalent to $1299–$1553 in financial incentives per year, whereas PC was only worth $408–$886 per year. Each minute of additional office time for patient education was only worth an average $265 per year. These estimates imply that prescribers could be paid an average of $1400, $650 or $265 per year to achieve the same results as SA, PC and additional time, respectively. These results also imply that SA is equivalent to an additional 5–6 min of office time, and PC 3 min. In contrast, prescribers in the control group would rather sacrifice $13 of pay per month (or $156 annually) than be required to enter a justification for every inappropriate antibiotic prescription (JA).

    Compared to the results of the DCE, the exit survey yielded an overwhelmingly ambivalent attitude towards the BEARI study as a whole, and towards each individual intervention. While 60% of prescribers agreed that providing feedback on clinician performance and using electronic decision support tools are effective ways to improve quality care, 42% of prescribers felt that the BEARI interventions were neither useful nor useless in improving antibiotic prescribing practices. When asked about the usefulness of the PC emails, 37% of prescribers responded neutrally, while 14% felt the information was not at all useful, and only 7% found it very useful. For SA and JA, 30% of prescribers responded neutrally, ∼20% felt the interventions were not at all useful and only 10% found the interventions to be very useful.

    Discussion

    Overall, the DCE-elicited preferences did not reflect actual behaviour as revealed in the RCT of prescriber interventions to alter antibiotic prescribing. Although the DCE overwhelmingly favoured SA as the most effective method for reducing inappropriate antibiotic prescriptions across treatment and controls, this intervention yielded similar antibiotic prescribing rates to controls throughout the duration of the BEARI trial.18 On the other hand, most prescribers generally felt that JA was ineffective in reducing their ARI antibiotic prescribing, despite its significant impact on inappropriate antibiotic prescription rates in the trial. PC was generally significantly preferred to provider education, although this preference was not as strong as that for SA. Regardless of the intervention group to which participants were assigned in the trial, the general trends in stated preference were similar across all groups in the DCE survey, while the trial showed that the trajectory of antibiotic prescribing rates for JA and PC relative to the control group consistently decreased compared to SA.

    Differences between stated and revealed preferences may be due to a number of factors. Prescribers did not get to choose their intervention in the randomised trial; therefore, prescribing behaviour may not represent the prescriber's true ‘revealed preference’. Another consideration is that in the DCE task, prescribers may not be choosing the intervention that best reduces inappropriate antibiotic prescribing; instead, the stated preference might be for the intervention that is the least inconvenient for the prescriber. The DCE results are also reflected in the exit survey, which indicated that the majority of prescribers rated the usefulness of each intervention as 1, 2 or 3 on a scale of 1–5 (1 being not at all useful, 5 being very useful). Only 10% of prescribers found SA to be very useful (rating of 5 on a scale of 1–5), while just 8% and 6% found JA and PC to be very useful interventions, respectively. Given the lack of enthusiasm in the survey responses, it is unsurprising that even SA, the most preferred intervention according to the DCE, was only worth up to $1500 annually to clinicians, and the other interventions even less.

    SA may be the most preferred intervention overall because it is a clinical decision support tool, as opposed to a socially motivated intervention such as JA or PC. The stated preference for SA is consistent with design recommendations that if Electronic Health Record (EHR) alerts are used, they should include actionable information.32 ,33 On the other hand, socially motivated interventions force the prescriber to conform to ‘social norms’ due to a concern about his/her social reputation and/or an awareness of the social norm of cooperation.34 Rather than applying social pressure, SA bears no social consequences if the prescriber chooses to ignore the alternatives presented. In contrast, prescribers' dislike of JA is consistent with findings describing the phenomenon of ‘alert fatigue’, and relatively poor efficacy of alerts that do not include actionable recommendations.33 ,35 However, JA in BEARI was actually highly effective in the RCT, and differs from similar studies that do not include peer accountability.33

    One limitation of the BEARI study is that there were many incomplete responses to the DCE task. However, results for the entire sample were compared to results for those who completed six or more choice tasks, and there were no significant differences. In addition, geographic variation in prescribing habits and patient attitudes may have affected the rate at which inappropriate antibiotics were prescribed, regardless of the presence of a behavioural economic intervention. However, it is assumed that such attitudes and behaviours would have been balanced among all treatment groups after randomisation. Responses may have been different had the DCE been administered before the RCT took place, since prescribers would not have actually experienced the interventions prior to being asked about them. The pilot DCE was administered to prescribers who did not receive any of the interventions, yet it showed similar statistically significant stated preferences to those elicited from prescribers who participated in the trial. This, along with the fact that actual intervention assignment did not affect the overall trend of stated preferences, suggests that prescribers appear to express the same preferences regardless of exposure to the BEARI interventions. Finally, contrary to consensus regarding the inclusion of an opt-out option in DCE to reflect a respondent's choice in real life, no opt-out option was included.36 ,37 However, one study empirically evaluated the effect of an opt-out option on attribute preference, and found that while attribute value estimates differed, there were no notable differences in the relative order of the attributes (when compared to each other).36 Based on their findings, the authors recommended that an opt-out option always be included in a DCE if non-participation is an option in real life as well. In BEARI, prescribers did not have the option to opt out of participation, and it is likely that if the BEARI interventions were implemented in practice, prescribers would also be unable to opt out.

    Compared to previous studies attempting to evaluate the external validity of DCEs in healthcare, this study provides stronger evidence that stated and revealed preferences can be non-concordant. Specifically, revealed preferences were determined through a RCT, and the same individuals were then asked to complete the DCE. Participants were not given a choice as to which intervention they wished to be implemented in clinic; instead, real-life behaviour was captured in the trial. Prescribers were also from various types of practice settings across the country, increasing the external validity of the results, rather than eliciting responses from a single practice site. Previous studies did not use an experimental design to determine revealed preferences, and thus, the results are not as robust.

    Ultimately, had the BEARI trial relied upon the DCE preferences of the clinicians unexposed to the interventions to inform intervention selection, the least effective intervention (SA) would have been adopted and one of the effective interventions (JA or PC) would have been rejected. Relying on the exit survey responses would have led to none of the interventions being adopted. Our findings are consistent with recommendations that quality improvement interventions combining local participatory approaches with expert-driven approaches are likely to be most effective.38 In particular, approaches that engage targets in the implementation strategy rather than the design and development of interventions may optimally ensure that interventions are deployed in a pragmatic way without being influenced by the stated preferences of clinicians.

    Conclusion

    This study is part of a growing body of literature comparing stated and revealed preferences in a healthcare setting. Consistent with similar studies evaluating the external validity of DCE-stated preferences, this study showed that they are not concordant, and suggests that quality improvement interventions should not rely on frontline staff input alone. Future work should continue to test the external validity of DCE preferences in healthcare and the factors that contribute to differences between stated and revealed preferences.

    References

    1. ↵
      1. Shapiro DJ,
      2. Hicks LA,
      3. Pavia AT, et al
      . Antibiotic prescribing for adults in ambulatory care in the USA, 2007–09. J Antimicrob Chemother 2014;69:234–40. doi:10.1093/jac/dkt301
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Steinman M
      . Changing use of antibiotics in community-based outpatient practice, 1991–1999. Ann Intern Med 2003;138:525–33. doi:10.7326/0003-4819-138-7-200304010-00008
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Grijalva CG,
      2. Nuorti JP,
      3. Griffin MR
      . Antibiotic prescription rates for acute respiratory tract infections in US ambulatory settings. JAMA 2009;302:758–66. doi:10.1001/jama.2009.1163
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Ranji SR,
      2. Steinman MA,
      3. Shojania KG, et al
      . Antibiotic prescribing behavior Vol. 4: closing the quality gap: a critical analysis of quality improvement strategies. Technical Review 9 (Prepared by Stanford University-UCSF Evidence-based Practice Center under Contract No. 290-02-0017. Rockville (MD): Agency for Healthcare Research and Quality, 2006.
    5. ↵
      1. Van Bokhoven MA,
      2. Kok G,
      3. Van Der Weijden T
      . Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care 2003;12:215–20. doi:10.1136/qhc.12.3.215
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Demakis J,
      2. Mcqueen L,
      3. Kizer KW, et al
      . Quality Enhancement Research Initiative (QUERI): a collaboration between research and clinical practice. Med Care 2000;38(Suppl 1):I17. doi:10.1097/00005650-200006001-00003
      OpenUrlPubMed
    7. ↵
      1. Rubenstein LV,
      2. Mittman BS,
      3. Yano EM, et al
      . From understanding health care provider behavior to improving health care: the QUERI framework for quality improvement. Quality Enhancement Research Initiative. Med Care 2000;38(Suppl 1):I129–41. doi:10.1097/00005650-200006001-00013
      OpenUrlPubMed
    8. ↵
      1. Robinson JC
      . Theory and practice in the design of physician payment incentives. Milbank Q 2001;79:149–77, III. doi:10.1111/1468-0009.00202
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Scott A
      . Eliciting GPs’ preferences for pecuniary and non-pecuniary job characteristics. J Health Econ 2001;20:329–47. doi:10.1016/S0167-6296(00)00083-7
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Armour BS,
      2. Pitts MM,
      3. Maclean R, et al
      . The effect of explicit financial incentives on physician behavior. Arch Intern Med 2001;161:1261–6. doi:10.1001/archinte.161.10.1261
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Town R,
      2. Kane R,
      3. Johnson P, et al
      . Economic incentives and physicians’ delivery of preventive care: a systematic review. Am J Prev Med 2005;28:234–40. doi:10.1016/j.amepre.2004.10.013
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Loewenstein G,
      2. Brennan T,
      3. Volpp K
      . Asymmetric paternalism to improve health behaviors. JAMA 2007;298:2415–17. doi:10.1001/jama.298.20.2415
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Frolich A,
      2. Talavera JA,
      3. Broadhead P, et al
      . A behavioral model of clinician responses to incentives to improve quality. Health Policy 2007;80:179–93. doi:10.1016/j.healthpol.2006.03.001
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Thaler RH,
      2. Sunstein CR
      . Nudge: improving decisions about health, wealth, and happiness. New York: Penguin Books, 2009.
    15. ↵
      Who We Are (cited 19 May 2016). http://www.behaviouralinsights.co.uk/about-us/
    16. ↵
      1. Hallsworth M,
      2. Chadborn T,
      3. Sallis A, et al
      . Provision of social norm feedback to high prescribers of antibiotics in general practice: a pragmatic national randomised controlled trial. Lancet 2016;387:1743–52. doi:10.1016/S0140-6736(16)00215-4
      OpenUrlCrossRefPubMed
    17. ↵
      1. Persell S,
      2. Friedberg M,
      3. Meeker D, et al
      . Use of behavioral economics and social psychology to improve treatment of acute respiratory infections (BEARI): rationale and design of a cluster randomized controlled trial—study protocol and baseline practice and provider characteristics. BMC Infect Dis 2013;13:290–300. doi:10.1186/1471-2334-13-290
      OpenUrlCrossRefPubMed
    18. ↵
      1. Meeker D,
      2. Linder JA,
      3. Fox CR, et al
      . Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices. JAMA 2016;315:562–70. doi:10.1001/jama.2016.0275
      OpenUrlCrossRefPubMed
    19. ↵
      1. Ryan M,
      2. Gerard K
      . Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy 2003;2:55–64.
      OpenUrlPubMed
    20. ↵
      1. Whitty JA,
      2. Lancsar E,
      3. Rixon K, et al
      . A systematic review of stated preference studies reporting public preferences for healthcare priority setting. Patient 2014;7:365–86. doi:10.1007/s40271-014-0063-2
      OpenUrlPubMed
    21. ↵
      1. De Bekker-Grob E,
      2. Ryan M,
      3. Gerard K
      . Discrete choice experiments in health economics: a review of the literature. Health Econ 2012;21:145–72. doi:10.1002/hec.1697
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Clark MD,
      2. Determann D,
      3. Petrou S, et al
      . Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics 2014;32:883–902. doi:10.1007/s40273-014-0170-x
      OpenUrlCrossRefPubMed
    23. ↵
      1. Krucien N,
      2. Gafni A,
      3. Pelletier-Fleury N
      . Empirical testing of the external validity of a discrete choice experiment to determine preferred treatment option: the case of sleep apnea. Health Econ 2015;24:951–65. doi:10.1002/hec.3076
      OpenUrl
    24. ↵
      1. Linley W,
      2. Hughes D
      . Decision-makers’ preferences for approving new medicines in wales: a discrete-choice experiment with assessment of external validity. Pharmacoeconomics 2013;31:345–55. doi:10.1007/s40273-013-0030-0
      OpenUrlCrossRefPubMed
    25. ↵
      1. Ryan M,
      2. Watson V
      . Comparing welfare estimates from payment card contingent valuation and discrete choice experiments. Health Econ 2009;18:389–401. doi:10.1002/hec.1364
      OpenUrlCrossRefPubMed
    26. ↵
      1. Lambooij MH,
      2. Irene A,
      3. Veldwijk J, et al
      . Consistency between stated and revealed preferences: a discrete choice experiment and a behavioural experiment on vaccination behaviour compared. BMC Med Res Methodol 2015;15:19. doi:10.1186/s12874-015-0010-5
      OpenUrlCrossRefPubMed
    27. ↵
      1. Mark TL,
      2. Swait J
      . Using stated preference modeling to forecast the effect of medication attributes on prescriptions of alcoholism medications. Value Health 2003;6:474–82. doi:10.1046/j.1524-4733.2003.64247.x
      OpenUrlPubMed
    28. ↵
      1. Arrow K,
      2. Solow R,
      3. Portney PR, et al
      . Report of the NOAA Panel on Contingent Valuation, US National Oceanic and Atmospheric Administration (NOAA), 1993. http://www.economia.unimib.it/DATA/moduli/7_6067/materiale/noaa%20report.pdf
    29. ↵
      1. Louviere J,
      2. Flynn T,
      3. Carson R
      . Discrete choice experiments are not conjoint analysis. J Choice Model 2010;3:57–72. doi:10.1016/S1755-5345(13)70014-9
      OpenUrl
    30. ↵
      1. Louviere J,
      2. Hensher D,
      3. Swait J
      . Stated choice methods analysis and applications. Cambridge (UK): Cambridge University Press, 2000.
    31. ↵
      1. Lancaster KJ
      . A new approach to demand theory. J Polit Econ 1966;74:132–57. doi:10.1086/259131
      OpenUrlCrossRefWeb of Science
    32. ↵
      1. Maviglia SM,
      2. Zielstorff RD,
      3. Paterno M, et al
      . Automating complex guidelines for chronic disease: lessons learned. J Am Med Inform Assoc 2003;10:154–65. doi:10.1197/jamia.M1181
      OpenUrlAbstract/FREE Full Text
    33. ↵
      1. Kawamoto K,
      2. Houlihan CA,
      3. Balas EA, et al
      . Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765. doi:10.1136/bmj.38398.500764.8F
      OpenUrlAbstract/FREE Full Text
    34. ↵
      1. Kerr N
      . Anonymity and social control in social dilemmas. In: Foddy M, Smithson M, Schneider S, et al. eds. Resolving social dilemmas: dynamics, structural, and intergroup aspects. Philadelphia (PA): Psychology Press, 1999:103–9.
    35. ↵
      1. Kesselheim AS,
      2. Cresswell K,
      3. Phansalkar S, et al
      . Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011;30:2310–7. doi:10.1377/hlthaff.2010.1111
      OpenUrlAbstract/FREE Full Text
    36. ↵
      1. Veldwijk J,
      2. Lambooij MS,
      3. De Bekker-Grob EW, et al
      . The effect of including an opt-out option in discrete choice experiments. PLoS ONE 2014;9:e111805. doi:10.1371/journal.pone.0111805
      OpenUrl
    37. ↵
      1. Lancsar E,
      2. Louviere J
      . Conducting discrete choice experiments to inform healthcare decision making: a user's guide. Pharmacoeconomics 2008;26:661–77. doi:10.2165/00019053-200826080-00004
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Parker LE,
      2. Ritchie MJ,
      3. Kirchner JAE, et al
      . Balancing health care evidence and art to meet clinical needs: policymakers’ perspectives. J Eval Clin Pract 2009;15:970–5. doi:10.1111/j.1365-2753.2009.01209.x
      OpenUrlPubMed

    Footnotes

    • Contributors CLG and JWH made substantial contributions to the conception and design of this project. CLG, JWH, DM and JND assisted with data acquisition, analysis and interpretation of data for this manuscript. CLG drafted the manuscript, and JWH, DM and JND revised it critically for important intellectual content. All authors approved of the final version to be published. CLG is the guarantor.

    • Funding The original BEARI trial was supported by the American Recovery & Reinvestment Act of 2009 (RC4 AG039115) from the National Institutes of Health/National Institute on Aging and Agency for Healthcare Research and Quality (Dr Doctor, University of Southern California). The project also benefited from technology funded by the Agency for Healthcare Research and Quality through the American Recovery & Reinvestment Act of 2009 (R01 HS19913-01) (Dr Ohno-Machado, University of California, San Diego). Data for the project were collected by the University of Southern California's Medical Information Network for Experimental Research (Med-INFER) which participates in the Patient Scalable National Network for Effectiveness Research (pSCANNER) supported by the Patient-Centered Outcomes Research Institute (PCORI), Contract CDRN-1306-04819 (Dr Ohno-Machado). JND has support from NIH and AHRQ for the submitted work.

    • Competing interests None declared.

    • Ethics approval The study was approved by the University Park Institutional Review Board at the University of Southern California (HS 11-00249).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.