Article Text
Abstract
Allocation decisions in emergency medicine must occur when demand for emergency services exceeds supply. In many circumstances, strong clinical or cost evidence upon which to base allocation decisions is lacking. In these circumstances, patient or community preference may be used to inform decisions. If preference is to be incorporated into allocation decision-making, scientifically rigorous quantitative methods should be chosen for measuring preference. This article describes the theoretical background, advantages, risks and applications of discrete choice experiments for measuring patient preference in emergency medicine.
- Choice behaviour
- cost effectiveness
- emergency care systems
- emergency medicine
- patient preference
- resource allocation
Statistics from Altmetric.com
- Choice behaviour
- cost effectiveness
- emergency care systems
- emergency medicine
- patient preference
- resource allocation
A woman presents to a tertiary emergency department (ED) late on a Sunday evening with a contaminated laceration on the sole of her foot, having stepped on glass at the beach. She is triaged with a low urgency score to the waiting room, to be seen in a section of the department staffed by junior medical staff without dedicated specialist staff. She is informed of the potential long wait to be seen in the ED and the availability of a high fee primary care clinic 500 m away that can see her within 20 min. She also knows that she can see her regular family doctor tomorrow morning.
Economics is the science of choice. The choices that individuals such as this patient make are related to both their preferences and to any constraints that limit the options from which they can choose. In making a choice, needs and wants are weighed against preferences and constraints. Choice is driven by preference, and economists often refer to strength of preference as ‘utility’. Individuals are assumed to weigh up a number of alternatives and make choices that maximise their utility. Cumulatively, these choices create demand for finite resources.
In this example, a patient is faced with a number of possible options, all of which have pros and cons. She can wait in the ED to see junior staff and incur minimal cost; she can pay a large cost out of her own pocket to be seen quickly in the private clinic; or she can go home and wait until tomorrow to see her regular doctor, with whom she is comfortable and familiar. She will choose the option that she most prefers, after weighing up the advantages and disadvantages of each option.
Although not as obvious, all aspects of the care of this patient to this point have also been determined by choices. At a societal or policy level, choices have been made to fund health services in preference to others such as education or welfare. Within the health system, highly specialised tertiary emergency services have been funded in preference to other health services. Within the ED, resources are distributed in favour of some patients over others: consider in this example the triage score, the assignment of the patient to the waiting room and the decision to have her seen by junior rather than senior staff. It is uncertain how, if at all, community or patient preferences have been incorporated into these allocation decisions.
In this article, we will provide an overview of the methods that can be used to assess preferences and how they can be applied in the context of emergency medicine. We will focus on quantitative methods, in particular, the rationale for using discrete choice experiments (DCE) to enable informed decisions regarding many systems of care problems that confront ED.
Why might determining patient preferences be relevant in the ED?
Demand for ED services is growing rapidly—in the USA between 1993 and 2003 the number of ED attendances rose 26%; comparatively the population increased by 12%.1 2 Comparable trends are seen worldwide. Several broad responses to this surge in demand for ED care are possible.
Current standard of care is maintained, but only with an increase in resources commensurate with the increased demand. This is generally the solution advocated by clinicians. It ignores the basic economic principle of opportunity cost, namely that in the context of a finite budget, increased funding to ED must be accompanied by reduced funding to another area within the hospital, within the health budget, or outside the health budget.
Current standard of care is maintained, but with no commensurate increase in resources. In the absence of reducing demand back to pre-surge levels, this can only be achieved by more efficient use of available resources. This is the solution most likely to be advocated by government or private funders of services, as it does not involve any additional expenditure.
Standard of care is revised, for example by providing fewer services. This is analogous to rationing care and must occur whenever demand exceeds available resources. Rationing is a daily clinical reality for the ED physician, although it is often unplanned or ad hoc. Unpredictably delayed waiting times for patients is an example of ad hoc rationing.
Rationing and priority setting is rarely explicitly advocated by clinicians, funders or patients. However, healthcare rationing is essentially inevitable3 and therefore we need to consider how best to set priorities. A systematic and evidence-based approach to priority setting in the ED, as opposed to ad hoc rationing, should be encouraged. In some circumstances, rationing decisions may appropriately be made without reference to patient preferences when the clinical and economic evidence in favour of one strategy is overwhelming. This is sometimes known euphemistically as ‘stewardship’;4 the Ottawa ankle rules rationing ankle x-rays are a well-known example. However, in many circumstances such evidence is equivocal or lacking, and public consultation and preferences are one important source of evidence upon which to base allocation decisions in health care, given they are the end-users of the service; other factors are also likely to play a role, but will not be discussed here.
Methods for determining patient preferences in ED research
If patient preferences are used as a source of evidence for allocation and priority-setting decisions, the most methodologically rigorous means of evaluating preferences should be adopted. Ryan et al5 have conducted a detailed review of the techniques in the medical literature for determining patient preferences, which they broadly classify as qualitative and quantitative techniques. Only a limited number of methods described under these two headings are well represented in emergency medicine research.
Qualitative techniques enable an exploration of behaviour that does not lend itself to numerical analysis—preferences are then uncovered through thematic analysis. Qualitative explorations of preferences do not allow the estimation of the relative importance of particular aspects of a service, but can provide an understanding of patient perceptions about their experiences. ED researchers have used focus groups6 or one-on-one interviews7 8 as qualitative methods of determining patient preferences.
For quantitative determination of patient preferences in ED research, studies have predominantly adopted the simplistic approach of rating or ranking exercises using questionnaires9–13 or other closed question surveys.14 15 Statistical analysis in these studies is largely descriptive. However, a small number of ED researchers have used choice analysis rating exercises with more sophisticated analysis to quantify preference, whether through the forced ranking of scenarios16 17 or the use of a DCE.18 These more sophisticated techniques can quantify the relative importance of various attributes or factors of ED care and how much they contribute individually and collectively to decision-making.
Background and conceptual framework of DCE for determining patient and consumer preferences
DCE methods have developed from work that began in the 1960s, for which US econometrician Daniel McFadden shared the 2000 Nobel Prize in economics. In DCE respondents are provided with scenarios in which they choose between two or more alternatives. The alternatives are defined by multiple attributes, and the analysis (often a logit analysis) estimates the relative influence of attributes on the choices made by respondents. The levels of the attributes are varied across a plausible range, and this enables estimates of both the overall contribution of each attribute to the choice, and the trade-off respondents are willing to accept between attributes.
A DCE assumes respondents choose the option that is most preferred, or has the highest ‘utility’ or ‘value’. By observing respondent choices for different scenarios, it is possible to determine a utility function that describes numerically the value that respondents attach to different healthcare services. These utility functions estimate the weights that individuals attach to each of the attributes used in the experiment and, as such, allow for an estimation of the relative importance of each attribute. Covariates such as gender or age may also enter the utility functions of the model as explanatory variables. DCE can thus statistically relate the choice made by each person to the attributes of the person as well as the attributes of the alternatives within the scenario. Importantly, DCE models provide numerical outputs that enable preferences to be measured and quantified, and allow the trade-offs between attributes to be assessed, setting DCE apart from other quantitative methods for determining patient preference.
The most common form of utility function is Unsj=Vnsj+εnsj. Here Unsj denotes the utility of alternative j as perceived by respondent n in choice situation s. Unsj is made up of two separate components, an observed (Vnsj) component and unobserved and unmodelled (εnsj) component. εnsj are the factors that influence choice that are not measured by the analyst as they are neither attributes that make up the alternatives or characteristics of the individual making the choice that can be modelled as covariates.
Vnsj is assumed to be the (often linear) relationship of observed attribute levels of each alternative, x, and their corresponding parameter weights, β. In the simplest form of model, the multinomial logit model, parameter weights for each attribute do not vary over respondents, such that Vnsj may be represented as the sum of the weighted attributes. However, a more complex model specification in which some or all of the parameter weights are represented as distributions over the sampled population (rather than point estimates) can also be specified—the mixed multinomial logit model.
Both of these models assume the unobserved components of all alternatives are identically and independently distributed (and so not correlated with each other). A number of advanced models exist that relax this assumption, but to date, they have been rarely used in health economics literature.
Methodology of DCE
Figure 1 summarises the steps involved in the conduct of DCE.19 20 It is important that the alternatives presented, the attributes making up the alternatives, and the range of levels ascribed to the attributes (stage 2) are realistic. Previously published research, audits, qualitative interviews and expert opinion can all be used as sources to determine what attributes and levels are most important to be represented in choice sets.
A large amount of debate exists regarding the optimal method for designing DCE (stage 3). Some experts advocate that designs are constructed in such a way as to maximise the difference of the attribute levels shown across different alternatives.21 The appeal of this approach is that respondents are forced to make trade-offs on every attribute of the design because no two attributes in any choice situation will take the same value. Others believe this approach is not paramount.22 In either case, it is important to note that the design of the experiment is vitally important for the subsequent gathering and interpretation of data. The actual choice sets to be presented to respondents are then generated using specialist software. The order in which any individual respondent is given the choice sets will typically be randomised to minimise ordering or learning effects that might occur if all respondents receive the sets in the same order (stage 4). Usually, analysts will pilot an experiment on a small number of respondents and refine it before its application in the full respondent sample. Consideration must also be given to how the context of delivering information to individuals in the course of conducting the experiment (stage 5) may influence results.23 Data may be collected in a variety of ways, for example, postal surveys, personal interviews, web-based computer programs.
Example of DCE
Let us reconsider the original scenario in the context of a DCE. Figure 2 contains two typical choice sets of three alternatives each comprising four attributes, based upon some plausible service options that might be available for the scenario. Depending on experimental design and sample size calculations, up to many hundreds of respondents will be given a substantial number of such choice sets in which the levels of the attributes making up the alternatives, or the attributes themselves, are varied. Interactions between attributes and covariates such as age, gender, socioeconomic status, education and previous recent experience with ED can be determined. Utility functions are derived for each choice set and responses statistically pooled, enabling the calculation of a number of relevant and useful pieces of information.
Estimates of the marginal effect (importance) of each attribute on overall preference. Here, as waiting time is an attribute, one can estimate the relative importance of waiting time in determining patient preference for acute care.
Estimates of marginal rates of substitution between attributes giving an indication of the extent to which respondents are prepared to trade off one attribute for another. Here, as waiting time and cost are offered as attributes, the marginal rate of substitution between these reflects the total waiting time patients are willing to accept to access a low-cost service, or alternatively how much they are willing to pay for a reduction in waiting time.
An indication of the predicted values or ‘market shares’ associated with different parameter levels within the estimated utility functions. This can be used to ascertain how far attribute levels need to change from current values in order to improve the acceptability of the services offered to a point at which someone would choose the new service over the existing service.
Although cost is presented here as an attribute, some health systems such as in the UK have no effective private alternative to publically funded free health care. Cost is not an essential attribute for the use of DCE; in such circumstances choice sets can be constructed that enable respondents to trade off on other attributes that define alternatives (figure 3).
Important assumptions and limitations of DCE relevant to ED research
Several assumptions are fundamental to DCE methodology, restricting the application of DCE in some circumstances.
The number of alternatives from which to choose is finite—the dependent variable in a discrete choice model cannot be continuous.
Alternatives are collectively exhaustive (all possible alternatives are presented) and mutually exclusive (only one choice per choice set may be made). Conjoint analyses16 17 involving the ranking of scenarios do not have these two restrictions but are associated with other mathematical and psychological shortcomings.24
When modelling both observed and unobserved effects, individual respondents are assumed to have ‘perfect’ information upon which they base their choice.
Furthermore, a number of weaknesses of DCE have been described.25 As with any clinical research, bias will arise if there is an unrepresentative selection of participants; the experiment has unrealistic context (eg, combinations or levels of attributes); there is incomplete repeatability of the experiment; or there is poor experiment design threatening the internal validity of the study. Some have questioned the theoretical underpinnings of DCE, arguing that DCE do not predict behaviour because a body of psychological research suggests individuals often employ cognitive shortcuts based on ‘rules of thumb’, known as heuristics, in making decisions rather than trade between attributes or their levels.26 Much of the cutting edge research in this field of economics is directed at improving experimental design and statistical modelling in response to these criticisms. It is inherent upon those using DCE to be aware of these limitations and seek expert advice at all stages of the DCE (figure 1).
Potential application of DCE in emergency medicine
In the example used throughout this article, we have shown how DCE can be used to quantify preferences when comparing ED to alternative (competing) acute primary care services, estimating the degree of preference respondents have for attributes that describe the alternatives. This will enable the optimal design of ED services, such that resources are allocated appropriately to ensure that neither too much nor too little resourcing is provided to manage those patients when alternative services exist.
In many situations, patients have no real choice but to come to ED for their illness or injury. In this situation DCE are still highly useful as a rigorous tool to maximise quality of care and patient wellbeing within ED, providing information that may not be readily apparent to emergency clinicians with an interest in quality on what matters most to patients. DCE have been used as a tool in chronic disease management to enhance shared decision-making between patients and physicians,27 28 and the same concept can be applied to the acute care setting. Table 1 contains a summary of measures of quality.29 30 DCE can conceivably be applied to answer questions in these areas, even in regard to questions of technical quality. For example, a DCE could be conducted to estimate how sensitive a cranial CT scan would need to be before it could be predicted that a certain percentage of patients, given accurate risk information, would decline a lumbar puncture if they had a thunderclap headache and a normal CT. While the response of any one individual to this or any other scenario is still unpredictable, accurate predictions about population behaviour can be made.
Conclusions
Allocation decisions in emergency medicine are inevitable. In addition to evidence from clinical or cost effectiveness studies, evidence of patient and community preferences can be used to inform allocation decisions. DCE are a highly rigorous tool for measuring preferences with potentially wide application in emergency medicine research.
References
Footnotes
Competing interests None to declare.
Provenance and peer review Not commissioned; externally peer reviewed.