Discrete choice experiments

Discrete choice experiments are based on the premise that, firstly, any good or service can be described by its characteristics (or attributes) and, secondly, the extent to which an individual values a good or service depends upon the nature and levels of these characteristics. The technique involves presenting individuals with choices of scenarios described in terms of characteristics and associated levels. For each choice they are asked to choose their preferred scenario. Response data are modelled within a benefit (or satisfaction) function which provides information on whether or not the given characteristics are important; the relative importance of characteristics; the rate at which individuals are willing to trade between characteristics; and overall benefit scores for alternative scenarios.^1–4

DISCRETE CHOICE EXPERIMENTS IN HEALTH CARE

Ryan et al⁵ have systematically reviewed the application of DCEs in health care. The technique was initially applied in an economic evaluation framework in an attempt to go beyond health outcomes and to take account of “non-health outcomes” and “process attributes” in the delivery of health care.⁴ Following this, the technique has been applied to address a wide range of issues including estimation of benefits within health technology assessments; analysis of patient/consumer and professional decision making; and developing prioritisation frameworks.⁵

The increased number of applications has been accompanied by investigation of methodological and theoretical issues. Few difficulties have been reported when answering DCEs and the technique has been well received by policy makers.⁵ Validity has been addressed at a number of levels. High levels of internal validity—that is, results consistent with a priori expectations—have been recorded and convergent validity—that is, results move in line with those of other instruments measuring the same construct—has been demonstrated with respect to standard gamble and willingness to pay.⁵ The technique has been shown to be relatively insensitive to both the ordering and levels of attributes.⁶ At the theoretical level three key axioms underlying the technique—completeness, stability, and rationality—have been investigated with encouraging results.^7–9

Case study: preferences for a specialist nurse in the provision of rheumatology care

BACKGROUND

This application considers patient preferences for potential benefits from a change in the organisation of service delivery. The setting is a rheumatology outpatient clinic but the approach has general applications. The issue arose from difficulties in meeting the demand for outpatient services against a background of changes in junior doctors' hours and constrained resources. One possible solution was the introduction of a specialist nurse practitioner to hold review clinics for patients with stable rheumatoid arthritis.

The DCE approach was chosen to value benefits because it allows consideration of health outcomes (such as pain levels) and process attributes (such as waiting time and continuity of care). More traditional means of measuring benefits have concentrated on improvements in health outcomes using either hard clinical outcomes or Quality Adjusted Life Years.¹⁰¹¹

SUBJECTS AND METHODS

The study population comprised patients attending the rheumatology outpatient clinics at Aberdeen Royal Infirmary. A researcher (AB) attended clinics to collect the data on various days of the week and at various times. Two hundred questionnaires were handed out and the researcher was available to answer questions. This number was sufficient to carry out statistical analysis on both the attributes and subgroups (see below).¹²¹³

The attributes for the study were determined by reviewing the literature on extending the role of the specialist nurse in rheumatology.^10111415 These are shown in table 1 together with the levels assigned to them (and their coding for the regression analysis). The attributes chosen relate to both health outcomes and process attributes. Levels were chosen to reflect both the current situation with doctor led care and the likely situation if nurse led care was introduced.

The attributes and levels chosen resulted in 192 (2⁴ × 3¹ × 4¹) possible clinic configurations. The experimental design package speed was used to reduce these to a manageable number while still being able to infer benefit scores for all possible configurations.¹⁶ A main-effects linear model was assumed. This technique resulted in a design of 16 orthogonal scenarios (absence of multicollinearity between scenarios) which were converted into eight discrete choices and an attempt was made to maintain orthogonality in differences. Respondents were presented with these eight choices and, for each, asked whether they preferred clinic A or B (see table 2 for an example of a discrete choice).

From the response data the following equation was estimated:

ΔB = α₁ΔStaff + α₂ΔWait + α₃ΔContinuity + α₄ΔPain + α₅ΔPhone + α₆ΔLength + e where ΔB is the change in benefit in moving from clinic A to clinic B and the independent variables are the differences in the attributes of the two clinics, as defined in table 1. Response data were analysed in limdep¹⁷ using a random effects probit regression model (to take account of multiple observations from respondents). A general to specific approach was used. The general model included all the attributes and the specific model included only those significant at the 5% level. From the above equation the following were investigated:

• the relative importance of the attributes (as indicated by the significance of the coefficients α and their size);

• how individuals trade between these attributes—that is, the rate at which they give up one unit of an attribute for an increase in another. This is shown by the ratio of the coefficients—for example, α₁/α₂ shows how much waiting time an individual is willing to trade to have their most preferred or least preferred member of staff;

• benefit (satisfaction) scores for alternative ways of providing the services.

For more details on the underlying economics of the model the reader is referred to Louviere et al.¹

Information was also collected on respondents' age, sex, length of time attending clinic and time since last appointment, current physical and mental health rating, time to complete the questionnaire, and ease of completion (on a scale from 1 to 5, where 1 = very easy and 5 = very difficult). For policy purposes it may be useful to know how preferences vary across individuals. Response data were analysed separately according to length of time attending the rheumatology clinic: first time attendees; 1 week–12 months; 13–36 months; 37–96 months, and 97+ months.

“Rationality” of responses was assessed by including two dominant options—that is, choices where one option was “better” than another on all levels. Respondents were expected to choose the “better” configurations; those who “failed” one test were assumed to have done so through random error whereas those who “failed” both tests were defined as “irrational” and were dropped from the regression analysis.

RESULTS

Of the 200 questionnaires distributed, 189 (94.5%) were returned. The characteristics of the respondents are shown in table 3. The higher prevalence of women in the study sample is not surprising since rheumatoid arthritis is more prevalent in women. Of the 189 respondents, 41 “failed” one rationality test and 15 “failed” both tests. Given that the questionnaire was distributed in a clinic setting (reducing the chance of the sample being biased through non-response by those not understanding the questionnaire as can be the case for postal methods), the high response rate, and the high proportion of older respondents, these rationality results are encouraging.

Estimated coefficients and their standard errors for the estimated benefit equation are shown in table 4. The results from the specific model are discussed here. Given the coding in table 1, the positive signs indicate that patients prefer continuity in staffing, reduced pain levels, and the introduction of a phone-in/advice line service. The negative sign on “wait” indicates that patients prefer to have shorter waiting times. These results support the theoretical validity of the technique.

Introducing a phone-in/advice line service increases benefit by 0.97 and having staff continuity increases benefit by 0.45. Although waiting time has the smallest coefficient, it must be noted that this attribute is measured in minutes. While a change in waiting time of 1 minute may not be as important as a marginal change in any of the other three attributes, assuming a linear utility function, the change in benefit resulting from a 30 minute change is equal to 0.477 (0.0159*30), which outweighs the benefit of a marginal change in both staff continuity and improvement in pain.

The value of individual attributes can be estimated in terms of the time respondents are willing to give up for that attribute. For example, respondents are willing to wait 61 minutes longer in the waiting area if a phone-in/advice line is available (0.974/0.0159) and 29 minutes longer in the waiting room to have staff continuity (0.454/0.0159). This implies that a phone-in service is twice as important as staff continuity (also indicated by the relative size of the coefficients). The monetary value patients would be willing to pay for improvements in these attributes could be estimated by assuming that waiting time in clinics has a similar value to waiting time for public transport, which has been estimated to be £0.15 per minute.¹⁸ This implies a marginal value of £9.15 per visit for the introduction of a phone-in service and £4.35 per visit for staff continuity.

Benefit scores and time values are presented in table 5 for different ways of providing the service. Example 1 illustrates the case when moving from the current situation in Aberdeen to a hypothetical clinic that also provides staff continuity and a phone-in/advice line. The benefit score is 1.4279, with an associated value of time of 90 minutes. A benefit score and time value greater than zero indicates that patients would benefit from this change. Thus, assuming other attributes remain constant, the introduction of these services would benefit patients. The second example highlights moving from a situation where staff continuity is provided (with small improvements in pain) to a drop in outpatient clinic where there is no waiting time and no staff continuity. This benefit score is negative (–0.1358) with an associated value of time of –8 minutes, indicating that the lower waiting time is not sufficient to compensate for the loss of staff continuity. These conclusions are echoed by the implied monetary value of the attributes. At a value of £0.15 per minute the implied value of the move in example 1 is £13.50 per visit and £–1.28 per visit in example 2, indicating that respondents would be willing to pay to avoid this move.

Generally, a negative benefit score means that patients are worse off from the service change, a benefit score of zero would leave overall benefit unchanged, and a benefit score greater than zero implies an increase in benefit. Policy makers should attempt to reach the highest benefit score within the available budget.

Table 6 shows the results for the segmented model. All five groups valued staff continuity and the introduction of a phone-in/advice service. Waiting time was important to all groups other than first time attendees. This latter group also preferred to see a specialist nurse (as indicated by the positive sign on the coefficient). Pain was only important to the group who had been attending for 1–12 months. The results indicate that different groups may have different preference structures. Future work should investigate the reasons for these differences.

Discrete choice experiments: research issues for the future

The case study illustrates how DCEs can be used to address policy relevant issues. Given the absence of a monetary attribute, willingness to pay was implied from the value of reducing waiting time for public transport. The evaluation of staff continuity and phone-in services are likely to be important in the coming years.¹⁹²⁰ However, a number of issues are raised in the design and analysis of DCE studies of which the reader should be aware.

EXPERIMENTAL DESIGN

Experimental methods employed in the health economics literature have mainly been modifications of standard linear experimental designs. Such designs result in a number of orthogonal scenarios. For example, in the above case study an orthogonal fractional factorial design was created using speed.¹⁶ When using DCEs these scenarios must be placed into choice sets which are orthogonal in differences, have minimum overlap, and be balanced (occurring an equal number of times).²¹²² Pairing these choice sets in order to maintain these properties can be challenging. Various methods are currently adopted, including pairing the choices manually and checking the statistical properties; comparing each scenario with the same base comparator; and the use of computer software. Future work should examine the sensitivity of results to the experimental design employed.

DATA COLLECTION AND METHODS OF ANALYSIS

Once an efficient choice set has been devised, a number of methods exist for eliciting responses. While simple binary response models, analysed using random effects probit and logit models, have proved the most common approach, future studies should consider offering respondents more than two options. While more information is obtained by the researcher, issues are raised concerning the method of analysis. The standard approach to analysing such data is conditional probit or logit models. Although the latter is computationally easier, it is known to violate the assumption of independence compared with irrelevant alternatives assumption (IIA)—that is, the ratio of probabilities for any two alternatives is assumed to be independent of the attribute levels in the third alternative. Furthermore, multinomial logit models do not account for multiple observations from individuals. Revelt and Train²³ have developed the random parameter logit model to take account of these limitations and future research should explore these modelling techniques with a DCE framework.²⁴

ESTIMATION OF BENEFIT/UTILITY SCORES

For simplicity the model used in this study assumed an additive linear relationship between the choice of clinic and attribute levels. The additive assumption implies that there are no interaction terms between attributes—that is, the level at which one attribute is set does not effect preferences for another. The linear assumption implies that the effects of the attributes on choice do not change as the level of that attribute changes—that is, each additional unit change in waiting time has the same marginal effect on choice. Research from outside health economics has shown that alternatives to the linear additive model seldom result in a significantly better fit.⁴⁵ Future work applying DCEs to health care should explore going beyond the linear additive model. Such modelling needs to be built into the experimental design of the study and requires data to be collected on a larger choice set.

Given that cost was not included as an attribute in this study, a benefit score was estimated for different ways of providing the service. Environmental economists have developed a similar scoring method termed the “attractiveness index”.²⁶²⁷ While this index is potentially useful at the policy level, questions have been raised concerning its properties—is it cardinal or ordinal? Future work should explore this.

This study also valued attributes in terms of waiting time. Users of health care may be more used to trading this attribute and future work should explore its use in estimating the value of alternative ways of providing a service. From such data it is possible to estimate the monetary value of time. The method adopted here assumed that the value of time when waiting for public transport is the same as waiting time in a healthcare setting. Such an assumption should be tested.

ESTIMATING WILLINGNESS TO PAY

Including cost as an attribute in DCEs allows willingness to pay (a monetary measure of benefits) to be indirectly estimated.⁴ The contingent valuation method, which has most commonly been used to elicit maximum willingness to pay,²⁸ has been shown to be subject to a number of problems including lack of scope sensitivity,²⁹ strategic biases,³⁰ and warm glow.³¹ Future work should examine the extent to which DCEs overcome these limitations.³²³³

TESTING THEORETICAL AXIOMS

This study tested for rationality of responses by including dominance tests. A number of issues are raised here. Firstly, when considering responses, research should investigate the reasons for “irrational responses”. There is a growing literature from both psychology and economics indicating that apparently “irrational” responses can be rationally explained,⁸³⁴³⁵ and future work should explore “irrational” responses in more detail. Qualitative research techniques will prove useful here.³⁶ Secondly, it may be argued that conventional dominance test are “easy” to satisfy and that different results regarding rationality may be reached if different tests are employed. Alternative definitions of rationality should be explored. Possibilities include testing for transitivity and Sen's contraction (α) and expansion (β) properties. Such tests are arguably more demanding than dominance tests and are therefore more difficult to satisfy.⁷⁹

DCEs also assume a compensatory decision making process—that is, when presented with choice sets, respondents consider all attributes. Tests of this axiom have been limited to examining whether individuals always make decisions according to the best level of a given attribute. However, such a decision making process may be simply a result of the attribute levels presented in the experiment. Alternative methods of examining the trading axiom should be examined. The psychology literature indicates that respondents often employ simplifying heuristics (decision making rules) in decision making, employing “fast and frugal heuristics”.³⁷ Such decision making heuristics may also explain apparent irrationalities.³⁸ Future work should explore the compensatory decision making assumption in detail, with consideration given to the relationship of cognitive strategies to the complexity of the choice sets presented.

Conclusions

This paper has considered the role of DCEs when eliciting preferences in the delivery of health care. While DCEs have been applied in a number of healthcare settings and potentially offer useful information to aid decision making, methodological issues should continue to be addressed. Important areas of future research relate to experimental design, alternative methods of data collection and analysis, and investigation of the underlying axioms of economic theory. Collaborative work with psychologists and qualitative researchers will prove useful when investigating these issues.