Some probabilistic models of best, worst, and best–worst choices
Introduction
Finn and Louviere (1992) proposed a discrete choice task in which a person is asked to select both the best and the worst option in an available (sub)set of options. They presented an analytical model for data from that task, and considered its evaluation using an experimental design ( fractional factorial) that ensures that each option, and each pair of distinct options, is presented equally often across the selected subsets of size j. Since the publication of that paper, interest in, and use of, such best-worst choice tasks has been increasing, with two recent empirical applications receiving “best paper” awards (Cohen, 2003, Cohen and Neira, 2003). In principle, best–worst tasks have a number of advantages over traditional discrete choice tasks: (1) a single pair of best–worst choices contains a great deal of information about the person's ranking of options (e.g., if there are 3 items in a set, one obtains the entire ranking of that set; if there are 4 items in a set, one obtains information on the implied best option in 9 of the 11 possible non-empty, non-singleton subsets;1 and if there are 5 items in a set, one obtains information on the implied best option in 18 of the 26 possible non-empty, non-singleton subsets); (2) best–worst tasks take advantage of a person's propensity to identify and respond more consistently to extreme options; and (3) best–worst tasks seem to be easy for people. Despite increasing use of the approach, the underlying models have not been axiomatized, leaving practitioners without clear guidelines on appropriate experimental designs, data analyses, and interpretation of results.
This paper develops theoretical results for three overlapping classes of probabilistic models for best, worst, and best–worst choices, with the models in each class proposing specific relationships between such choices. The model classes are called random ranking and random utility, joint and sequential, and ratio scale. We consider models that belong simultaneously to one or more of these classes, with the best known being the maximum-difference (maxdiff) model, which is introduced later in this section, and we formulate a number of open theoretical problems.
We now illustrate the framework, and the basic models, through the maximum-difference (maxdiff) model of best–worst choice. To do so we require some basic notation. Let T with denote the finite set of potentially available choice options, and for any subset , with , let denote the probability that the alternative x is chosen as best in X, the probability that the alternative y is chosen as worst in X, and the probability that, jointly, the alternative x is chosen as best in X and the alternative is chosen as worst in . Thus and We assume throughout the paper that for each , .
For motivational purposes only, we assume now that best and worst choices are in some sense more basic than simultaneous best–worst choices, and develop a model of the latter based on the former. So suppose that when asked to choose the best and the worst option in a (finite) set , the person simultaneously, but independently, chooses the best, respectively, the worst, option in . If the resulting options are distinct, these are reported as the best–worst pair of options in X, otherwise the person re-samples. As we show later in detail, such a process gives rise to the following representation for the best–worst choice probabilities in terms of the best and the worst choice probabilities:2 for
Now, suppose that the multinomial logit (MNL or Luce's choice) model holds separately for the best and the worst choice probabilities, i.e., there exist ratio scales b and w such that for ,Then direct substitution of (2) in (1) yields that for ,
Notice that this combined set of representations has three interesting properties—the best–worst choice probabilities are represented in (1) directly in terms of the best and worst choice probabilities, and are represented in (3) in terms of the ratio scale values that determine the best and the worst choice probabilities, with the same functional form in both representations. Thus, this aggregation method is plausible, and interesting, in that it works both at the level of the choice probabilities and at the level of the scale values. We are immediately lead to the first theoretical question, namely, are there other (ratio scale) models that satisfy this type of aggregation property, and, if so, how large is this class. The detailed formulation of this problem requires extensive further notation, and its solution the use of complex functional equation techniques. The conjectured final result is that the class of solutions is large, but in an interesting sense the individual solutions do not differ greatly from the above model.
It is important to note that the above example was for choices from a given fixed set X. Often probabilistic models of choice (or, briefly, probabilistic choice models) are assumed to be consistent over all the subsets of a finite master set . If that is assumed in the present context, and we consider the binary choice probabilities and , then it is reasonable to assume that, and to test whether, which with (2) gives that3 for each for ,and so in particular any scale transform of b is linked to the scale transform of . In the following work, we consider separately the cases where b and w are independent ratio scales, and where they are subject to common scale transforms.
The paper has the following structure. Each of the three main sections develops one of three classes of models, gives examples of that class, and presents solved, and open, theoretical problems about that class. Section 2 introduces the terminology and basic conditions, Section 3 presents random ranking and random utility models, Section 4 joint and sequential choice models, and Section 5 ratio scale models. Section 6 summarizes the results and restates the main open theoretical problems.
Section snippets
General terminology and conditions
Given a finite master set T, and a particular set X, , we refer to the set , , , , respectively, as a set of best, worst, best–worst choice probabilities (on X). We have a complete set of best, worst, best–worst choice probabilities, respectively, (on a master set T) when we have a set of best, worst, best–worst choice probabilities on each X, . Unless stated otherwise, we assume that we have a complete set of best, worst, and best–worst choice
Random ranking and random utility models
We are naturally interested in relations between these various choice and ranking probabilities. Since the focus of this paper is best–worst choice, we concentrate on how ranking probabilities might determine such best–worst choice probabilities, though we do consider one result in the other direction. There is extensive related work on how best choice probabilities determine, or are determined by, ranking probabilities (see Falmagne, 1978, Fishburn, 1994, Fishburn, 2002).
For each and rank
Joint and sequential best–worst choice models
We now motivate models through the idea that a person, in selecting the best–worst pair of options, approaches the selection of the best and the worst option independently, and follows up on these choices, as needed, to ensure that the same option is not selected as both best and worst. We first consider models where the best and worst choices are made jointly, then models where these choices are made sequentially.
A referee asked whether such an independence assumption is reasonable and
Ratio scale models
We now return to the maxdiff model, i.e., the example in Section 1, and use it to motivate a theoretical question concerning when a set of best, worst and best–worst choice probabilities are “of the same form” with the best–worst choice probabilities determined by some “functions” of the best and worst choice probabilities. Unfortunately, the notation required for the general formulation is complex, so we use the earlier example to illustrate the ideas, and include details in the Appendix. We
Summary and conclusions
We derived and discussed theoretical results for a number of best, worst and best–worst choice models, with the focus on the latter. Our results include a number of interesting theoretical relationships between these types of models, which in turn suggest a variety of tests to determine which model is most consistent with choice data. For example, the maximum-difference model is of the well-known Luce (1959), equivalently Multinomial Logit (McFadden, 1974), form with ratios of scale values, and
Acknowledgments
This research has been supported by Australian Research Council Discovery Grant DP034343632 to the University of Technology, Sydney, for Louviere, Street and Marley, and Natural Science and Engineering Research Council Discovery Grant 8124-98 to the University of Victoria for Marley. It was begun while Marley was a Visiting Researcher at Systems, Organizations and Management of the University of Groningen and supported by the Netherlands’ Organization for Scientific Research for the period July
References (23)
- et al.
Consistent aggregation of scale families of selection probabilities
Mathematical Social Sciences
(1997) - et al.
Consistent aggregation of simply scalable families of choice probabilities
Mathematical Social Sciences
(2000) - et al.
Probability models on rankings
Journal of Mathematical Psychology
(1991) A representation theorem for finite random scale systems
Journal of Mathematical Psychology
(1978)A short proof of a theorem of Falmagne
Journal of Mathematical Psychology
(2004)On “choice” probabilities derived from ranking distributions
Journal of Mathematical Psychology
(1994)Some probabilistic models of simple choice and ranking
Journal of Mathematical Psychology
(1968)The relationship between Luce's choice axiom, Thurstone's theory of comparative judgment, and the double exponential distribution
Journal of Mathematical Psychology
(1977)Generalized Thurstone models for ranking: Equivalence and reversibility
Journal of Mathematical Psychology
(1980)Categorical data analysis
(2002)
Falmagne and the rationalizability of stochastic choices in terms of random orderings
Econometrica
Cited by (455)
Bidding against the wind: A choice experiment in green energy, green jobs and offshore views in North Carolina, USA
2024, Journal of Environmental ManagementEuropean consumers' preferences for fresh fruit and vegetables – A cross-country analysis
2023, Journal of Agriculture and Food ResearchBest, worst, and best&worst choice probabilities for logit and reverse logit models
2023, Journal of Choice ModellingA variation of the cube model for best–worst choice
2023, Journal of Mathematical PsychologyCube model: Predictions and account for best–worst choice situations with three choice alternatives
2023, Journal of Choice Modelling