Measures of importance for economic analysis based on decision modeling
Introduction
When making choices between alternate uses of scare resources concern focuses both on the expected benefits and costs and the confidence that can be placed on these estimates. Thus, an important component of an economic evaluation is an analysis of the degree of uncertainty around estimates of the outcome of choice and the risk of making an incorrect decision [1], [2], [3]. This is especially the case for those studies based within a modeling framework where data inputs have been obtained from disparate sources [4].
A distinction has been made between two specific types of uncertainty [5], [6], [7], [8]. First-order uncertainty (also referred to as variability or stochastic uncertainty) relates to the randomness in the population and cannot be reduced—rather, information can allow stratification into more homogeneous population groups. Thus, variability is of importance with respect to policy decisions only in terms of the stratification of potential recipients of treatment. Second-order uncertainty (also referred to as either knowledge or subjective uncertainty) relates to the lack of confidence in a parameter estimate due to lack of knowledge. Knowledge uncertainty can be addressed through better measurement; although further knowledge may not lead to less uncertainty over the optimal decision. Knowledge uncertainty, unlike variability, can be represented by probability distributions.
Related to uncertainty around parameter estimates and the associated uncertainty in outcome measurement is the concept of risk. In this context, risk recognizes the binary nature (yes/no) of the decision regarding the decision to adopt a particular therapy for each particular patient group. Risk relates to the probability and consequences of an incorrect decision; that is, the likelihood and effects of the adoption of one option when another is superior.
Importance analysis involves the use of techniques to determine how different input parameters contribute to the uncertainty over outcomes of interest [9], [10]. Such techniques are called importance measures, and involve ranking input parameters by their contribution to uncertainty. These measures provide information on which parameters it may be worthwhile obtaining more information, although only certain techniques can actually assess the value for such information.
Importance analysis differs from standard sensitivity analysis where the aim of the analysis is to address the degree of uncertainty around an outcome measure [1]. Instead, the focus of importance analysis is to identify quantitatively those parameters that contribute most to the uncertainty. This can be argued to be more pertinent to decision makers in that it addresses decisions that have to be taken; that is, given the evidence available what is the most appropriate action and what further information should be collected.
Probabilistic methods such as Monte Carlo simulation techniques have been identified as suitable bases for the conduct of importance analysis, in that the techniques allow for the estimation of the likelihood of various output values based on a wide number of sets of input parameters generated by sampling from their probability density functions [7], [8], [10], [11].
Alternative methods for identification of the importance have been applied to models dealing with uncertainty relating to decisions addressing a wide spectrum of public and private policy issues—for example, health interventions, nuclear safety, fire safety, radioactive waste [7], [8], [9], [12], [13]. Few of these measures have been considered in economic analysis. This article presents a range of techniques available, and apply these to an economic model. The objective is to compare the techniques through a comparison of the rankings based on each technique and an analysis of the appropriateness of each technique in this specific context.
Section snippets
Identification and classification of importance measures
A thorough review of the literature using databases such as Medline and HealthStar, Internet-free text searching, and hand searching of relevant journals and articles relating to health economics, risk assessment, safety analysis, and environmental appraisal was conducted to identify potential importance measures. Measures were classified as either variance/correlation-based measures, elasticity-based measures, probability-based measures, entropy, or information value-based measures.
In defining
Importance of individual parameters
Table 1 details the rank ordering of the importance of parameters based on a threshold value of $50,000. All techniques ranked utility values as the most important parameters followed by transition probabilities, with costs being the least important. However, within these broad categories there is divergence in ranking.
For utility parameters, the techniques that consider the probability of incorrect decision and the associated consequences had different rankings compared to techniques that
Conclusions
The degree of uncertainty around parameter estimates within economic decision models generally leads to uncertainty propagation whereby the “true value” of the net benefits of a therapy is unknown. Thus, it is necessary when interpreting the results of the analysis not just to determine what is the optimal treatment choice given the information available, but also to assess the relative importance of parameters based on their contribution to such uncertainty. Techniques for assessing the
References (36)
Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal
Reliab Eng Syst Saf
(1993)- et al.
Importance measures for nuclear waste repositories
Reliab Eng Syst Saf
(2000) - et al.
Priority assessment of toxic substances in life cycle assessmentPart II: assessing parameter uncertainty and human variability in the calculation of toxicity potentials
Chemosphere
(2000) - et al.
Eutrophication in peel inlet,II. Identification of critical uncertainties via generalized sensitivity analysis
Water Res
(1980) Addressing uncertainty in medical cost effectiveness analysis. Implications of expected utility maximization for methods to perform sensitivity analysis and the use of cost-effectiveness analysis to set priorities for medical research
J Health Econ
(2001)- et al.
The economic evaluation of pharmacotherapies for Parkinson's disease treatment
Parkinson's and related disorders
(2003) - et al.
Uncertainty in the economic evaluation of health care technologies—the role of sensitivity analysis
Health Econ
(1994) Handling uncertainty in cost-effectiveness models
Pharmacoeconomics
(2000)- et al.
Sensitivity analysis in health economic and pharmacoeconomic studies. An appraisal of the literature
Pharmacoeconomics
(1997) Economic evaluation of pharmaceuticals. Frankenstein's monster or vampire of trials?
Med Care
(1996)
Representing both first- and second-order uncertainties by Monte Carlo simulation for groups of patients
Med Decis Making
Estimating CE ratios under second-order uncertainty—the mean ratio versus the ratio of means
Med Decis Making
Fire safety design based on calculations
Fire Saf J
Sensitivity analysis
Probabilistic sensitivity analysis using Monte Carlo simulation: a practical approach
Med Decis Making
A comparison of sensitivity analysis techniques
Health Phys
Sensitivity analysis for healthcare models fitted to data by statistical methods
Health Care Manag Sci
Parameter uncertainty and variability in evaluative fate and exposure models
Risk Anal
Cited by (34)
Global sensitivity analysis for a prediction model of soil solute transfer into surface runoff
2021, Journal of HydrologyCitation Excerpt :The sensitivity analysis (SA), which involves testing the relative impact of model parameters on the model predictions, has been proved to be an efficient way to rank model parameters importance in prediction models, and has been applied in many fields. Hamby (1994) had reviewed techniques for parameter SA of environmental models, and Coyle et al. (2003) had discussed the SA measures employed in the field of economics. Saltelli et al. (2005); (2012;) had focused on SA in chemical models.
Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications
2015, Journal of HydrologyCitation Excerpt :Many reviews of SA methods have been conducted in different fields. In particular, Hamby (1994) reviewed the literature on parameter SA for environmental models; Frey and Patil (2002) and Mokhtari and Frey (2005) reviewed the SA methods used for food safety; Coyle et al. (2003) discussed the SA measures employed in the field of economics; Saltelli et al. (2005, 2012) focused on SA in chemical models; Borgonovo (2007) investigated sensitivity and uncertainty measures; Mishra et al. (2009) reviewed the global SA methods used in groundwater models; Peter and Dwight (2010) discussed numerical SA approaches for aerodynamic optimization; Perz et al. (2013) reviewed the global SA and UA methods applied to ecological resilience; Tian (2013) summarized the application of SA methods to building energy analysis; and Wu et al. (2013) review recent advances in SA for infectious disease models. Some of these reviews explicitly highlighted the advantages and disadvantages of various methods and they provided very good summaries of these topics.
Multi-method global sensitivity analysis of flood inundation models
2008, Advances in Water ResourcesEfficient computation of partial expected value of sample information using Bayesian approximation
2007, Journal of Health Economics