Article Text
PDF
Abstract
Objectives To evaluate the incremental value of new drugs across disease areas receiving favourable coverage decisions by the UK’s National Institute for Health and Care Excellence (NICE) over the past decade.
Design, setting, and participants This cross-sectional study assessed favourable appraisal decisions of drugs between 1 January 2010 and 31 December 2020. Estimates of incremental benefit were extracted from NICE’s evidence review groups reports.
Primary outcome measure Incremental benefit of novel drugs relative to the best alternative therapeutic option, expressed in quality-adjusted life-years (QALYs).
Results 184 appraisals of 129 drugs provided QALYs. The median incremental value was 0.27 QALY (IQR: 0.07–0.73). Benefits varied across drug-indication pairs (range: −0.49 to 5.22 QALY). The highest median benefits were found in haematology (0.70, IQR: 0.55–1.22) and oncology (0.46, IQR: 0.20–0.88), the lowest in ophthalmology (0.09, IQR: 0.04–0.22) and endocrinology (0.02, IQR: 0.01–0.06). Eight appraisals (4.3%) found contributions of more than two QALYs, but one in four (50/184) drug-indication pairs provided less than the equivalent of 1 month in perfect health compared to existing treatments.
Conclusions In our review period, the median incremental value of novel drugs approved for use within the English National Health System, relative to the best alternative therapeutic option, was equivalent to 3–4 months of life in perfect health, but data were heterogeneous. Objective evaluations of therapeutic value helps patients and physicians to develop reasonable expectations of drugs and delivers insights into disease areas where medicinal therapeutic progress has had the most and least impact.
- therapeutics
- health policy
- general medicine (see internal medicine)
Data availability statement
All data are available on the NICE website. Aggregated data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We systematically compared quality-adjusted life-year (QALY) data from National Institute for Health and Care Excellence appraisals of all novel pharmaceuticals recommended for use within the English National Health System between 1 January 2010 and 31 December 2020.
Incremental QALYs were calculated based on the best alternative therapy.
We analysed expected health benefits from the individual patient’s perspective and did not consider effects on the population level, for example, disease prevalence and market share.
Our analysis is limited to appraisals that disclose information on incremental QALYs.
Introduction
Before a novel treatment is allowed on the market, its clinical benefit is assessed by regulatory agencies such as the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA). However, clinical benefit evaluations do not provide insight into issues deemed relevant by payers, such as comparative effectiveness, cost effectiveness, or lifetime benefit. Therefore, several countries have created independent health technology assessment bodies to conduct drug value assessments, commonly referred to as cost effectiveness analyses.1 Through these value assessments, publicly funded experts help to clarify the incremental clinical benefit and incremental costs of selected new therapies according to their approved indications, which professional societies may then rely on when revising treatment guidelines to include the new drug.
Despite the increased focus on incremental drug value, surprisingly little attention has been devoted to understanding the magnitude and distribution of their clinical benefits across disease areas. The limited scholarship in this area can be explained in part by the fact that, until recently, it has been difficult to compare the benefit of drugs intended to treat widely divergent diseases or conditions.
However, the emergence of official government drug value assessments over the past two decades, rigorously conducted following a consistent set of health economic modelling guidelines, now makes such comparisons feasible. These assessments utilise the quality-adjusted life-year (QALY), a common metric of patient health. One QALY, for example, represents the equivalent of one additional year of life in perfect health, or some longer period of time in less-than-perfect health.2 3 Although the QALY has long been available as a measure and is frequently used in individual economic evaluations,4 the QALY can, in combination with forecasts over the lifetime of patients from health economic models, be used to compare health benefits across medical disciplines in a consistent and transparent manner. QALYs are primarily used to calculate incremental cost-effectiveness ratios (ICER), which signals the efficiency with which a health technology produces health by dividing incremental costs by incremental benefits expressed as QALYs. However, it is often overlooked that the QALY part of an ICER is, in and of itself, a parameter that provides relevant insights into the size of forecasted health benefit. In the case of the UK, QALYs are produced following specific modelling guidance by the National Institute for Health and Care Excellence (NICE), enhancing their comparability across diseases.
NICE is a non-departmental public body that assesses the value of novel drugs and the impact on the English National Health System (NHS) of adopting them. Since NICE was established in 1999, drug manufacturers have been invited to submit evidence on the health benefits and costs of their drugs in comparison to the standard of care.5 An evidence review group—generally a group of university based researchers contracted by NICE—then appraises the evidence in ‘single technology appraisals’ and produces independent estimates of health benefits, measured in QALYs.
Using data from NICE evidence review groups, we sought to better understand the incremental value of all new therapies assessed from 2010 to 2020. Although these data are used to inform public health decisions, we here present their implications from a patient’s perspective. Specifically, we sought to identify disease areas where the greatest gains from novel therapies have occurred, and the differing average amounts of gain per drug for individual patients in each disease area.
Materials and methods
We identified all single technology appraisals of novel pharmaceuticals that were submitted to NICE between 1 January 2010 and 31 December 2020.6 Data were extracted on 1 May 2021. We excluded drug appraisals resulting in negative coverage decisions, appraisals for which no data were available because of termination, withdrawal or reconsideration and appraisals that addressed only cost-saving issues and lacked QALY data.
Two authors (TBP and DGJC) independently extracted QALY estimates from each drug’s appraisal documents. Discordance was resolved by discussion with the last author (MMV). As per NICE guidance,7 QALYs are calculated over the remainder lifetime of patients, and future health benefits are discounted at a 3.5% annual rate. We extracted these ‘net present’ values. When appraisal documents included multiple comparators, we extracted the QALY value that corresponded to the best alternative therapy. As a sensitivity analysis, in the case of multiple comparators, we also computed the added value compared with the next-best alternative. We disregarded cost, as we focused on health gains for individual patients and not on healthcare systems.
The evidence review group usually specified which of the modelled QALYs was its preferred estimate of health benefit (ie, which modelling assumptions were deemed most appropriate to the review group). If the evidence review group did not clearly document their preference and this could not be determined after deliberation with the last author (MMV), we discarded the appraisal from our analysis. Although manufacturers frequently report the ICER in cost (British pounds) per QALY, they are not required to disclose the individual components of this ratio. We, therefore, removed appraisals in which the manufacturer redacted all estimates of incremental QALYs (also see: online supplemental material). A schematic overview of our appraisal selection and data extraction method is depicted in figure 1.
Supplemental material
Flow diagram of the selection and retrieval of estimates of quality-adjusted life-years (QALYs) from NICE technology appraisals between 1 January 2010 and 31 December 2020. NICE, National Institute for Health and Care Excellence.
Each appraisal was categorised according to its medical discipline: cardiology, endocrinology, gastroenterology, haematology, neurology, oncology, ophthalmology, rheumatology, vascular medicine, infectious diseases and other (benign haematology, dermatology, internal medicine, nephrology, psychiatry, pulmonology, urology). Summary statistics were calculated and visualised in R V.4.0.5.
Patient and public involvement
No patients were involved during the planning and writing of this work; all data were derived from NICE single technology appraisals.
Results
Between 1 January 2010 and 31 December 2020, 436 single technology appraisals were submitted to NICE associated with 212 drugs. No documentation was available for 115 appraisals, including 14 that were withdrawn, 56 that were terminated, and 45 that were later reconsidered or updated. Another 37 appraised drug-indication pairs received a negative reimbursement determination, meaning they were not considered a cost-effective use of NHS resources and thus did not become available to patients in the UK. An estimate of QALY gain could not be extracted in 19 appraisals, because QALYs were not reported in cost-saving appraisals or because the evidence review group did not specify its preferred estimate out of several reported outcomes. After these exclusions, 265 appraisals were available for evaluation, associated with 171 drugs. Of these appraisals, 81 had their incremental QALY estimates redacted (online supplemental material), which can occur at the company’s request, leaving 184 appraisals associated with 129 drugs for inclusion in our data set (different appraisals can review the same drug for different indications).
Of the 184 drug-indication pairs, the median incremental QALY gain relative to the best alternative therapy was 0.27 QALY (IQR: 0.07–0.73) (figure 2). The highest median benefits were associated with drugs developed for medical disciplines such as haematology (0.70, IQR: 0.55–1.22), oncology (0.46, IQR: 0.20–0.88) and neurology (0.45, IQR: 0.13–1.15), and the lowest for drugs associated with medical disciplines such as vascular medicine (0.11, IQR: 0.01–0.19), ophthalmology (0.09, IQR: 0.04–0.22) and endocrinology (0.02, IQR: 0.01–0.06). Of note, QALY estimates were redacted in 26.7% of neurology, 28.6% of ophthalmology, 37.2% of oncology and 44.9% of haematology appraisals, whereas for vascular medicine and endocrinology, QALY estimates were available in all appraisals (also see online supplemental material).
The added value of novel pharmaceuticals approved by NICE from 2010 to 2020 display of the distribution (boxplot) of added value in QALYs of novel pharmaceuticals per medical discipline that have received a positive coverage decision of NICE between 1 January 2010 and 31 December 2020, compared with their next-best alternative. Medical disciplines with fewer than eight appraisals were classified as ‘other’. NICE, National Institute for Health and Care Excellence.
In our review period, eight (4.3%) positive coverage decisions were granted to drugs contributing more than the equivalent of two life-years in perfect health. Both dinutuximab beta to treat neuroblastoma and nusinersen used to treat children with spinal muscular atrophy led patients to accumulate 5.2 incremental QALYs.
On the other hand, 50 (27%) drugs contributed no more than the equivalent of 1 month in perfect health over the best alternative therapeutic option (≤0.082 QALY) (table 1). Eight drugs were estimated to provide lower QALY gains than their next best alternative. Government decision-makers may nevertheless be willing to pay for such products thanks to the uncertainty around point estimates, together with strategic pricing by manufacturers. For example, one drug, venetoclax, was estimated to be inferior to its direct comparator (ibrutinib) in the treatment of chronic lymphocytic leukaemia. Although this negative point estimate was considered most plausible by the evidence review group, there was still considerable uncertainty remaining as the group also provided higher estimates (an incremental benefit of 0.51 when idelalisib was the comparator) and lower estimates (−1.75 when treatment effects of venetoclax were assumed to be waning faster than expected) under varying assumptions. Venetoclax was offered at a lower price than ibrutinib, and NICE concluded that the new drug was likely a cost-effective use of NHS resources in the treatment of lymphocytic leukaemia.8
Pharmaceuticals that produced most and least incremental health benefit, ranked according to their added quality-adjusted life-years (QALYs) extracted from NICE technology appraisals (TAs)
When selecting the next-best drug as a comparator instead of the best available comparator, the median added value slightly increases (0.31, IQR: 0.09–0.73), suggesting our results are robust under these different choices of comparators.
Discussion
Novel pharmaceuticals that became publicly available to patients in the NHS over the past eleven years and that were favourably evaluated by NICE contributed the net present equivalent of between 3 and 4 months of life in perfect health relative to the best alternative therapy. The added benefit varied greatly, including eight drugs that were inferior in some cases to its already-available counterpart, and two that provided the equivalent of over 5 years in perfect health. To our knowledge, this analysis is the first to compare the therapeutic value of drugs across diverse disease areas using QALYs extracted from independent cost-effectiveness analyses conducted through a standardised framework.
The largest benefits were observed in areas such as haematology or oncology, where drugs were shown to improve quality or duration of life by 0.70 and 0.46 QALY. Patients have least profited from pharmaceutical innovations in endocrinology and ophthalmology, where novel pharmaceuticals were associated with a median incremental benefit of 0.02–0.09 QALY.
The nature of each treatment (curative, palliative, symptomatic, preventive) may impact the incremental QALY. For example, adult patients that have undergone total hip or knee replacements may be treated with apixaban (TA245) to prevent venous thromboembolism. When used for this indication, apixaban provides an incremental benefit of 0.0016 QALY over the standard of care (low-molecular-weight heparin), equivalent to an additional fourteen hours of life in perfect health. The very low benefit reflected estimates that one venous thromboembolism event would be prevented for every 110–250 patients treated prophylactically for 10 days following surgery.9–11 Although apixaban may prevent serious outcomes (death) in some patients, outcome heterogeneity led to the extremely low average incremental QALY.
QALY evaluations are necessarily based on the data available at the time of drug approval, which are in turn increasingly based on earlier-phase trials, but later-generated evidence often fails to confirm promising early results.12 Furthermore, most (59%) drugs are now approved on the basis of surrogate endpoints,13 such as progression free survival, which for purposes of QALY calculations are assumed to correlate with clinical outcomes such as increased survival. However, studies have shown that this correlation is often poor or fair, particularly in oncology.14 15 Additionally, data on infrequent or longer-term harms cannot be known with certainty or incorporated in the appraisals, as these data only become apparent when the drug is available for broader use. Furthermore, fitter patients are often recruited for clinical trial participation and the outcomes for more vulnerable patients are not known. Factors such as these could cause QALY values to be lower than NICE estimates suggest.
Three additional issues can also lead to overestimations in incremental therapeutic benefit. First, during the time it takes to plan and conduct a trial, approve a drug and complete a cost-effectiveness assessment, the standard of care may have shifted and the best available comparator may no longer provide the relevant baseline for comparison. Second, a drug may have different benefits for different indications, a factor of particular relevance when off-label use is widespread or where marketing authorisation is granted for a population that is broader than the tested population. Third, trials may be designed to demonstrate incremental benefit even when available treatments might demonstrate similar efficacy if tested with a different trial design.
Our findings should be interpreted with caution and cannot easily be interpreted from a population health perspective, as drug-indication pairs may be reimbursed within some health systems only for specific patient populations. For example, some of these large incremental benefits mainly occur for drugs that were not considered cost-effective in earlier lines of therapy—but when all prior therapies fail, these drugs are estimated to provide substantial benefit. From the examples in table 1, sofosbuvir-velpatasvir-voxilaprevir is estimated to generate 3.76 incremental QALYs for patients who have previously been treated with direct-acting antivirals. However, the marketing authorisation has been granted to treat patients regardless of cirrhosis status and treatment history. These benefits must be seen in this larger context.
Our study has a few limitations. First, our analysis was restricted to data presented to NICE of drugs that subsequently obtained a positive coverage decision, excluding medicine that may be accessed via private health insurance. Therefore, drugs in our review are a subset of the drugs covered in other analyses of medication approved by the FDA or EMA, a subset that is likely to be associated with higher QALY estimates than the average new drug. Not all FDA-approved drugs are subsequently approved by the EMA, and not all EMA-approved drugs are assessed by NICE. A recent assessment of oncology drugs approved via the FDA’s accelerated approval pathway demonstrated that only half (48%, 45/93) of drug-indication pairs subsequently became reimbursed within the English NHS, suggesting their therapeutic benefit was not sufficiently important or well established in relation to the associated cost to receive a positive reimbursement decision.16
Second, we could not retrieve all estimates of health benefit as some were concealed by the manufacturer, the implications of which are unclear. It seems some companies maintain a policy of not disclosing QALY figures for any indications or drugs, whereas other companies consistently provide full disclosure. The desire to maintain in confidence the incremental cost of their treatment, which would implicitly be made evident if both cost/benefit ratios and QALY values were simultaneously disclosed, may be the driving force behind redactions. In the online supplemental material, we provide examples where we could retrieve estimates due to ineffective redaction. We also list the number of redacted estimates by disease area. The rates of redaction in oncology (37.2%) and haematology (44.9%), compared with other disease areas (such as cardiology, vascular medicine, endocrinology) where none of the values were redacted, may either represent the unwillingness to disclose high drug prices in these indications,17 or the unwillingness to disclose low benefits, the latter of which may make average QALY figures appear larger than they are for these disease areas. For withdrawn or terminated appraisals, no detailed information is available to the public on cost or QALYs. Although speculative, it is unlikely these appraisals discussed drugs that were cheaper and more effective than the current standard of care.
Third, QALY estimates of individual products are sensitive to the choice of relevant comparator. Our results, however, show that the choice of comparator does not significantly affect the overall estimated QALY gain in our dataset. Alternatively, one may not be interested in the overall population, but only in specific (sub)populations reported in the appraisal documentation. This may give more specific estimates for individual patients, but impedes the comparison of drugs across diseases.
Fourth, estimates of median incremental QALY for each drug are associated with varying degrees of uncertainty. Although we have extracted the ‘preferred’ estimate from the evidence review group, the variance of these estimates is not routinely reported. Furthermore, distinct preferences in modelling choices, may result in substantial differences in benefit estimates.
Our findings provide insight into the relative benefits of new pharmaceuticals across therapeutic areas. Additional health gains may be hindered by the difficulty of developing novel drugs for specific diseases, perhaps because major improvements have already been generated prior to our review period,18 19 or because scientific breakthroughs have not yet occurred. QALYs are a useful tool for comparison, but the measure omits important health-related variables, such as the extent to which a patient remains unable to live out a ‘normal’ life expectancy or achieve complete health. Other factors, such as lack of fundamental understanding of disease pathologies,20 21 or the abundance or absence of sufficient research funding may also limit health gains.22 Our figures evaluate the net present health-related benefits of drugs that are considered cost-effective by NICE over the past decade. In combination with indices measuring health needs, such as the Global Burden of Disease,23 as well as cost-effectiveness/cost-saving data of novel drugs that might produce similar QALYs as already available therapies, our findings can help provide context for the allocation of research funding and thereby shape health policy.
Eight drugs improved life by more than two incremental QALYs, which may justify their superlative epithets of ‘ground-breaking’ or ‘game-changing’.24 Half of the drugs in our study were likely to improve life by the equivalent of 3–4 months in perfect health, and 84.8% of novel drugs did not add more than one such year. Unfortunately, 25% of appraisals have covered drugs that contributed the equivalent of no more than 1 month in perfect health, and 23 (12.5%) drug-indication pairs were estimated to add several hours to just a week of perfect health. For example, eluxadoline for prevention of diarrhoea and abdominal pain in patients with irritable bowel syndrome yielded a total QALY gain of 0.015—equivalent to 5.5 days in perfect health—compared with placebo. Given the uncertainty around cost-effectiveness estimates—models that require ample assumptions and extrapolations over lifetime horizons can hardly be expected to accurately forecast a week of health gained—drafting extensive cost-effectiveness reports in these situations is not likely to be a cost-effective use of time.
Drugs that have little health benefit relative to the best alternative may still promote price competition and thereby free funds for other public health initiatives or treatments. To avoid wasting public resources in needless evaluations, guideline committees could determine a threshold of incremental benefit that is clinically relevant to each disease area.25 Drugs that do not pass this threshold based on early assessments of their value should be rejected without a full evaluation unless they are offered at lower cost.
Patients and physicians can use the QALY data presented here to put the effectiveness of treatments in perspective. The frequently employed metric of’number needed to treat’ provides important information about the effectiveness of drugs on the principal disease-specific outcome. For example, the efficacy of eluxadoline could be described in terms of the number of patients that would need to be treated 3 months to avoid one episode of abdominal pain or diarrhoea, in this case between 8 and 33 patients over 3 months.26 Metrics such as this, however, do not account for adverse events. Using the incremental QALY estimate that integrates gains and losses into a single measure (for eluxadoline, 0.015), it is possible to calculate that 67 patients would need to be treated over their lifetime horizons to gain the equivalent of 1 year in perfect health. As such, the QALY provides an estimate of both duration and quality of life, which are arguably the two most important factors from the perspective of a patient.
Conclusions
Novel pharmaceuticals that received a positive coverage decision by NICE from 2010 to 2020 provided patients with an average of 0.27 additional QALYs over the best alternative therapy, the equivalent of 3–4 additional months of life in perfect health. One in four drugs does not improve quality and quantity of life by more than 1 month, and incremental benefit varies greatly across disease areas and compounds. Several novel drugs do not provide additional QALY gains over available therapies, but if offered at a lower price could still be of interest from a public cost-saving perspective even if not from the patient’s perspective. Providing transparent information on the added value of novel therapies enables patients and physicians to have reasonable expectations about the average net benefits of therapies at their disposal. Objectively evaluating the benefits contributed by novel pharmaceuticals provides insight not only into whether a given drug is worth its price once approved, but also into the therapeutic return on investment reaped by society from the substantial public and private sums expended on research and development. Finally, these figures provide a benchmark for future innovations.
Data availability statement
All data are available on the NICE website. Aggregated data are available on reasonable request.
Ethics statements
Patient consent for publication
Acknowledgments
We thank Dr. Noor Gieles and Ms. Rozemarijn Berends for critically reviewing the manuscript.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @TBPolak
Contributors TBP and MMV conceived the research. TBP and DGJC collected, analysed and visualised the data. TBP drafted the first version of the manuscript; JJD and DGJC critically revised the first version of the paper; all authors discussed the results and contributed to the final manuscript. All authors had full access to all the data in the study and accept responsibility for submission for publication.TBP acts as guarantor and accepts full responsibility for the work, had access to the data, and controlled the decision to publish.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests TBP is employed part-time by myTomorrows, which facilitates expanded access programs for the pharmaceutical industry. TBP holds stock in myTomorrows (<0.1%). TBP has received funding for research from the Dutch government (grant EMCLSH20012). The research of TBP is conducted independently and TBP is contractually free to publish any results for all the conducted work. None of the recent work concerns the topic in this analysis. DGJC received speaker fees from Takeda. MMV is director of the institute of Medical Technology Assessment (iMTA). iMTA conducts cost-effectiveness research funded by international governments, pharmaceutical industry, and med-tech industry. All research is conducted independently and iMTA is contractually free to publish any results for all conducted work. None of the recent work concerns the topic in this analysis. JJD receives funding from Arnold Ventures, the Greenwall Foundation, the Kaiser Permanente Institute for Health Policy, West Health, and the Novo Nordisk Foundation (grant for a scientifically independent Collaborative Research Programme; grant NNF17SA0027784).
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.