The quality of published health economic analyses in digestive diseases: A systematic review and quantitative appraisal

doi:10.1053/j.gastro.2004.04.020

Gastroenterology

Volume 127, Issue 2, August 2004, Pages 403-411

https://doi.org/10.1053/j.gastro.2004.04.020 Get rights and content

Abstract

$Background & Aims :$ Health economic analyses are increasingly common in the digestive diseases literature and often are cited to frame practice guidelines. Although clinical trials are subjected routinely to critical appraisal, there has been no attempt to appraise the quality of health economic analyses with a validated instrument. We sought to appraise the quality of health economic analyses in digestive diseases, and to identify predictors of study quality. $Methods :$ We performed a systematic review to identify digestive disease health economic analyses published since 1980. We assessed these studies using the Quality of Health Economic Studies (QHES), a validated quality-scoring instrument (score range = 0–100; >75 = high quality). We conducted regression analysis to identify predictors of high quality. $Results :$ Of 186 identified analyses, 29% were high quality, 71% failed to address potential model biases, 52% failed to disclose conflicts of interest, and 74% failed to describe methods for deriving the model assumptions. Four factors predicted high quality in logistic regression: (1) one or more authors had an advanced degree in health services or a related field (odds ratio for high quality, 5.0; 95% confidence interval, 2.6–9.3); (2) the study used decision-analysis software package (odds ratio, 2.4; 95% confidence interval, 1.2–4.7); (3) the study was federally funded (odds ratio, 2.2; 95% confidence interval, 1.2–4.1); and (4) the study cited the National Panel on Cost Effectiveness guidelines (odds ratio, 2.1; 95% confidence interval, 1.1–4.2). $Conclusions :$ Less than one third of health economic analyses in digestive diseases meet criteria for high quality. Study quality is limited by factors that potentially can be remedied. These data may be used to focus the attention of journal editors and peer reviewers to ensure the future high quality of health economic analyses in digestive diseases.

Section snippets

Systematic review

We performed a systematic review of English-language gastroenterology, endoscopy, hepatology, and internal medicine journals with an impact factor greater than 1.0 to identify health economic analyses in digestive diseases published between January 1980 and January 2004. Table 1 lists the journals subjected to our search and provides the key words and search strings used to perform the systematic review. Three reviewers with experience in health economic analyses independently assessed the

Interrater agreement

The mean scores (±SD) for the 10% random sample training set for the 3 reviewers were: 60.8 ± 14, 56.0 ± 16, and 59.5 ± 15 (P value for ANOVA, 0.58). The agreement for dichotomous scoring was high (κ = 0.8). The 3 reviewers scored 85% of the studies in the test set within 10 points. Therefore, the reviewers achieved excellent interrater agreement.

Descriptive analyses

Table 4 shows the results of the descriptive analyses. There were 186 studies included in the final dataset (135 from gastrointestinal journals, 51

Conclusions

In light of the increasing reliance on health economic analyses to formulate policy decisions in digestive diseases, it is important to systematically rate and describe the quality of this body of literature. Our analysis reveals that less than one third of health economic analyses in digestive diseases meet criteria for high quality as measured by a validated instrument. Traditional surrogate markers of quality (e.g., presence of an editorial, number of Web of Science citations per year) do

References (15)

S. Winawer et al.
Colorectal cancer screening and surveillanceclinical guidelines and rationale—update based on new evidence
Gastroenterology
(2003)
M.M. Goetghebeur et al.
Towards a European consensus on conducting and reporting health economic evaluations—a report from the ISPOR inaugural European conference
Value Health
(1999)
J.K. Marshall et al.
Use and misuse of cost-effectiveness terminology in the gastroenterology literaturea systematic review
Am J Gastroenterol
(2002)
M. Dubinsky et al.
Suspected inflammatory bowel disease—the clinical and economic impact of competing diagnostic strategies
Am J Gastroenterol
(2002)
M.R. Chassin et al.
Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures
JAMA
(1987)
D. Provenzale et al.
A reader’s guide to economic analysis in the GI literature
Am J Gastroenterol
(1996)
D. Provenzale et al.
Cost-effectivenessdefinitions and use in the gastroenterology literature
Am J Gastroenterol
(1996)

There are more references available in the full text version of this article.

Cited by (73)

Assessment of Extent and Quality of Pharmacoeconomic Studies in India Using Quality of Health Economic Studies Score: A Targeted Literature Review
2024, Value in Health Regional Issues
We assessed the quality of pharmacoeconomic studies conducted in India to report key areas of focus on the findings from the reviewed studies.
A targeted literature review was conducted using well-defined search strategy in PubMed to identify economic studies conducted in India from May 2017 to April 2022. Only economic evaluation studies were included, whereas trial-based cost analyses were excluded. The quality of included studies was assessed using the Quality of Health Economic Studies tool, which comprised 16 evaluation criteria related to objectives, source, funding, perspective, subgroup analysis, scales, and economic modeling related parameters. Based on scores (100 points), studies were rated as good (≥75), fair (50-74), and poor (≤49) quality.
Search strategy provided 888 studies; 95 of these were economic studies, and 74 were included in the analysis. These 74 studies included budget impact analysis (n = 4), burden of illness (n = 8), cost-benefit analysis (n = 5), cost-consequences analysis (n = 1), cost-effectiveness analysis (n = 55), and cost-utility analysis (n = 1). The average quality score of studies was 64.08. Of the studies, 15 studies were rated as “good,” 51 “fair,” and 8 “poor.” It was observed that primary outcome measures, stating negative outcomes, reporting bias, and implementing statistical and sensitivity analysis significantly affected the quality score.
Most of the health economic studies conducted in India are of fair quality, and there is a need for standardization of guidelines and increase in number of Indian peer-reviewed health economics journals. A collaborative effort from pharma companies, policy makers, education experts, curriculum planners, and medical faculty is needed to promote quality economic studies.
The Cost of ARDS: A Systematic Review
2022, Chest
ARDS is an inflammatory condition of the lungs and is a common condition in adult ICUs. The resources required and costs of care for patients with ARDS are significant because of the severity of the illness and extended ICU lengths of stay.
What are the costs associated with ARDS?
We systematically searched the literature through April 29, 2021, for articles relevant to ARDS and costs. MEDLINE, Embase, Central, and EconLit databases were searched, and articles that reported on cost data from an original publication in adult patients with ARDS were included. Two authors independently assessed articles for inclusion and extracted data elements related to costs, methodology, health care system type, economic perspective, and clinical data. Publication quality was assessed using a modified version of the Quality of Health Economic Studies Instrument.
Four thousand six hundred sixty-three publications were found, of which 110 were included for full-text review (κ = 0.72). A total of 22 publications (49,483 patients) were suitable for data extraction. The publications represented a broad range of health care systems, economic perspectives, costing methodology, and time frames. Mean inpatient costs ranged from $8,476 (2021 US dollars [USD]) to $547,974 (2021 USD) and were highest in publications of lower quality and in American health systems and were associated with trauma cohorts. Outpatient costs were highest in publications with higher readmission rates, longer durations of follow-up, and in American health systems.
A wide range of costing data is available for ARDS. A comprehensive synthesis of this literature frames the reasons for this and allows estimates to reflect the context in which they were assessed. This information will be of value to researchers and administrators interested in the economics of caring for patients with ARDS.
PROSPERO; No.: CRD42020192487 https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=192487
Noninvasive Fetal RhD Blood Group Genotyping: A Systematic Review of Economic Evaluations
2021, Journal of Obstetrics and Gynaecology Canada
Noninvasive fetal rhesus D (RhD) blood group genotyping may prevent unnecessary use of anti-D immunoglobulin (RhIG) in non-alloimmunized RhD-negative pregnancies and can guide management of alloimmunized pregnancies. We conducted a systematic review of the economic literature to determine the cost-effectiveness of this intervention over usual care.
Systematic literature searches of bibliographic databases (Ovid MEDLINE, Embase, and Cochrane) until February 26, 2019, and auto-alerts until October 30, 2020, and of grey literature sources were performed to retrieve all English-language studies.
We included studies done in serologically confirmed non-alloimmunized or alloimmunized RhD-negative pregnancies, comparing costs and effectiveness of the intervention versus usual care.
Two reviewers extracted data from the eligible studies and assessed their methodological quality (risk of bias) using the Quality of Health Economic Studies (QHES) and Drummond tools. We narratively synthesized findings. Our review included 8 economic studies that evaluated non-invasive fetal RhD genotyping followed by targeted RhIG prophylaxis in non-alloimmunized pregnancies. Five studies further considered a subsequent alloimmunized pregnancy. The cost-effectiveness of the intervention versus usual care (e.g., universal RhIG or prophylaxis conditional on results of paternal testing) for non-alloiummunized pregnancies was inconsistent. Two studies indicated greater benefits and lower costs for the intervention, and another 2 suggested a trade-off. In 4 studies, the intervention was less effective and costlier than alternatives. Three studies were determined to be of high quality by both tools. Two of these studies favoured the intervention, and one assessed benefits in quality-adjusted life-years. No study clearly examined the cost-effectiveness of repetitive use of fetal genotyping in multiple non-alloimmunized or alloimmunized pregnancies. The cost of genotyping was the most influential parameter.
The cost-effectiveness of noninvasive fetal RhD genotyping for non-alloimmunized pregnancies varies between studies. Potential savings from targeted management of alloimmunized pregnancies requires further research.
Le génotypage non invasif du rhésus D (RhD) fœtal peut prévenir l'administration inutile d'immunoglobulines anti-D (IgRh) en cas de grossesses non allo-immunisées RhD négatif et guider la prise en charge des grossesses allo-immunisées. Nous avons effectué une revue systématique de la littérature économique pour déterminer le rapport coût-efficacité de cette intervention par rapport aux soins habituels.
Des recherches documentaires systématiques ont été effectuées dans les bases de données bibliographiques (Ovid MEDLINE, Embase et Cochrane) jusqu'au 26 février 2019, avec des alertes automatiques jusqu'au 30 octobre 2020, et dans les sources de littérature grise afin d'en extraire toutes les études publiées en anglais.
Nous avons inclus des études réalisées auprès de femmes enceintes allo-immunisées ou non allo-immunisées RhD négatif avec confirmation sérologique pour comparer les coûts et l'efficacité de l'intervention par rapport aux soins habituels.
Deux évaluateurs ont extrait des données des études admissibles et ont évalué leur qualité méthodologique (risque de biais) à l'aide de la grille QHES (Quality of Health Economic Studies) et des méthodes de Drummond. Nous avons synthétisé les résultats sous forme narrative. Notre revue comprend 8 études économiques évaluant le génotypage non invasif du RhD fœtal suivi d'une prophylaxie ciblée par IgRh en cas de grossesses non allo-immunisées. Cinq études se sont également penchées sur une grossesse allo-immunisée subséquente. Les rapports coût-efficacité de l'intervention par rapport aux soins habituels (p. ex., IgRh universel ou prophylaxie conditionnelle aux résultats du génotypage paternel) pour les grossesses non allo-immunisées n’étaient pas uniformes. Deux études ont indiqué des bénéfices supérieurs et des coûts inférieurs pour l'intervention, tandis que deux autres ont suggéré un compromis. Dans quatre études, l'intervention était moins efficace et plus coûteuse que les solutions de rechange. Trois études ont été évaluées comme étant de qualité élevée selon les deux outils. Deux de ces études ont privilégié l'intervention et une a évalué les bénéfices en années de vie pondérées par la qualité. Aucune étude n'a clairement examiné le rapport coût-efficacité de l'utilisation répétée du génotypage fœtal en cas de multiples grossesses allo-immunisées ou non allo-immunisées. Le coût du génotypage était le paramètre ayant la plus grande influence.
Le rapport coût-efficacité du génotypage non invasif du RhD fœtal pour les grossesses non allo-immunisées varie d'une étude à l'autre. D'autres études sont nécessaires pour bien évaluer les économies potentielles de la prise en charge ciblée des grossesses allo-immunisées.
Cost-effectiveness analyses in shoulder arthroplasty: a critical review using the Quality of Health Economic Studies (QHES) instrument
2021, Journal of Shoulder and Elbow Surgery
Citation Excerpt :
A low-quality score was deemed below the 25th percentile (<82 points), whereas a high-quality score was deemed above the 75th percentile (>94 points). An analysis to evaluate factors associated with achieving both low- and high-quality QHES scores was conducted, based on both scientific and general factors previously reported within high-quality cost literature.30,32,33,38,39 Scientific factors of interest were inclusive of the following: use of randomized cohort data, cohort size >100 patients, use of sensitivity analysis, use of societal perspective for analysis, application of a cost discounting rate, application of an incremental analysis, and use of Markov modeling for analysis.
The purpose of this study was to perform a systematic review to identify cost-analysis studies pertaining to shoulder arthroplasty, provide a comprehensive review of published studies, and critically evaluate the quality of the available literature using the Quality of Health Economic Studies (QHES) instrument.
A systematic review of the literature was performed to identify cost analyses examining shoulder arthroplasty. The inclusion criteria included studies pertaining to either shoulder hemiarthroplasty (HA), total shoulder arthroplasty (TSA), or reverse TSA. Articles were excluded based on the following: nonoperative studies, nonclinical studies, studies not based in the United States, and studies in which no cost analysis was performed. The quality of studies was assessed using the QHES instrument. One-sided Fisher exact testing was performed to identify predictors of both low-quality (ie, QHES score < 25th percentile) and high-quality (ie, QHES score > 75th percentile) cost analyses based on items within the QHES checklist.
Of the 196 studies screened, 9 were included. Seven studies conducted cost analyses comparing reverse TSA vs. arthroscopic rotator cuff repair, HA, or total hip arthroplasty, and 2 studies examined TSA vs. HA for primary glenohumeral arthritis. The average QHES score among all studies was 86.22 ± 13.39 points. Failure to include an annual cost discounting rate was associated with a low-quality QHES score (P = .03). In addition, including a discussion of the magnitude and direction of potential biases was associated with a high-quality score (P = .03).
Shoulder arthroplasty is a cost-effective procedure when used to treat a multitude of shoulder pathologies. The overall quality of cost analysis in shoulder arthroplasty is relatively good, with an average QHES score of 86.22 points. Studies failing to include an annual cost discounting rate are more likely to score below the 25th percentile, whereas those including a discussion of the magnitude and direction of potential biases are more likely to achieve a score in excess of the 75th percentile.
Economic analyses of mental health and substance use interventions in the workplace: a systematic literature review and narrative synthesis
2020, The Lancet Psychiatry
Mental illness and substance use disorders in the workplace have been increasingly recognised as a problem in most countries; however, evidence is scarce on which solutions provide the highest return on investment. We searched academic and grey literature databases and additional sources for studies that included a workplace intervention for mental health or substance abuse, or both, and that did an economic analysis. We analysed the papers we found to identify the highest yielding and most cost-effective interventions by disorder. On the basis of 56 studies, we found moderate strength of evidence that cognitive behavioural therapy is cost-saving (and in some cases cost-effective) to address depression. We observed strong evidence that regular and active involvement of occupational health professionals is cost-saving and cost-effective in reducing sick leave related to mental health and in encouraging return to work. We identified moderate evidence that coverage for pharmacotherapy and brief counselling for smoking cessation are both cost-saving and cost-effective. Addressing mental health and substance misuse in the workplace improves workers' wellbeing and productivity, and benefits employers' bottom line (ie, profit). Future economic analyses would benefit from the consideration of subgroup analyses, examination of longer follow-ups, inclusion of statistical and sensitivity analyses and discussion around uncertainty, and consideration of potential for bias.
Cost-effectiveness analysis of Xpert in detecting Mycobacterium tuberculosis: A systematic review
2020, International Journal of Infectious Diseases
To report the cost-effectiveness of Xpert in detecting Mycobacterium tuberculosis (MTB) through a comprehensive systematic review.
Specialized bibliographic databases were searched. Study quality was evaluated by commonly-used industry standards. Due to heterogeneity, evidences were synthesized narratively.
Four studies from intermediate-to-low tuberculosis (TB)-burdern areas and 17 studies from high-TB-burden areas were included. Smear microscopy, clinical diagnosis and chest radiography were mostly used for comparison. Cost elements varied considerably depending on the perspectives. Cost-effectiveness and cost-utility analyses were used by seven and fourteen studies, respectively. All studies were of high quality (CHEERS score of 78.4 and QHES score of 86.9). Average cost per test was 29.8 US$ for Xpert compared with 3.83 US$ for smear microscopy. Cost-effectiveness analyses mostly supported application of Xpert into areas under varying TB burdens.
Xpert seems cost-effective under respective willingness-to-pay thresholds in nations with differences in socioeconomy, HIV stress and geographical distribution. Nevertheless, policymakers will benefit from localized studies since regional economic/financial statuses and health-care system should also be considered apart from the reports of cost-effectiveness.

View all citing articles on Scopus

^☆: Supported by a National Institutes of Health training grant (DK-07180 to B.M.R.S.), an American Association for the Study of Liver Diseases (AASLD) Advanced Hepatology Fellowship Award (to F.K.), National Institutes of Health K23 Career Development Award (RR-16188 to G.S.D.), and a VA Health Services Research and Development (HSR&D) (IIR 01-191-1 to I.M.G.)

View full text

Clinical-alimentary tractThe quality of published health economic analyses in digestive diseases: A systematic review and quantitative appraisal☆

Abstract

Section snippets

Systematic review

Interrater agreement

Descriptive analyses

Conclusions

Gastroenterology

Value Health

Am J Gastroenterol

Am J Gastroenterol

Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures

JAMA

A reader’s guide to economic analysis in the GI literature

Am J Gastroenterol

Cost-effectivenessdefinitions and use in the gastroenterology literature

Am J Gastroenterol

Clinical-alimentary tract
The quality of published health economic analyses in digestive diseases: A systematic review and quantitative appraisal☆