Article Text
PDF
Abstract
Introduction Cumulative network meta-analysis (NMA) is a method to provide a global comparison of multiple treatments with real-time update to evidence users. Several studies investigated the ranking of cumulative NMA and the recommendations of practice guidelines. However, to the best of our knowledge, no study has evaluated the cumulative NMA ranking and prescription patterns. Here, we present a protocol for a meta-epidemiological investigation to compare the results of cumulative NMA with the recommendations in postmenopausal osteoporosis practice guidelines and with the actual prescriptions.
Method and analysis We will use the data of primary trials from the upcoming postmenopausal osteoporosis clinical practice guideline of the Endocrine Society. We will conduct cumulative NMA using all eligible trials and generate hierarchy of treatment rankings by using the surface under the cumulative ranking curve. We will search practice guidelines in relevant society websites. Two review authors will extract the practice recommendations. We will use data from the Medical Expenditures Panel Survey, a US representative sample of the non-institutionalised population, to determine the prescription patterns.
Ethics and dissemination Because all data will be retrieved from public databases, institutional review board approval is not required. We will publish our findings in a peer-reviewed journal and present key findings at conferences.
Trial registration number UMIN000031894: Pre-results.
- practice guideline
- network meta-analysis
- process assessment (health care)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study is novel because it will compare rankings of drugs based on cumulative network meta-analysis with both clinical practice guideline recommendations and actual prescriptions.
This study will delineate the time lag between the body of evidence and guideline recommendations, and actual practice.
Physicians’ choice would be affected by reasons other than evidence about efficacy, which cannot be considered in this study.
Introduction
Recommendations in clinical practice guidelines (CPGs) should be ‘informed by a systematic review (SR) of evidence and an assessment of the benefits and harms of alternative care options’.1 Incorporating SR and meta-analysis (MA) into CPGs has greatly improved the credibility of CPGs in the past decades.2–4 However, conventional pairwise MAs that compare two interventions at a time is insufficient for analysing the increasing number of treatment options in a coherent manner because not all treatment alternatives have been compared directly in randomised trials.5–7 Network MA (NMA) combines direct and indirect evidence and generates the relative effects of three or more treatment alternatives in a single analysis. Cumulative NMA can provide global comparisons of multiple treatment options with repeated updates.7 8 Moreover, because NMAs use both direct and indirect evidence, they can provide answers earlier than conventional pairwise MAs.8
Whether successive revisions of CPGs in various fields of medicine have been able to incorporate such rigorous evidence updates remains an open question. One recent study compared the rankings by cumulative NMAs and recommendations of CPGs for open-angle glaucoma and found that cumulative NMAs can contribute to more timely recommendations than had traditionally been possible.9 This study did not intend to examine the influence of the updated evidence and the CPGs on the actual prescriptions by physicians for the disease. The translation of clinical knowledge from randomised controlled trials (RCTs) through cumulative NMAs and CPGs to actual prescription patterns by physicians is at the heart of evidence-based practices and therefore deserves greater scrutiny.
Postmenopausal osteoporosis is a common disease worldwide and in the USA; its prevalence is increasing with the ageing of the populations.10 11 The US Food and Drug Administration (FDA) has approved 12 classes of drugs for this condition.12 Several US and international societies and organisations have developed CPGs for the use of these drugs,12–14 but the real-world prescription patterns vary widely.15 This study aims to compare the results of cumulative NMAs with the recommendations in postmenopausal osteoporosis practice guidelines and with the actual prescription practices.
Methods and analysis
SR and cumulative NMAs of osteoporosis drugs
Study identification and data extraction
We will retrieve eligible original articles and data from the upcoming postmenopausal osteoporosis CPG of the Endocrine Society. We will use a recently completed search for relevant studies (last search date: 7 July 2017) that we have conducted for the guideline. The inclusion criteria are:
Parallel-group RCTs.
Trials studied postmenopausal women with primary osteoporosis or osteopenia at risk of developing fragility fractures.
Trials evaluated commonly used medications including bisphosphonates, teriparatide, selective oestrogen receptor modulators, denosumab, oestrogen with or without progesterone, calcitonin, lasofoxifene, strontium ranelate, tibolone or intact parathyroid hormone (1–84). We will also include nutritional supplements commonly recommended for osteoporosis including calcium and vitamin D. Control conditions may include placebo, no treatment or treatment as usual.
Trials must have evaluated the primary outcome of interest in this study, namely, new hip fractures at the time of the longest follow-up in the included studies. Hip fracture was designated as primary because of its clinical impact of patients’ prognosis.16
We did not limit language, geographical location or publication date.
Two of 10 review authors independently examined each title and abstract identified in the search to exclude obviously irrelevant reports, then independently examine full‐text articles to determine eligibility. If there were any disagreements, the same authors discussed disagreements; a third author helped reach consensus if necessary. The same independent pairs of reviewers also evaluated the risk of bias following the Cochrane risk of bias tool.17 They resolved any disagreement through discussion of the two assessors or, where necessary, in consultation with a third assessor.
Statistical analyses
We will conduct random-effects cumulative NMAs of the identified network of trials at 5-year intervals (see below for Comparisons of NMA rankings, CPG recommendations and actual prescriptions).18 Each drug as well as each combination of drugs will be treated as a node in this network. We will assess the transitivity assumption of the whole data set in the final NMA; if confirmed, we will not validate it at every time point reanalysis. We will use a multilevel hierarchical model with components at the following levels: study, individual drug and drug class. This model accounts for the within-study correlation of multigroup trials and also incorporates class effect.19–21 Given the clinical and methodological heterogeneity of the populations and methods among the included trials in NMAs, we will use the random-effects model in our primary analyses. We will examine the consistency of the total network through both local and global tests of inconsistency. We will test small study effects and publication bias using the comparison-adjusted funnel plot taking placebo as the common comparator.22
We will examine the hierarchy of treatment rankings by using the surface under the cumulative ranking curve (SUCRA).22 23 A SUCRA value can indicate a ranking of the treatment while accounting both for the location and the variance of all relative treatment effects. The larger the SUCRA value, the better the ranking of the treatment.22 23 We will also show the relative treatment effects of all active medications in comparison with placebo in ranked forest plots. We will not adjust for multiple comparisons in successive NMAs as we are not interested in establishing superiority or inferiority of particular comparisons.
We will use Stata V.15.1 (StataCorp to conduct NMAs.24 25 We will conduct the cumulative NMA in a frequentist framework using Stata, and therefore, no prior distributions and relevant sensitivity analyses will be employed.
Identification of practice guideline recommendations
We will search the website of Agency for Healthcare Research and Quality (AHRQ)’s National Guideline Clearinghouse,26 American Association of Clinical Endocrinologists,27 American College of Physicians,28 Endocrine Society29 and The North American Menopause Society30 using the following term: ‘osteoporosis’. One author (YK) will select guidelines for the treatment of postmenopausal osteoporosis from US-based organisations because we will evaluate the US prescriptions. Two of five independent authors (YK, YL, AO, MK and YT) will extract data from each guideline. We will extract publication year, developers, drug treatment recommendations and their strength, and whether the recommendations were based on SRs or not. We will resolve disagreements through discussion and, if necessary, though arbitration by another author (YK, YL, AO, MK or YT).
Real-world prescriptions
Medical Expenditure Panel Survey (MEPS) is a survey from nationally representative samples of the US non-institutionalised civilian population. MEPS uses sampling weights reflecting adjustments for survey non-response and population totals from the Current Population Survey31 and can therefore be used to derive nationally representative estimates. We will use the Household Component Files which contain detailed information about demographic information and prescribed medicines for respondents.31 We will include all female respondents aged 50 years and older who have been classified as ‘206 osteoporosis’. The cut-off value of 50 is in accordance with previous reports.32 33 We will exclude those who have been classified as ‘202 rheumatoid arthritis and related disease’, because they may have steroid-induced osteoporosis.34 We will also exclude those who have been classified as ‘158 chronic renal failure’, because they would have mineral and bone disorders.35 We will also exclude those who have been classified as ‘cancer’ (the codes are from 11 to 44), because they sometimes have bone metastasis which need to be treated with bone modifying agents.36
The prescription proportions and rankings will be determined by the 5-year prescription proportion of each drug category. The proprietary and non-proprietary names will be searched using the following terms from pharmacological class of National Drug Code Directory37: Bisphosphonate (EPC), Parathyroid Hormone Analog (EPC), Selective Estrogen Receptor Modulators (MoA), RANK Ligand Inhibitor (EPC), Estrogen (EPC), Progestin (EPC), Calcitonin (EPC), Calcium (Chemical/Ingredient), Vitamin D2 Analog(EPC) and Vitamin D3 Analog (EPC).
The numerator will be the number of patients who were prescribed each specific drug within 5 years. The denominator will be the number of patients who were female, over 50 years, and diagnosed as osteoporosis within the same 5 years. The greater proportion will mean the higher ranking.
We anticipate that we can start retrieving data in December 2018.
We will use Python V.3.6 (Python Software Foundation) and STATA V.15.1 (StataCorp) to handle data from MEPS.
Comparisons of NMA rankings, CPG recommendations and real-world prescriptions
We will compare results from cumulative NMAs with recommendations by CPGs and with actual prescriptions at 5-year intervals. MEPS started in 1996. We, therefore, chose 1996 as the first year of prescription ranking.
Because there is bound to be some time lag as randomised evidence is generated, synthesised, integrated into recommendations and translated into practice, the time frame for the comparisons will be set as shown in table 1. First, because the median time from last search to publication of SRs has been found to be 8.0 months (range: 0–47), we will include trials published up to 1 year prior to conducting the cumulative NMA.38 As there should be no time lag between the latest evidence synthesis and the CPG recommendations, we expect the NMA results to be reflected in the CPGs published in the ensuing 5 years. In 2000 a meta-epidemiological study showed a delay by 9.3 years between evidence review and its implementation.39 This delay may have been shortened in recent years.40 We will therefore compare the results from NMA and the CPG recommendations with actual prescriptions 1 or more years later than them.
Time frame for comparisons
This is a descriptive study. We will visually explore the differences between evidences from NMA, CPG and actual prescriptions. We will not conduct statistical tests for comparison.
In comparing cumulative NMA rankings based on best-available evidence in the world literature and the CPG recommendations and the prescriptions in USA, we will take into consideration the drug approval dates for osteoporosis by FDA as well as the dates when each drug became off-patent. To examine the influence of drug costs, we will tabulate the approval and off-patent date of each drug while on patent and also conduct a sensitivity analysis by limiting the analyses to patients with insurance.
Patient and public involvement
The study group developed this study protocol without patient involvement. This study will use only anonymised public data without new patient recruitment. We will disseminate the results via web sites and social network services to patients with osteoporosis.
Ethics and dissemination
We will prepare the publication in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guideline41 and its adaptation for meta-epidemiological studies.42 We will publish our findings in a peer-reviewed journal and also may present them at conferences.
Discussion
We have presented the study protocol to compare the results of cumulative NMA with the recommendations in CPGs, and with the actual prescriptions.
To our knowledge, this is the first effort to evaluate the influence of cumulative evidence to CPGs and physicians’ attitude simultaneously. By using cumulative NMA, the real-time trend of cumulative evidence and comprehensive network of available treatments will be presented.6 7 By using the MEPS, it is possible to estimate the representative prescription trends in the USA.31
There are several limitations for this study. First, physicians’ choice would be affected by reasons other than evidence, such as the policy of insurance companies or the marketing efforts of pharmaceutical company.43 44 These factors are difficult to quantify and will warrant a separate study. Second, we should not prescribe teriparatide and bisphosphonate for long term because of their harm.45 In this study, we plan to compare the proportion of prescriptions in MEPS, which will therefore likely underestimate the rankings of the teriparatide and bisphosphonate in comparison with its incident prescriptions.
In conclusion, this study will provide useful empirical evidence to compare the results of cumulative NMA with the recommendations in CPGs and with the actual prescriptions. The expected findings will show the magnitude of the impact of comprehensive evidence in CPGs and real-world prescriptions.
Acknowledgments
We would like to thank the valuable comments from the members of Research Group on Meta-epidemiology at The Kyoto University School of Public Health (Tomoko Fujii, Yusuke Tsutsumi, Aran Tajika and Kenji Omae).
References
Footnotes
Contributors YK and TAF contributed to the conception and design of the research. YK and TAF are fully responsible for writing the protocol. TAF supervised the research, and TL, AO, MK, YT, MHM, TL and AC contributed important intellectual contents to the revised protocol and gave final approval of the protocol before submission. After the publication of the protocol, we plan the following contributions of each author: YK and AC will conduct statistical analyses. YK, YL, AO, MK and YT will retrieve data from guidelines. YK will write the draft manuscript. TAF, YL, AO, MK, YT, MHM, TL and AC will revise the manuscript critically for important intellectual content. TAF will supervise the research, and YL, AO, MK, YT, MHM, TL, AC and TAF will give final approval of the manuscript before submission.
Funding This study is supported in part by JSPS KAKENHI (Grant-in-Aid for Scientific Research) Grant Number 17K19808 to TAF.
Competing interests TAF has received lecture fees from Janssen, Meiji, Mitsubishi-Tanabe, MSD and Pfizer. He has received royalties from Igaku-Shoin and Nihon Bunka Kagaku-sha publishers. He has received research support from Mitsubishi-Tanabe and Mochida.
Patient consent Not required.
Ethics approval All data will be retrieved from public databases, hence this study does not require institutional review board approval.
Provenance and peer review Not commissioned; externally peer reviewed.