Objectives To develop a longitudinal statistical model to indirectly estimate the comparative efficacies of two drugs, using model-based meta-analysis (MBMA). Comparison of two oral dipeptidyl peptidase (DPP)-4 inhibitors, sitagliptin and linagliptin, for type 2 diabetes mellitus (T2DM) treatment was used as an example.
Design Systematic review with MBMA.
Data sources MEDLINE, EMBASE, http://www.ClinicalTrials.gov, Cochrane review of DPP-4 inhibitors for T2DM, sitagliptin trials on Food and Drug Administration website to December 2011 and linagliptin data from the manufacturer.
Eligibility criteria for selecting studies Double-blind, randomised controlled clinical trials, ≥12 weeks’ duration, that analysed sitagliptin or linagliptin efficacies as changes in glycated haemoglobin (HbA1c) levels, in adults with T2DM and HbA1c >7%, irrespective of background medication.
Model development and application A Bayesian model was fitted (Markov Chain Monte Carlo method). The final model described HbA1c levels as function of time, dose, baseline HbA1c, washout status/duration and ethnicity. Other covariates showed no major impact on model parameters and were not included. For the indirect comparison, a population of 1000 patients was simulated from the model with a racial composition reflecting the average racial distribution of the linagliptin trials, and baseline HbA1c of 8%.
Results The model was developed using longitudinal data from 11 234 patients (10 linagliptin, 15 sitagliptin trials), and assessed by internal evaluation techniques, demonstrating that the model adequately described the observations. Simulations showed both linagliptin 5 mg and sitagliptin 100 mg reduced HbA1c by 0.81% (placebo-adjusted) at week 24. Credible intervals for participants without washout were −0.88 to −0.75 (linagliptin) and −0.89 to −0.73 (sitagliptin), and for those with washout, −0.91 to −0.76 (linagliptin) and −0.91 to −0.75 (sitagliptin).
Conclusions This study demonstrates the use of longitudinal MBMA in the field of diabetes treatment. Based on an example evaluating HbA1c reduction with linagliptin versus sitagliptin, the model used seems a valid approach for indirect drug comparisons.
- STATISTICS & RESEARCH METHODS
this is an open-access article distributed under the terms of the creative commons attribution non-commercial license, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. see: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
In the absence of evidence from head-to-head trials, indirect and mixed treatment comparisons can be used for drug comparisons.
The aim of this study was to develop an approach, using Bayesian methodology (Markov Chain Monte Carlo method) to indirectly estimate the comparative efficacy of two compounds, incorporating longitudinal dose–response data.
A longitudinal statistical model was developed for the indirect comparison of two pharmaceutical compounds (oral DPP-4 inhibitors linagliptin and sitagliptin), with respect to changes in glycated haemoglobin (HbA1c) levels in patients with type 2 diabetes mellitus (T2DM).
The model was evaluated by comparing model predictions with observed values.
The model demonstrated that both linagliptin and sitagliptin reduced HbA1c levels by 0.8% (placebo-adjusted) when administered to patients with T2DM for 24 weeks, irrespective of background medication.
Strengths and limitations of this study
This study represents a novel use of longitudinal model-based meta-analysis in the field of diabetes treatment, being the only instance to date that adequately accounts for longitudinal correlations in each treatment arm, which is a prerequisite to the correct characterisation of uncertainty in estimation of drug effects.
When relevant head-to-head comparisons are not available, the model described in this study could have an important role in treatment decision-making.
Although the analysis included a large sample of 11 234 patients with T2DM, its applicability to the general population of patients with T2DM might be limited by the relatively selected patient populations in the included trials. Additionally, while our analysis adjusts for key differences in study designs, there remains the possibility of bias attributable to covariate effects that could not be estimated with the available data.
Ideally, head-to-head, randomised controlled trials should be conducted to estimate the comparative efficacy of different treatments. However, it is not always feasible to conduct direct comparisons among all available treatment options. Network meta-analysis (mixed treatment comparisons) has been used to estimate relative efficacy when there are no direct comparative data, to provide the best available evidence to facilitate decision-making by physicians and other stakeholders, such as payers. However, these approaches have certain limitations, including the risk of bias arising from inherent differences in the designs of the included studies, and the difficulties of finding appropriate summary statistics to compare the findings of individual trials.1 ,2 In particular, endpoint-based approaches cannot be sensibly applied when the studies involved in the review vary substantially with respect to treatment duration.
An approach, recently described as model-based meta-analysis (MBMA), has been developed and used to estimate the comparative efficacy of two medications. MBMA can be used to provide a mechanism for integrating information from heterogeneously designed trials and thus to evaluate outcomes with different drugs that have not been compared directly.3 MBMA is distinguished from the methodology of conventional meta-analysis by the manner in which it incorporates longitudinal and/or dose–response data. By modelling the response as a parametric function of time, MBMA allows the integration of information from trials of different durations and with different sampling time-points. This enables the use of less restrictive inclusion/exclusion criteria for study selection, and more efficient use of data from the studies that are selected, thereby resulting in a particularly comprehensive summary of all relevant data.3
In response to the growing worldwide epidemic of diabetes mellitus, new antihyperglycaemic agents are continuously being developed. The dipeptidyl peptidase (DPP)-4 inhibitors are a relatively new class of oral antihyperglycaemic drugs developed for the treatment of type 2 diabetes mellitus (T2DM) that are increasingly being used in clinical practice because of their clinically meaningful efficacy, promising tolerability, safety and convenience—in particular, a virtually absent risk of hypoglycaemia or weight gain.4 Although several DPP-4 inhibitors are already available in many countries, to date, only one published trial has been conducted to directly compare individual drugs within this class.5 Therefore, further research is needed to understand the comparative effects of the drugs within this class.
The model developed in this study incorporates Bayesian methodology and aims to provide a valid approach to estimate the comparative efficacy of different compounds. Bayesian approaches are acknowledged by the Cochrane collaboration to have a role in meta-analysis, particularly in the setting of indirect comparison.1
This approach to drug comparison employs a mathematical model to describe the timecourse of glycated haemoglobin (HbA1c) levels, and is being increasingly used to characterise longitudinal data. The general meta-analytic methodology of Ahn and French3 has previously been used to successfully describe longitudinal metadata from clinical trials in Alzheimer's disease,6 ,7 rheumatoid arthritis,8 lipid disorders,9 glaucoma10 and chronic obstructive pulmonary disease.11 Similar approaches have been used to perform dose–response meta-analyses in a range of therapeutic areas, including migraine,12 postoperative anticoagulant therapy13 and rheumatoid arthritis.14 This analytical approach has also been used in the field of diabetes in a recent study by Gibbs et al,15 which evaluated the relationship between DPP-4 inhibition and HbA1c reduction using data obtained from clinical trials of four drugs in this class.
To use an MBMA approach to develop a longitudinal statistical model for the comparison of the efficacy of two oral DPP-4 inhibitors, shown by changes in HbA1c levels, in patients with T2DM who had started treatment with one of two DPP-4 inhibitors, regardless of background medication. The two drugs evaluated were linagliptin, which has recently been approved for clinical use in several jurisdictions, and sitagliptin, the most commonly used DPP-4 inhibitor.
Sitagliptin studies were identified from a systematic search in MEDLINE, EMBASE, studies listed on http://www.ClinicalTrials.gov that included a reference to publication, the latest-date Cochrane review of DPP-4 inhibitors for T2DM16 and details of sitagliptin trials on the Food and Drug Administration website, to December 2011.17 Details of the search strategy used are provided in the appendix (see online supplementary table S1).
Included studies were double-blind, randomised controlled trials of ≥12 weeks’ duration that analysed the efficacy of sitagliptin or linagliptin in the reduction of HbA1c levels in adults with T2DM and HbA1c >7%, irrespective of background medication. Excluded studies were: open-label studies (and data from open-label extensions to double-blind studies) and extension studies that used patient response in the initial study to determine eligibility in the extension phase of the study (eg, if the extension phase included only those who did not require rescue medication during the initial study). Other excluded study types were special population studies (eg, studies in patients with declining renal function) and phase IV studies or study arms in which patients were randomly assigned to initial combination therapies.
Two independent reviewers extracted aggregated data from all selected studies, according to treatment arm (sitagliptin, linagliptin or placebo). We extracted data on: the first author's name, year of publication of the trial, comparator, dose(s) of sitagliptin or linagliptin evaluated, trial duration, number of participants and their gender, ethnicity, duration of T2DM, mean age, baseline HbA1c (%), HbA1c at evaluated time-points, baseline body mass index (BMI, kg/m2), fraction of patients on previous antihyperglycaemic therapy, and the presence and duration of washout and concomitant medication. A common data template was defined. The main outcome of interest was HbA1c, the primary end-point of all included studies. Intention-to-treat populations were included whenever possible and group means, as reported, were used or were calculated, using the last observation carried forward approach. The analyses were conducted using the maximum licensed dose of sitagliptin (100 mg) and the licensed dose of linagliptin (5 mg). However, when data at other dose levels were available, they were included in the analysis, and appropriate adjustments were made via the dose–response terms in the model.
Data selection process
For the linagliptin studies, the dataset was built from the original Boehringer Ingelheim database using SAS scripting. The quality of the dataset was assured by an independent script review. For the sitagliptin studies, the dataset was built manually by collecting information given in the different source publications. If the results were available as numbers in the publications, these numbers were included in the dataset. Where the results were only available as graphics, the corresponding data were collected using GetData Graph Digitilizer, V.2.24 software (http://www.getdata-graph-digitizer.com). The quality of the manually built sitagliptin dataset was assured by an independent second reviewer. The initial dataset consisted of HbA1c data, presented as either the change from baseline and/or the actual HbA1c measurements, depending on the information provided in the publication. R scripting (R V.2.10.1, The R Foundation for Statistical Computing, Vienna, Austria) was then used to obtain an analysis-ready dataset with consistent encoding of information (eg, baseline values were added to changes from baseline in order to obtain actual HbA1c measurements for all records).25
The statistical models that were considered represent a particular class of non-linear mixed-effects models in which model precision terms are scaled according to sample sizes. Sample size adjustments are carried out in a manner that approximately estimates and adjusts for longitudinal correlations, following an approach described elsewhere.3
Initial exploratory data analyses were used to derive a suitable parametric (algebraic) description of the average HbA1c trends as a function of time, dose, washout status/duration and ethnic origin. Qualitative prior information was also used to guide the initial selection of parametric forms. The following assumptions were made: (1) given the known properties of measured HbA1c, it was assumed that in the absence of additional interventions, HbA1c levels for patients washing out prior antidiabetes medication (during the study washout/run-in phase) would rise for some time until achieving a plateau, and (2) the incremental (placebo-adjusted) effect of DPP-4 inhibitors on HbA1c was expected to approach a plateau during the time frame of interest (24 weeks). Bayesian prior distributions for parameters describing the magnitude and onset of drug effects were specified separately and independently for linagliptin and sitagliptin. Magnitudes of drug effect were parameterised as fractional reductions from baseline and were assigned uniform prior distributions between zero and one, implying that both drugs have some beneficial effects (a defensible assumption for marketed drugs) and that neither can reduce HbA1c levels below zero (patently true), and assigning equal likelihood to all possibilities between these two extremes.
The model was fitted using Bayesian Markov Chain Monte Carlo methodology. The computations were carried out using OpenBUGS V.3.2.1 (2010) software (Free Software Foundation, Boston, Massachusetts, USA). Final inferences were based on 1000 approximately independent draws from the posterior (after discarding burn-in samples and thinning to de-correlate samples26). The model was adjusted for baseline HbA1c and washout status/duration. Other covariates considered were: standard covariates including demographics, such as ethnicity, age, BMI and gender, antihyperglycaemic background medication, duration of T2DM and the fraction of patients who underwent washout of previous antihyperglycaemic therapy. The OpenBUGS code is available from the authors on request.
Model selection and evaluation
Following a ‘full model estimation approach’,27 ,28 initial preference was given to a full model, meaning one that includes all terms of potential interest. In order to achieve stable parameter estimation, selective simplifications were applied, guided by exploratory data analysis, to the full model until we obtained satisfactory convergence diagnostics. Covariates were excluded from the model for the purpose of achieving stable parameter estimation; however, each excluded covariate was evaluated graphically to ensure that it was not associated with model residuals (differences between the observed values and those predicted by the model). A graphic representation of the final model, for patients with or without a prerandomisation washout period, is shown in figure 1A,B.
The final model was evaluated using posterior predictive check methodology26 in order to assess whether the observed data were consistent with the range of expectation implied by the model. This model inherently adjusted for baseline HbA1c and washout status/duration. The other covariates (see previous page), with the exception of ethnicity, showed no major impact on the model parameters and were therefore not included in the final model. Further details of the mathematical and statistical specifications of the final model are presented in the online supplementary technical appendix.
Model summary and inference
Since the mean predicted values are not directly available as model parameters, these were estimated by taking averages of values that were simulated from the fitted model. In the same way that variances can be appropriately scaled according to sample size during model fitting, variances were scaled during simulation to simulate trial arms of different sizes. This included scaling simulation variances to correspond to n=1, which we conceptualised as the simulation of an individual patient.
In order to assess the efficacy of the two DPP-4 inhibitors in comparable patients under similar conditions, a population of 1000 patients was simulated from the model under reference conditions and the average HbA1c level was computed at each time-point for this simulated population. Data for each patient were simulated as if arising from an individual trial, so that the resulting inference represents an average over the expected range of intertrial variation. The simulation of this population average was then repeated for each of the 1000 different parameter configurations represented in the posterior sample (the entire posterior simulation therefore involved a total of 106 simulated patients), resulting in inferences that reflect posterior parameter uncertainty as well as intertrial and interpatient variation. The reference racial composition for this simulated population was 61.5% White, 1.5% Black and 37% Asian, reflecting the average enrolled distribution in linagliptin trials. The median simulated baseline HbA1c (%) in this population was 8 Results are expressed as mean differences, with 95% credible intervals (the Bayesian equivalent of CIs).
A total of 31 sitagliptin studies were assessed for eligibility for inclusion in the analysis, and 16 were excluded on the basis of the study design that did not meet our inclusion criteria (see online supplementary table S2). A further 10 linagliptin studies were included.
Data from a total of 11 234 participants were included in the analysis, arising from 25 randomised trials (10 linagliptin and 15 sitagliptin). The mean age at baseline of all study participants was 56.5 years, with reported means for treatment arms of the included studies ranging from 50.9 to 62 years; the proportion of women across all study participants was 45.5%, with reported proportions for study groups ranging from 22.8% to 64%; and the mean BMI was 29.7 kg/m2, with reported means for treatment arms ranging from 24.1 to 32.7 kg/m2. Mean baseline HbA1c was 8%, with reported means for treatment arms ranging from 7.49% to 8.87%. The most commonly used background medication was metformin monotherapy. Metformin was also used in combination with glimepiride or pioglitazone, and one study40 included patients receiving initial monotherapy with pioglitazone.
Figure 2A,B depict the application of the statistical model to each individual study, demonstrating that the observed data from the studies fall mostly within the 90% prediction interval (between 5% and 95% prediction bounds), with no overall systematic overprediction or underprediction. Both change from baseline and placebo- corrected change from baseline HbA1c percentage points are presented to demonstrate longitudinal model performance for each therapy. Similarly, figure 3A–D show the 90% credible intervals at the endpoint for the linagliptin and sitagliptin change from baseline and placebo-corrected change from baseline, demonstrating accurate prediction of the effect, on average.
The simulations performed using the model show that both linagliptin 5 mg and sitagliptin 100 mg reduce HbA1c levels by 0.81% (placebo-adjusted), at week 24, when administered to patients with T2DM for 24 weeks (figure 4A,B). Credible intervals for participants without washout were −0.88 to −0.75 (linagliptin) and −0.89 to −0.73 (sitagliptin). For those who underwent washout of previous antihyperglycaemic therapy, the credible intervals were −0.91 to −0.76 (linagliptin) and −0.91 to −0.75 (sitagliptin). Figure 5 shows simulated differences in the true effect at 24 weeks between linagliptin 5 mg and sitagliptin 100 mg with no washout, demonstrating that the model predicted difference lies almost entirely within 0.2 percentage points, less than the previously suggested margins for non-inferiority of 0.3–0.4 percentage points.46 ,47
As a post hoc assessment, a t test was used to compare the HbA1c difference from placebo residuals (unexplained variations after fitting of the model) for linagliptin and sitagliptin. A p value of 0.14 was generated, suggesting no evidence of a systematic bias in favour of linagliptin by conventional thresholds (p<0.05).
The model developed in this study incorporates Bayesian methodology and provides a tangible approach to indirectly estimating the comparative efficacy of two compounds. The findings presented suggest that the model developed in this study provides a valid alternative approach to indirect drug comparisons. The findings of this MBMA show that linagliptin is as effective as sitagliptin in the reduction of HbA1c levels, both showing a mean, placebo-adjusted reduction of approximately 0.81% after 24-weeks' treatment of patients with T2DM. In this study, evidence was gathered from the results of randomised, double-blind trials of sitagliptin and linagliptin. The sensitivity analyses performed in this study, using various prior distributions, support the robustness of the model. Although the use of MBMA is relatively new in the field of diabetes therapy, this method is nonetheless being increasingly recognised as an important tool in the evaluation of pharmaceutical therapies.48 ,49
There might be some limitations in applying the findings of the present analysis to the general population of patients with T2DM because of the relatively selected patient populations in the included trials, which included mostly white, middle-aged patients with mean baseline HbA1c <9%. The participants in the analysed trials would have been further restricted during pretrial run-in periods, which would exclude those with poor treatment adherence. Furthermore, the analysis was performed retrospectively, using data from different trials. As with all meta-analyses based on published data, there is a potential for publication bias. In the context of the present analysis, this potential bias pertains only to our estimates of the effects of sitagliptin, as our linagliptin data sources were not subject to publication selection. However, this is unlikely to have a substantial impact on the findings for sitagliptin, as current practice in clinical research mandates that all clinical trials are published regardless of their results and several sources were searched, including trial registries and documents used in the regulatory process.
The model includes the assumption that HbA1c levels are maintained after the full effect of treatment has been reached. This is based on observations in previous 24-week trials, where HbA1c levels have been shown to be maintained for this period,19–21 ,50 and the known pharmacological properties of DPP-4 inhibitors.4 ,51 ,52 The final model was adjusted for baseline HbA1c, ethnic origin and washout duration. Other covariates (concurrent medications, fraction of patients on previous oral antidiabetic drugs, BMI, age, gender, duration of T2DM) were not included in the final model because they did not show significant impact on the model parameters. Reasons for this might be either that only mean covariate values were available, or that some covariates are confounded (eg, BMI was shown to vary as a function of ethnic origin, making it difficult to isolate the independent effects of these covariates). It is important to recognise that these covariates might be of clinical importance, and their exclusion from the model could simply reflect an inability to reliably estimate the independent effect of these factors with the data available.
To date, four standard meta-analyses of the DPP-4 inhibitor class have been published, none of which has provided any results on the comparative efficacies of linagliptin and sitagliptin.16 ,53–55 These analyses confirm the efficacy of DPP-4 inhibitors, in terms of HbA1c reduction, and their tolerability, in particular resulting from the absence of weight gain and low risk of hypoglycaemia associated with monotherapy. The findings also indicate that therapy with DPP-4 inhibitors reduces HbA1c reductions to a similar extent to comparator drugs.53 Several of the limitations associated with traditional meta-analysis arise from the fact that only study end-point data are used in these analyses. For example, difficulties in selecting an appropriate summary statistic are often encountered because the treatment effect of interest varies as a function of the duration of treatment. Similarly, it might be difficult to appropriately adjust for the effect of covariates on treatment response when response is assessed at different time-points in different studies. To address the limitations of traditional meta-analysis, a general methodology has recently been proposed for the statistically valid use of MBMA.3 The advantage of this approach, also used in the present study, is that it enabled the synthesis of longitudinal data from multiple studies with different durations and different sampling schedules, resulting in analyses that are both more comprehensive (including a greater number of studies) and more efficient (incorporating more of the relevant data within each study) than previous methods. The unique MBMA approach in the current study also allows adjustment for covariates (eg, differences in the use of washout or racial composition in individual trials) to allow comparison of treatment response in comparable patients under similar conditions. One limitation of the study by Gibbs et al15 was that the MBMA used did not account for correlations across time points within treatment arms, which could lead to an overestimation of the intertrial variability in drug effect. In contrast, our approach takes account of longitudinal correlations, in accordance with previously published methods,3 which is a prerequisite to the correct characterisation of uncertainty in the estimation of drug effects.
As the clinical use of DPP-4 inhibitors increases, patients, prescribers and payers will require information on the relative benefits of the individual drugs within this class. Based on the model developed in this study, it is apparent that the efficacy of the two DPP-4 inhibitors, sitagliptin and linagliptin, is virtually indistinguishable, in terms of changes in mean HbA1c levels, in patients with T2DM treated with a range of background antihyperglycaemic therapies. Both linagliptin and sitagliptin act by inhibiting the DPP-4 enzyme that rapidly inactivates the intestinal hormone, glucagon-like peptide (GLP)-1. GLP-1 stimulates insulin secretion in a glucose-dependent manner. Sitagliptin is largely excreted via the kidneys, with the major portion of the oral dose (87%) being excreted in the urine.56 Unlike sitagliptin and other DPP-4 inhibitors, linagliptin has a largely non-renal route of excretion (only ∼5% excreted renally), with the majority being eliminated via the bile and gut57 ,58; it therefore does not require dose adjustment in patients with declining renal function.59 In view of the similar efficacy of these two drugs, treatment choices might, therefore, be made on the basis of other differences between the drugs and consideration of patient clinical characteristics, such as the patient's renal function.
Broadening the use of MBMA has the potential to improve the comparison of individual drug therapies, compared with older statistical methods, and could provide a new way of generating results for populations that have not yet been studied.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
Contributors All authors were fully responsible for all content and editorial decisions. They were involved at all stages of manuscript development, including reviewing and revising the manuscript for scientific content, and have approved the final version. In addition: JG contributed to the data analysis and interpretation of the findings. JR, DP and WG shared primary responsibilities for developing the statistical analysis plan, executed all statistical analyses (including model development, model selection and model summary) and interpreted the findings. SP monitored data collection, and contributed to data selection, the statistical analysis plan and interpretation of the results. CF contributed to the analysis concept, the statistical analysis plan and interpretation of the findings. BM contributed to data collection and the statistical analysis plan, and interpreted the findings. YG contributed to the interpretation of the findings. AS contributed to the analysis strategy, the statistical analysis plan and interpretation of the results. SR contributed to analysis strategy, the statistical analysis plan, data collection and interpretation of the results.
Funding Medical writing assistance, supported financially by Boehringer Ingelheim, was provided by Jennifer Edwards, MBBS, of Envision Scientific Solutions during the preparation of this article.
Competing interests All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: JG has received fees for Board membership from Boehringer Ingelheim, Novo Nordisk and Eli Lilly, and he has also received research grants from Boehringer Ingelheim, Eli Lilly, GlaxoSmithKline and Janssen. JR, DP and WG have received fees for participation in review activities, and for manuscript writing and reviewing from Boehringer Ingelheim. CF, YG, BM, SP, AS and SR are employees of Boehringer Ingelheim, the manufacturer of linagliptin.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Previous presentations Abstracts based on this study have been presented as posters at the 72nd Scientific Sessions of the American Diabetes Association, 8–12 June 2012, Philadelphia, Pennsylvania, USA, and at the Population Approach Group Europe conference, Venice, Italy, 5–8 June 2012.