Intended for healthcare professionals

  1. Research
  2. Potential impact on...
  3. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review
CCBYNC Open access
Research

Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review

BMJ 2012; 344 doi: https://doi.org/10.1136/bmj.e2809 (Published 18 May 2012) Cite this as: BMJ 2012;344:e2809
  1. Elie A Akl, associate professor12,
  2. Matthias Briel, assistant professor23,
  3. John J You, assistant professor24,
  4. Xin Sun, assistant professor25,
  5. Bradley C Johnston, assistant professor26,
  6. Jason W Busse, scientist27,
  7. Sohail Mulla, student2,
  8. Francois Lamontagne, adjunct professor8,
  9. Dirk Bassler, associate professor9,
  10. Claudio Vera, assistant professor10,
  11. Mohamad Alshurafa, student2,
  12. Christina M Katsios, student4,
  13. Qi Zhou, statistician2,
  14. Tali Cukierman-Yaffe, researcher411,
  15. Azim Gangji, assistant professor4,
  16. Edward J Mills, chair in global health12,
  17. Stephen D Walter, professor2,
  18. Deborah J Cook, professor24,
  19. Holger J Schünemann, department chair2413,
  20. Douglas G Altman, professor of statistics in medicine14,
  21. Gordon H Guyatt, professor24
  1. 1Departments of Medicine and Family Medicine, State University of New York at Buffalo, Buffalo, NY, USA
  2. 2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada
  3. 3Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, Switzerland
  4. 4Department of Medicine, McMaster University, Hamilton, ON
  5. 5Center for Health Research, Northwest, Kaiser Permanente Northwest, Portland, OR, USA
  6. 6Research Institute, Child Health Evaluative Sciences, Hospital for Sick Children, Toronto, ON, Canada
  7. 7Institute for Work and Health, Toronto, ON
  8. 8Centre de Recherche Clinique Étienne-Le Bel, Université de Sherbrooke, Sherbrooke, QC, Canada
  9. 9Centre for Paediatric Clinical Studies and Department of Neonatology, University Children’s Hospital Tuebingen, Tuebingen, Germany
  10. 10Division of Obstetrics and Gynaecology, Pontificia Universidad Católica de Chile, Santiago, Chile
  11. 11Gertner Institute for Epidemiology and Health Policy Research, Endocrinlogy Institute, Sheba Medical Center, Sackler school of Medicine, Tel-Aviv University, Tel-Aviv, Israel
  12. 12Faculty of Health Sciences, University of Ottawa, Ottawa, ON
  13. 13Institut für Medizinische Informatik und Biometrie, University of Freiburg, Germany
  14. 14Centre for Statistics in Medicine, University of Oxford, Oxford, UK
  1. Correspondence to: E A Akl, Department of Medicine, State University of New York at Buffalo, ECMC-CC 142, 462 Grider Street, Buffalo, NY 14215, USA elieakl{at}buffalo.edu
  • Accepted 9 March 2012

Abstract

Objective To assess the reporting, extent, and handling of loss to follow-up and its potential impact on the estimates of the effect of treatment in randomised controlled trials.

Design Systematic review. We calculated the percentage of trials for which the relative risk would no longer be significant under a number of assumptions about the outcomes of participants lost to follow-up.

Data sources Medline search of five top general medical journals, 2005-07.

Eligibility criteria Randomised controlled trials that reported a significant binary primary patient important outcome.

Results Of the 235 eligible reports identified, 31 (13%) did not report whether or not loss to follow-up occurred. In reports that did give the relevant information, the median percentage of participants lost to follow-up was 6% (interquartile range 2-14%). The method by which loss to follow-up was handled was unclear in 37 studies (19%); the most commonly used method was survival analysis (66, 35%). When we varied assumptions about loss to follow-up, results of 19% of trials were no longer significant if we assumed no participants lost to follow-up had the event of interest, 17% if we assumed that all participants lost to follow-up had the event, and 58% if we assumed a worst case scenario (all participants lost to follow-up in the treatment group and none of those in the control group had the event). Under more plausible assumptions, in which the incidence of events in those lost to follow-up relative to those followed-up is higher in the intervention than control group, results of 0% to 33% trials were no longer significant.

Conclusion Plausible assumptions regarding outcomes of patients lost to follow-up could change the interpretation of results of randomised controlled trials published in top medical journals.

Introduction

Loss to follow-up in randomised controlled trials could bias results if the unavailability of data is associated with the likelihood of outcome events. For example, patients might fail to return for assessment because of deterioration in their medical condition, resulting in a higher frequency of adverse outcomes of interest associated with that condition. If the distribution of such patients differs between study arms, the prognostic balance created by randomisation will be disturbed.1 2 Although analysis of patients for whom outcome data are available in the groups to which they are randomised will avoid bias as a result of factors such as non-adherence, it will not protect against potential bias associated with loss to follow-up.3

Although investigators strive to reduce the amount of missing data, in most instances they will fail to achieve complete follow-up.3 4 5 Indeed, 60-89% of randomised controlled trials have some missing outcome data.6 7 8 Interpretation of results is compromised when, as is often the case, investigators do not report strategies for handling such data.8 9 The most commonly reported strategy among trials that do report their approach is to restrict analyses to participants with full outcome information (complete case analysis).6 7

Most previous analyses have not found significant associations between the extent of loss to follow-up and the magnitude of treatment effect.10 11 12 13 14 These findings suggest that substantial loss to follow-up sometimes leads to overestimates of treatment effects and sometimes to underestimates. Another approach to examining the possibility of bias associated with loss to follow-up in randomised controlled trials is to evaluate the potential impact on effect estimates of various assumptions regarding the outcomes of participants lost to follow-up. We examined this question in a sample of randomised controlled trials from five general medical journals with the highest impact factors. We also described the reporting, extent, and handling of loss to follow-up.

Methods

We have reported the protocol for this study and the full details of our methods elsewhere.15 For this study, we defined loss to follow-up as incomplete ascertainment of the primary outcome for some participants in a randomised controlled trial. If the authors excluded some participants from the analysis but still provided their primary outcome data, we considered that loss to follow-up did not occur. If the authors did not provide the primary outcome data of those excluded participants, however, we considered that loss to follow-up did occur.

Eligibility criteria

Eligible studies were randomised controlled trials published in one of the five general medical journals with the highest impact factors: Annals of Internal Medicine, BMJ, JAMA, Lancet, and New England Journal of Medicine. All eligible studies reported a significant binary primary patient important outcome (P<0.05 or confidence interval not including 1).16 We focused on these trials because they are the most likely to influence clinical practice and to have lower rates of loss to follow-up. Results would therefore reflect a conservative estimate of risk of bias associated with loss to follow-up in randomised trials in general. We excluded cluster trials, crossover trials, “n of 1” trials, and trials reported in research letters.

We defined a patient important outcome as one for which one would answer the following question with “yes”: “if the patient knew that this outcome was the only thing to change with treatment, would the patient consider receiving this treatment if it was associated with adverse effects, inconvenience, or cost?” Such outcomes included mortality, morbidity, and outcomes reported by patients. We considered surrogate outcomes (such as changes in blood pressure, HbA1c) as not patient important.

Literature search

We used the Cochrane Collaboration’s highly sensitive search strategy to identify reports of randomised controlled trials in Medline (OVID interface) (see appendix 1). We restricted the search to human trials published in the selected journals during 2005-07.

Review process

After formal calibration exercises, pairs of reviewers trained in health research methodology performed each of the review stages (screening title and abstract, screening full text, selection of the primary outcome, and data abstraction) independently and in duplicate. They used standardised pilot tested forms and detailed written instructions and resolved disagreements by discussion and with the assistance of an arbitrator when needed. We contacted authors of all included reports to verify the results of our data abstraction and made corrections accordingly.17

Abstracted data

For each trial, we abstracted data relating to general characteristics, methodological characteristics, and the reporting, handling in the analysis, and extent of loss to follow-up.15 We recorded circumstances associated with loss to follow-up: mistakenly randomised with inappropriate post-randomisation exclusion; did not receive the intervention with inappropriate post-randomisation exclusion; withdrew consent; did not adhere to treatment; crossed over to the other treatment; or lost contact. We judged post-randomisation exclusion as inappropriate unless the information about ineligibility was available at baseline and those making the decision regarding exclusion were blinded to allocation.18 For each trial, including those using survival analysis, we also collected data to construct 2×2 tables (intervention/control×event/no event).

Analysis

We assessed agreement between reviewers for both the title and abstract screening stage and the full text screening stage using the κ statistic.

Reporting, handling, and extent of loss to follow-up

We conducted a descriptive analysis of the different aspects of reporting and handling of loss to follow-up. To estimate the extent of loss to follow-up, we calculated the percentage of loss to follow-up in each trial and the median and interquartile range of that percentage across trials. For each trial we calculated the ratio of the total number of participants classified as lost to follow-up to the number of primary outcome events: the “lost to follow-up to events ratio.” We then calculated the mean and standard deviation of this ratio across trials.

We conducted a multiple linear regression analysis with “percentage of participants lost to follow-up” as the dependent variable and the following independent variables:

  • General trial characteristics:

    • Number of centres

    • Type of funding

    • Type of outcome (mortality v other)

    • Clinical specialty (medical v surgical)

    • Type of intervention (pharmacological v surgery/invasive procedure v other)

    • Length of follow-up

  • Methodological trial characteristics:

    • Concealment of allocation

    • Blinding of patients

    • Stopping early for benefit

    • Reporting the use of intention to treat analysis

    • Actual analysis of participants in the groups to which they were randomised.

Potential impact of loss to follow-up

We evaluated the effect of several assumptions about the outcomes of participants lost to follow-up on the estimate of effect for the primary outcome (see appendix 2 for an illustrative example for each assumption). We first evaluated four common assumptions:

  • None of the participants lost to follow-up had the event

  • All the participants lost to follow-up had the event

  • None of those lost to follow-up in the treatment group had the event and all those lost to follow-up in the control group did (best case scenario)

  • All participants lost to follow-up in the treatment group had the event and none of those in the control group did (worst case scenario).

While these assumptions are commonly used in the literature,19 they are not plausible. Thus, we evaluated the more plausible assumptions that the incidence of events among participants lost to follow-up is higher by a specific ratio relative to the observed event incidence among participants followed up. For this purpose, we defined RILTFU/FU as the event incidence among those lost to follow-up relative to the event incidence among those followed up (see appendix 2). LTFU refers to “lost to follow-up” and FU refers to “followed up.” The assumptions we evaluated combine a range of what we believe are plausible RILTFU/FU values (1, 1.5, 2, 3.5) in the intervention group and control group. We chose an upper limit of 5 for RILTFU/FU as it represents the highest ratio reported in the literature. Geng et al used a community tracker to evaluate the incidence of death among participants in scale-up programmes of antiretroviral treatment in Africa who were lost to follow-up.2 They found the mortality rate to be five times higher in patients lost to follow-up compared with patients who were followed up.2 We converted all trial outcome data so that all events were regarded as negative (for example, we converted “survival” to “death” and “absence of pain” to “presence of pain”).

Percentage of trials losing significance—For each of the trials, we used data from the 2×2 tables to calculate the relative risk associated with each of the assumptions (see appendix 2). We then calculated the percentage of trials whose primary outcome was no longer significant for each of the assumptions. We excluded trials in which no loss to follow-up occurred but included them in a subsequent sensitivity analysis. We also excluded trials that lost significance with a complete case analysis (that is, analysis excluding participants lost to follow-up from both the denominator and numerator). Such a finding might indicate that loss of significance with any of the assumptions could simply be because of our use of a 2×2 table calculation. Trials in which the investigators used either a survival analysis or an adjusted analysis might be particularly prone to such an effect. Not excluding these studies would inflate the results.

Mean ratio of relative risks—For each of the above assumptions and for each trial we calculated the ratio of the relative risk based on the specific assumption to the relative risk based on a complete case analysis. We then used the inverse variance method to pool the ratio of relative risks across the trials. We did not calculate the mean change in effect estimate as initially planned because of considerable diversity in the effect measures and associated estimates of precision reported among eligible trials.15

Sample size

We estimated that our search strategy would identify about 200 eligible studies during 2005-07. We judged that such a sample size would result in acceptable 95% confidence intervals for three hypothesised proportions (10%, 20%, and 30%) of studies losing significance based on our assumptions: 5.8 to 14.2 for 10%; 14.5 to 25.5 for 20%; and 23.7 to 36.4 for 30%.

Results

We included 235 eligible reports, close to our target number (figure⇓). Agreement between reviewers during the screening process was high: κ 0.93 for title and abstract screening and 0.78 for full text screening. Authors of 107 reports responded to our request to verify abstracted data (46% response rate).17 The percentage accuracy across trials for the different abstracted items varied from 84% to 100%. Inaccuracies were related to unclear reporting. Tables 1⇓ and 2⇓, respectively, present the general and methodological characteristics of included trials.

Figure1

Fig 1 Identification of articles to include in study of effect of loss to follow-up

Table 1

 General characteristics of 235 included trials in study to determine effect of loss to follow-up on outcomes

View this table:
Table 2

 Methodological characteristics of 235 included trials in study to determine effect of loss to follow-up on outcomes

View this table:

Reporting of loss to follow-up

Table 3⇓ shows the percentage of trials with different information regarding reporting of loss to follow-up. Of the 235 trials, 204 (87%) had either an explicit statement about loss to follow-up or a CONSORT flow diagram showing loss to follow-up, or both.

Table 3

 Reporting of information regarding loss to follow-up in included trials in study to determine effect of loss to follow-up on outcomes

View this table:

Extent of loss to follow-up

Of the included trials, 191 (81%) reported some loss to follow-up. Among these 191 trials, the medians and interquartile ranges for the reported percentage of loss to follow-up were 6% (2-14%) overall; 6% (1-14%) in the intervention group; and 7% (2-15%) in the control group (the difference between intervention and control groups was not significant). The medians and interquartile ranges for the “lost to follow-up to events ratio” were 0.26 (0.09-0.76); 0.30 (0.09-1.25) in the intervention group; and 0.23 (0.08-0.65) in the control group. A value of 0.26 means that one participant was lost to follow-up for every four participants experiencing the primary outcome (the difference between intervention and control groups was not significant).

In our regression analysis, a higher percentage of participants lost to follow-up was associated with inadequate concealment of allocation, a longer length of follow up, and a non-medical non-procedural intervention (see appendix 3).

Handling of loss to follow-up

Table 4⇓ shows the analytical methods used for handling loss to follow-up in the primary analysis of the 191 trials that reported some loss to follow-up. In about a fifth of trials, the method used was unclear.

Table 4

 Analytical methods for handling loss to follow-up applied by authors in their primary analysis in 191 studies

View this table:

Potential impact of loss to follow-up

Of the 191 trials reporting some loss to follow-up, results of 160 remained significant in a complete case analysis.

Percentage of trials losing significance—For the four common assumptions, the percentage of trials that lost significance varied from 0% (best case scenario) to 9% (none of the participants lost to follow-up had the event) to 17% (all participants lost to follow-up had the event) to 58% (worst case scenario). Table 5⇓ shows the percentage of eligible trials that lost significance across a range of assumptions for the event incidence among those lost to follow-up relative to those followed-up. This percentage varied from 0% to 33% for the range of plausible assumptions. The percentages in the sensitivity analyses that included trials in which no loss to follow-up occurred were relatively lower across all assumptions by about 20% (see appendix 4).

Table 5

 Percentage of 160 trials in which results would no longer be significant under different assumptions about outcomes of participants lost to follow-up in intervention and control groups

View this table:

Mean ratio of relative risks—The mean ratio of relative risk across randomised controlled trials varied from 0.73 (best case scenario) to 1.51 (worst case scenario) (see appendix 5). A ratio of 1.51 signifies a relative increase in relative risk by 51% (for example, if the relative risk is 0.8 with the complete case, it would be 1.2 in the worst case scenario). Appendix 5 also presents the findings for the assumptions that the event incidence among participants lost to follow-up decreased or increased relative to those followed-up. The ratio varied from 0.79 to 1.23 (that is, the relative risk increased by up to 23%).

Discussion

Summary of findings

Up to a third of trials published in five top general medical journals and reporting significant results for binary primary outcomes that are patient important lose significance if one makes plausible assumptions about their loss to follow-up. Thirteen percent of trials did not report whether loss to follow-up occurred. In those that did report loss to follow-up, the median percentage of loss to follow-up was 6%; a fifth of the trials did not report on how missing data from participants were handled.

Interpretation of findings

The plausibility of our assumptions about the outcomes of patients lost to follow-up varied. The first two assumptions (none of the participants lost to follow-up had the event and all participants lost to follow-up had the event) are commonly used but are implausible. The third assumption (worst case scenario) can be used to verify the robustness of a trial results but is extreme and generally unrealistic.20 In our sample, results of only 42% of randomised controlled trials would retain significance under a worst case scenario.

We designed the remaining assumptions (that is, the combinations of RILTFU/FU) to be more plausible on the basis of limited evidence that patients who are lost to follow-up tend to have worse outcomes.1 2 20 We would have tested other imputation methods (such as multiple imputations and regression models) if individual patient data had been available, which, as is typical in most study reports, it never was.

The most plausible assumption can depend on the question being examined by the trial. For example, an assumption that all patients lost to follow-up experienced an adverse event could be reasonable when patients are expected to comply with the trial protocol and follow-up. This might be the case in a trial used to evaluate a drug to prevent rejection after cardiac transplantation.21 Indeed, the investigators of such a trial implicitly made this assumption by including loss to follow-up as a component of a composite primary end point along with morbidity and mortality outcomes.21 Smoking cessation trials generally make the assumption that those lost to follow-up have failed to quit.22

The most plausible assumption also depends on the reason for loss to follow-up. For example, participants who were lost to follow-up because they “moved” are likely to have better outcomes than those who were excluded because of “failure to improve.” Higgins et al proposed choosing the assumption based on the reason for loss to follow-up and, if necessary, using different assumptions for different groups of loss to follow-up in the same trial.23

The most appropriate assumption could also be determined by using empirical data from studies assessing the outcomes of patients lost to follow-up from related trials.1 Investigators could also use data from population based studies.24 25 For example, a systematic review of studies tracing the outcomes of patients lost to follow-up from antiretroviral treatment programmes found that mortality was inversely associated with the rate of loss to follow up.26 Caution and judgment are needed when these observations are applied to randomised controlled trials. Baseline characteristics of participants lost to follow-up is another factor to consider. When these characteristics suggest poorer prognosis relative to participants followed up (such as older people and higher percentages of comorbidities), participants lost to follow-up would probably have poorer outcomes.

Loss to follow-up with no bias reduces power because the effective sample size is reduced. By imputing some events in those lost to follow-up, the total number of events rises and some power is regained, particularly if one assumes the same risk in those lost to follow-up and their respective treatment groups (such as when a value of 1 for the RILTFU/FU is used in both groups). Given, however, that the investigators are making up the data, one could argue that the apparent increased precision is misleading. Indeed, Higgins et al have highlighted the need to take into account the uncertainty about the imputed data.23 Taking into account the uncertainty would result in wider confidence intervals; if we had done this, results of more trials would have lost significance. Our results are therefore conservative estimates of the percentage of trials losing significance.

The association we found between the length of follow-up and the extent of loss to follow-up is not surprising as longer follow-up will inevitably result in difficulties retaining all randomised patients. The association between inadequate concealment of allocation and the extent of loss to follow-up could represent less methodological rigor in both aspects of design and conduct. One might have expected that the extent of loss to follow-up would be associated with the type of outcome (that is, more loss to follow-up with non-fatal compared with fatal outcomes); we did not find this to be the case.

Strengths and limitations of study

Our study has several strengths. Our a priori publication of the study protocol made our objectives and methods transparent and subjected them to peer review.15 We used transparent and systematic methods to search for and select eligible studies, select the primary outcomes, and abstract data. We also ensured rigorous data abstraction by using detailed written instructions, conducting formal calibration exercises, conducting duplicate abstraction, measuring agreement, and implementing a consensus approach to resolve disagreement. We contacted authors to verify our abstracted data and achieved a 46% response rate and most data were verified as accurate.17 While other authors have proposed many of these assumptions,23 we think that this is the first study to propose and test approaches based on RILTFU/FU.

One limitation of our study is its generalisability because of its focus on reports published in five top general medical journals rather than a wider range of journals. Randomised controlled trials published in lower profile journals, however, might report smaller effects than those published in top medical journals27 and might be of lower methodological quality28 and thus have higher rates of loss to follow-up. Lower effect estimates and higher rates of loss to follow-up would inflate the potential impact of loss to follow-up. Thus, our findings are more likely an underestimate of the impact of loss to follow-up in a wider range of randomised controlled trials. Our results do not apply to continuous data, which present specific challenges that need to be dealt with separately.29

We focused on reports of trials with significant effect estimates because these studies are more likely to influence clinical practice. Also, unless event rates in those lost to follow-up are greater relative to those followed up in control groups rather than intervention groups, ignoring missing data will not result in misleading inferences. The reasonably narrow time range of the included studies (2005-07) was determined to a large extent by our sampling method; we first sampled all eligible trials published in 2007 and included trials from preceding years until we reached our target sample size. If trials in these years are idiosyncratic, or if strategies for avoiding loss to follow-up have improved in the years since 2007, our results could be unrepresentative. Neither of these possibilities, however, is likely.

Finally, we used a frequentist statistical approach to explore the impact of loss to follow-up on effect estimates. An alternative would have been a Bayesian approach.30 31

Implications of findings

This study has important implications for trialists, editors of medical journals, systematic reviewers, and users of medical literature. Investigators should of course aim to reduce the extent of loss to follow-up in the design and implementation of their trials.5 They should also be transparent and detailed in reporting loss to follow-up (such as, extent, timing, reasons, and baseline characteristics of those lost to follow-up, all by study arm) and describe the potential implications for their primary analysis.19 Specifically, conducting sensitivity analyses with reasonable assumptions about loss to follow-up is necessary to test the robustness of their results.19 The assumptions we have made could be a reasonable standard from which trialists could deviate if they have compelling reasons to do so. Our study was limited to relatively simple assumptions that do not require individual participant data. If this level of data is available then investigators should consider more sophisticated statistical methods such as multiple imputation.32 33 Editors of medical journals have the opportunity to improve the quality of the medical literature by enforcing the use of the CONSORT statement, particularly as it relates to reporting the patient flow diagram and the number of patients lost to follow-up, the reasons for loss to follow-up, and the number of patients analysed.34

Systematic reviewers should consider all available information about the extent of loss to follow-up and the assumptions used in the primary analysis of original reports. They should also routinely conduct sensitivity analyses with reasonable assumptions about the outcomes of those lost to follow-up to test the robustness of the results of their meta-analyses.

Users of published medical literature should be aware of the potential vulnerability of apparently positive results to loss to follow-up. Important factors that might be associated with higher vulnerability include a small magnitude of effect, a high number of participants lost to follow-up (particularly when compared with the number of events), differential loss to follow-up in study arms (in terms of numbers and reasons), poorer baseline prognosis of participants lost to follow-up, reasons for loss to follow-up likely to be associated with poorer prognosis, and loss of statistical or clinical significance, or both, when reasonable assumptions about participants lost to follow-up are applied.

Future research should include collection of empirical evidence to define the most reasonable assumptions about the outcomes of patients lost to follow-up. Assumptions will probably vary with the population involved, the nature of the intervention, and the outcome under consideration. Similar work is also needed to inform the impact of loss to follow-up for continuous outcomes. For now, authors of individual randomised controlled trials and of systematic reviews should test their results against various reasonable assumptions. Only when the results are robust to all reasonable assumptions can inferences from those results be viewed as secure.

What is already known on this topic

  • Loss to follow-up in randomised controlled trials can bias results if the unavailability of data is associated with the likelihood of outcome events

  • Substantial loss to follow-up can lead to overestimates and underestimates of treatment effects

What this study adds

  • Reporting the extent, the reasons for, and the handling of loss to follow-up in randomised clinical trials published in top general medical journals remains suboptimal

  • In up to a third of cases, positive findings in these trials lose significance when plausible assumptions about the outcomes of participants lost to follow-up are made

Notes

Cite this as: BMJ 2012;344:e2809

Footnotes

  • We thank Ann Grifasi, Deborah Maddock Shelley Anderson, and Monica Owen for their administrative assistance and Aravin Duraik for developing the study electronic forms.

  • Contributors: EAA, MB, JJY, XS, FL, MA, EJM, SDW, DJC, HJS, DGA, and GHG were responsible for study conception and design. EAA, MB, JJY, XS, BCJ, JWB, SM, FL, DB, CV, MA, CMK, TC-Y, and AG acquired the data. EAA, QZ, and GHG analysed the data. EAA drafted the manuscript and is guarantor. All authors critically revised the manuscript and approved the final version.

  • Funding: This study was funded by Pfizer. The funder had no role in the study design, in the writing of the manuscript, or in the decision to submit this or future manuscripts for publication. MB is supported by a scholarship for advanced researchers from the Swiss National Foundation (PASMA-112951/1) and the Roche Research Foundation. JJY is supported by a career scientist award from the Ontario Ministry of Health and Long-Term Care. XS is supported by the National Natural Science Foundation of China (70703025). BCJ is supported by a SickKids Foundation postdoctoral fellowship. JWB is funded by a new investigator award from the CIHR and Canadian Chiropractic Research Foundation. AG is supported by the Kidney Foundation of Canada/Canadian Society of Nephrology fellowship award. DJC and EJM are supported by research chairs from CIHR. DGA is supported by Cancer Research UK.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not required.

  • Data sharing: No additional data available.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

References

  1. ↵
    Te Riele WW, Boerma D, Wiezer MJ, Borel Rinkes IH, van Ramshorst B. Long-term results of laparoscopic adjustable gastric banding in patients lost to follow-up. Br J Surg2010;97:1535-40.
    OpenUrlCrossRefPubMedWeb of Science
  2. ↵
    Geng EH, Emenyonu N, Bwana MB, Glidden DV, Martin JN. Sampling-based approach to determining outcomes of patients lost to follow-up in antiretroviral therapy scale-up programs in Africa. JAMA2008;300:506-7.
    OpenUrlCrossRefPubMed
  3. ↵
    Montori VM, Guyatt GH. Intention-to-treat principle. 2001;165:1339-41.
  4. ↵
    Fleming TR. Addressing missing data in clinical trials. Ann Intern Med2011;154:113-7.
    OpenUrlCrossRefPubMedWeb of Science
  5. ↵
    Altman DG. Missing outcomes: addressing the dilemma. Open Med 2009;3:e21-3.
    OpenUrl
  6. ↵
    Wood AM, White IR, Thompson SG. Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clinical Trials2004;1:368-76.
    OpenUrlCrossRefPubMed
  7. ↵
    Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ1999;319:670-4.
    OpenUrlAbstract/FREE Full Text
  8. ↵
    Baron B, Boutron Bruno I, Ravaud G. Violation of the intent-to-treat principle and rate of missing data in superiority trials assessing structural outcomes in rheumatic diseases. Arthritis Rheum2005;52:1858-65.
    OpenUrlCrossRefPubMedWeb of Science
  9. ↵
    Gravel J, Opatrny L, Shapiro S. The intention-to-treat approach in randomized controlled trials: Are authors saying what they do and doing what they say? Clinical Trials2007;4:350-6.
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA2002;287:2973-82.
    OpenUrlCrossRefPubMedWeb of Science
  11. ↵
    Kjaergard LL, Villumsen J, Gluud C, Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med2001;135:982-9.
    OpenUrlCrossRefPubMedWeb of Science
  12. ↵
    Schulz KF, Chalmers I, Hayes RJ, Altman DG, Schulz KF, Chalmers I, et al. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA1995;273:408-12.
    OpenUrlCrossRefPubMedWeb of Science
  13. ↵
    Tierney JF, Stewart LA. Investigating patient exclusion bias in meta-analysis. Int J Epidemiol 2005;34:79-87.
    OpenUrlAbstract/FREE Full Text
  14. ↵
    Hewitt CE, Kumaravel B, Dumville JC, Torgerson DJ. Assessing the impact of attrition in randomized controlled trials. J Clin Epidemiol2010;63:1264-70.
    OpenUrlCrossRefPubMed
  15. ↵
    Akl E, Briel M, You J, Lamontagne F, Gangji A, Cukierman-Yaffe T, et al. LOST to follow-up Information in Trials (LOST-IT): a protocol on the potential impact. Trials2009;10:40.
    OpenUrlCrossRefPubMed
  16. ↵
    Guyatt G, Montori V, Devereaux PJ, Schunemann H, Bhandari M. Patients at the center: in our practice, and in our use of language. ACP J Club2004;140:A11-2.
    OpenUrlPubMed
  17. ↵
    Akl EA, Sun X, Busse JW, Johnston BC, Briel M, Mulla S, et al. Specific instructions for estimating unclearly reported blinding status in randomized trials were reliable and valid. J Clin Epidemiol2011;65:262-7.
    OpenUrlPubMed
  18. ↵
    Fergusson D, Aaron SD, Guyatt G, Hebert P. Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis. BMJ2002;325:652-4.
    OpenUrlFREE Full Text
  19. ↵
    White IR, Horton NJ, Carpenter J, Pocock SJ. Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ2011;342:d40.
    OpenUrlFREE Full Text
  20. ↵
    Unnebrink K, Windeler J. Intention-to-treat: methods for dealing with missing values in clinical trials of progressively deteriorating diseases. Stat Med2001;20:3931-46.
    OpenUrlCrossRefPubMedWeb of Science
  21. ↵
    Hershberger RE, Starling RC, Eisen HJ, Bergh C-H, Kormos RL, Love RB, et al. Daclizumab to prevent rejection after cardiac transplantation. N Engl J Med2005;352:2705-13.
    OpenUrlCrossRefPubMedWeb of Science
  22. ↵
    Twardella D, Brenner H. Implications of nonresponse patterns in the analysis of smoking cessation trials. Nicotine Tob Res2008;10:891-6.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    Higgins JP, White IR, Wood AM. Imputation methods for missing outcome data in meta-analysis of clinical trials. Clin Trials2008;5:225-39.
    OpenUrlAbstract/FREE Full Text
  24. ↵
    Weigel R, Hochgesang M, Brinkhof MW, Hosseinipour MC, Boxshall M, Mhango E, et al. Outcomes and associated risk factors of patients traced after being lost to follow-up from antiretroviral treatment in Lilongwe, Malawi. BMC Infect Dis2011;11:31.
    OpenUrlCrossRefPubMed
  25. ↵
    Geng EH, Bangsberg DR, Musinguzi N, Emenyonu N, Bwana MB, Yiannoutsos CT, et al. Understanding reasons for and outcomes of patients lost to follow-up in antiretroviral therapy programs in Africa through a sampling-based approach. J Acquir Immune Defic Syndr2010;53:405-11.
    OpenUrlCrossRefPubMedWeb of Science
  26. ↵
    Brinkhof MW, Pujades-Rodriguez M, Egger M. Mortality of patients lost to follow-up in antiretroviral treatment programmes in resource-limited settings: systematic review and meta-analysis. PLoS One2009;4:e5790.
    OpenUrlCrossRefPubMed
  27. ↵
    Montori VM, Devereaux PJ, Adhikari NKJ, Burns KEA, Eggert CH, Briel M, et al. Randomized trials stopped early for benefit: a systematic review. JAMA2005;294:2203-9.
    OpenUrlCrossRefPubMedWeb of Science
  28. ↵
    Lee KP, Schotland M, Bacchetti P, Bero LA. Association of journal quality indicators with methodological quality of clinical research articles. JAMA2002;287:2805-8.
    OpenUrlCrossRefPubMedWeb of Science
  29. ↵
    Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ2009;338:b2393.
    OpenUrlFREE Full Text
  30. ↵
    Eddy DM, Hasselblad V, Shachter R. An introduction to a Bayesian method for meta-analysis: the confidence profile method. Med Decis Making1990;10:15-23.
    OpenUrlAbstract/FREE Full Text
  31. ↵
    Greenland S. Bayesian perspectives for epidemiologic research: III. Bias analysis via missing-data methods. Int J Epidemiol2009;38:1662-73.
    OpenUrlAbstract/FREE Full Text
  32. ↵
    Rubin DB. Multiple imputation for nonresponse in surveys. John Wiley, 2004.
  33. ↵
    Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol2009;9:57.
    OpenUrlCrossRefPubMed
  34. ↵
    Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ2010;340:c869.
    OpenUrlFREE Full Text
View Abstract
Back to top