Objectives We sought to understand why randomised controlled trials in septic shock have failed to demonstrate effectiveness in the face of improving overall outcomes for patients and seemingly promising results of early phase trials of interventions.
Design We performed a retrospective analysis of large critical care trials of severe sepsis and septic shock. Data were collected from the primary trial manuscripts, prepublished statistical plans or by direct communication with corresponding authors.
Setting Critical care randomised control trials in severe sepsis and septic shock.
Participants 14 619 patients randomised in 13 trials published between 2005 and 2015, enrolling greater than 500 patients and powered to a primary outcome of mortality.
Intervention Multiple interventions including the evaluation of treatment strategies and novel therapeutics.
Primary and secondary outcome measures Our primary outcome measure was the difference between the anticipated and actual control arm mortality. Secondary analysis examined the actual effect size and the anticipated effect size employed in sample size calculation.
Results In this post hoc analysis of 13 trials with 14 619 patients randomised, we highlight a global tendency to overestimate control arm mortality in estimating sample size (absolute difference 9.8%, 95% CI −14.7% to −5.0%, p<0.001). When we compared anticipated and actual effect size of a treatment, there was also a substantial overestimation in proposed values (absolute difference 7.4%, 95% CI −9.0% to −5.8%, p<0.0001).
Conclusions An interpretation of our results is that trials are consistently underpowered in the planning phase by employing erroneous variables to calculate a satisfactory sample size. Our analysis cannot establish if, given a larger sample size, a trial would have had a positive result. It is disappointing so many promising phase II results have not translated into durable phase III outcomes. It is possible that our current framework has biased us towards discounting potentially life-saving treatments.
- clinical trials
- septic shock
- randomized control trials
- sample size calculation
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Contributors JLCW: substantial contribution towards conception, design analysis and interpretation of the data. AJM: substantial contribution towards analysis and interpretation of data. ACG: substantial contribution in drafting work and revising it critically for important intellectual content. SJB: substantial contribution towards conception, design analysis and interpretation of the data. All authors approved of the final version. We agreed to be accountable for all aspects of the work and ensure accuracy and integrity.
Funding This study was supported by the National Institute for Health (NIHR) Comprehensive Biomedical Research Centre based at Imperial College Healthcare NHS Trust and Imperial College London. ACG is funded by a National Institute of Health Research (NIHR) Research Professorship award (RP-2015-06-018)
Disclaimer The views expressed are those of the authors and not necessarily those of the NIHR, the NHS or the UK Department of Health.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data in this paper are available from the published peer-reviewed literature.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.