Article Text

Download PDFPDF

Counterintuitive results from observational data: a case study and discussion
  1. Erik Doty1,
  2. David J Stone2,3,
  3. Ned McCague3,4,
  4. Leo Anthony Celi5
  1. 1 Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
  2. 2 University of Virginia School of Medicine, Charlottesville, Virginia, USA
  3. 3 Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
  4. 4 Verily, Cambridge, Massachusetts, USA
  5. 5 Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
  1. Correspondence to Dr Leo Anthony Celi; lceli{at}


Objective To explore the issue of counterintuitive data via analysis of a representative case in which the data obtained was unexpected and inconsistent with current knowledge. We then discuss the issue of counterintuitive data while developing a framework for approaching such findings.

Design The case study is a retrospective analysis of a cohort of coronary artery bypass graft (CABG) patients. Regression was used to examine the association between perceived pain in the intensive care unit (ICU) and selected outcomes.

Setting Medical Information Mart for Intensive Care-III, a publicly available, de-identified critical care patient database.

Participants 844 adult patients from the database who underwent CABG surgery and were extubated within 24 hours after ICU admission.

Outcomes 30 day mortality, 1 year mortality and hospital length of stay (LOS).

Results Increased pain levels were found to be significantly associated with reduced mortality at 30 days and 1 year, and shorter hospital LOS. A one-point increase in mean pain level was found to be associated with a reduction in the odds of 30 day and 1 year mortality by a factor of 0.457 (95% CI 0.304 to 0.687, p<0.01) and 0.710 (95% CI 0.571 to 0.881, p<0.01) respectively, and a 0.916 (95% CI −1.159 to –0.673, p<0.01) day decrease in hospital LOS.

Conclusion The finding of an association between increased pain and improved outcomes was unexpected and clinically counterintuitive. In an increasingly digitised age of medical big data, such results are likely to become more common. The reliability of such counterintuitive results must be carefully examined. We suggest several issues to consider in this analytic process. If the data is determined to be valid, consideration must then be made towards alternative explanations for the counterintuitive results observed. Such results may in fact indicate that current clinical knowledge is incomplete or not have been firmly based on empirical evidence and function to inspire further research into the factors involved.

  • pain
  • mortality
  • length of stay

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

View Full Text

Statistics from


  • Contributors ED was responsible for the data extraction, the initial statistical analysis and writing and editing the manuscript. NM was involved in validating the statistical models and participated in editing the manuscript. DS was responsible for assisting with background information and editing the manuscript. LC was the project supervisor, responsible for project conception and manuscript editing.

  • Funding Leo Anthony Celi receives extramural funding from the National Institute of Health.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement The datasets generated for the current study were derived from the MIMIC-III Database available at The data subsets and statistical code used in this project can be found at

  • Patient consent for publication Not required.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.