Population

We selected patients from the MIMIC database who met all of the following inclusion criteria and none of the exclusion criteria. Inclusion criteria included: (1) Adult >18 years old, (2) who underwent CABG surgery and (3) was extubated within 24 hours after arrival to the intensive care unit (ICU). Exclusion criteria were: (1) Non-CABG surgical procedure and (2) missing data on confounding variables. Patients were identified using Current Procedural Terminology (CPT) codes: The following CPT codes corresponded to the CABG procedure: 33 510 to 33 516 for venous grafting only for coronary artery bypass and 33 533 to 33 548 for arterial grafting for coronary bypass. The final study cohort contained 844 patients (figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Shows selection of patient cohort from MIMIC database. After selecting those who underwent CABG procedure and excluding those with no pain measurements, 844 patients were extubated within 24 hours following surgery and included in the cohort. CABG, coronary artery bypass graft; MIMIC, Medical Information Mart for Intensive Care.

The MIMIC-III database included 1917 patients who underwent CABG with 844 meeting the study criteria. CABG was chosen for the investigation as it is a common procedure with the majority of patients having no or few postoperative complications and relatively predictable recoveries.5 Due to the nature of the surgical procedure which requires sternal spreading for exposure, there is an expected high analgesic burden immediately after surgery.

Outcomes

The primary outcome assessed was mortality at 30 days. Secondary outcomes were mortality at 1 year and hospital LOS. In the MIMIC database, mortality data for patients who die after hospital discharge is derived from the social security death registry.4

Exposures

The exposures of interest were pain levels reported by the patient immediately and in the subsequent interval after ICU extubation. Pain levels on a scale of 0 to 10 were regularly self-reported by patients to ICU nurses and recorded in the database, generating a continuum of measurements for each patient. The mean, median and maximum pain levels were used for separate analyses. Concomitant measurements of heart rates, respiratory rates and systolic blood pressures were also compared against their simultaneously recorded pain measurement.

Intravenous (IV) opiate administration was extracted from the database. MIMIC contained data for the following medications: Morphine, fentanyl, hydromorphone and meperidine. The was no data in MIMIC corresponding to the administration of oral analgesics.

We also looked for an association of pain and nausea for use in falsification hypothesis testing. The presence of nausea was derived from the nursing notes stored in the database. A positive nausea exposure was defined as the mention ‘nausea’ or ‘nauseous’ in the nursing note with no negative descriptor, such as ‘not nauseous’ or ‘denies nausea’, attached.

Covariates

Several variables found to be linked to worse patient outcomes in previous studies were included to control for confounding in the regression models; demographic factors, comorbid conditions and illness severity score on admission to the ICU.8 9 Comorbid burden was represented by the Elixhauser index which is determined by the aggregate presence or absence of 30 different comorbid conditions as detected by International Classification of Diseases (ICD)-9 codes.10 These conditions include but are not limited to cardiovascular disorders such as hypertension, congestive heart failure, coronary artery disease and peripheral vascular disease; pulmonary disorders such as chronic obstructive pulmonary disease; endocrine disorders such as diabetes and hypothyroid; obesity; drug and alcohol use disorders; renal disease; liver disease. Illness severity was captured using the Oxford Acute Severity of Illness Score (OASIS), which is calculated on admission to the ICU and takes into account age, heart rate, Glasgow coma scale, mean arterial pressure, temperature, respiratory rate, ventilatory status, urine output, pre-ICU in-hospital LOS and whether or not the patient underwent elective surgery. Studies have shown OASIS is comparable to other illness severity ratings in predicting outcomes such as mortality and length of stay.11

Patient and public involvement

This research was done without patient or public involvement. They were not invited to contribute to the development of our methodology, our outcomes, nor the writing of our paper.

Statistical analysis

Analysis was carried out using R V.3.4.0 and SAS V.9.4. Binomial logistic regression models were fitted using maximum likelihood estimation to compare the pain measures with 30 day and 1 year mortality. Linear regression was used to model the relationship between mean pain scores and hospital LOS. Age, gender (male reference), Elixhauser index and OASIS score were included in the models to account for potential confounders. In a separate regression, mean pain levels were categorised into four ordinal groups of no pain (0/10), mild pain (1 to 3), moderate pain (3 to 6) and severe pain (7 to 10) in accordance with the National Institutes of Health (NIH) Pain Consortium.12

Analysis of variance (ANOVA) was used to determine if there was a significant variation in heart rate, respiratory rate and/or systolic blood pressure, when compared with the concurrent pain assessment.

IV analgesia medications were converted to their morphine equivalents based on the National Pharmaceutical Council guidelines.13 The IV analgesia was subdivided into total dose in the first 24 hours, mean dose per ICU course day and total dose during ICU course. ANOVA models were used to determine if there were any significant variation in administration of IV analgesics among the four categorised pain groups.

Two sensitivity analyses were performed to assess the robustness of the observed effects. The first included the same statistical tests in all postoperative CABG patients regardless of duration of intubation. The second sensitivity analysis excluded patients who died in the hospital.

To add validity to the potential observed associations, falsification hypothesis testing using Prasad and Jena’s methodology was employed. A distinct and highly unlikely hypothesis is tested against the exposure of interest, pain in this case.14 We used nausea, a symptom with no known correlation to pain, and tested it against the four different pain metrics.

Case study

We will first discuss our unexpected results and then discuss the general issue of counterintuitive data. Our results that increasing levels of patient-reported pain severity post-CABG surgery are associated with better clinical outcomes were not consistent with our initial hypothesis that better outcomes correlate with better pain control as per the reported literature. In fact, prior studies have found increased levels of pain in the hospital to be associated with increased mortality.15

The difference in the study cohort between our study and others may explain some of the discordance. Our initial analysis was limited to ‘fast-tracked’ patients who did not have intra-operative complications and were extubated early in their ICU course. These patients made up 44% of the database patients. Studies that have reported worse clinical outcomes associated with postoperative pain did not select for a relatively healthy sub-cohort of patients. Why would patients with higher levels of pain have better outcomes? It is well documented that an increased inflammatory reaction is associated with increased pain. Pro-inflammatory cytokines such as interleukin-1 beta, interleukin-6 and tumour necrosis factor-alpha have been directly implicated in the physiology of pain.16 17 These cytokines have also been found to be directly involved in wound healing through the stimulation of processes such as keratinocyte and fibroblast proliferation and synthesis and breakdown of extracellular matrix proteins.18 We speculate that those patients who demonstrated better outcomes mounted a more robust inflammatory response leading to more pain, but also to increased healing ability.

Another possibility is that higher perceived pain levels represent a proxy for a generally better state of health, including superior physiological function of the cardiovascular, respiratory, renal and hepatic systems. In tandem, these systems act to metabolise and eliminate anaesthetic and analgesic drugs so that the net pharmacokinetic result would likely be increased susceptibility to pain due to less administered agent remaining at active sites. Furthermore, patients with better cardiovascular function would likely have better cerebral perfusion with improved central neurological function, and thereby have a pharmacodynamic reason for perceiving more pain. Also patients who are generally in better overall condition would be expected to manifest better outcomes. These thoughts are admittedly speculative and additional research is needed to explore these possibilities.

It is important to point out that the goal of clinicians should not be in any way to maximise pain to optimise outcomes. Conventional approaches that aim to control pain adequately should be employed. Our observation is just that - an observation of an association and conjectures of possible linking mechanisms but is not intended in any way to drive pain management policy in the direction of tolerating undertreated pain.

We performed sensitivity analyses, one including all patients regardless of postoperative ventilation duration and another excluding patients who died during hospitalisation and reached similar conclusions. When excluding in-hospital deaths, we discovered the 30 day mortality rate had a similar OR, but was no longer statistically significant. This is most likely due to the low mortality rate after hospital discharge following CABG, making it difficult to detect a statistically significant effect.

We believe that researcher bias is a non-issue as these findings were not expected, but rather, the opposite. Sampling bias was also minimal. Our inclusion criteria were predefined prior to database sampling and only 28 patients needed to be excluded due to missing data. We performed multiple sensitivity analyses to determine if those that were excluded would have influenced our results. However, the study has several limitations inherent in any retrospective data analysis. We acknowledge that correlation does not equal causation and further research is needed to determine the underlying physiological mechanism for the results seen. Due to the self-reported nature of the pain scores, reporting bias is a concern. Some patients may have over-reported and others under-reported their pain. We also recognise that analgesic administration is a confounder and were unable to completely control for this due to lack of information regarding oral analgesics in the database. However, with respect to intravenous analgesics, we attempted to limit this potential confounder by excluding those with prolonged intubations who would inherently have received and required greater doses of sedatives and analgesics. We also compared the amount of narcotics that patients were receiving and did not observe any significant differences among the various pain groups. Despite measures taken to guarantee internal validity, we anticipate appropriate scepticism with regard to generalisability of the findings. This, of course, is of genuine concern given the current state-of-affairs where clinicians are already inundated with conflicting studies of questionable quality. We therefore invite other investigators to replicate (and expand) our analysis in other databases.

Counterintuitive results and examples

As noted, our findings were contrary to clinical expectations and to most published works which associate increased pain with worse outcomes.15 19 20 Encountering counterintuitive results is not unique to retrospective data analysis. Clinicians encounter unexpected, possibly aberrant, values in situations such as the evaluation of laboratory and monitor data. When a possibly spurious lab result is obtained, the usual response is to repeat the test. When the second test comes back with a more acceptable value, we generally then ignore the unexpected value. But what if the repeat value is also aberrant? Do we repeat it again or do we begin to believe that the value is ‘real’ and start to formulate a response to a clinical problem? In this case, it is the consistency and reproducibility of the counterintuitive value that drives its possible validity. The details of this process are determined by the overall clinical risks involved. The consistency we found in the pain score values drove us to consider the possibility that the values were ‘real’ even though they were counterintuitive in terms of our expectations.

Another issue in evaluating to counterintuitive values is whether they are possible. Impossible values would include a potassium of 64.5, one incompatible with life. But a potassium of 7.3 is a possible value. The pain values associated with better outcomes were unexpected, but not so high that they were impossible in the observed context.

One question that would arise with a potassium of 7.3 would be that of continuity - did the value occur suddenly or gradually in a stream of normal values? Were surrounding values similarly abnormal? In the context of persistently abnormal values, for example, untreated uraemia, a normal value would be counterintuitive. So that while most counterintuitive values will tend to be out of the ‘normal range’, they will not necessarily be so. In the context of increasing values, it might simply be the first one that was not only out of the normal range, but that crossed the line into a critical range.

The fundamental question is whether counterintuitive results are actually false results or does the problem lie in our perception of what should be. Table 3 displays a categorisation of error types that could result in faulty data. We are not able to attribute the counterintuitive data we observed to any of these factors, however.

How can counterintuitive results be approached in secondary data analyses? Table 4 displays characteristics that may distinguish reliable (but counterintuitive) from truly faulty data. With consideration of these factors, the first investigative step is to retrace the process and work flow involved in data entry so far as possible. Our data was obtained at the institution of several of the authors where nurses are trained to assess pain on a standard scale from 0 to 10. There are several potential faults to this method. The nursing staff could neglect to regularly assess pain or neglect to enter the information into the medical record generating the database. While this may alter a few data points, it is unlikely to systematically affect all data unless there was an obvious glaring institutional issue affecting every nurse and every data entry.

After determining that the data source is valid, additional statistical tests can be run on the patient cohort. Tests such as the falsification hypothesis testing we used, add validity to the results as they show that the cohort follows other generally known principles. In our study, falsification analysis provided support for our findings.

Concurrent contextual data can also help to confirm the veracity of data - for example, one could examine ECGs if hyperkalaemia was being analysed. We examined concomitant vital signs during the time of pain measurements. We expected to observe significant increases with higher pain levels, but did not: With the combination of analgesics, residual anaesthetics and the concurrent use of drugs that directly affect vital signs such as beta-blockers, the lack of correlation is probably not surprising. In fact, we learnt that in this setting, it appears to be inadvisable to use vital sign changes as a proxy for the presence of unvoiced pain. Finally, one can attempt to physiologically explain the disparity between the observed and expected results as we did above for the case of post - CABG pain.

The use of lower thresholds for blood transfusions in the ICU is an example of a counterintuitive finding. ICU target haemoglobin levels were historically set at greater than 10 g/dL, theoretically to ensure adequate oxygen delivery.21 This led to increased transmission of blood borne diseases, unnecessary healthcare expenditures and actually worse outcomes.22 Later research showed that this rule was not necessary for most patients, but only for selected patients such as those with acute coronary syndrome actively experiencing chest pain. The initially counterintuitive findings that lower haemoglobin levels were not only acceptable but preferable in most cases, served as research triggers to more fully elucidate optimal clinical practice. Our case may serve as an analogous research trigger in terms of optimally managing postoperative pain. Outcomes such as mortality and LOS are complex phenomena driven by many factors - to observe a clear and robust statistical effect such as we did is strongly suggestive that something ‘real’ is occurring even if the data were counterintuitive.

The final step when dealing with counterintuitive data is to look for additional evidence that confirms the reliability of the results (perhaps this could be termed ‘confirmatory metadata’). With respect to our CABG case, the analysis should be rerun on additional databases and in different settings. Just as clinicians continued to manage intensive care unit anaemia as they always had until more definitive results were reported, our results should not impact the analgesic care of patients at this point. However, we hope that we have raised the issue in the appropriate minds that outcomes may benefit from approaches slightly different from usual. After all, one can easily eliminate all pain from postoperative patients, but they would have to remain sedated and ventilated for an indefinite period of time to do so and after they are extubated, pain management should not be so aggressive that it leads to apnoea and respiratory arrest. In other words, there may be a detectable level of tolerable pain that leads patients to their best outcomes and no honest clinician will guarantee a patient that they will have no pain at all after a procedure like a sternal-disrupting CABG.