Article Text
PDF
Abstract
Objective To assess the value added to the National Early Warning Score (NEWS) by mid-regional pro-adrenomedullin (MR-proADM) blood level in predicting deterioration in mild to moderately ill people.
Design Prospective observational study.
Setting The Medical Admissions Suite of the Royal Victoria Infirmary, Newcastle.
Participants 300 adults with NEWS between 2 and 5 on admission. Exclusion criteria included receiving palliative care, or admitted for social reasons or self-harming. Patients were enrolled between September and December 2015, and followed up for 30 days after discharge.
Outcome measure The primary outcome measure was the proportion of patients who, within 72 hours, had an acuity increase, defined as any combination of an increase of at least 2 in the NEWS; transfer to a higher-dependency bed or monitored area; death; or for those discharged from hospital, readmission for medical reasons.
Results NEWS and MR-proADM together predicted acuity increase more accurately than NEWS alone, increasing the area under the curve (AUC) to 0.61 (95% CI 0.54 to 0.69) from 0.55 (95% CI 0.48 to 0.62). When the confounding effects of presence of chronic obstructive pulmonary disease or heart failure and interaction with MR-proADM were included, the prognostic accuracy further increased the AUC to 0.69 (95% CI 0.63 to 0.76).
Conclusions MR-proADM is potentially a clinically useful biomarker for deterioration in patients admitted to hospital with a mild to moderately severe acute illness, that is, with NEWS between 2 and 5. As a growing number of National Health Service hospitals are routinely recording the NEWS on their clinical information systems, further research should assess the practicality and use of developing a decision aid based on admission NEWS, MR-proADM level, and possibly other clinical data and other biomarkers that could further improve prognostic accuracy.
- internal medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study to use rigorous statistical methods to assess the value added by mid-regional pro-adrenomedullin (MR-proADM) to the admission National Early Warning Score for predicting clinically important deterioration in mild to moderately ill patients.
Overall prognostic accuracy might have been greater had more severely ill patients been included, but the aim of this study was to predict deterioration in less severely ill patients who could benefit from closer observation.
This was an observational study and thus could not directly assess the use of more accurate prediction of deterioration.
Initial evidence for MR-proADM as a biomarker for deterioration appears promising, but requires further validation for clinical use.
Introduction
The National Early Warning Score (NEWS) is recommended for assessing severity of illness in patients presenting in primary or secondary National Health Service (NHS) care and for surveillance of patients in hospital.1 2 Six physiological parameters (which can be measured at the bedside) are scored: respiratory rate, oxygen saturation, temperature, systolic blood pressure, pulse rate and level of consciousness. The scores are aggregated, and, if the patient requires oxygen, the total is increased. NEWS predicts death, cardiac arrest and unplanned intensive care unit (ICU) admission within 24 hours.3–5 However, NEWS does not identify all patients who turn out to be seriously ill,6–8 and there are also patients whose NEWS is usually elevated and who do not require the level of observation that the NEWS tool would suggest. For example, people with chronic obstructive pulmonary disease (COPD) or chronic heart failure (HF) have higher baseline NEWSs than those without these comorbidities. The prognostic accuracy of NEWS for patients presenting to the emergency department (ED) has been confirmed in a wide range of severity of illness,9 10 as has its reduced accuracy in people with COPD.11 However, no previous studies of the prognostic accuracy of NEWS in the ED/medical admissions unit (MAU, also called Assessment Suite (AS)) have focused on patients admitted with mild to moderately severe illness. Since a clinically important proportion of these patients do deteriorate unexpectedly, improved risk stratification would be useful.
Mid-regional pro-adrenomedullin (MR-proADM) is one of several promising biomarkers for severe illness and deterioration.12–16
MR-proADM is a precursor of adrenomedullin (ADM), a member of the calcitonin peptide family. ADM is widely expressed and has roles in vasodilation, immune modulation and metabolic regulation. It is upregulated in severe infections, inflammation, vasodilation, stimulation of diuresis, increased cardiac output and stroke.17–19 ADM has a short half-life, but MR-proADM is more stable and directly reflects ADM concentrations in blood. Both ADM and MR-proADM levels are strongly associated with risk of mortality, regardless of aetiology.20–26 In people presenting with acute chest pain, MR-proADM has been reported to improve the Global Registry of Acute Coronary Events risk classification by 41%.27 As with the NEWS, people with COPD or chronic HF have higher baseline levels of MR-proADM.
The aim of this study was to assess whether the MR-proADM level used alongside the NEWS would improve prediction of deterioration over NEWS alone in patients admitted to the MAU with mild to moderately severe illness.
Methods
Patient and public involvement
Patients and the public were not specifically involved in the planning and execution of this study. However, the National Institute for Health Research now requires that the research it supports includes active involvement and engagement with patients and the public.
Study participants and study design
This was a prospective observational cohort study. Patients were enrolled between September and December 2015 at the Royal Victoria Infirmary, Newcastle, and followed up for 30 days after discharge. If the patient died within 30 days of follow-up, this and the cause of death were recorded. Adults admitted to the MAU were recruited for the study between 9:00 and 16:00 on weekdays.
Sample size was based on a pragmatic recruitment target for a 3-month observational study. A recent unpublished audit conducted in the MAU at the Royal Victoria Infirmary found a deterioration rate of 20%. With 300 patients and complete data collection, 60 events would be anticipated. With this number of events, a multivariable prediction model could include up to six independent predictors. This is based on a widely accepted rule of thumb that models with fewer than 10 events per predictor tend to be overfitted.28 However, recent research suggests that the ‘ten events per variable’ rule of thumb may be optimistic.29 Because the aim of this study was to assess if further research would be indicated, 60 is considered an acceptable number of events, even if the rule of thumb is optimistic.
Patients were considered eligible for inclusion in the study if their NEWS on admission was at least 2 and not greater than 5, and all NEWS parameters were recorded. Patients were excluded from the study if they were receiving palliative care, were admitted for social reasons only, or were self-harming, or overdosing with drugs or other substances.
Recorded data
Demographic and admission data included gender, year of birth, reason for admissions, diagnosis on discharge and the presence of comorbidities in which baseline MR-proADM levels are chronically raised: COPD with hypoxia (PaO2 <10 kPa)7; HF30; acute brain injury6; acute coronary syndrome27; acute venous thromboembolism21; high international normalised ratio (>2); acute kidney injury; electrolyte disturbances (Na+ <130 or >150 mmol/L; K+ <3.0 or >5.5 mmol/L); hyperglycaemia in type 1 diabetes (random glucose >10 mmol/L).
The NEWS was assessed at admission and over the next 72 hours, and the scores and assessment times were recorded. The seven clinical parameters used to determine the NEWS were recorded for the baseline (admission) assessment only. Baseline NEWSs were used to determine eligibility for this study. Subsequent NEWSs were used in the analyses to identify deterioration.
Blood samples were taken at hospital admission for assessment of MR-proADM, C reactive protein (CRP) and white blood cell (WBC) count.
Laboratory tests
Plasma was obtained from blood samples (collected in EDTA) that were no longer clinically required. Plasma was stored in aliquots at –80°C.
MR-proADM was assayed in the on-site Blood Sciences Laboratory using the B·R·A·H·M·S Kryptor system according to the manufacturer’s instructions.
Blood samples were analysed in batches by personnel blinded with regard to the condition and NEWS of the patient. Nurses who assessed the NEWS and healthcare professionals managing patients in the MAU were blinded to MR-proADM results.
Outcome measures
Outcome measure 1: acuity increase
A patient was classified as having an acuity increase if one or more of the following occurred within 72 hours from admission:
Transfer to a higher level of care (ICU or high-dependency unit).
Readmission to hospital for reasons related to the initial admission.
Death for reasons related to the initial admission.
NEWS increased by at least 2 compared with the admission score.
Outcome measure 2: deterioration event
For most of the observed acuity increase cases, the reason for classification was an increase in the NEWS (table 1). An increase in NEWS reflects both measurement variation and physiological variation, so additional exploratory analyses were carried out to assess the performance of MR-proADM in predicting deterioration. Deterioration events were classified as the occurrence of one or more of the following:
Characteristics of the study population, classified by outcome 1 (acuity increase), outcome 2 (deterioration event) and all patients
Transfer to higher level of care within 72 hours from admission;
Death (for reasons related to the admission) within 30 days;
Readmission to hospital (for the same reason as the previous admission) within 30 days from first admission.
Classification based on this definition is unlikely to be subject to clinically important measurement variation. This analysis, therefore, should optimise the prognostic accuracy for events which are both clinically and economically important.
Outcome measure 3: length of stay
Length of stay was defined as the duration (in days) from admission to discharge or death.
Statistical analysis
All data analyses were performed using the R language, V.3.2.0,31 with the support of RStudio, V.0.99.896 (RStudio). The following R packages were used: ggplot2, pROC, psych, PredictABEL, Hmisc and rms.
Logistic regression models were compared for their accuracy in predicting deterioration outcome measures as prespecified in an analysis plan. Analyses are presented as unadjusted parameter estimates of risk (OR, with CIs) and estimates adjusted for identified clinical confounding factors. The aims of the multivariable analyses were twofold: first, to estimate the effect size and significance adjusted for other identified influential predictors and interactions; and second, to investigate whether the addition of other predictors improved the goodness of fit and accuracy of prediction.
Only complete cases were analysed since missingness was minimal: 10 records without data on comorbidities (details in footnote in table 1).
For each measure of deterioration (acuity increase, deterioration event and length of stay), logistic regression models were compared for the following sets of predictor variables:
Predictor set a. Comparator (base case): NEWS on admission.
Predictor set b. Primary analysis: NEWS, MR-proADM.
Predictor set c. Secondary analyses: NEWS and MR-proADM always included. Age, gender, CRP, WBC, presence of COPD or HF, presence of other comorbidities and interactions between predictors when appropriate.
Predictors (and the underlying assumption of linearity of their relationship with the outcome of interest) were initially investigated through univariate analyses based on simple log and quadratic functions. Transformations were applied if they improved the goodness of fit as assessed by the Akaike information criterion (AIC) and were retained in the multivariable setting. NEWS was treated as an ordinal variable. We assessed interactions through visual data exploration without significance testing as the study was not powered for this. For the multivariable regression models, the set of predictors was assessed for independence through backward elimination, based on changes in AIC.
The analysis plan for the secondary outcome of length of stay was similar: using multiple linear regressions based on transformed outcomes to address non-normality. Dependent and exploratory variables were log-transformed if not normally distributed. Normality was assessed by visualising the data. More details on the methods used are reported in online supplementary material 1.
Supplementary file 1
Goodness of fit of logistic regression models was assessed with the C-statistic (which is the area under the receiver operating characteristic curve and is used as a measure of discrimination) presented with 95% CIs. To assess the value added by including the MR-proADM level with the NEWS in predicting deterioration, continuous net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated.32 33
For internal validation of the statistical models, the C-statistic was evaluated after correcting for optimistic predictions through bootstrapping with 10 000 resamples.
Results
Study enrolment
The process of recruitment and enrolment of patients for the study is shown in figure 1. The study recruited 300 patients, and 292 were included in the analysis. Five patients were excluded because the blood samples for MR-proADM were taken more than 12 hours from baseline NEWS assessment; three patients were excluded from the primary outcome due to missing follow-up NEWSs.
Patient recruitment process. AS, Assessment Suite, also called as Medical Admissions Unit (MAU); GP, general practitioner; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score.
Patient characteristics
Patient demographics and mean biomarker levels for each covariate are reported in table 1. The cohort was evenly divided in gender and had a mean age of 63 years and mean NEWS on admission of 3, with the majority of patients having NEWS of 2. COPD or HF was present in 28%, and 25% had other comorbidities.
The study population was homogeneous across acuity increase and no acuity increase outcomes in terms of gender, age and NEWS on admission. Table 2 shows the frequencies of criteria determining acuity increase and deterioration event. Notably, around 95% of acuity increases were the result of an increase in NEWS, while readmission was the reason for around 70% of deterioration events.
Criteria met by patients classified with an acuity increase or deterioration event
Patients who experienced acuity increase had higher MR-proADM and CRP levels at admission, and longer length of stay in the hospital and in the MAU.
The prevalence of acuity increase was 29% (somewhat higher than the anticipated 20%). The prevalence of deterioration events was 11%. The numbers of events provided sufficient statistical power to assess statistical significance for the primary outcome, acuity increase, but not for the secondary outcome, deterioration event.
Accuracy of MR-proADM for predicting acuity increase
Potentially useful predictors with univariate analysis of acuity increase were MR-proADM (OR 1.27, 95% CI 1.02 to 1.62; p=0.037), age2 (OR 1.00, 95% CI 0.99 to 1.00; p=0.023) and the presence of COPD or HF (OR 2.25, 95% CI 1.30 to 3.91; p=0.004; online supplementary figure S1). The prognostic accuracy of CRP, WBC and NEWS did not reach the threshold of significance (p=0.88, p=0.090, table 3, and p=0.416, table 4, respectively).
Univariate regression analyses for predicting the three outcomes of interest: acuity increase, deterioration event and length of stay
Multivariable regression analyses for the outcomes of interest: acuity increase, deterioration event and length of stay (outcomes 1, 2 and 3, respectively) with NEWS comparator group
The prognostic accuracy for acuity increase of NEWS on its own was limited and not significant (area under the curve (AUC) 0.55, 95% CI 0.48 to 0.62), but when MR-proADM was included as an additional predictor, the accuracy of the model increased (AUC 0.61, 95% CI 0.54 to 0.69; OR 1.28, 95% CI 1.02 to 1.63; p=0.039) (tables 4 and 5, figure 2A). When including MR-proADM with NEWS, the reclassification of patients was also significant, especially for the NRI (NRI 0.3, SE 0.1, p=0.007, IDI 0.017; table 4).
Model comparisons
(A) Prognostic accuracy for acuity increase. Predictor set a: NEWS; predictor set b: NEWS, MR-proADM; predictor set c: NEWS, MR-proADM, COPD/HF, interaction between MR-proADM and COPD/HF. (B) Comparisons as for panel (A) but for predicting a deterioration event. Predictor set c: NEWS, MR-proADM level, age2, other comorbidities. (C) Length of stay predicted by MR-proADM level. COPD, chronic obstructive pulmonary disease; HF, heart failure; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score.
The prognostic accuracy of MR-proADM and the additional value it provides to the NEWS was confirmed for deterioration events and length of stay (tables 4 and 5, and figure 2B,C).
For MR-proADM alone, the AUCs were for acuity increase 0.58 (0.51 to 0.66) and deterioration event 0.64 (0.54 to 0.74). For length of stay, the R2 was 0.12.
Effect on prognostic accuracy when clinical information is added to the set of predictors
Secondary multivariable modelling evaluated the prognostic accuracy of MR-proADM when adjusted for the clinical factors in predictive set c: age, gender, CRP, WBC, presence of COPD or HF, and presence of other comorbidities,
For acuity increase, COPD or HF comorbidity status and its interaction with MR-proADM level improved the prognostic accuracy of the model: AUC increased from 0.61 (95% CI 0.54 to 0.69) to 0.69 (95% CI 0.63 to 0.76) and net reclassification index from 0.3 to 0.4 (table 5).
For deterioration events, the presence of other comorbidities (excluding COPD and HF) and age2 increased the prognostic accuracy of MR-proADM (tables 4 and 5). The prognostic accuracy of length of stay (outcome 3) of MR-proADM is also increased by including age in the model (tables 4 and 5, online supplementary figure S2).
Potential confounding effects
Shorter-term outcomes
NEWS and MR-proADM were less accurate in predicting acuity increase within 24 and 12 hours from admission than in predicting acuity increase within 72 hours (online supplementary table S1 and S2).
Interval between admission NEWS scoring and blood collection
Because ward processes did not allow the times of scoring NEWS and collecting blood to be specified for research, we assessed for a confounding effect from variation in the timings, but found no evidence for it (online supplementary table S3).
Correlations among biomarkers
Diagnostic plots, shown in online supplementary figure S1 and S3, show no multicollinearity in the data, no autocorrelation, no heteroscedasticity and no data points that stood out in terms of their influence on results.
Sensitivity and specificity
As overall measures of accuracy, sensitivity and specificity were calculated (where appropriate) for each model using Youden’s index. The results are shown in online supplementary table S4. In practice, the trade-off between sensitivity and specificity would depend on the type of clinical decision to be made on the result (ie, ‘rule-in’ or ‘rule out’) and this would differ from the approach in Youden’s Index, which gives equal weight to false-positive and false-negative results.
Internal validation
C-statistic values after correcting for optimistic predictions (ie, bootstrapped average of the AUC for each model) were as follows: for acuity increase—predictor set a, C-stat 0.53; predictor set b, C-stat 0.59; predictor set c, C-stat 0.66; for deterioration events—predictor set a, C-stat 0.52; predictor set b, C-stat 0.61; predictor set c, C-stat 0.68; for length of stay— predictor set a, R2=0.003; predictor set b, R2=0.12; predictor set c, R2=0.13. AUCs decreased slightly with the bootstrapped averages, but the differences between the AUCs for predictor sets a, b and c were constant. These results are an internal validation, and further validation on an external dataset is required.
Discussion
Accuracy of prediction of deterioration by MR-proADM
This study shows that MR-proADM may be a clinically useful biomarker for predicting deterioration (ie, acuity increase) within 72 hours from admission to hospital in mild to moderately ill patients with admission NEWS between 2 and 5. By design, NEWSs in this range imply a low risk of deterioration, and our data are consistent with this. Previous evaluations of the NEWS assessed on admission have found that it predicts deterioration,3–5 34 which may seem inconsistent. But these studies included all patients admitted to ED, whatever their NEWS.
For most of the observed acuity increase events, the reason for classification was an increase in the NEWS. Because an increase in NEWS reflects both measurement variation and physiological variation, additional exploratory analyses were carried out to assess the performance of MR-proADM, using an operational definition of deterioration, deterioration event, designed to minimise measurement variation. NEWS on its own had low prognostic accuracy for deterioration events. However, MR-proADM level and NEWS together predicted deterioration events with an AUC of 0.65. Considering baseline patient characteristics further increased the accuracy of the model (AUC=0.73).
Comorbidities and interactions with MR-proADM levels
MR-proADM levels in people with COPD and/or HF are chronically raised and are not predictive of deterioration. However, in other people whose MR-proADM levels are not chronically raised, high levels are predictive of acuity increase (online supplementary figure S1). Including these comorbidities and their interaction with MR-proADM level increased the prognostic accuracy of the logistic regression model.
Limitations
This study included only patients who were admitted with a NEWS between 2 and 5. The prognostic accuracy of the MR-proADM would perhaps have been greater if more extreme cases had been included. However, patients with NEWSs more than 5 are already known to be severely ill and to require close monitoring and/or management at higher levels of care.
Internal validation found that the uncorrected C-statistics are optimistic, which implies that external validation in an independent study would be useful. However, after correction for optimistic predictions, the study’s conclusions remain unchanged.
Interpretations and implications
The contributions of MR-proADM to the accuracy of the prognostic models suggests that it could provide additional prognostic information over and above NEWS.
Secondary analyses suggest that a potentially useful clinical decision aid could be based on the NEWS, MR-proADM level and clinical features.
Future research and development
As a growing number of NHS hospitals are implementing the NEWS on their clinical information systems, it should be practical to develop a decision aid based on admission NEWS, MR-proADM level and clinical features. Other biomarkers may further improve prognostic accuracy for deterioration, for example, lactate,3 peroxiredoxin-4 and copeptin,22 35 36 and soluble urokinase plasminogen activator receptor.37 The feasibility, cost-effectiveness and acceptability of such decision aids needs to be evaluated in further research.
A rapid point-of-care test for MR-proADM could facilitate the assessment process and reduce delays in arranging optimal levels of care and intensity of monitoring. Future research could identify the threshold MR-proADM level corresponding to the optimal combination of sensitivity and specificity for a binary test (eg, ‘present’ or ‘absent’) for deterioration.
Acknowledgments
The authors would like to thank the patients who accepted to be part of the study and the team of research nurses for the data collection: Laura Shewan, Louise Taylor, Janine Gradwell, Karen Martin, Katherine Cullen, Gerry Jones, Graham Soulsby, Carmen Scott and Helen Reed. We also thank the reviewers for their helpful suggestions and comments.
References
Footnotes
Contributors AJS, DAP, MP and DDS devised the study. RAO, SG and AJA managed the project. SG performed the statistical analyses with advice from DS. All authors contributed to the final manuscript.
Funding The study was funded by B·R·A·H·M·S GmbH. The NIHR funds the NIHR Newcastle In Vitro Diagnostics Co-operative.
Disclaimer The views expressed are those of the authors and not necessarily those of B·R·A·H·M·S GmbH, the NHS, the NIHR or the Department of Health.
Competing interests None declared.
Patient consent Obtained.
Ethics approval The study was approved by the Newcastle and North Tyneside Research Ethics Committee (15/NE/0120), and by the R&D Committee of the Newcastle upon Tyne Hospitals NHS Foundation Trust (reference no. 7495).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.