Article Text


Clinical implications and validity of nursing assessments: a longitudinal measure of patient condition from analysis of the Electronic Medical Record
  1. Michael J Rothman1,
  2. Alan B Solinger2,
  3. Steven I Rothman1,
  4. G Duncan Finlay1
  1. 1Rothman Healthcare Corp., Sarasota, Florida, USA
  2. 2ABS Professionals, Sarasota, Florida, USA
  1. Correspondence to Dr Michael J Rothman; michael{at}


Objectives This study investigates risk of mortality associated with nurses’ assessments of patients by physiological system. We hypothesise that nursing assessments of in-patients performed at entry correlate with in-hospital mortality, and those performed just before discharge correlate with postdischarge mortality.

Design Cohort study of in-hospital and postdischarge mortality of patients over two 1-year periods.

Setting An 805-bed community hospital in Sarasota, Florida, USA.

Subjects 42 302 inpatients admitted for any reason, excluding obstetrics, paediatric and psychiatric patients.

Outcome measures All-cause mortalities and mortality OR.

Results Patients whose entry nursing assessments, other than pain, did not meet minimum standards had significantly higher in-hospital mortality than patients meeting minimums; and final nursing assessments before discharge had large OR for postdischarge mortality. In-hospital mortality OR were found to be: food, 7.0; neurological, 9.4; musculoskeletal, 6.9; safety, 5.6; psychosocial, 6.7; respiratory, 8.1; skin, 5.2; genitourinary, 3.0; gastrointestinal, 2.3; peripheral-vascular, 3.9; cardiac, 2.8; and pain, 1.1. CI at 95% are within ±20% of these values, with p<0.001 (except for pain). Similar results applied to postdischarge mortality. All results were comparable across the two 1-year periods, with 0.85 intraclass correlation coefficient.

Conclusions Nursing assessments are strongly correlated with in-hospital and postdischarge mortality. No multivariate analysis has yet been performed, and will be the subject of a future study, thus there may be confounding factors. Nonetheless, we conclude that these assessments are clinically meaningful and valid. Nursing assessment data, which are currently unused, may allow physicians to improve patient care. The mortality OR and the dynamic nature of nursing assessments suggest that nursing assessments are sensitive indicators of a patient's condition. While these conclusions must remain qualified, pending future multivariate analyses, nursing assessment data ought to be incorporated in risk-related health research, and changes in record-keeping software are needed to make this information more accessible.

Statistics from

Article summary

Article focus

  • This study investigates risks of mortality associated with entry and last predischarge nurses’ assessments of patients’ conditions by physiological system.

  • It is the first quantitative study of the validity and clinical implications of nurses’ head-to-toe clinical assessments.

Key messages

  • Entry nursing assessments (other than pain) are strongly correlated with in-hospital mortality, and final predischarge nursing assessments (other than pain) are strongly correlated with postdischarge mortality, independent of diagnosis and medical history. It is evident that most nursing assessments are clinically meaningful and valid.

  • The dynamic nature of in-hospital nursing assessments and the large mortality OR associated therewith suggest not only are nursing assessments sensitive indicators of a patient's condition, but they may also aid in detection of clinical problems as they develop during the course of a patient's stay.

  • Nursing assessment data, which are now essentially unused, provide additional information on patients’ conditions and should allow physicians to improve patient care and reduce in-hospital mortality.

Strengths and limitations of this study

  • This is the first quantitative study of the clinical validity of nursing assessments. More than 42 000 patient visits over two 1 year periods give the study a strong statistical base.

  • All the OR reported are both statistically and clinically significant, with none of the 95% CI's overlapping one (except for pain).

  • It is internally consistent, with in-hospital and postdischarge OR's for all time periods yielding similar results, providing evidence of reliability of nursing assessments.

  • There are some limitations to our study. No multivariate analysis was performed, making the associations found subject to possible unknown confounders. The work has been done at a single site and that site has a population skewed older, which raises questions about generalisability.


Nursing budgets constitute a major part of a hospital's operating costs, accounting for some 25% of the total operating budget and 44% of direct care costs.1 ,2 Recent studies have demonstrated that higher staffing ratios of registered nurses are associated with fewer hospital-related deaths, failures to rescue, cardiac arrests, hospital-acquired pneumonia and other adverse events, as well as having positive effects on patients’ safety in intensive care units and in surgical patients.3–6 Increased registered nurse hours spent on direct patient care were associated with decreased risk of hospital-related death and shorter lengths of stay.3–6 In the course of providing direct patient care, nurses assess each patient by physiological system and record those assessments in the electronic medical record (EMR). Once recorded however, nursing notes are not often read by attendants or residents.7

There have been several studies of the relationship between registered nurse staffing and in-hospital mortality.8 ,9 However, no previous studies have demonstrated a direct connection between nursing assessments and patient risk of mortality. Furthermore, while the nursing literature is replete with studies of nursing diagnostic terminology and its standardisation,10–19 there is a paucity of quantitative studies of the validity and clinical implications of nurses’ head-to-toe clinical assessments. In this article we investigate clinical associations of 12 simplified nursing assessments, one for each physiological systems, with both in-hospital and postdischarge mortality data. Our hypotheses are that nursing assessments performed at entry of in-patients are predictors of in-hospital mortality, and nursing assessments performed just before in-patient discharges are predictors of postdischarge mortality.


This research was initiated in an attempt to understand issues of continuity of care in hospitals. What piqued our interest in nursing assessments is the fact that other than laboratory tests and vital signs, they are the only clinical variables in the EMR that are reflective of patient condition that are not static, and their values are updated regularly. The results of this research are incorporated in the ‘Rothman Index’, which is a new measure of patient condition. This is the first foundational paper in a series of studies related to the scientific basis of the Rothman Index. Approval for the work was granted by the Sarasota Memorial Hospital Institutional Review Board.

Nursing assessment data for the periods 1/2004–12/2004 and 7/2005–6/2006 were extracted from the Electronic Hospital Record at Sarasota Memorial Hospital, an 805-bed community hospital. Our cohort for this study were all patients admitted for any reason during these periods, excluding obstetrics, paediatric and psychiatric patients, which determined the study size of 42 302 inpatient visits. For the discharge study, we had complete data for 39 964 inpatient visits in which the patient was discharged alive. Demographic data and diagnostic data have not been collected for this population; however, our subject community hospital serves a population skewed older than the US average.

In general, nurses’ assessments are entered into the Electronic Medical Record in one of two ways: either the nurse answers a series of detailed questions to document each assessment, such as for respiratory, ‘What are the breath sounds?’, ‘What colour are the nail beds’, etc; or, the ‘charting by exception’ method is used, where the nurse records a simple overall answer of ‘met’ or ‘not met’ for each physiological system, such as ‘Does the patient's respiratory function meet a minimum standard?’ (if the answer is ‘not met’, only then are detailed follow-on questions considered). In this study the binary charting by exception method was used, and the assessments were characterised as either having ‘met’ the standard or having ‘not met’ the standard in each of the following 12 areas: food, neurological, safety, skin, genitourinary, musculoskeletal, respiratory, cardiac, peripheral vascular, gastrointestinal, psychosocial and pain. Definitions of the relevant standards are shown in the appendix. Nursing assessments were generally performed at least once per shift. For each area of nursing assessment, the all-cause in-hospital death rates and mortality OR's, associated with patients’ entry nursing assessments, were computed and the all-cause death rates and mortality OR's, associated with patients’ last assessments prior to discharge, were computed for patients living at the time of discharge with deaths within the time periods 2 days, 30 days and 1 year from discharge. Mortality was established by comparison with the Social Security Administration Death File. The reproducibility of outcome measures between the 2004 data and the 2005–2006 data was assessed using an intraclass correlation coefficient. The data analysis was carried out utilising SAS V.9.2 (SAS Institute, Cary, North Carolina, USA) and Systat V.13 (Systat Corp., Chicago, Illinois, USA).


The population studied had a mean of 4.7 days and median 3.1 days length of stay, with an SD of 5.1 days. An example of the data and associated results for 30-days postdischarge mortality is given in table 1, illustrating our calculations. Results for in-hospital mortality OR's associated with failing an entry nursing assessment are given in table 2 for 42 302 patients of whom 1086 died in the hospital. OR's for deaths within postdischarge periods of 2 days, 30 days and 1 year for 39 964 patients for whom we had data and who were living at the time of discharge are given in table 3. There were less than 0.3% missing data for any result. Although generally, about 90% of patients passed the assessments, in all categories and for both in-hospital and postdischarge deaths, not meeting standards for an assessment resulted in significantly higher death rates than meeting standards and very large mortality OR's, with the single exception of the pain assessment. Except for the pain assessments, all results are statistically significant (p<0.001) and none of the 95% CI's overlap one. The implications of pain assessments are examined in the discussion section below.

Table 1

Nursing assessment data with resulting 30-day mortality OR

Table 2

In-hospital death OR associated with entry nursing assessments

Table 3

Postdischarge mortality OR for final nursing assessments

To evaluate the agreement between 2004 and 2005–2006 values, first we calculated all the OR's for the 2004 and 2005–2006 subsets of data, and then an intraclass correlation coefficient was calculated comparing all OR's for 2004 to their counterparts from 2005 to 2006 across all categories and all time points (ie, in-hospital and 2, 30, and 365 days postdischarge) and found to be 0.85; values greater than 0.75 indicate excellent reproducibility.20 Thus, there is excellent reproducibility between the results for 2004 and those for 2005–2006 across all our measures of interest.


There is a growing body of evidence that suggests that nurses have a separate and identifiable effect on patient hospital outcomes, irrespective of other medical care (cf Kane et al21 ,22 and references cited therein). An important subject area in the literature is whether nursing data collected during the hospital stay can be used to explain commonly studied variables such as costs/charges, lengths of stay and mortality. However, until now, nursing assessments have not been shown to have clinical predictive validity. Our study combines over 40 000 cases, encompassing all admissions for any reason except for paediatrics, psychiatric and rehab, and we have shown that patients failing to meet minimum standards for any nursing assessment had significantly higher in-hospital and postdischarge death rates than patients who did meet these standards, regardless of medical history and diagnosis. This is evidence of the clinical validity of nursing assessments. In fact, the medical literature is replete with studies of critically ill patients for whom variables with mortality OR in the range 2–4 are considered sensitive and important predictors of mortality.23–26 The OR's we report demonstrate that these measures of a patient's functionality are significant and sensitive indicators of a patient's condition. Thus, nursing assessments may aid in physician care and possibly reduce hospital patient mortality, and future risk-related health research ought to consider incorporation of nursing assessments. Further study is clearly warranted.

We have studied deaths without regard to diagnosis, age, gender or severity of illness that occur in-hospital as well as within short (2 and 30 days) and long (1-year) periods of time for two reasons: (1) in order to learn whether there might be similar or different short and long-term relationships of assessments to mortality and (2) as a way to test validity and reliability. We have found that failing a nurse's assessment has both short-term and long-term correlations with mortality. The fact that our results are consistent over short and long periods, over all assessments, and from a medically diverse patient population, demonstrates that nursing assessments have valid and significant clinical implications irrespective of medical histories and diagnoses. While we have not compared survivor demographics with those who died, it is unlikely that such large OR's would be explained by underlying demographic factors like age or race or gender.

One might ask whether the high OR's associated with these assessments may simply represent high-risk diagnoses. We have not built a multiple linear regression model to determine the added predictive power of nursing assessments to other indicators of mortality. However, this is an area of current investigation. Preliminary results, which we shall publish in subsequent studies, show that among patients who were admitted with a diagnosis of congestive heart failure, those who failed any of the food, neurological, musculoskeletal or skin nursing assessments at admission had a mortality OR of 6–9, versus those who passed. Further, failing multiple nursing assessments had a greater implication in terms of in-hospital mortality than failing just one. Generally, patients who fail nursing assessments may well be those who also have serious or even terminal diagnoses. Nonetheless, here we have demonstrated that nursing assessments at both entry and discharge capture the seriousness of the patient's condition irrespective of diagnosis. Given the size of the study, it is clear that these variables are meaningful indicators of both in-hospital and postdischarge mortality.

The highest in-hospital mortality OR's, 7 or greater, are associated with those patients not passing the food, neurological, psychosocial or musculoskeletal assessments. It is important to comprehend the simple clarity of these findings. For example, if a newly admitted patient is not able to move independently in his/her bed, or behaves inappropriately, or cannot chew and swallow food—that patient is almost seven times more likely to die in the hospital than patients who have none of these problems. If a patient is incoherent or is not oriented, that the patient is almost 10 times more likely to die in the hospital than those who are coherent and alert, no matter what the diagnosis may be. In contradistinction, pain is not a significant indicator of mortality. This may be because pain, unlike the other problems indicated by nursing assessments, can often be controlled independent of the patient's general condition, so one does not expect it to have similar clinical implications. Further studies may clarify this issue.

Since we have shown that initial and final nursing assessments contain clinical information, it is reasonable to infer that all nursing assessments (other than pain) gathered throughout the patient's stay in the hospital contain significant clinical information. Nursing assessments are generally recorded every 12 h, and sometimes more frequently, throughout the patient's stay. Therefore, these data represent a changing indicator of patient condition. Thus, the importance of this work is in alerting physicians to a source of longitudinal clinical information about the patient's condition that no static measurement, such as demographics or principal diagnosis, can provide and which is not currently being routinely utilised.

Although our results show excellent reproducibility across two 1-year time periods, we are aware of questions about the reliability of nursing data, specifically intra-rater and inter-rater reliability. However, reproducible and consistent results such as shown here are only possible with reproducible and consistent nursing assessments. One has to conclude that by and large, the nurses get it right, and that they provide an important and valuable tool for assessing patient condition, irrespective of medical diagnosis and history. Thus, it is suggested that hospital physicians make special effort to ascertain whether patients have passed or failed their nursing assessments, a practice not widely followed currently by attendants or residents.7

We have established that simplified nursing assessments, gathered throughout a patient's stay and which are now essentially unused by physicians, have clinical validity. If physicians were to utilise these data, they then would be adding an important and largely ‘new’ source of clinical information to their evaluations. For example, when a physician sees a patient in the hospital, he or she often consults with the bedside nurse. However, this consultation is not always possible or practical. In many cases, consulting with the nurse from the previous shift, or the previous day, or even the day before that, to understand changes in the patient's condition, is not feasible. And even if it were, getting a verbal report on a patient's condition about the previous several days would likely be incomplete. These circumstances make use of nursing assessments, recorded in the EMR, the only effective way to gain access to this clinically relevant information. And though immediate observations are the most important in determining care, the prior observations provide an important and meaningful context, allowing the physician to assess the changes in patient condition. It must be noted however, that current EMR technology does not facilitate quick and easy physician access to nurses’ observations and assessments. Either changes in current record-keeping software or adjuncts to it would make access to this information more accessible.


Entry nursing assessments of in-hospital patients are strongly associated with in-hospital mortality, and predischarge nursing assessments of in-hospital patients are associated with postdischarge mortality. No multivariate analysis has yet been performed, and will be the subject of a future study, and thus there may be confounding factors. However, it is difficult to hypothesise any alternative factors that might confound our results, given the high OR's and the variety, multiple time intervals, and extent of data considered. We infer that these nursing assessments have valid and significant clinical meaning irrespective of medical histories and diagnoses. It is reasonable to infer further that nursing assessments taken throughout a patient's stay are also clinically meaningful. These assessments, which are part of what is termed as the ‘head-to-toe’ patient assessment, and which are a standard part of nursing school curricula, are collected and recorded at all hospitals, and simplified summaries of assessments, as we have analysed, can be constructed. Since nursing assessments are recorded at least every 12 h throughout the patient's stay, they represent a changing indicator of patient condition. Thus, they make available real-time longitudinal sensitivity that no static measurement, such as demographics or principal diagnosis, can provide.

The large OR's suggest that nursing assessments are sensitive indicators of clinical problems during the course of a patient's hospital stay. This compact clinical data source in the EMR is a natural longitudinal source of information, providing physicians access to the insights of nurses as recorded throughout the patient's entire stay. Such dynamic information should allow physicians to improve patient care. While these conclusions must remain tentative, pending detailed multivariate analyses, we believe nursing assessment data ought to be incorporated along with standard diagnoses in future risk-related health research. Current EMR technology does not allow quick and easy access to nurses’ observations and assessments, so changes in current record-keeping software or adjuncts to it will be necessary to make this information more accessible.


The authors thank Sarasota Memorial Hospital for making available the data upon which this investigation is based, and for the cooperation of the staff, especially Steven Goodman, Donna Jackson, Patti Reynolds and Denis Baker. Alex Kiss is thanked for checking calculations, a critical reading of the manuscript, and general statistical advice. Peter Herbert, Chief Medical Officer, Yale New Haven Hospital, is thanked for discussions regarding doctors’ current utilisation of nursing assessments. We also thank Susan Horn and Professor Robert Kane for their insights in reviewing the study, which resulted in significant improvements.


Nursing assessments

The nursing assessments are generally performed at least once per shift. When documented as described above, they consist of binary data, characterised either as having ‘met’ the standard or having ‘not met’ the standard. For each physiological system, definitions for the relevant standards at Sarasota Memorial Hospital are shown below. Although standards vary across hospitals, basically the same data are being collected at every hospital. It is generally possible to construct similar binary variables from any hospital's nursing data.

Cardiac standard: Pulse regular, rate 60–100 bpm, skin warm and dry. Blood Pressure less than 140/90 and no symptoms of hypotension.

Food/nutrition standard: No difficulty with chewing, swallowing or manual dexterity. Patient consuming >50% of daily diet ordered as observed or stated.

Gastrointestinal standard: Abdomen soft and non-tender. Bowel sounds present. No nausea or vomiting. Continent. Bowel pattern normal as observed or stated.

Genitourinary standard: Voids without difficulty. Continent. Urine clear, yellow to amber as observed or stated. Urinary catheter patent if present.

Musculoskeletal standard: Independently able to move all extremities and perform functional activities as observed or stated (includes assistive devices).

Pain standard: Without pain or visual analogue scale<4 or experiencing chronic pain that is managed effectively.

Neurological standard: Alert, oriented to person, place, time and situation. Speech is coherent.

Peripheral/vascular standard: Extremities are normal or pink and warm. Peripheral pulses palpable. Capillary refill <3 s. No oedema, numbness or tingling.

Psychosocial standard: Behaviour appropriate to situation. Expressed concerns and fears being addressed. Adequate support system.

Respiratory standard: Respiration 12–24/min at rest quiet and regular. Bilateral breath sounds clear. Nail beds and mucous membranes pink. Sputum clear if present.

Safety/fall risk standard: Safety/fall risk factors not present. Patient is not a risk to self or others.

Skin/tissue standard: Skin clean, dry and intact with no reddened areas. Patient is alert, cooperative and able to reposition self independently. Braden scale>15.


View Abstract


  • Contributors MJR, SIR and GDF conceived this research issue, and performed the preliminary investigations. MJR performed the data extraction. ABS extended the scope of the investigation, analysed the extended dataset, searched the literature and wrote the first draft of the manuscript. All authors contributed to the interpretation of the results and critical revision of the manuscript for important intellectual content and approved the final version of the manuscript. MJR is the guarantor.

  • Funding This research was funded by Rothman Healthcare Corporation, of which Michael and Steven Rothman are cofounders, in an attempt to find ways to improve communication between caregivers in the hospital, following the avoidable hospital death of Florence Rothman in 2003 (which led to the creation of the company). This research was also funded by the Sarasota Memorial Health Care System, and by grants from the Sarasota Memorial Healthcare Foundation and the Goldsmith Fund of the Greenfield Foundation.

  • Competing interests All authors have completed the Unified Competing Interest form at and submitted herewith, and declare: we have received support from Rothman Healthcare Corporation (RHC) for the submitted work; and we all have financial relationships with RHC, which has an interest in the submitted work, but no other relationships or activities that could appear to have influenced the submitted work. All the authors performed this work as employees of RHC, except ABS, who worked on this project as an independent contractor. RHC utilises the results of this research in the Rothman Index, which is used in commercial products, and thereby has an interest in having the research reviewed by an impartial peer journal.

  • Ethics approval Sarasota Memorial Hospital IRB.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Data will be made available to qualified researchers upon request to the corresponding author.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.