You are here

  • Home
  • Archive
  • Volume 20, Issue 1
  • A human factors framework and study of the effect of nursing workload on patient safety and employee quality of working life

Article Text

Article menu

Download PDFPDF

ORIGINAL RESEARCH
A human factors framework and study of the effect of nursing workload on patient safety and employee quality of working life
  1. Richard J Holden1,2,
  2. Matthew C Scanlon3,
  3. Neal R Patel4,
  4. Rainu Kaushal5,
  5. Kamisha Hamilton Escoto6,
  6. Roger L Brown7,
  7. Samuel J Alper1,
  8. Judi M Arnold8,
  9. Theresa M Shalaby8,
  10. Kathleen Murkowski9,
  11. Ben-Tzion Karsh1
  1. 1Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA
  2. 2Psychology, University of Wisconsin-Madison, Madison, Wisconsin, USA
  3. 3Pediatrics, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
  4. 4Pediatrics, Vanderbilt University, Nashville, Tennessee, USA
  5. 5Pediatrics and Public Health, Weill Cornell Medical College, New York, USA
  6. 6Analytics & Research Unit, Ingenix, Eden Prairie, Minnesota, USA
  7. 7Nursing, University of Wisconsin-Madison, Madison, Wisconsin, USA
  8. 8Pediatric Critical Care, Vanderbilt Children's Hospital, Nashville, Tennessee, USA
  9. 9Critical Care, Children's Hospital of Wisconsin, Milwaukee, Wisconsin, USA
  1. Correspondence to Dr Ben-Tzion Karsh, Industrial and Systems Engineering, University of Wisconsin-Madison, 1513 University Avenue, Room 3218, Madison, WI 53706, USA; bkarsh{at}engr.wisc.edu

Abstract

Background Nursing workload is increasingly thought to contribute to both nurses' quality of working life and quality/safety of care. Prior studies lack a coherent model for conceptualising and measuring the effects of workload in healthcare. In contrast, we conceptualised a human factors model for workload specifying workload at three distinct levels of analysis and having multiple nurse and patient outcomes.

Methods To test this model, we analysed results from a cross-sectional survey of a volunteer sample of nurses in six units of two academic tertiary care paediatric hospitals.

Results Workload measures were generally correlated with outcomes of interest. A multivariate structural model revealed that: the unit-level measure of staffing adequacy was significantly related to job dissatisfaction (path loading=0.31) and burnout (path loading=0.45); the task-level measure of mental workload related to interruptions, divided attention, and being rushed was associated with burnout (path loading=0.25) and medication error likelihood (path loading=1.04). Job-level workload was not uniquely and significantly associated with any outcomes.

Discussion The human factors engineering model of nursing workload was supported by data from two paediatric hospitals. The findings provided a novel insight into specific ways that different types of workload could affect nurse and patient outcomes. These findings suggest further research and yield a number of human factors design suggestions.

  • Workload
  • mental workload
  • patient safety
  • medication error
  • quality of working life
  • human factors
  • nurses
  • medication safety

https://doi.org/10.1136/bmjqs.2008.028381

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Quality of care and quality of working life in healthcare are oft-cited targets for improvement.1–5 Substantive improvements in both can be achieved by combining a thorough understanding of the healthcare work system with human factors design and redesign.6–13 Excessive nursing workload is among the many work system contributors to quality of care and working life problems that has been internationally recognised as needing further study and remediation.14–21

    Aiken's seminal study reported that workload affected patient safety and nurses' well-being,21 findings corroborated by studies in the USA22–29 and abroad,30–37 and supported by anecdotal experiences.38–42 Despite increasing interest in nursing workload, much work remains to be done in the conceptualisation and measurement of nursing workload, its causes and its effects.19 43 44

    Reviews of nursing workload measurement16 44–47 show that workload is most often defined in terms of staffing ratios. However, staffing ratios are not clearly representative of nurses' actual or perceived workload.47 48 Furthermore, occupational workload research suggests that staffing ratios measure only one type of workload43 49 (figure 1). Generally, workload can be thought of as the ratio of demands (‘task load’) to available resources.50 At one level of analysis, patient-to-nurse ratios might be a good representation of workload. However, there are other levels at which demands and resources interact to produce workload.8 29 43

    Figure 1
    Figure 1

    A multi-level human factors framework of nursing workload.

    Based on the human factors models of Carayon and Gürses43 and Karsh et al,8 we propose that at least three types of workload exist, resulting from different demands and resources (see figure 1). Unit-level workload refers to the staffing ratios mentioned above. Job-level workload51 52 refers to general and specific demands of the job, including the general amount of work to be done in the day, the difficulty of the work and the amount of concentration or attention required to do it. Resources at this level include time given to complete work, rest breaks and available human (eg, unlicensed assistive personnel) or technological resources. Task-level workload53–55 refers to the demands and resources for a specific nursing task, such as medication administration. Demands might include the need to concentrate or multitask, while resources include training, cognitive capacity, technologies, staff support and more.

    Each type of workload is measured in a distinct manner, and each might have a different impact on outcomes such as quality of care, patient safety, nurse behaviour or nurse job dissatisfaction and burnout. Importantly, each type can be addressed through distinct approaches to policy change, training/education and systems design.

    The objectives of the current study were (1) to measure each of the three types of workload experienced by nurses at two paediatric hospitals, and (2) to assess whether and which measures of workload were related to three important outcomes: nurses' self-reported job dissatisfaction and burnout, and the perceived likelihood of an error occurring during medication administration. It was expected that some, but not other, types of workload would be associated with each of the outcomes.

    Methods

    Study design

    The study is based on cross-sectional survey data collected from registered nurses at two urban academic tertiary care free-standing paediatric hospitals. The study was approved by both hospitals’ Institutional Review Boards (IRBs).

    Sample and setting

    Hospital A, in the Midwest, had 222 beds. Hospital B, in the South, had 162 beds. In each hospital, three inpatient units were studied: paediatric intensive care (PICU), haematology–oncology–transplant (HOT), and general medical/surgical. Hospital A, had a 24 bed PICU, 24-bed HOT unit, and 48-bed general medical/surgical unit. In Hospital B, each unit had 24 beds.

    Full-time registered nurses (24 h/week or more) in the study units were eligible to participate. Temporary nurses, float nurses or nurses who did not directly provide patient care were not included in the target sample, leaving 203 and 144 eligible nurses from Hospital A and B, respectively.

    Measures

    All items underwent extensive evaluation procedures56–60 including cognitive interviewing with non-study nurses.61–65 Survey items were measured on 7-point Likert-type scales ranging from 0 to 6, with the category labels ‘not at all,’ ‘a little,’ ‘some,’ ‘a moderate amount,’ ‘pretty much,’ ‘quite a lot’ and ‘a great deal,’ and an option to mark ‘don't know.’ Participants were instructed to think of the past 30 days.

    Workload and outcome measures are described in table 1. Both general and specific measures of job-level workload were obtained, the latter focussing on attention demands of the job and comprising two highly intercorrelated subscales that were combined to form a single variable (results were nearly identical using the subscales separately). The two task-level workload scales were developed by the researchers based on the two most valid subjective task workload measures, the NASA-TLX53 and SWAT.54 Using separate internal and external task-level workload scales was a post-hoc decision based on a poor fit between the data and a one-factor confirmatory model; exploratory factor analysis strongly suggested a two-factor solution. The task of interest was medication administration because of the importance of this task to nursing care and patient safety.66 67 Indeed, on a scale of 0 to 6, these same nurses reported that, compared with all nursing interventions, this task had an impact of 5.2 (SD=1.0) on patient outcomes.

    View this table:
    Table 1

    Measures used, internal consistency, and examples

    Survey administration

    Eligible nurses were informed about the study and hand-delivered surveys packets during inservices, staff meetings, or shift-change meetings. Each survey packet was individually labelled with a nurse's name and a unique ID; each contained a personalised cover letter, survey, informational sheet/consent form, stamped reply envelope, and US$5 cash incentive. Nurses were instructed to complete the survey on their own time. Reminder postcards were placed in nurse mailboxes after 1 week, followed by a new survey packet 7–10 days after, and another reminder postcard 7–10 days after that. Data were collected during November–December 2005 and March–May 2006 at Hospitals A and B, respectively.

    Analysis plan

    A multilevel logit model68 was constructed, as depicted in figure 2, and path loadings were assessed. Exogenous variables were measures of unit-level, general job-level, specific job-level, task-level internal and task-level external workload. Endogenous variables were job dissatisfaction, burnout and medication error likelihood. The path loading between the three outcome variables were estimated. The model was constructed using the MPlus software (Muthén & Muthén, Los Angeles, California). Parameter estimated standard errors took into account non-independence of observations due to the cluster sampling within units.69 Statistical adjustment was made for hospital, unit, shift, number of hours worked per week, experience (total time with current employer) and age. Due to a lack of variation in gender, ethnicity and education, those variables were not adjusted for. Adding covariates did not change the pattern of results or any outcomes of significance tests. Thus, only statistical values adjusted for covariates are reported. An a priori alpha criterion of 0.05 was used. Where noted, reported correlations were adjusted using the False Discovery Rate approach to counteract the increased chance of Type I error with multiple statistical assessments.70–72

    Figure 2
    Figure 2

    Multilevel logit model. Arrows depict tested path loadings between workload, nurse outcomes, and patient outcomes.

    Results

    The overall response rate was 57.3%. Table 2 provides sample characteristics. Compared with nurses in the 2004 US National Sample Survey of Registered Nurses (NSSRN),73 nurses in our sample were slightly more likely to be female (NSSRN=93.8% female), more likely to be white (NSSRN=81.2%), more likely to be younger (NSSRN average=45.4 years) and worked fewer hours (NSSRN=43.7). These discrepancies are not unexpected, as in the national sample, 56.2% of nurses worked in hospitals, and only 6.5% worked full-time with paediatric patients, and hospital nurses tended to differ from others.73

    View this table:
    Table 2

    Response rates and sample characteristics for both hospitals

    For the present analyses, 12 cases were excluded due to missing data on key variables. An additional 11 respondents reported working fewer than 24 h/week and were excluded, although including their data did not change the results of the analyses. The final analysed sample was 176: 99 from Hospital A and 77 from Hospital B.

    Table 3 reports descriptive statistics. Self-reported medication error likelihood was assessed with a single item, and the left-skewed (skewness=0.67) distribution of responses on this item suggested dichotomising the variable as follows: responses reporting the likelihood of a medication error to be ‘not at all’ (n=9), ‘a little’ (n=59) or ‘some’ (n=59) were recoded as ‘Low error likelihood;’ responses reporting the likelihood to be ‘a moderate amount’ (n=42), ‘pretty much’ (n=3), ‘quite a lot’ (n=3), or ‘a great deal’ (n=1) were recoded as ‘Moderate error likelihood.’ (Results did not change when medication errors were analysed as a continuous variable).

    View this table:
    Table 3

    Descriptive statistics, Pearson correlations and ORs

    Table 3 also reports false discovery rate-adjusted Pearson correlations. Workload measures were positively intercorrelated across levels. Job dissatisfaction and burnout were significantly correlated, but neither was significantly associated with the likelihood of medication error. The bivariate correlations between workload and outcome measures demonstrate fairly consistent relationships between the two.

    Table 4 summarises results from a test of the statistical model in figure 2. Results show a positive relationship between external mental workload and medication error likelihood (γ25=1.04, 95% CI (0.78 to 1.30)), with a corresponding OR of 2.82 (95% CI (2.17 to 3.67)). There was a significant effect of Hospital on medication error likelihood (OR=0.58; 95% CI (0.37 to 0.93)), such that Hospital B nurses were almost half as likely to report a moderate-or-higher error likelihood, and of unit (OR=1.93; 95% CI (1.04 to 3.58)), such that PICU nurses were nearly twice as likely to report a moderate-or-higher error likelihood, compared with nurses in other units.

    View this table:
    Table 4

    Results of model test

    Dissatisfaction was positively associated with unit-level workload measure of staffing adequacy (γ11=0.31, 95% CI (0.18 to 0.45)). Burnout was positively associated with both unit-level workload (γ31=0.45, 95% CI (0.24 to 0.66)) and task-level external mental workload (γ35=0.25, 95% CI (0.07 to 0.44)). Dissatisfaction and burnout were also positively associated (Ψ31=0.52, 95% CI (0.42 to 0.63)). All remaining relationships were non-significant. The findings described above were identical to multiple linear and logistic regression equations carried out separately for the three dependent variables (not reported here).

    Discussion

    This study set out to test the workload–outcome relationships suggested by the multiple-level model in figure 1. The findings provide an understanding of how these relationships may play out: medication errors may be best predicted by task-level workload; dissatisfaction by unit-level workload; and burnout by both unit- and task-level workload.

    Representing multiple types of workload jointly as we did here has implications for interpretation and design. Previous studies have demonstrated a relationship between quality/safety outcomes and a single measure of workload such as staffing ratios, as in table 3. A designer might surmise from any significant value from table 3 that a design intervention addressing staffing/resource adequacy, general job demands or task-level (external) workload conditions would improve patient and nurse outcomes. However, the causal inferences and design prospects change when one examines a full model, as in figure 2, where each workload–outcome relationship is adjusted for the effect of the other workload types, and outcomes are free to covary. Following such adjustment, only unit- and task-level (external) workload uniquely relate to the outcomes, and not uniformly so. The multivariate findings suggest to the scientist or designer a deeper understanding of causal mechanisms: for example, the effect of job demands on errors might be mediated by external workload (ie, related to interruptions, divided attention and rushing) during the medication administration task. Thus, a cognitive performance mechanism could be posited to explain medication errors,74 and the designer might realise that for an intervention at the job design level to be successful, it would have to address the task performance factors associated with the medication administration task. This would not be evident from univariate findings.

    This is not the first study in which the findings question whether staffing ratios are truly predictive of patient safety,16 31 75 76 raising the possibility that staffing interventions may not be as effective for reducing errors, morbidity or mortality as previously thought, or that only staffing interventions that affect task-level workload will be effective. On the other hand, as expected, workload associated with the medication administration task was related to perceived likelihood of medication error.77–80 Unexpectedly, the task-level workload measure turned out to have two components, an internal one related to mental effort and concentration, and an external one related to exogenous demands of the task. Only the external component was related to patient safety outcomes. This finding requires further analysis and study. One hypothesis might be that mental effort and concentration are beneficial consequences of certain work demands. Perhaps in nursing, some amount of this makes work more satisfying, buffers against burnout and improves patient outcomes through superior performance. Figure 3 shows a comparison of the effect on medication error likelihood of external (upper panel) and internal (lower panel) workload. As suggested here, increases in external workload have deleterious effects on error likelihood, whereas the parabolic error-by-internal workload function is in line with the idea that there are performance benefits to concentration and mental effort at higher magnitudes of these conditions. (This finding is reminiscent of, but not necessarily the same as, the parabolic arousal–performance relationship of the Yerkes–Dodson Law.81) If true, there will be implications for introducing automation and work methods that reduce concentration and effort and make work simpler.82 For example, researchers and organisations studying bar-coded medication administration (BCMA) systems should consider evaluating their impact on clinicians' workload.

    Figure 3
    Figure 3

    Perceived likelihood of medication error as a function of external (upper panel) and internal (lower panel) task-level workload.

    Limitations

    The use of subjective measures, such as self-reported workload in this study, is the topic of much discussion.83 Because workload is a construct that is internal to a worker,84 85 some argue ‘there is no other way to measure such concepts as workload … directly unless we ask the participant.’86 What is most important in respect to measuring subjective workload is intersubjectivity, or, roughly, the degree to which multiple respondents interpret the questions of the measurement instrument in a similar way.87 The current study took several steps to achieve acceptable intersubjectivity. First, all questions were based on widely used instruments with established validity; second, most constructs were faceted; third, question wording was carefully chosen to reflect commonly understood terminology; fourth, and most importantly, numerous cognitive interviews were conducted in order to evaluate how nurses interpreted survey questions.

    A subjective measure of medication error merits careful attention. On the one hand, an internal assessment of error likelihood might have as great an impact as does ‘objective reality.’ Safety-related behaviour such as error reporting, taking risks and taking safety precautions is to a great extent based on risk perception, accurate or not.88–92 On the other hand, patient safety experts are most interested in actual error rates, and subjective assessments of error likelihood are most valuable if they reflect this actuality. Even allowing that nurses' assessments reflect actual error likelihood, the use of subjective error measurement is a limitation of this study. Ongoing analysis of medication error observation at both hospitals will, in the future, provide an objective unit-based error likelihood value that can be used to validate and supplement the self-reported measure.

    A further limitation is the inability to assess the direction of causality beyond the theoretical plausibility of the proposed causal model. Another limitation is that workload, a construct purported to exist at multiple levels, was measured by asking individuals. In future studies, this limitation can be addressed through aggregation, consideration of nesting and measurement of workload directly at the level of interest.93 Our specification of task-level workload was limited in that it focused on a single, albeit important, task and that it focused primarily on mental demands of the task. Nevertheless, the findings presented here speak to the importance of medication administration-related (external) mental workload.

    Conclusion

    In sum, a study carried out at six nursing units at two paediatric hospitals provided interesting possibilities for how different types of workload may relate to common patient and employee problems in paediatric clinical settings. The findings inspire further workload research and a continued multiple-level approach to the study and policy- and design-based resolution of workload in healthcare.

    Based on study findings, human factors design solutions include:

    • Base staffing, assignment and work design decisions on multiple measures of workload, not only on patient census, staffing counts or patient classification systems.

    • Reduce workload during medication administration by minimising interruptions, concurrent demands and time pressure.29 94

    • When administration tasks cannot be redesigned, technological interventions such as point-of-care BCMA systems might help detect workload-induced errors before they reach the patient. However, in order to be effective, technologies such as BCMA must be designed and implemented according to good usability and implementation practices.95–99

    • Do not indiscriminately reduce demands for mental effort and concentration, as these may be beneficial for patient and employee safety. Instead, a task analysis can identify instances in which concentration and effort are important and should be supported, and instances during which they can be alleviated with periods of low mental activity. Related, technological aids or automation may inadvertently reduce the need for concentration or effort, which may not be advisable.

    • Maintain staffing adequacy. However, if adding staff is not feasible, human factors solutions can be directed at better allocation of work on the unit100 101 or better communication and teamwork.

    • When redesigning work, changes must be made to the proper level of workload: for example, to reduce errors, general job redesign must include support of medication administration tasks. Care should be taken not to increase one type of workload in an attempt to reduce another.

    Finally, further work would benefit from measures of system causes of workload, a broader set of workload measures (eg, capturing physical and emotional demands,49 including physiological indicators of strain102 or focussing on other clinical tasks such as extubation103), and measures of other outcomes of interest, such as safety protocol violations, technology use, perceived quality of care, medication-related injury, turnover and patient satisfaction. Work should continue, especially, in high-risk populations such as paediatrics, critical care and haematology–oncology. Although earlier studies were sufficient to alert us to workload problems, much more remains to be done in the conceptualisation and measure of workload in order to understand the mechanisms at play, a prerequisite for design or engineering interventions to succeed.

    Acknowledgments

    We thank the study participants and everyone involved in the ‘Bar Coding and Patient and Employee Safety’ study. We also thank the anonymous reviewers, A Alonso and Y Donchin for their helpful feedback. A presentation at the 2007 Human Factors and Ergonomics Society Annual Meeting contained a preliminary presentation of these workload data.

    References

    1. ↵
      Institute of Medicine. To err is human: building a safer health system. Institute of Medicine Report on medical errors. Washington, DC: National Academies Press, 2000.
    2. ↵
      Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press, 2001.
    3. ↵
      Institute of Medicine. Patient safety: achieving a new standard for care. Washington, DC: National Academies Press, 2004.
    4. ↵
      Institute of Medicine. Keeping patients safe: transforming the work environment of nurses. Washington, DC: National Academies Press, 2004.
    5. ↵
      Institute of Medicine. Building a better delivery system: a new engineering/health care partnership. Washington, DC: National Academies Press, 2005.
    6. ↵
      1. Bogner MS
      . Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum Associates, 1994.
    7. ↵
      1. Reason J
      . Human error: models and management. BMJ 2000;320:768–70.
      OpenUrlFREE Full Text
    8. ↵
      1. Karsh B,
      2. Holden RJ,
      3. Alper SJ,
      4. et al
      . A human factors engineering paradigm for patient safety—designing to support the performance of the health care professional. Qual Saf Healthcare 2006;15:i59–65.
      OpenUrlCrossRef
    9. ↵
      1. Picket RM,
      2. Triggs TJ
      , eds. Human factors in health care. Lexington, MA: DC Heath & Co., 1975.
    10. ↵
      1. Berwick DM
      . Not again! Preventing errors lies in redesign—not exhortation. BMJ 2001;322:247–8.
      OpenUrlFREE Full Text
    11. ↵
      1. Carayon P
      , ed. Handbook of human factors and ergonomics in patient safety. Mahwah, NJ: Lawrence Erlbaum Associates, 2007.
    12. ↵
      1. Gosbee J
      . Human factors engineering and patient safety. Qual Saf Healthcare 2002;11:352–4.
      OpenUrlCrossRef
    13. ↵
      1. Goldmann D,
      2. Kaushal R
      . Time to tackle the tough issues in patient safety. Pediatrics 2002;110:823–6.
      OpenUrlFREE Full Text
    14. ↵
      1. Page A
      , ed. Keeping patients safe: transforming the work environment of nurses. Washington, DC: National Academies Press, 2004.
    15. ↵
      1. Aspden P,
      2. Corrigan JM,
      3. Wolcott J,
      4. et al.
      , eds. Patient safety: achieving a new standard for care. Washington, DC: National Academies Press, 2004.
    16. ↵
      1. Hurst K
      . Relationship between patient dependency, nursing workload and quality. Int J Nurs Stud 2005;42:75–84.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Trueland J
      . Scottish workload report receives warm welcome. Nurs Stand 2004;18:8.
      OpenUrl
    18. ↵
      Workload measure to benefit all. Author's name withheld. Aust Nurs J 2004;12:11.
      OpenUrlPubMed
    19. ↵
      1. Brady A-M,
      2. Byrne G,
      3. Horan P,
      4. et al
      . Measuring the workload of community nurses in Ireland: a review of workload measurement systems. J Nurs Manag 2007;15:481–9.
      OpenUrlCrossRefPubMed
    20. ↵
      Healthcare Commission. Ward staffing. London: Commission for Healthcare Audit and Inspection, 2005.
    21. ↵
      1. Aiken LH,
      2. Clarke SP,
      3. Sloane DM,
      4. et al
      . Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002;288:1987–93.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Davidson H,
      2. Folcarelli RH,
      3. Crawford S,
      4. et al
      . The effects of healthcare reform in job satisfaction and voluntary turnover among hospital-based nurses. Med Care 1997;35:634–45.
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Kazanjian A,
      2. Green C,
      3. Wong J,
      4. et al
      . Effect of the hospital nursing environment on patient mortality: a systematic review. J Health Serv Res Policy 2005;10:111–17.
      OpenUrlAbstract/FREE Full Text
    24. ↵
      1. Lang TA,
      2. Hodge M,
      3. Olson V,
      4. et al
      . Nurse-patient ratios: a systematic review on the effects of nurse staffing on patient, nurse employee, and hospital outcomes. J Nurs Adm 2004;34:326–37.
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. McVicar A
      . Workplace stress in nursing: a literature review. J Adv Nurs 2003;44:633–42.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Needleman J,
      2. Buerhaus P,
      3. Mattke S,
      4. et al
      . Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 2002;346:1715–22.
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Mark BA,
      2. Harless DW,
      3. McCue M,
      4. et al
      . A longitudinal examination of hospital registered nurse staffing and quality of care. Health Serv Res 2004;39:279–300.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Sochalski J
      . Is more better?: the relationship between nurse staffing and the quality of nursing care in hospitals. Med Care 2004;42:ii67–73.
      OpenUrlPubMed
    29. ↵
      1. Weissman JS,
      2. Rothschild JM,
      3. Bendavid E,
      4. et al
      . Hospital workload and adverse events. Med Care 2007;45:448–55.
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Lambert VA,
      2. Lambert CE,
      3. Petrini M,
      4. et al
      . Predictors of physical and mental health in hospital nurses within the People's Republic of China. Int Nurs Rev 2007;54:85–91.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Seki Y,
      2. Yamazaki Y
      . Effects of working conditions on intravenous medication errors in a Japanese hospital. J Nurs Manag 2006;14:128–39.
      OpenUrlCrossRefPubMed
    32. ↵
      1. Czapilinksi C,
      2. Diers D
      . The effect of staff nursing on length of stay and mortality. Med Care 1998;36:1626–38.
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Tarnow-Mordi WO,
      2. Hau C,
      3. Warden A,
      4. et al
      . Hospital mortality in relation to staff workload: a 4-year study in an adult intensive-care unit. Lancet 2000;356:185–9.
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Rafferty AM,
      2. Clarke SP,
      3. Coles J,
      4. et al
      . Outcomes of variation in hospital nurse staffing in English hospitals: cross-sectional analysis of survey data and discharge records. Int J Nurs Stud 2007;44:175–82.
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Estabrooks CA,
      2. Midodzi WK,
      3. Cummings GG,
      4. et al
      . The impact of hospital nursing characteristics on 30-day mortality. Nurs Res 2005;54:74–84.
      OpenUrlPubMedWeb of Science
    36. ↵
      1. Aiken LH,
      2. Clarke SP,
      3. Sloane DM,
      4. et al
      . Nurses' reports on hospital care in five countries. Health Aff 2001;20:43–53.
      OpenUrlAbstract/FREE Full Text
    37. ↵
      1. Aiken LH,
      2. Clarke SP,
      3. Sloane DM
      . Hospital staffing, organization, and quality of care: cross-national findings. Int J Qual Health Care 2002;14:5–13.
      OpenUrlAbstract/FREE Full Text
    38. ↵
      Workload a safety risk. Aust Nurs J 2007;14:p3.
      OpenUrl
    39. ↵
      1. McGillis Hall L,
      2. Kiesners D
      . A narrative approach to understanding the nursing work environment in Canada. Soc Sci Med 2005;61:2482–91.
      OpenUrlCrossRefPubMedWeb of Science
    40. ↵
      1. Hurley ML
      . Workload, UAPs, and you. RN 2000;63:47–9.
      OpenUrlPubMed
    41. ↵
      1. Bathen S
      . Nursing ratios. State Net California Journal 2004;59:16.
      OpenUrl
    42. ↵
      1. Sparks SA,
      2. Corcoran KJ,
      3. Nabors LA,
      4. et al
      . Job satisfaction and subjective well-being in a sample of nurses. J Appl Soc Psychol 2005;35:922–38.
      OpenUrlCrossRefWeb of Science
    43. ↵
      1. Carayon P,
      2. Gurses AP
      . A human factors engineering conceptual framework of nursing workload and patient safety in intensive care units. Intensive Crit Care Nurs 2005;21:284–301.
      OpenUrlCrossRefPubMed
    44. ↵
      1. O'Brien-Pallas L,
      2. Irvine D,
      3. Peereboom E,
      4. et al
      . Measuring nursing workload: understanding the variability. Nurs Econ 1997;15:171–82.
      OpenUrlPubMedWeb of Science
    45. ↵
      1. Unruh LY,
      2. Fottler MD
      . Patient turnover and nursing staff adequacy. Health Serv Res 2006;41:599–612.
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      1. O'Brien AJ,
      2. Abas M,
      3. Christensen J,
      4. et al
      . Nursing workload measurement in acute mental health inpatient units. Auckland, New Zealand: Health Research Council of New Zealand, 2002.
    47. ↵
      1. Pearson A,
      2. O'Brien-Pallas L,
      3. Thomson D,
      4. et al
      . Systematic review of evidence on the impact of nursing workload and staffing on establishing healthy work environments. Int J Evidence Based Healthcare 2006;4:337–84.
      OpenUrlCrossRef
    48. ↵
      1. Adomat R,
      2. Hicks C
      . Measuring nursing workload in intensive care: an observational study using closed circuit video cameras. J Adv Nurs 2003;42:402–12.
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      1. Carayon P,
      2. Alvarado CJ
      . Workload and patient safety among critical care nurses. Crit Care Nurs Clin North Am 2007;19:121–9.
      OpenUrlCrossRefPubMed
    50. ↵
      1. Xie B,
      2. Salvendy G
      . Review and reappraisal of modelling and predicting mental workload in single- and multi-task environments. Work Stress 2000;14:74–99.
      OpenUrlCrossRefWeb of Science
    51. ↵
      1. Karasek RA
      . Job demands, job decision latitude, and mental strain, implications for job redesign. Adm Sci Q 1979;24:285–307.
      OpenUrlCrossRefWeb of Science
    52. ↵
      1. Sauter S,
      2. Hurrell JJ,
      3. Cooper CL
      1. Hurrell JJ,
      2. McLaney MA
      . Control, job demands, and job satisfaction. In: Sauter S, Hurrell JJ, Cooper CL, eds. Job control and worker health. Chichester, NY: John Wiley and Sons, 1989:97–103.
    53. ↵
      1. Hancock PA,
      2. Meshkati N
      1. Hart SG,
      2. Staveland LE
      . Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds. Human mental workload. Amsterdam, The Netherlands: North-Holland, 1988:139–83.
    54. ↵
      1. Hancock PA,
      2. Meshkati N
      1. Reid GB,
      2. Nygren TE
      . The subjective workload assessment technique: A scaling procedure for measuring mental workload. In: Hancock PA, Meshkati N, eds. Human mental workload. Amsterdam, The Netherlands: North-Holland, 1988:185–218.
    55. ↵
      1. Rubio S,
      2. Díaz E,
      3. Martín J,
      4. et al
      . Evaluation of subjective mental workload: a comparison of SWAT, NASA-TLX, and workload profile methods. IAAP 2004;53:61–86.
      OpenUrl
    56. ↵
      1. Presser S,
      2. Rothgeb JM,
      3. Couper MP,
      4. et al.
      1. Forsyth BH,
      2. Rothgeb JM,
      3. Willis GB
      . Does pretesting make a difference? In: Presser S, Rothgeb JM, Couper MP, et al., eds. Questionnaire development evaluation and testing methods. New York: Wiley, 2004.
    57. ↵
      1. Groves RM,
      2. Fowler FJ,
      3. Couper MP,
      4. et al
      . Survey methodology. Hoboken, NJ: Wiley-Interscience, 2004.
    58. ↵
      1. Marsden P
      1. Presser S,
      2. Blair J
      . Survey pretesting: do different methods produce different results? In: Marsden P, ed. Sociology methodology. Washington, DC: American Sociological Association, 1994:73–104.
    59. ↵
      1. Schwarz N,
      2. Sudman S
      1. Graesser A,
      2. Bommareddy S,
      3. Swamer S,
      4. et al
      . Intergrating questionnaire design with a cognitive computational model of human question answering. In: Schwarz N, Sudman S, eds. Answering questions. San Francisco, CA: Jossey-Bass, 1996:143–74.
    60. ↵
      1. Schwarz N,
      2. Sudman S
      1. Lessler JT,
      2. Forsyth B
      . A coding system for appraising questionnaires. In: Schwarz N, Sudman S, eds. Answering questions. San Francisco, CA: Jossey-Bass, 1996:259–92.
    61. ↵
      1. Lessler JT,
      2. Tourangeau R,
      3. Salter W
      . Questionnaire design research in the cognitive research laboratory. Vital and health statistics, series 6 (No. 1). Hyattsville, MD: US Department of Health and Human Services, DHHS Publication No, (PHS) 89-1076, 1989.
    62. ↵
      1. Tourangeau R,
      2. Rips L,
      3. Rasinski K
      . The psychology of survey response. Cambridge, UK: Cambridge University Press, 2000.
    63. ↵
      1. Willis GB
      . Cognitive interviewing: a ‘how to’ guide. Research Triangle Park, NC: Research Triange Institute, 1999.
    64. ↵
      1. Willis GB
      . Cognitive interviewing: a tool for improving questionnaire design. Thousand Oaks, CA: Sage, 2004.
    65. ↵
      1. Presser S,
      2. Rothgeb JM,
      3. Couper MP,
      4. et al.
      1. Willis GB
      . Cognitive interviewing revisited: a useful technique, in theory? In: Presser S, Rothgeb JM, Couper MP, et al., eds. Methods for testing and evaluating survey questionnaires. New York: Wiley, 2004.
    66. ↵
      1. Aspden P,
      2. Wolcott J,
      3. Bootman JL,
      4. et al.
      , eds. Preventing medication errors: quality chasm series. Washington, DC: National Academies Press, 2007.
    67. ↵
      1. Cohen MR
      . Medication errors. 2nd edn. Washington, DC: American Pharmaceutical Association, 2006.
    68. ↵
      1. Guo G,
      2. Zhao H
      . Multilevel modeling for binary data. Annu Rev Sociol 2000;26:441–62.
      OpenUrlCrossRefWeb of Science
    69. ↵
      1. Huber PJ
      . The behavior of maximum likelihood estimates under non-standard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability 1967;1:221–33.
      OpenUrl
    70. ↵
      1. Benjamini Y,
      2. Hochberg Y
      . Controlling the false discovery rate: a practical and powerful approach to multiple testing. J Roy Stat Soc B 1995;57:289–300.
      OpenUrl
    71. ↵
      1. Storey JD
      . A direct approach to false discovery rates. J Roy Stat Soc B 2002;64:479–98.
      OpenUrlCrossRef
    72. ↵
      1. Storey JD
      . The positive false discovery rate: a Bayesian interpretation and the q-value. Ann Stat 2003;31:2013–34.
      OpenUrlCrossRefWeb of Science
    73. ↵
      Health Research and Services Administration. The registered nurse population: findings from the 2004 National Sample Survey of Registered Nurses. http://bhpr.hrsa.gov/healthworkforce/rnsurvey04/ (accessed 2 Jul 2010).
    74. ↵
      1. Grasha AF,
      2. Schell K
      . Psychosocial factors, workload, and human error in a simulated pharmacy dispensing task. Percept Mot Skills 2001;92:53–71.
      OpenUrlCrossRefPubMedWeb of Science
    75. ↵
      1. Shojania K,
      2. Duncan B,
      3. McDonald K,
      4. et al.
      1. Seago JA
      . Nurse staffing, models of care delivery, and interventions. In: Shojania K, Duncan B, McDonald K, et al., eds. Making health care safer: a critical analysis of patient safety practices. Rockville, MD: AHRQ, 2001:423–46.
    76. ↵
      1. Potter P,
      2. Barr N,
      3. McSweeney M,
      4. et al
      . Identifying nurse staffing and patient outcome relationships: a guide for change in care delivery. Nurs Econ 2003;21:158–66.
      OpenUrlPubMed
    77. ↵
      1. Blegen MA,
      2. Goode CJ,
      3. Reed L
      . Nurse staffing and patient outcomes. Nurs Res 1998;47:43–50.
      OpenUrlCrossRefPubMedWeb of Science
    78. ↵
      1. Tang F-I,
      2. Sheu S-J,
      3. Yu S,
      4. et al
      . Nurses relate the contributing factors involved in medication errors. J Clin Nurs 2007;16:447–57.
      OpenUrlCrossRefPubMedWeb of Science
    79. ↵
      1. Rogers AE,
      2. Hwang W-T,
      3. Scott LD,
      4. et al
      . The working hours of hospital staff nurses and patient safety. Health Aff 2004;23:202–12.
      OpenUrlAbstract/FREE Full Text
    80. ↵
      1. Burke RJ
      . Hospital restructuring, workload, and nursing staff satisfaction and work experiences. Health Care Manag 2003;22:99–107.
      OpenUrl
    81. ↵
      1. Yerkes RM,
      2. Dodson JD
      . The relation of strength of stimulus to rapidity of habit-formation. J Comp Neurol Psychol 1908;18:459–82.
      OpenUrlCrossRef
    82. ↵
      Technology's role in addressing Maryland's nursing shortage: innovations & examples: Technology Workgroup of the Maryland Statewide Commission on the Crisis in Nursing, 2004.
    83. ↵
      1. Stanton NA,
      2. Stammers RB
      . Creative (dis)agreement in ergonomics. Ergonomics 2002;45:963–5.
      OpenUrlCrossRefPubMedWeb of Science
    84. ↵
      1. Moray N
      1. Sheridan TB,
      2. Stassen HG
      . Definitions, models and measures of human workload. In: Moray N, ed. Mental workload: its theory and measurement. New York: Plenum, 1979.
    85. ↵
      1. Moray N
      . Mental workload since 1979. Int Rev Ergon 1988;2:123–50.
      OpenUrl
    86. ↵
      1. Drury CG
      . Measurement and the practising ergonomist. Ergonomics 2002;45:988–90.
      OpenUrlCrossRefPubMedWeb of Science
    87. ↵
      1. Annett J
      . Subjective rating scales: science or art? Ergonomics 2002;45:966–87.
      OpenUrlCrossRefPubMedWeb of Science
    88. ↵
      1. DeJoy DM
      . Managing safety in the workplace: An attribution theory analysis and model. J Safety Res 1994;25:3.
      OpenUrlCrossRefWeb of Science
    89. ↵
      1. Slovic P,
      2. Fischhoff B,
      3. Lichtenstein S
      . Behavioral decision theory perspectives on risk and safety. Acta Psychol 1984;56:183–203.
      OpenUrlCrossRefWeb of Science
    90. ↵
      1. Wilder DA
      . Perceiving persons as a group: effects on attributions of causality and beliefs. Soc Psychol 1978;41:13–23.
      OpenUrlCrossRef
    91. ↵
      1. Amenolda A
      . Recent paradigms for risk informed decision making. Saf Sci 2001;40:17–30.
      OpenUrlCrossRefWeb of Science
    92. ↵
      1. Rundmo T,
      2. Iversen H
      . Risk perception and driving behaviour among adolescents in two Norwegian counties before and after a traffic safety campaign. Saf Sci 2004;42:1–21.
      OpenUrlCrossRefWeb of Science
    93. ↵
      1. Karsh B,
      2. Brown R
      . The impact of levels on theory, measurement, analysis, and intervention in medical error research: the case of patient safety health information technology. Appl Ergon 2007;41:674–81.
      OpenUrlCrossRef
    94. ↵
      1. Tucker AL,
      2. Spear SJ
      . Operational failures and interruptions in hospital nursing. Health Ser Res 2006;41:643–62.
      OpenUrlCrossRef
    95. ↵
      1. Holden RJ,
      2. Karsh B
      . A theoretical model of health information technology behavior. Behaviour & Information Technology 2009;28:21–38.
      OpenUrlCrossRefWeb of Science
    96. ↵
      1. Karsh B
      . Beyond usability for patient safety: designing effective technology implementation systems. BMJ: Quality and Safety in Healthcare 2004;13:388–94.
      OpenUrlCrossRef
    97. ↵
      1. Carayon P
      1. Karsh B,
      2. Holden RJ
      . New technology implementation in health care. In: Carayon P, ed. Handbook of human factors and ergonomics in patient safety. Mahwah, NJ: Lawrence Erlbaum Associates, 2007:391–408.
    98. ↵
      1. Or KL,
      2. Valdez RS,
      3. Casper GR,
      4. et al
      . Human factors, ergonomics, and health information technology in home care – a perspective from work system analysis. WORK 2008;33:201–9.
      OpenUrlWeb of Science
    99. ↵
      1. Karsh B,
      2. Scanlon M
      . When is a defibrillator not a defibrillator? When it is like a clock radio…. The challenge of usability and patient safety in the real world. Ann of Emerg Med 2007;50:433–5.
      OpenUrlCrossRefPubMedWeb of Science
    100. ↵
      1. Mullinax C,
      2. Lawley M
      . Assigning patients to nurses in neonatal intensive care. J Oper Res Soc 2002;53:25–35.
      OpenUrlCrossRef
    101. ↵
      1. Burke EK,
      2. De Causmaecker P,
      3. Vanden Berghe G,
      4. et al
      . The state of the art of nurse rostering. JOS 2004;7:441–99.
      OpenUrl
    102. ↵
      1. Nuikka M-L,
      2. Paunonen M,
      3. Hänninen O,
      4. et al
      . The nurse's workload in care situations. J Adv Nurs 2001;33:406–15.
      OpenUrlCrossRefPubMedWeb of Science
    103. ↵
      1. Krayem A,
      2. Butler R,
      3. Martin C
      . Unplanned extubation in the ICU: impact on outcome and nursing workload. Ann Thorac Med 2006;1:71–5.
      OpenUrlCrossRef
    104. ↵
      1. Aiken LH,
      2. Patrician PA
      . Measuring organizational traits of hospitals: the revised Nursing Work Index. Nurs Res 2000;43:146–53.
      OpenUrl
    105. ↵
      1. Jackson PR,
      2. Wall TD,
      3. Martin R,
      4. et al
      . New measures of job control, cognitive demand, and production responsibility. J Appl Psychol 1993;78:753–62.
      OpenUrlCrossRefWeb of Science
    106. ↵
      1. Seashore S,
      2. Lawler E,
      3. Mirvis P,
      4. et al.
      1. Cammann C,
      2. Fichman M,
      3. Jenkins D,
      4. et al
      . Assessing the attitudes and perceptions of organizational members. In: Seashore S, Lawler E, Mirvis P, et al., eds. Assessing organizational change: a guide to methods, measures and practices. New York: Wiley, 1983.
    107. ↵
      1. Maslach C,
      2. Jackson SE
      . Maslach burnout inventory: University of California, Berkeley, CA, 1981.
    View Abstract

    Footnotes

    • Funding This work was supported in part by a grant from the Agency for Healthcare Research and Quality (1 R01 HS013610) to author B-TK. The Agency for Healthcare Research and Quality had no role in the study or publication except for providing funding for the study. Author RJH is supported by a training grant from the US National Institutes of Health (1 TL1 RR025013-01).

    • Competing interests None.

    • Ethics approval Ethics approval was provided by the University of Wisconsin IRB, IRBs at each of the two participating hospitals.

    • Provenance and peer review Not commissioned; externally peer reviewed.