Article Text

A study protocol for performance evaluation of a new academic intensive care unit facility: impact on patient care
  1. Mauricio Ferri1,2,
  2. David A Zygun3,
  3. Alexandra Harrison2,
  4. Henry T Stelfox1,2,4
  1. 1Department of Critical Care Medicine, University of Calgary, and Alberta Health Services, Calgary, Alberta, Canada
  2. 2Department of Community Health Sciences, Institute for Public Health, University of Calgary, Calgary, Alberta, Canada
  3. 3Critical Care Medicine, University of Alberta, and Alberta Health Services, Edmonton, Alberta, Canada
  4. 4Department of Medicine, University of Calgary, and Alberta Health Services, Calgary, Alberta, Canada
  1. Correspondence to Dr Henry T Stelfox; tstelfox{at}


Background Healthcare facility construction is increasing because of population demand and the need to replace ageing infrastructure. Research suggests that there may be a relationship between healthcare environment and patient care. To date, most evaluations of new healthcare facilities are derived from techniques used in other industries and focus on physical, financial and architectural performance. However, few studies have evaluated the impact of healthcare facility design on processes and outcomes of patient care.

Study aims The primary objective of this study was to investigate the impact of relocation to a new intensive care unit (ICU) facility on clinical performance measures. This study also proposes to develop and test a framework for facility performance evaluation using accepted ICU design guidelines and Donabedian's model for healthcare quality.

Methods and analysis We will utilise a mixed-methods, observational, retrospective, controlled, before-and-after design to take advantage of the quasiexperimental conditions created with the construction of a new ICU facility in Calgary, Canada. For the qualitative substudy, we will conduct individual interviews with end-users to understand their impressions and experiences with the new environment and perform thematic analysis. For the quantitative substudy, we will compare process of care indicators and patient outcomes for the 12-month period before and after relocation to the new facility. Two other local ICU facilities that did not undergo structural change during the study period will serve as controls. We will triangulate qualitative and quantitative results utilising a novel framework.

Ethics and dissemination The results of this study will contribute in understanding the impact of new ICU facilities on clinical performance measures centred on patients, their families and healthcare providers. The framework will complement existing building performance evaluation techniques and help healthcare administrators plan new ICU facilities. The University of Calgary Research Ethics Board approved this study protocol.

  • Health Services Administration & Management

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

View Full Text

Statistics from


Healthcare facility construction industry

The Hospital Survey and Construction Act, also known as the Hill-Burton Act of 1946, was the first governmental programme designed to provide funding for construction and renovation of healthcare facilities in the USA. It was a partnership between federal and state governments to address the shortage of healthcare facilities resulting from the Great Depression and World War II. It also stimulated the development of standards for design and construction of such facilities.1 ,2 Similarly, in Canada, after World War II, the federal government proposed measures to promote a more comprehensive social security plan that included the National Health Grants Program. This programme also established a Dominion-Provincial partnership and offered financial support to the provinces to plan and build new healthcare facilities.3 Recently, the healthcare facility construction industry has grown exponentially to accommodate both demand for healthcare access and ageing infrastructure.1 ,2

Construction of intensive care unit (ICU) facilities is heavily regulated by building codes and guidelines that recommend minimum standards for building materials, designated spaces and appropriate unit size.4 Codes and guidelines also address functionality, patient circulation and overall physical disposition, but often neglect social, psychological or cultural aspects of facility performance. Such aspects also comprise the environment of care and may play an important role in either facilitating or hindering best practices and delivery of high-quality care.5 ,6 In addition to expanding recommendations and guidelines to include non-technical items, another important fact driving the need for a more comprehensive approach to healthcare facility design and construction is the mandate to incorporate new technology and to use the best available clinical evidence to improve layout.1 The confluence of demographic, cultural and technological aspects poses the following questions to the critical care community: does the design of healthcare facilities impact clinical performance measures? If it does, how can we measure it?

Impact of healthcare facility design on clinical performance measures: the concepts of healing environment and evidence-based design

A growing body of research demonstrates an association between healthcare environment and clinical performance measures (process of care indicators and patient outcomes).7–11 Research by healthcare and design professionals and the development of the performance improvement movement led to the ‘evidence-based design’ approach.12 ,13 Evidence-based design is defined as ‘a process for the conscientious, explicit, and judicious use of current best evidence from research and practice in making critical decisions, together with an informed client, about the design of each individual and unique project’.12 This novel approach when applied to healthcare facility design and construction, involves applying scientific methodology to find associations between design interventions and healthcare outcomes, and goes beyond traditional prescriptive guidelines that recommend minimum standards.

Using evidence-based design, healthcare administrators, providers and designers developed the related concept of healing environments, which are healthcare spaces that support the needs of patients, family members and providers, and facilitate the delivery of high quality care and better patient outcomes.12 ,14 Evidence-based design is the main force behind the concept of healing environments and an important tool in determining the best design solutions for healthcare facilities in any clinical area.12 ,14 Table 1 summarises research to date on the relationship between design of healthcare facilities and impact on healthcare clinical performance.2 ,15 ,16 For example, a recent systematic review of single-occupancy versus multiple-occupancy rooms reported that the former reduced the risk of hospital-acquired infections and had positive impact on patient satisfaction, corroborating the need to further investigate these relationships.8

Table 1

Evidence of association between structure (design elements), processes and outcomes in healthcare

Conceptual frameworks for facility performance evaluation

The most common approach to evaluate facility performance is a postoccupancy evaluation (POE) project,5 which has been successfully used to evaluate ICU facilities.8–10 ,17 ,18 POEs focus on end-user impressions after occupying the facility, not on clinical performance measures (process of care indicators and patient outcomes), which some authors find do not produce actionable recommendations to improve care.19–21 Recently, the scope of healthcare facilities POEs has been expanded in a framework called ‘Building Performance Evaluation’ to encompass other dimensions of the facility, including a possible impact on health clinical performance measures.13 ,19 Despite general agreement that the physical environment is instrumental to quality of care, there is not a validated, widely accepted process to systematically evaluate the impact of healthcare facility design in clinical performance measures; thus, many approaches have been used.13 ,19

Our study will adopt the widely accepted Donabedian conceptual model that describes quality-of-healthcare delivery as a function of three domains: structure, the physical environment in which healthcare occurs; process, the methods utilised and outcome, the effect on health status of a patient or population.22 ,23 Donabedian's model will be applied to this project by examining the relationship between a complex structural intervention (a new ICU facility) and possible impacts on processes and outcomes of patient care.

Another relevant framework for our study is the Society of Critical Care Medicine (SCCM) Guidelines for ICU Design. This design-focused framework divides the ICU into four functional zones with interrelated activities: the Patient Care Zone: areas of direct patient care; the Clinical Support Zone: areas related to diagnosis and treatment of patients (including elsewhere in the hospital); the Unit Support Zone: areas of administrative and staff support functions and the Family Support Zone: areas outside of the patient room that are oriented to serve visitors.4 The SCCM guidelines for ICU Design do not provide a structured approach to the evaluation of the impact on end-users of construction decisions made based on them.

Evaluating ICU facilities: a unique opportunity

ICUs receive vulnerable patients with high illness severity and little physiological reserve in a complex system composed of multiple providers, sophisticated technologies and frequent time-sensitive interventions. They are often called a ‘hostile environment’ because of data overload, stressful working conditions, poor lighting, high noise levels and lack of privacy.24 ,25 Facility design may negatively impact provider performance and contribute to fatigue, stress and burnout resulting in errors.6 ,17 Conversely, facility design may also play a key role in mitigating some of the hostile aspects of the ICU environment.

Considering that ICU facility infrastructure is a long-term investment, design interventions that impact delivery of care will have repercussions for facility end-users for many years;1 ,19 thus, there is increasing interest and need to investigate optimal design.13 ,19 Despite the large number of healthcare facilities undergoing construction, most healthcare teams will rarely have the opportunity to take part in planning and building a new facility, adding to the importance of developing a standardised and validated process for new facility evaluation.

Study aims

The primary aim of this study was to investigate the impact of relocation to a new ICU facility on clinical performance measures. We define clinical performance measures as process of care indicators and outcomes measures from the perspectives of patients, patient families and healthcare providers. A secondary aim was to develop and test an ICU facility performance evaluation framework that integrates process of care indicators and outcome measures with current design guidelines and widely accepted quality-of-care determinants. The goal is to develop a framework that will complement existing building performance evaluation techniques and help healthcare planners understand how new ICU facilities impact patient care. The specific objectives of this study were:

  • To investigate the impact of relocating to the new ICU facility on 11 coprimary quantitative outcome measures (box 1). We will compare these measures before and after the relocation, as well as the results from control facilities that did not have any structural interventions during the study period.

  • To understand end-users’ impressions, needs and experiences with the new facility design early (2–3 months) and late (12–15 months) after relocating to the new ICU using qualitative methods.

  • To develop, describe and test a comprehensive evaluation framework that integrates a widely accepted quality of care conceptual model with the current functional guidelines for ICU facility design.

Box 1

Quantitative study outcome measures per ICU functional zone*

Patient Care Zone

  1. Number of ICU-acquired infections (Ventilator Associated Pneumonia + Blood Stream Infection >48 h postadmission).

  2. Proportion of ‘Excellent + Very Good’ answers in the FS-ICU question ‘The atmosphere in the ICU was?’.

  3. Reported ‘Device/Equipment/Product’ adverse events, count and rate per 1000 patient-days.

  4. Proportion patients that received neuroleptics (Haloperidol, Olanzapine, Quetiapine) in the ICU.

    Clinical Support Zone

  5. MET (intrahospital emergencies) response time, median in minutes.

  6. Reported ‘Medication/IV fluids’ adverse events, count and rate per 1000 patient-days.

    Family Support Zone

  7. Total score on FS-ICU, median.

  8. Proportion of ‘Excellent + Very Good’ answers in FS-ICU question ‘The atmosphere in the ICU waiting room was?’.

    Unit Support Zone

  9. Nurse absenteeism. Total ‘Sick calls’ hours and proportion of nursing shifts.

    General measures

  10. ICU length of stay, median in days.

  11. Reported adverse event (any) rate per 1000 patient-days.

*Co-primary outcome measures.

FS-ICU, family satisfaction survey; ICU, intensive care unit.

Methods and analysis

This study is a mixed-methods, observational, retrospective, controlled, before-and-after design. The quantitative and qualitative substudies will be conducted concurrently. This manuscript reports on the early components of the project that have already been completed such as the selection of appropriate conceptual models, definition of exposure and outcome status and individual interviews. It also describes the proposed next steps in data collection and analysis for the qualitative as well as quantitative substudies.

Conceptual model

We will use a conceptual model developed by the research team that merges the Donabedian model and the SCCM Guidelines for ICU design (table 2) since no other validated framework exists.

Table 2

Conceptual model for measuring impact of ICU design on patient care*

Quantitative substudy design

Retrospective assessment of clinical performance measures (box 1) will be conducted for 12 months before (1 April 2010 to 31 March 2011) and 12 months after (1 May 2011 to 30 April 2012) relocation of the Foothills Medical Centre Intensive Care Unit (FMC-ICU) to the new facility, which took place in April 2011. Two other general ICUs in the Calgary regional department of critical care medicine, the Peter Lougheed Centre (PLC) and the Rockyview General Hospital (RGH), were identified as controls. These facilities were chosen because they follow the same protocols, have interchangeable physician staff, cover the same patient population catchment area and have not experienced any healthcare construction within the study period.

Identifying exposure components and outcome measures

To assess the clinical impact of a complex, multilayered exposure such as relocation of an ICU to a new facility, we identified specific design elements or features with a plausible link between structure, processes and outcomes. We used a multifaceted approach to identify such features, including a focused literature review, examination of local guidelines for ICU design, clinical judgement and thematic analysis of individual interviews with four key participants who served on the local design and planning committee. Table 2 displays the features that were selected, arranged according to the four ICU zones described in the ‘SCCM Guidelines for intensive care unit design’4 and Donabedian's Quality Improvement approach of Structure, Process and Outcome. Box 1 identifies the data sources for the outcome measures.

Study participants

We will include consecutive patients admitted to one of the study ICUs between 1 April 2010 and 31 May 2012 (12 months before and 12 months after the exposure in April 2011) for the quantitative analysis. We will exclude admissions from April 2011, since it was the transitional month from the old to new unit, and patients may have been exposed to both facilities. The total number of admissions available for analysis will be 5641. Of those, FMC-ICU will have 2801 (1306 before/1495 after), PLC 1508 (735 before/773 after) and RGH 1105 (490 before/615 after).

Data collection and analysis

All quantitative data elements such as admission and discharge time, vital status at ICU and hospital discharge, laboratory values, severity score and demographic attributes will be derived from clinical and administrative databases collected by the regional department of critical care medicine for quality assurance and clinical purposes (see online supplementary additional file I). Statistical analyses will be performed using Stata V.11.2 for Mac (Stata Corp; College Station, Texas, USA). The distribution characteristics of all continuous variables will be visually inspected for distribution characteristics using histograms and other graphic tools. We will report normally distributed variables as means±SD and non-normal variables as medians with IQR. We will compare before and after exposure means and also study and control group means with the student’s t or Mann-Whitney tests as appropriate. For categorical variables, we will assess for differences in frequencies and proportions using χ2 test for multiple groups or Fisher’s exact test (small cell numbers). We anticipate the study groups to have comparable performance indicators and exposure to quality improvement cointerventions. Quality of care improvement trends should also be similar since all three units function within in the same department. However, we expect some differences in the baseline characteristics explained by case mix and the clinical programmes available at each site. We will utilise multivariable logistic regression to investigate the relationship between the proposed study exposure and outcomes. As an example of statistical power to perform our analysis, we expect the proportion of ICU-acquired bloodstream infection will be 5%.26 This will result in approximately 280 cases, allowing for the inclusion of up to 28 variables in the regression model to adjust the estimations of the OR for potential sources of confounding and/or effect modification when we apply the 10 outcome cases per variable rule.27 ,28 The following variables may be included as part of the multivariate regression models as they characterise usual confounders or effect modifiers: age, gender, severity of illness (APACHE II score), admission classification type, admission time and day, workload (TISS 28 score) and the first recorded limitation on goals of care (resuscitative, medical or comfort). The feasibility of using an interrupted time series analysis will be explored when the number of events for process and outcome measures has been determined.

Qualitative substudy design

We will analyse two sets of verbatim interview transcripts performed by one of the authors (MF) using a similar interview guide and audiotaped at two different time points (see online supplementary additional file II). For the early phase interview analysis (2–3 months after relocation), we will have transcripts of previously recorded sessions that were part of a local quality improvement initiative. For the late phase (12–15 months after relocation), we will analyse transcripts of interviews carried out specifically for this project and compare the findings with early end-user impressions. This time lag will allow for adaptation to the new physical environment,19 accounting for both the ‘settling-in’ phase when problems are most frequent and the ‘halo effect’ associated with moving to a new facility.

Study participants

Qualitative substudy participants were recruited at FMC-ICU by one of the authors (MF) during typical shift working hours (day and night shift teams) to facilitate participation. Participants could also schedule a specific time in advance if this was more convenient for them. We utilised a combination of purposeful, convenience and snow-ball sampling strategies to obtain a representative group to answer our research question. The end-user groups included providers (physicians, nurses, respiratory therapists, physiotherapists, social workers, janitors, unit clerks, students) and patient family members. We have 24 interview transcripts in the early phase and 15 transcripts in the late phase.

Data analysis

The qualitative study intends to achieve contextual understanding of impressions and experiences of FMC-ICU end-users. We will utilise traditional qualitative research methods with an iterative and reflexive process; the findings will be inductively derived directly from the analysis of the raw data after multiple readings.29 ,30 This process will involve identifying broad topics and coding them as themes through careful reading and rereading of the data for pattern recognition. Subsequently, themes will be divided into subthemes. Subthemes are defined as common ideas across many participants’ responses, and may describe different aspects of the themes. One author (MF) will code and analyse six transcripts prior to meeting with an experienced qualitative researcher (AH) for auditing and revision of the coding process. This interim analysis meeting is intended to achieve an independent and rigorous process to define, code and synthesise the core ideas expressed by the participants. The results of the thematic analysis will be discussed with participants, aiming at respondent validation of the identified themes and subthemes (member checking). Finally, we will group the themes that emerged according to the SCCM ICU zones to better understand the impact of the design elements within each zone on the end-users’ impressions. This will also facilitate triangulation with quantitative results.

Anticipated impact

This study will contribute to understanding the impact of new ICU facilities on clinical performance measures centred on healthcare providers, patients and their families. This study will produce and test a performance evaluation framework for healthcare environment interventions in ICU facilities that integrates measures of processes and outcomes of care with current design guidelines and widely accepted quality-of-care determinants. This framework will complement existing building performance evaluation techniques and help healthcare planners understand how new ICU facilities impact patient care. In addition, we will develop and implement a multifaceted knowledge translation strategy that will involve engagement of regional multidisciplinary stakeholders in the discussion of the results with unit end-users, this will enhance linkages necessary for dissemination of our results within the Department of Critical Care Medicine. We will publish in critical care medicine and health services research journals. We will also encourage presentation of findings at discipline-based conferences at national and international meetings including International Symposium of Intensive Care and Emergency Medicine and Critical Care Congress of the Society of Critical Medicine among others.

Anticipated challenges

This research project will not encompass the design, construction and occupation phases. This limitation will be mitigated by analysing paper records of the design principles (eg, meeting minutes) that guided the early phases of planning and by conducting interviews with key design planners. A second limitation of this study will be the retrospective design, which will restrict the number and choice of data variables. However, the databases available (see online supplementary additional file I) contain key patient demographic and clinical variables that are appropriate to answer our research question. We will include carefully selected exposure components and outcome measures derived from plausible hypothesised relationships between structural exposures, process of care indicators and outcome measures. In addition, we will adopt a widely accepted quality of care conceptual model that was merged with a novel framework for ICU design, this will enable triangulation of inference using both qualitative and quantitative analyses. Recall bias in the qualitative substudy is another potential challenge. To help attenuate this risk, we will analyse the transcripts of interviews performed both in the early (2–3 months postrelocation) and late (12–15 months postrelocation) phases following relocation of the unit to the new facility.

There could be residual confounding in the relationship between measured health outcomes and the relocation of the new FMC-ICU that is inherent to all observational studies. To minimise this problem, this study will have a controlled before-and-after quasiexperimental design, using concomitant controls to help minimise the effects of secular changes in health services processes and outcomes. Despite these limitations, we believe that the opportunity to study the impact of moving to a new facility on healthcare providers, patients and families will be invaluable to influence planning and construction of future healthcare facilities.


The authors would like to thank Melissa Haveroen for her comments on an earlier version of the manuscript. The authors would also like to thank Alberta Health Services for allowing healthcare providers to participate in the study during their regular work hours.


View Abstract

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:


  • Contributors MF and HTS conceived the research question. HTS is the principal investigator for this study. MF, DAZ, AH and HTS drafted the quantitative and qualitative aspects of the study protocol. MF was responsible for drafting this paper and all authors read, provided important revisions and approved the final version of the manuscript.

  • Competing interests None.

  • Ethics approval The Conjoint Health Research Ethics Board at the University of Calgary approved the study protocol.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.