Exploratory study of staff perceptions of shift safety in the critical care unit and routinely available data on workforce, patient and organisational factors

Objectives To explore bedside professional reported (BPR) perceptions of safety in intensive care staff and the relationships between BPR safety, staffing, patient and work environment characteristics. Design An exploratory study of self-recorded staff perceptions of shift safety and routinely collected data. Setting A large teaching hospital comprising 70 critical care beds. Participants All clinical staff working in adult critical care. Interventions Staff recorded whether their shift felt ‘safe, unsafe or very unsafe’ for 29 consecutive days. We explored these perceptions and relationships between them and routine data on staffing, patient and environmental characteristics. Outcome measures Relationships between BPR safety and staffing, patient and work environment characteristics. Results 2836 BPR scores were recorded over 29 consecutive days (response rate 57.7%). Perceptions of safety varied between staff, including within the same shift. There was no correlation between perceptions of safety and two measures of staffing: care hours per patient day (r=0.13 p=0.108) and Safecare Allocate (r=−0.19 p=0.013). We found a significant, positive relationship between perceptions of safety and the percentage of level 3 (most severely ill) patients (r=0.32, p=0.0001). There was a significant inverse relationship between perceptions of safety and the percentage of level 1 patients on a shift (r=−0.42, p<0.0001). Perceptions of safety correlated negatively with increased numbers of patients (r=−0.44, p=0.0006) and higher percentage of patients located side rooms (r=0.63, p<0.0001). We found a significant relationship between perceptions of safety and the percentage of staff with a specialist critical care course (r=0.42. p=0.0001). Conclusion Existing staffing models, which are primarily influenced by staff-to-patient ratios, may not be sensitive to patient need. Other factors may be important drivers of staff perceptions of safety and should be explored further.

Line 30: You mention "a preliminary study". Add reference or state "unpublished".

Marty van Beuzekom Leiden University Medical Center
The Netherlands REVIEW RETURNED 01-Oct-2019

GENERAL COMMENTS
In general: The study is about exploring, but so many data and correlations are presented that the question Exploring staff perceptions or safety for all professions, on a shift by shift basis, using a bedside professional reported (BPR) safety estimate is not clearly answered, Although the authors indicate this in the Limitations of the study to be unable to explore the meaning behind why staff felt a particular shift was "safe" or not. In a next version of the manuscript It is important.
An exploratory study of staff perceptions of shift safety in the critical care unit and routinely available data on workforce, patient and organisational factors. Methods Although the results do not make a distinction between professional groups, for the interpretation of the results I advise to describe where staff consists of. On page 5 something is said about physiotherapists, but that is not described earlier.
Results I think Figure 1 is not mentioned in the text I have a question: Was examined whether there is a difference between day and night shift and the perception on safety because of the adverse effects or sleep deprivation and fatigue on staff. Interesting to know if this influences the perception of safety. "We noted that staff perceptions frequently varied within the same shift; for only 13% of shifts did all staff who responded report the same perception of safety for their site" What does this mean for performed statistics and the interpretation of the results? The distribution of the BPR shift scores for 170 shifts is illustrated in Figure 1b. But what is the significance of the figure presented that 1.8 times occurs 8. Describing the results below is confusing, there is talk about units (Table 4) and level patients, but you do not know how many, for example, level 1 patients were on unit 1, which makes it difficult to interpret the results "Most responses were green "safe" (61%), 18 % of responses were amber, "unsafe" and 21% of responses were red, "very unsafe". Total responses and responses by site are illustrated in Supplementary Table 4. I think this is an important table Red (very unsafe) for Unit 1 37%, for unit 2: 5% and for Unit 3: 10%. We found a significant, inverse relationship between perceptions of safety and the percentage of Level 1 patients on a shift (p=<0.0001, r= -0.42 Figure 3a) We found a significant inverse relationship between perceptions of safety and numbers of patients in the largest unit (Unit 1) where number of patients ranged between 27 and 32 (figure 4a r=-0.44, p=<0.0006). You can also interpret that from table 4. We did not find a significant relationship between these variables when we included doctors' hours. This is the first time doctors are mentioned, they are not described by characteristic data.
In general many correlations are described and explained in figures, but what is the significance of this, moreover, they are usually moderate correlations with the exception of perception or safety and patient characteristics. Supplementary We have added in more detail in response to this comment.
We also refer to the examples cited in references 3-11.
Page 3, line 11 Lines 22-23: "The mechanisms that mediate these relationships (…)." Did you consider research on quality of the work environment/Nurse work index, e.g. from Aiken et al.?
We have adopted the Reviewers recommendation and included Aikens' conclusions on work environment.
Citations 4 and 5 also refer to Aikens' study of workforce composition which we believe to be the most relevant to this paper.
We thank the Reviewer for identifying this and have included more detail. We accept that the text was confusing.
We have added clarity in two ways.
Firstly, by detailing in page 5 line 8 that we collected number of staff for all professions.
Secondly we have changed the title from Staff Characteristics data to a more specific one "Data regarding nurse training" , line 2 We chose to measure this as it was routinely available; the smaller size of other staffing groups meant this data could not usefully be broken down further.
Page 5, line 8 Page 6 line 3 Page 7, lines 36-37: It may read "We also explored the relationships between (…)." We thank the Reviewer for this helpful rewording and have made this adjustment.
Page 7 line 39 Line 59: "Both studies support an argument that (…)." In the sentence before, you refer to one other study, which is the second study. Please specify.
We thank the Reviewer for identifying this, we accept our original wording was unclear and have changed this. We have included more detail regarding the education program that we have referenced.
We have amended this sentence to include the word unpublished.
Page 9 , line 29 Reviewer Two

General Comment
The study is about exploring, but so many data and correlations are presented that the question Exploring staff perceptions or safety for all professions, on a shift by shift basis, using a bedside professional reported (BPR) safety estimate is not clearly answered, Although the authors indicate this in the Limitations of the study to be unable to explore the meaning behind why staff felt a particular shift was "safe" or not. In a next version of the manuscript It is important.
We acknowledge that the term "explore" can be interpreted in several ways. We have taken the opportunity to clarify this by altering the wording of the first aim objective to "record" and changing the syntax to aid clarity We consider this work exploratory in that it is looking to see if there are relationships between staff perceptions of safety and other factors. This study identifies relationships which may be of interest to the wider community.
Page 4, line 5 Methods Although the results do not make a distinction between professional groups, for the interpretation of the results I advise to We thank the Reviewer for this comment.
We have added more detail to the methods section to make it explicit that staff recorded their perceptions anonymously including not stating their professional group.
describe where staff consists of. On page 5 something is said about physiotherapists , but that is not described earlier.
We address in the limitations that staff did not initially want to record their profession on their responses.

Results
I think figure 1 is not mentioned in the text Figures 1a 1b and 1c  We thank Reviewer 2 for this suggestion. We have since analysed and found that there is no difference between scores during the day versus night.
Page 7 line 30-32 "We noted that staff perceptions frequently varied within the same shift; for only 13% of shifts did all staff who responded report the same perception of safety for their site" What does this mean for performed statistics and the interpretation of the results?
We interpreted the Reviewers comment to mean that he was anxious about the statistical resilience of this observation. In fact, we analysed the data in a number of different ways and have presented these analyses in supplementary Table 5.
If the Reviewer's comment referred to the heterogeneity of response within shifts, this is already discussed in page 8 line 10.
The distribution of the BPR shift scores for 170 shifts is illustrated in Figure 1b. But what is the significance of the figure presented that 1.8 times occurs 8.
We have reviewed figure 1b and do not understand the question posed.
Describing the results below is confusing, there is talk about units (Table 4) and level patients, but you do not know how many, for example, level 1 We acknowledge the description of the hospital structure could be confusing. The units are wards within a single institution, though, they vary by speciality they share staff, policies and deployment and processes. We found a significant inverse relationship between perceptions of safety and numbers of patients in the largest unit (Unit 1) where number of patients ranged between 27 and 32 (figure 4a r=-0.44, p=<0.006). You can also interpret that from table 4 We have reworded the methods section to clarify this.
We were primarily interested in the relationships between staff perceptions of safety and levels of patients . During the study period there were higher numbers of Level of 1 patients in one of the wards , however we were not able to perform linear regression or more complex mathematics to separate out the impact due to the very uneven distribution of Level 1 patients between wards. Thus we cannot comment on the statistical interaction of "ward" and "level of care".
We did not perform subgroup analysis that is perhaps suggested as this would have resulted in small subgroups, insufficient for statistical analysis. We have added an acknowledgement of this to the limitations section. This is something we believe other researchers may want to explore.
We feel, as raised in the discussion, that the results indicate that staff deployment by ratios, informed by the numbers of organs in failure, does not seem to hold.
Page 9 Line 21-23 We did not find a significant relationship The Reviewers comments highlight that we have not signposted clearly enough in the page 5 Lines 8 and 18 between these variables when we included doctors' hours. This is the first-time doctors are mentioned, they are not described by characteristic data.
methods section where we included other staff: We have added more detail into page 5 lines 8 and 18. We have also reworded the subheading "staff characteristic data" to "Data regarding nurse training" page 6 , line 3 to give greater clarity .
page 6 Line 3 In general many correlations are described and explained in figures, but what is the significance of this, moreover, they are usually moderate correlations with the exception of perception or safety and patient characteristics.
Supplementary Table 5 is easier to read that all figures.
We have used figures as we feel most clearly displays the original data and is therefore the most transparent method. We have included Tables in the supplementary data to summarise the findings for the readers convenience.
We have also re-ordered and clarified the bullet points in implications to emphasize the significance of the findings more intuitively to the reader.