Article Text

PDF

Staffs’ and managers’ perceptions of how and when discrete event simulation modelling can be used as a decision support in quality improvement: a focus group discussion study at two hospital settings in Sweden
  1. Helena Hvitfeldt-Forsberg1,
  2. Pamela Mazzocato1,
  3. Daniel Glaser1,
  4. Christina Keller2,
  5. Maria Unbeck3,6
  1. 1 Department of Learning Informatics Management and Ethics and Medical Management Center (MMC), Karolinska Institutet, Stockholm, Sweden
  2. 2 Department of Informatics, International Business School, Jönsköping, Sweden
  3. 3 Departments of Clinical Sciences, Danderyd Hospital, Karolinska Institutet, Stockholm, Sweden
  4. 6 Department of Orthopedics, Danderyd Hospital, Stockholm, Sweden
  1. Correspondence to Dr. Helena Hvitfeldt-Forsberg; helena.hvitfeldt.forsberg{at}ki.se

Abstract

Objective To explore healthcare staffs’ and managers’ perceptions of how and when discrete event simulation modelling can be used as a decision support in improvement efforts.

Design Two focus group discussions were performed.

Setting Two settings were included: a rheumatology department and an orthopaedic section both situated in Sweden.

Participants Healthcare staff and managers (n=13) from the two settings.

Interventions Two workshops were performed, one at each setting. Workshops were initiated by a short introduction to simulation modelling. Results from the respective simulation model were then presented and discussed in the following focus group discussion.

Results Categories from the content analysis are presented according to the following research questions: how and when simulation modelling can assist healthcare improvement? Regarding how, the participants mentioned that simulation modelling could act as a tool for support and a way to visualise problems, potential solutions and their effects. Regarding when, simulation modelling could be used both locally and by management, as well as a pedagogical tool to develop and test innovative ideas and to involve everyone in the improvement work.

Conclusions Its potential as an information and communication tool and as an instrument for pedagogic work within healthcare improvement render a broader application and value of simulation modelling than previously reported.

  • Simulation modeling
  • healthcare improvement
  • focus group
  • rheumatology
  • orthopedic care
  • emergency care

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Strengths and limitations of this study

  • Customised simulation models were developed by the research team to address the specific needs of two hospital settings.

  • Focus group discussions were conducted in close connection to real healthcare improvement efforts.

  • The trustworthiness of the findings was strengthened by the fact that trained facilitators conducted the focus group and used a moderator guide, and that two researchers conducted the qualitative analysis.

  • The study includes two clinical settings, which limit the transferability of the findings.

Introduction

Healthcare improvement can be defined as joint efforts to improve patients’ health, healthcare operations and staff development.1 The complexity of healthcare improvement includes the size and scope of the initiative and the numerous contextual factors, which have caused the outcomes of healthcare quality improvement to be questioned.2–4 Improvement work often builds on testing changes, initially on a small scale, to foster knowledge of how changes are implemented and affecting practice.5 6 This strategy may make it difficult to predict outcomes not in close connection in time or space.7

Computer simulation modelling, such as discrete event simulation of healthcare systems and processes, has been proved as a valid tool for attending to problems in healthcare such as resource allocation, patient flows, epidemiological concerns and utilisation of resources.8–12

All types of simulation aim to imitate reality to test, educate and increase learning. Regardless of the many models developed to elucidate important healthcare related issues, little research has been done on the use, implementation and value of simulation in the everyday context of healthcare.13–15 However, research has shown that simulation modelling can enable informed decision making, develop system knowledge, determine critical factors for the development of an organisation, supply scenario analysis and options to choose from, help understand complex problems, facilitate communication and form plans and directions for future work.11 16

In the context of healthcare improvement, simulation modelling can help generate not only diagnostic data, but also knowledge and perspectives that can lead to predictive capacity.7 Even so, the uptake of simulation modelling and other operation research tools have been slow in healthcare rendering few actual model implementation cases to learn from.13 More research is needed to fully understand the impact and value of simulation modelling on the improvement of healthcare.8 12 14 15

The aim of this study was to explore healthcare staffs’ and managers’ perceptions of how and when discrete event simulation modelling can be used as a decision support in improvement efforts.

Method

Study design and setting

A qualitative study design was chosen to explore healthcare staffs’ and managers’ perceptions of discrete event simulation modelling, hereafter called simulation modelling, in two healthcare organisations: a rheumatology department and an orthopaedic section at a central surgical unit (which includes an orthopaedic department, an anaesthetic and intensive care unit and a surgical and urological department). The two units were part of two hospitals located in two different county councils in Sweden. An overview of the two organisational settings is presented in table 1.

Table 1

Overview of the included organisational settings

Study participants

Purposeful sampling was used to select the study participants. Thus, all healthcare staff and managers involved in the project groups of the respective ongoing improvement efforts were deemed suitable to participate in this study because of their knowledge of the routines and needs of the organisations. All potential study participants were contacted via telephone and email. Nine employees at the rheumatology department were invited. Of those, seven agreed to participate in the study. Two declined the invitation due to time constraints. At the orthopaedic section, eight employees were invited. Of those, seven accepted the invitation and one of those had a late cancellation due to acute illness and one declined due to a planned vacation. An overview of the study participants is presented in table 2.

Table 2

Study participants

Data collection

Data were collected through two focus group discussions (FGDs) at the two units. Focus groups can be defined as organised, interactive group discussions that aim to explore a certain topic.17 The method was chosen because focus groups are quite suitable when investigating experiences, attitudes and emerging ideas from a group.18 19 According to Morgan,20 p2 “the hallmark of focus groups is their explicit use of group interaction to produce data and insights that would be less accessible without the interaction found in the group.”

For both cases, customised simulation models were developed by the research team to address the units’ specific needs. At the rheumatology department, improvement efforts aimed to improve the referral process for newly diagnosed patients. At the orthopaedic section, improvement efforts aimed to improve access to care for patients with hip fracture. Both contexts had previous research and improvement collaborations with the authors.

The process of developing the two simulation models for the two contexts was somewhat distinct. In the rheumatology department, where authors had been involved in the improvement work, the participants were introduced to simulation modelling during that work as an optional tool for improvement. During a workshop, they had the opportunity to test a simulation model representing the processes in focus of the ongoing improvement and its included variables. Immediately after the workshop, the FGD was performed.

In the orthopaedic context, the participants were included in the simulation model building and validation of the simulation model during several meetings. The workshop and FGD were held at the end of the project time.

In both cases, the workshops began with a short introduction to simulation modelling and the researchers focus and questions. The researchers then presented the findings of the simulation model and led the FGDs on how and when simulation modelling could be used as a decision support in improvement efforts.

The FGD was conducted at the respective unit during 1 hour, and two trained researchers acted as moderators (CK and HHF in the rheumatology case and HHF and PM in the orthopaedic case). The more experienced researcher moderated the FGD and the other took notes and asked follow-up questions. The interviews were also recorded.

Before the initiation of the FGD, the participants were informed that the model developer would only be observing the FGD to let the participants freely discuss the model given their experience. The FGDs were initiated by asking all participants to comment briefly on their experience of the simulation model and their work with their specific questions. This ensured that all participants had a chance to share, and that the recordings included each participant. Following this introduction, the moderator’s questions guided the rest of the FGD. The following themes were covered in the moderator guide:

  • How was your experience of using the simulation model and the results?

  • What problems/issues/questions are suitable to address with simulation modelling in healthcare?

  • How can simulation modelling connect to improvement work?

  • What are the values and trustworthiness of simulation modelling?

Data analysis

The FGDs were transcribed verbatim by transcribers, which were then analysed using qualitative content analysis. Content analysis is commonly used in social sciences21 and is a systematic analysis of text.22 Two researchers (HHF and PM) conducted the coding and categorisation together, following the three general steps of performing content analysis outlined by Graneheim and Lundman.23 The transcripts were read through before meaning units were extracted and coded, and, finally, the codes were organised into categories. Each step was performed individually before collaboratively reviewing the analysed content and reaching a consensus.

Findings

Results presented below are organised according to the different categories derived from the qualitative content analysis.

When can simulation modelling be used?

Improvement support

The FGD participants described simulation as a tool that could be used to motivate change for staff and management. This included changes that present challenges when implementing because of budget constraints, big investments, staff resistance or lack of consensus. Simulation modelling can support change implementation by visualising both problems and potential solutions. Simulation completes the picture of the limited financial or process aspects of change by visualising effects that are different dimensions of the same change.

Test and evaluate ideas for improvement

Participants expressed repeatedly in both FGDs that simulation modelling could be used to evaluate the effect of changes that have either been implemented or that are under development or at an idea stage. Simulation modelling was considered a quicker and more efficient way to test new ideas, compared with testing in reality. Furthermore, changes can be tested without influencing patient care.

“It is exciting to be able to test a hypothesis in a computer environment to see what results you can anticipate,” said one FGD participant. “It is often difficult to test changes in real life; it takes time, costs money and results are sometimes uncertain.”

Participants also emphasised that simulation modelling could more systematically test change ideas and evaluate them, rather than just implementing changes without following up. In addition, they also shared how the model could be used as a pedagogical tool to develop and test ideas that could motivate staff to get engaged and involved in the improvement work.

“It's a pedagogical tool to use in our work team in improvement work. By thinking that nothing is forbidden to suggest, we can test many different things and start thinking outside the box. It is so easy to get stuck in patterns,” another participant commented.

Moreover, testing of ideas got some of the participants to think about existing work routines, and new improvement ideas were created, while previous ideas were rejected.

“We thought that if we just had more rooms, all our problems would go away,” a participant said. “But we saw no substantial change when we tested it in the model. It all depends on the number of doctors that are available after all; that is the answer we got. We even saw that, right now, we are not using all the rooms efficiently.”

How can simulation modelling be used?

Questions suitable for simulation modelling

Simulation modelling can be used to address concrete and measurable questions, including questions related to planning, resource allocation and staff scheduling. One area of application that was raised several times during the FGDs was related to cost efficiency.

“The future will be more about budgets and finances due to our expensive treatments. This [simulation model] can be a way to present financial figures to the management,” commented one participant.

“It is easier to connect numbers with a cause rather than to just talk about the cause itself,” said another participant.

This [simulation model] includes showing the cost efficiency of changes implemented and to make staff more aware of costs related to certain operations. Further on, it can be used to find ways to use resources more efficiently.

Validity

Participants stressed the importance of building the model from valid and reliable data. Using incorrect data or data in an incorrect way will undermine the simulation model and the staff´s trust in the model. The simulation model must reflect the organisation and real processes and data. Input from staff is important to ensure the validity of the model as they are the ones with knowledge of processes and operations.

As one participant put it, “The method is fine; the difficult part is what you put into the model.”

Participants expressed that to use any simulation model, it is important that all processes and logics incorporated in the model are well known to the user. This is to prevent misinterpretation of the model output and results. Relying on and trusting the model requires a deep understanding of the model and participation during the process of building the model.

“Users must be familiar with the model in order to critically evaluate results. [It’s] dangerous if results are interpreted in the wrong way,” expressed one participant.

Conducting simulation modelling projects

Results from the FGDs revealed two aspects of simulation modelling use: local use at the clinic in their improvement work and to guide management on planned changes or responses to changes suggested by management. Staff must be involved from the beginning to inform the building of the model with data and questions to test in the model. Involving staff from the beginning also helps develop trust in the simulation model.

“A simulation project must be approved by management and well supported by the staff,” said one participant.

“In improvement work, involvement from everyone is essential, perhaps simulation modeling can act as a tool to inform and inspire colleagues,” commented another.

Simulation modelling projects must be approved by management and staff and be well communicated within the organisation. The group working with the simulation model cannot be too big and there must be a person in charge of the work. The importance of using the simulation model together, in multidisciplinary teams, was emphasised. Specific persons holding positions, such as schedulers, nurse managers and heads of departments were mentioned as potential users.

“Simulation modeling should be used locally, at our clinic and by us,” a person said. “It should be used to show how we work and our results, at meetings and externally as marketing.”

Another participant described the simulation model as, “a good tool to show management what we do and our plans.”

Value and opportunities

Overall, participants expressed a positive attitude towards simulation modelling. They identified different opportunities for simulation modelling to add value to the development of their organisation. These include identifying trends and understanding complex relationships between processes and systems. Moreover, simulation can help the staff face future challenges collaboratively.

The following quotes help illustrate the overall outlook of simulation modelling among the participants:

“Simulation modeling at the right time can be very valuable. When initiating improvement, it is beneficial to see how small changes can have great effects.”

“You get an overview; you see the overall picture that can be used to stimulate improvement work.”

“Simulation modeling can help you find new models and aid the individual learning when seeing the relation to their respective work. To the organization, it might be more on how to accomplish goals and see how the overall picture is affected.”

Discussion

This study sets out to explore healthcare staffs’ and managers’ perceptions of how and when simulation modelling can be used as a decision support in improvement efforts. Simulation was described as a tool that could be used to evaluate and develop improvement ideas and help motivate the need to implement certain changes both for staff and for management. Also, simulation modelling can motivate difficult change by visualising effects and also financial aspects. Simulation modelling was best valued as a way to address concrete and measurable questions related to planning, resource allocation, staff scheduling and cost efficiency. Two areas of simulation modelling use in healthcare improvement were stressed in the two focus groups: locally in the clinical improvement work and to guide management on planned changes. Also, the early involvement of staff in the simulation project and use of correct data to validate the model is crucial for staff to trust and use the model.

When comparing the results to previous literature on simulation modelling, there are several evident similarities. Concerning the potential users of the simulation model, at the clinical level or in management, it is important to consider model knowledge and the ability to present it to others, data availability and support from leadership.14 Regarding testing and exploring what is outside of the box, Pidd describes simulation as a vehicle for experimentation where trial and error can be performed without concerns for reality but with a great opportunity to learn.23 Simulation modelling offers a holistic view on addressing change and improvement in complex systems and its inherent components by displaying the effects of change immediately.7 Visualising the problem and a potential solution was, in the FGDs, considered powerful in engaging healthcare staff in improvement work. This interactive opportunity has been proposed to help motivate staff to embrace change.7 However, the formation of a representable clinical team to facilitate the simulation modelling work is not always easy.15 Our first simulation focus group revealed a wish to include management, and the second focus group wanted to include the staff further. Coproducing the model with staff, managers and modellers, also known as facilitated modelling, are important to build a step-by-step understanding of the model logic and its validity.24–29 The early and continuous involvement of stakeholder in a structured model development process increases stakeholders trust in the model.30 Depending on the identified users, simulation models can be used to aid communication in creating a shared mental model.26 31–33

Drawing from our results, we can see values emerging from simulation modelling, such as providing a way to work with change and improvement (especially when initiating improvement work), and visually communicate planned changes, operations and the subsequent consequences. Simulation modelling can be a tool to be used at different levels of healthcare, but according to our findings, it might be best suited for local clinical improvement and management planning and allocating resources. Before initiating a simulation modelling project, the formation of the project team is essential for ensuring model validity and reliability. Bringing in the aspects of implementation, evaluation and research requires even greater consideration of the project team, but might enable the transition from in silico to in reality and create value for the healthcare organisation.

Limitations in our study design include the fact that we have drawn conclusions from limited empirical data gathered from two FGDs only. Nevertheless, an exploratory study like this could serve as a pilot for further research. All staff and managers involved in the improvement project teams were invited to attend the FGDs and thus the concept of saturation was not applicable. FGDs themselves can be subject to different impediments if not moderated well. The group dynamics are essential for allowing everyone to talk and present their opinion.18 34 35 Using trained facilitators, a moderator guide and introductory questions for everyone to answer may have helped participants feel free to share their perceptions openly.

Conclusions

The emerging categories from the content analysis are presented according to the research questions of how and when simulation modelling can assist healthcare improvement. Regarding how, the participants mentioned that simulation modelling can act as a tool for support and a way to visualise problems and potential solutions, as an information and communication tool to show management planned improvements and their effect. Simulation modelling could be used both locally and by management, but the user should be familiar with model logic and data to interpret results correctly. Relating to when, participants thought simulation modelling could be used as a pedagogical tool to develop and test ideas, to think outside the box and to get everyone involved in the improvement work.

This study showed that simulation modelling has more to offer than has been described in previous literature. Its potential as an information and communication tool, an instrument for pedagogic work within healthcare improvement and as a way to allow thinking outside the box render a broader application and value of simulation modelling.

Supplementary Material

Supplementary data

Acknowledgments

The authors wish to thank the involved units for their participation in this study.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
View Abstract

Related Data

Footnotes

  • Contributors HHF, PM, MU, DG and CK designed the study. HHF, PM, MU and CK collected the data. HHF, MU, PM and CK drafted the manuscript. Everyone have been involved in reading and critically revising the manuscript. All authors approved the final manuscript and are accountable for all parts of the work.

  • Funding The study was supported by grants provided by the Stockholm County Council (ALF project).

  • Disclaimer The funder had no involvement in study design, data analysis and manuscript preparation or publication decision.

  • Competing interests None declared.

  • Ethics approval Stockholm Ethical Review Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi:10.5061/dryad.5f5m7.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.