Article Text

Original research
The kindness COVID-19 toolkit: a mixed-methods evaluation of a programme designed by doctors in training for doctors in training
  1. Madeleine C Ward1,2,
  2. Karen Crinall3,
  3. Rebecca McDonald1,2,
  4. William Crinall3,
  5. James Aridas2,4,
  6. Cheryl Leung2,
  7. Danielle Quittner2,
  8. Ryan J Hodges1,2,
  9. Daniel L Rolnik1,2
  1. 1Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
  2. 2Women’s & Newborn, Monash Health, Clayton, Victoria, Australia
  3. 3Crinall Consulting, Melbourne, Victoria, Australia
  4. 4The Ritchie Centre, Hudson Institute of Medical Research, Clayton, Victoria, Australia
  1. Correspondence to Dr Madeleine C Ward; madeleine.ward{at}monash.edu

Abstract

Objectives The impact of a coronavirus disease (COVID-19)-specific professional development programme on the well-being of obstetrics and gynaecology (O&G) doctors in training (DiT) working during the pandemic.

Design A mixed-method evaluation of a single group pre–post test design study.

Setting Melbourne, Australia between September 2020 and April 2021.

Participants 55 O&G DiT working across four healthcare sites of a major tertiary hospital in Victoria, Australia, were included in the programme.

Interventions The delivery of a codesigned peer-to-peer programme, which identified and addressed the well-being goals of O&G DiT. Seven interactive workshops were run alongside the implementation of a number of participant-led wellness initiatives.

Main outcome measures Repeated-measures analysis of WHO Well-being Index (WHO-5) and Copenhagen Burnout Innovatory (CBI) scores across three time points during the programme. Multilevel generalised linear mixed-effects models with random intercept were fit to the data, both in the entire population (intention-to-treat) and restricted to those who attended the workshop (‘per-protocol’ analysis). Participatory experiences and programme learning were captured using the Most Significant Change (MSC) technique, which included inductive thematic analysis.

Results We demonstrated an overall 31.9% improvement in well-being scores (p=0.006). The MSC evaluation captured a shift in workplace culture as a result of the programme, with improvement across the domains of connection, caring, communication, confidence and cooperation.

Conclusions We have successfully used a mixed-method approach to contextualise a productive programme to improve the well-being of COVID-19 front-line healthcare workers.

  • COVID-19
  • obstetrics
  • qualitative research

Data availability statement

Data are available on reasonable request. We commit to undertaking all reasonable requests to share relevant data. Deidentified raw data are securely stored within Qualtrics and with the research team. Open access evaluation reports are available at: https://leadingkindnesscovid19toolkit.wordpress.com/

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • Uses a mixed-method evaluation, enhancing the interpretation of outcomes and adding valuable information about the impact, acceptability and utility of the intervention.

  • Provides a detailed description of the peer-led and peer-to-peer participatory processes used to affect a codesign approach to programme development.

  • Use of the Most Significant Change methodology, a powerful qualitative evaluation tool which provokes broad reflections in participants and provides an avenue for organisational leadership to initiate improvements and build capacity.

  • Limitations included sample size and generalisability, with a need to further validate the programme via larger studies with the inclusion of other populations.

Introduction

The novel coronavirus disease (COVID-19) pandemic has had an unparalleled impact on the provision of healthcare, resulting in significant physical and emotional burden on those accessing and servicing the acute hospital setting.1 2 It has created further pressure on an already vulnerable group of doctors with additional risks from exposure to an infectious disease, reduced resources and high workload.1 3

Burnout is a recognised occupational hazard for healthcare workers and encompasses emotional exhaustion, withdrawal from patients and loss of job satisfaction.4 Ensuring the health of workers is vital for the delivery of high-quality service, with healthy workers demonstrated to provide better customer relations, have more positive attitudes and be more enthusiastic.3 5 In comparison to other professionals, doctors are 10 times more likely to suffer from psychological distress, especially those under the age of 30.6 7 These findings are not isolated to the Australian medical workforce, with 69% of US healthcare workers reporting workplace stress and 37% of UK doctors requiring additional care due to burnout.4 More than a decade ago, a national report highlighted the significant distress of Australian doctors in training (DiT) with less than 30% satisfied with their career, over half (54%) losing compassion for patients and more than two-thirds (69%) experiencing burnout.8 Obstetrics and Gynaecology (O&G) DiT work long hours and shift work, which are known to contribute to dissatisfaction and reduced well-being, as well as occupational stress, burnout and mental health issues.3

Despite safety and quality healthcare indicators being drivers for accountability, there is a failure to recognise the need to support the well-being of healthcare workers.7 In recognition of the immediate risk to the well-being of the Women’s Health DiT group and associated impacts on health service provision, we aimed to provide an appropriately resourced, efficient and effective COVID-19 pandemic-specific professional development programme focused on improving the well-being of O&G DiT. A programme, aimed at informing the future development and implementation of similarly focused well-being initiative, was codesigned by the O&G DiT group and delivered via a peer-to-peer (P2P) teaching model. The aims were to:

  1. Assess the well-being and symptoms of burnout among the Monash Women’s DiT in O&G.

  2. Provide immediate and practical tools and strategies to enhance the well-being of Monash Women’s DiT working during the COVID-19 pandemic.

  3. Generate an evidence base, informed by qualitative and quantitative data, to advise future implementation.

Monash Women’s, a major tertiary hospital in Victoria, Australia, has been a leader in the implementation of COVID-19 strategic management plans, policies and procedures with the safety and well-being of staff, patients and the community at the forefront.9

The aim, specific to the mixed-methods analysis, was to evaluate and explore the impact of the COVID-19-specific well-being programme, as a whole, on the cohort of O&G DiT.

Methods

Here we list the steps involved in the P2P programme development, including the details of the codesign processes followed. The integration of a mixed-methods analysis is described, an approach which takes advantage of the considerable impact qualitative research methods can have on reviewing health interventions.10 A quantitative evaluation was conducted on the WHO Well-being Index (WHO-5) and Copenhagen Burnout Innovatory (CBI) measures that were administered prior to and at multiple time points following the programme.11 12 Qualitative analysis using the Most Significant Change (MSC) technique is also described and the key outcomes listed.

Patient and public involvement

All 55 Monash Women’s DiT working across four sites—one tertiary and three secondary hospitals (Monash Medical Centre, Dandenong District Hospital, Moorabbin Health and Casey Hospital)—were invited to participate in the study. Our study used a convenience sample size without a formal power calculation. Recruitment was maximised via advertising using group email. An introduction and orientation session was held with opportunity to complete recruitment at this time, and reminder emails were sent to increase the uptake. Recruitment at the beginning of the first workshop also took place. Inclusion criteria included being a current O&G DiT who were both available and willing to attend the workshops and complete the questionnaires.

Intervention

An overarching self-selected group of senior DiT, known as ‘champions’, initiated the project, formulated broad goals (figure 1) and directed activities in line with those promoted by Beyond Blue ‘Protecting your mental health and wellbeing as a healthcare worker’.13 14 Monash Women’s executive leaders were also engaged to participate and act on organisational level solutions. A codesign process and P2P learning model were identified as essential to the effective development and implementation of the proposed programme. A multiformat toolkit was developed, incorporating workshops, activities and resources.

Figure 1

Goals of the ‘Monash Women’s leading kindness COVID-19 toolkit’. O&G DiT, Obstetrics and Gynaecology doctors in training.

Seven online workshops, each of 1-hour duration, were delivered by DiT to their peers during protected teaching time on 17 September 2020, 24 September 2020 and 5 November 2020. This model was selected to maximise positive impacts and enable rapid implementation of change.13 15 It is also a familiar style of learning given its similarity to the traditional ‘journal club’ style of medical education frequently used in the training of Australian medical doctors.

Workshop topics were modelled on the ‘Pyramid of needs’, for health workers’ well-being (figure 2) (based on Maslow’s hierarchy of needs),14 15 and covered six topic areas: basic needs, safety, love and belonging, esteem, contribution and leadership. The seventh workshop was devoted to reviewing topics presented. Workshop content dissemination was maximised with recordings, webmail links and online communities. In keeping with the codesign approach, brainstorming sessions were included, and ideas generated were disseminated to the O&G DiT cohort via email and posters. Participant-led initiatives were also encouraged and developed during the workshops, and advocating for personal solutions was supported.

Figure 2

Pyramid of needs. Adapted from: Pandemic Kindess Movement.16

The first workshop addressed meeting basic needs and was led by a more senior DiT. The session provided space for participants to acknowledge the importance of hydration, nutrition, shelter and sleep. Facilitated breakout groups worked together to share how they were being impacted by the COVID-19 pandemic and ideas on how improvements at both an individual and organisational level could be achieved. These ideas were central to informing the larger goals and actions of the programme, including the development of the policy on dealing with doctors in distress.

Activities involved the provision of a drink ‘hydration’ station to encourage breaks, improvements to the doctor office space and social online sessions by Zoom Cloud Meetings (Zoom Video Communications, San Jose, California, USA).

Resources included the creation and display of posters articulating the main workshop points and a team social media app. Further information on the programme, including a workbook and templates can be found in resources: ‘Monash Women’s Leading Kindness COVID-19 Toolkit Pilot Project Most Significant Change Evaluation Report’16; ‘Monash Women’s Leading Kindness COVID-19 Toolkit Pilot Project Quantitative Evaluation Report’ and ‘Start up: A Kindness COVID-19 Toolkit’.17 18

Evaluation strategy

Quantitative analysis

The CBI and WHO-5 questionnaires, both well-validated and standardised, were employed to evaluate the pre-existing well-being and symptoms of burnout in the cohort O&G DiT (September 2020, timepoint 1).12 14 Both were repeated at timepoint 2 (November 2020) and at timepoint 3, 6 months following the completion of the workshop (April 2021). Questionnaires were delivered and secured electronically via the online data collection tool, Qualtrics (Qualtrics, Provo, Utah, USA), with surveys labelled with an anonymous but memorable code, created by the participant, to preserve confidentiality.

The WHO-5 is a standardised questionnaire, which asks five questions focused on well-being in the preceding 2-week time period (online supplemental appendix 1). A total score of less than 50 is considered to represent reduced well-being.11 The CBI assesses the load of personal, work and patient-related factors on burnout, with its benefits being a readily available and brief evaluation tool. A score of 25–50 indicates an intermediate level of burnout, and a score of >50 indicates a high level of burnout.19

Statistical analyses were performed in Stata 16.1 (Stata 2019, Stata Statistical Software: Release 16; Stata, College Station, Texas, USA). To compare the CBI and WHO-5 scores after exposure to the programme (timepoints 2 and 3) with those before its implementation (timepoint 1) accounting for the repeated-measures design, multilevel generalised linear mixed effects models with random intercept were fit to the data, both in the entire population (intention-to-treat) and restricted to those who attended the workshops (‘per-protocol’ analysis). In these models, the measurements’ timepoints were treated as fixed effects and participants were treated as random effects. The assumptions of linear additivity and homogeneity of the residuals were assessed by inspection of residual versus fitted plots. Effect estimates are reported as the mean difference with 95% CIs, and p<0.05 were considered statistically significant.

Qualitative analysis

Qualitative evaluation using the MSC technique was chosen to gain insight into participants’ experience of the personal and professional impacts of the toolkit. Story-gathering interviews took place via Zoom sessions between 8 December 2020 and 4 January 2021 (online supplemental appendix 2). Nine stories of change were collected from interviews that lasted between 20 and 40 min. To ensure rigour and impartiality, external expertise in qualitative analysis was engaged. MSC was chosen because it seeks to learn about participants’ perceptions of programme impacts by evaluating their stories of significant change, and thereby complemented the programme’s participatory principles: codesign, peer leadership and P2P learning. MSC was also selected because of its focus on ‘what works well and how to do more of what works’.20

Consistent with the MSC technique, the evaluation incorporated three main steps:

  1. Collection of Significant Change stories via interviews.

  2. Selection of the MSC story by a stakeholder panel, and identification of key themes through manual inductive analysis of MSC stories and selection panel discussion.

  3. Documenting and communication of the MSC story and key themes.

The MSC technique involves a hierarchical selection process which narrows the data down to one story representing the MSC. A stakeholder panel consisting of trainees and supervisors, as well as interview participants who were contributors of stories of change, undertook a two-tiered process culminating in the selection of one significant story from a total of nine stories (figure 3). Prior to the selection panel, the nine stories were randomly divided into three groups of three. Stories from each group were then read aloud in the panel session and were shortlisted via an open voting process. Individual reasons for the selection were shared and recorded through open discussion, and agreement was reached on one MSC story.

Figure 3

Most Significant Change (MSC) selection process.

To gain a comprehensive picture of the range of significant changes experienced by participants, a manual inductive analysis of all nine stories and the transcript of the selection panel session discussion was conducted to capture emergent themes. This methodology is discussed in detail in other publications.16 21 22

Results

Programme overview

The programme, which was evaluated as a whole rather than the individual components, included seven 1-hour live remote workshops (covering each of the six themes and one review session); circulated recordings of the workshops; three online social sessions; a hydration station stocked with drinks for each work site; six laminated wall posters with the main concepts from the workshops posted at each work site; a senior trainee education session on supporting junior trainees; a meeting with senior management advocating for well-being initiatives; a business proposal for well-being officers; renovation of the doctor’s office spaces and the development of a social media app.

Participants

Forty-six (83.6%) DiT completed the initial WHO-5 and CBI at timepoint 1. Seventeen DiTs, including residents and registrars, attended at least one of the live workshops, and the recorded workshops were circulated to all 55 DiT; however, it is not known how many viewed them in their own time. All 55 DiT were exposed to the initiatives of the programme; however, specific details of their uptake are unknown. Following the completion of the workshops (timepoint 2), 27 responses were collected, of which 59.3% (n=16) were workshop participants (live or recorded viewing). At timepoint 3, 11 responses were collected, with 63.6% (n=7) being participants of the workshops.

Quantitative analysis: CBI and WHO-5

Copenhagen Burnout Inventory

Among all DiT (those who participated in the workshop and those who did not), there was a mean reduction of 2.0 points (95% CI: −7.4 to 3.3) at timepoint 2 compared with timepoint 1, although this reduction was not statistically significant (p=0.454) (figure 4). There was a mean reduction of 6.8 points (95% CI: −14.4 to 0.7) at timepoint 3 compared with timepoint 1, demonstrating a trend toward statistical significance (p=0.077). Similarly, among those who participated in the workshop, there was a mean reduction of 0.3 points (95% CI: −6.8 to 6.2) at timepoint 2 compared with timepoint 1, although this reduction was not statistically significant (p=0.935). There was a trend toward statistical significance (p=0.086) at timepoint 3 compared with timepoint 1 with a mean reduction of 8.2 points (95% CI: −17.5 to 6.2).

Figure 4

Repeated-measures Copenhagen Burnout Innovatory (CBI) and WHO Well-being Index (WHO-5) scores for all and restricted to those who participated in the ‘Monash Women’s leading kindness COVID-19 toolkit’ workshops.

WHO Well-being Index

Among all participants there was a statistically significant mean increase of 10.5 points (95% CI: 3.3 to 17.7, p=0.006) at timepoint 2 compared with timepoint 1, and a statistically significant mean increase of 14.9 points (95% CI: 0.5 to 29.3, p=0.006) at timepoint 3 compared with timepoint 1 (figure 4). Analysis restricted to those who participated in the workshop showed a mean increase of 9.2 points that trended toward statistical significance (95% CI: −0.2 to 18.5, p=0.054) at timepoint 2 compared with timepoint 1, and a significantly higher WHO score at timepoint 3, with a mean increase of 16.4 points (95% CI: 3.2 to 29.7, p=0.015) at timepoint 3 compared with timepoint 1.

Qualitative analysis

The complete methodology and outcomes of the qualitative analysis have been published elsewhere, and we list a summary of the findings below.16 17

The MSC story

The selected story ‘Team cohesiveness’ (online supplemental appendix 3) was contributed by a junior O&G DiT workshop participant. It was chosen by the panel because it described how the COVID-19 toolkit brought about cultural change and fostered a sense of kinship by enabling basic needs to be met, “break[ing] down hierarchical barriers within the Monash Women’s Health team” and building team cohesiveness multidirectionally.

With the introduction of the Wellbeing program there was a more organised sense of looking out for each other … A highlight was senior clinicians telling their own stories … Witnessing their fears and concerns, and their approaches to challenges makes you more impressed by their achievement, you feel like challenges are more approachable, the steps ahead are more attainable … The program was also an opportunity to address the things that make a cohesive team, that make us all better together.

Themes

Significant changes linked to five interconnected themes (figure 5) were woven throughout the nine stories and encapsulated in the MSC story selected by the panel.

Figure 5

Themes revealed by stories which came together to shift the workplace culture.

Connection: Connections between DiT were strengthened through the P2P learning model of the workshop design, with one participant reflecting that “hearing others talk about their experiences and feelings of not being okay and sharing my experiences and feelings … made me feel more connected and less alone.” Another participant observed: “One of the most important things to come out of the program during the pandemic was being closer to colleagues that I don’t work with every day.”

A participant story described a significant change for them as “a noticeable physical difference … the revamp of our doctors’ office, there is new furniture, and plants in there now, it is fresh and more open. We feel welcome; now I’ve somewhere that I belong at work.”

Communication played an important role in fortifying connections. The impact of the workshop session which explored ‘Esteem’ was recalled as motivating DiT to connect with one another by engaging in two-way feedback. An example of how this was enacted was explained by one participant as paying attention to asking others “about their shift, and how they felt they went, and … ask[ing] them for feedback on how they thought I had gone.”

Caring: Caring emerged as a significant change. Participants recognised “a more organised sense of looking out for each other.” The workshop on meeting basic needs, in particular, shifted how DiT thought about self-care, as well as encouraging greater care for each other. Participants explained that “people were asking, ‘Have you had water this morning? Have you had enough to eat?’.” They also reflected “the program reminded us to take care of ourselves, and to support each other, even on long shifts when we are very stressed at work.”

Another observed: “Overall, the general culture at work has changed, everyone is more mindful of each other’s wellbeing.”

Communication: The programme offered alternative ways to share information and opened up communication vertically and horizontally. Investing in the revamping of the doctor’s office space was experienced as a powerful gesture by participants. The programme was described as having created a “space to talk” allowing trainee doctors to “hear each other.”

Another participant explained, “The workshop on giving feedback; asking for feedback and how to give feedback in a constructive, rather than critical way, I took that on. I will definitely remember that in the future. The workshop normalised open conversation.” Others observed an increase in interactions with colleagues, more open communication facilitated supporting each other, “When I spoke about what I was going through, others then asked me for a coffee or a meal and shared their version of not being okay.”

Confidence: All participants felt more confident as a result of experiencing the programme. They were not only more confident about asking for help but also felt empowered to be leaders and bring about change.

One DiT reflected that they had “learnt that courage is not the absence of fear – it is the ability to act in the presence of fear.” Another explained that for them the “most important change brought about by the wellbeing program … was recognising my agency. I learned there were changes I could make.” The P2P delivery was highlighted as a major contributor to programme impact, with one participant observing “a highlight was senior clinicians telling their own stories. You can have grandiose ideas about others at work, especially the seniors you admire, how they know everything and do everything right. Witnessing their fears and concerns, and their approaches to challenges makes you more impressed by their achievement, you feel like challenges are more approachable, the steps ahead are more attainable.”

Cooperation: The programme aimed to bring O&G DiT together; to reflect, learn and grow skills to improve well-being and most importantly to have their concerns acknowledged and acted on at an organisational level. Participants of the programme observed a shift to more cooperative workplace practices, reporting that “the program was also an opportunity to address the things that make a cohesive team, that make us all better together. More than before the whole team stepped up to help each other make it through the day together.”

Discussion

‘The difference that made a difference.’

This study evaluated an educational programme to improve well-being designed by and for a group of 55 O&G DiT at a major tertiary hospital in Victoria, Australia, during the 2020 COVID-19 pandemic. We tracked the indicators of well-being and burnout with the collection of surveys before (n=46) and after (n=27 and 11) the implementation of seven workshops, which were one component of a collection of initiatives that comprised a toolkit of resources. Nine workshop participants were interviewed about their most significant experiences of change resulting from the programme. We demonstrated an overall 31.9% improvement in well-being scores. The qualitative evaluation identified the MSC, that is, ‘the difference that made a difference’ as an overall positive shift in workplace culture associated with change across five domains: connection, caring, communication, confidence and cooperation.22

The impact of COVID-19 on the safety and well-being of healthcare workers is well documented.2 23–25 Ellis et al2 reported an increase in surgical error due to the combined effects of COVID-19 on doctors’ sleep hygiene, concerns regarding infection exposure and the burden of personal protective equipment restricting movement and communication. However, despite well-known risks to DiT and an urgent call for interventions,6 26 there has been a lack of validated programmes aimed at improving their well-being.7 27

During the third month of the second Victorian COVID-19 pandemic lockdown, which lasted 112 days from 6 July 2020 to 26 October 2020,28 we observed that Monash Women’s DiT were experiencing significantly reduced well-being (mean score 46.7) and intermediate levels of burnout (mean score 48.1). Participants in the qualitative evaluation described finding themselves overwhelmed. For some, the pandemic exposed vulnerabilities more so than ever before. One participant shared that “the COVID-19 pandemic affected [their] life – personally and professionally – in every way.” In the workplace, trainees reflected that there was “a lot of fear across staff, patients, administrators; everyone.” They shared their experiences of being both “extremely grateful to have work and … at the same time … a level of resentment about having to go to work, and being expected to see people and just absorb the daily changes in PPE protocols, wearing visors, face masks and glasses, at times feeling claustrophobic; like you couldn’t escape, not being able to eat or drink during a busy shift.”

DiT working in O&G are responsible for providing care across a range of clinical areas within the health service site, including the emergency department, outpatient clinics, inpatient wards, operating theatres and birthing suites. In a recent survey, obstetricians and gynaecologists were among the highest at risk of physician burnout.29 Anticipating that the added workload generated by a pandemic situation would pose additional risk to the well-being of O&G DiT is not unwarranted, with Ochsmann et al30 identifying strain levels as directly related to overtime worked. The provision of safe care also depends on effective communication with patients, their families and across multidisciplinary teams spanning all levels of the organisational structure. These multidimensional care and communication challenges have also been demonstrated to increase the risk for emotional and physical fatigue.31

The Monash Women’s COVID-19 leading kindness toolkit programme demonstrated a 31.9% improvement in the Well-being Index for all DiT participants and contributed to safeguarding against worsening burnout symptoms. Those who attended a component of the integrated workshops experienced slightly greater impact (35.5%) on well-being over time. The overall achievement of the project, as expressed by participants in their stories of change, was a shift to a more caring and supportive workplace culture. Junior and senior DiT felt more connected as colleagues and were more confident about advocating for change and communicating with one another about their work. The impact of workplace friendships has been shown to be inversely related to workplace stress with healthcare workers relying most heavily on strong peer support, sharing with senior staff and supportive social networks when facing a crisis.3 The P2P style of our programme was highlighted as a strength by participants. This is consistent with the finding of Chanchlani et al6 who evaluated a P2P mentoring programme and demonstrated an improved sense of community and support. Similarly, Walton et al9 identified the important role of P2P interactions in the acquisition of complex non-technical skills.

Based on our experience in this initiative, we strongly advocate for a mixed-methods approach in the evaluation of healthcare programmes. Triangulating qualitative and quantitative methods enhances the interpretation of outcomes and provides valuable information about the impact, acceptability and utility of interventions.12 31 This integrative approach uses the complementary capacities of quantitative research in defining measurable variables and qualitative research in investigating complex social constructs.12 Exploring reasons for participant satisfaction or dissatisfaction, lack of adherence and causes of conflicting outcomes in different population groups are examples of the way qualitative analysis can enhance quantitative findings.32 Despite growing recognition of the value of mixed-method designs, there remains a lack of published medical studies employing a dual-analysis approach.12 This is supported by Lewin et al31 who conducted a review of the Cochrane register and identified just one-third undertook a combined quantitative and qualitative analysis. Recognised barriers to mixed-method designs in medical research include the need for adequate resourcing, the time-consuming nature of the research process and difficulty accessing appropriately experienced qualitative researchers.31

Implications and limitations

The programme was enthusiastically welcomed, and our findings attest to the benefits received by participants. The triangulation of quantitative and qualitative results demonstrates DiT were provided with tools to address burnout and improve well-being. Although our results suggest an improvement in burnout over time, this effect did not reach significance. Given the small numbers, our study may have been underpowered to detect a significant improvement in burnout. Alternatively, the programme may have limited impact on improving burnout. However, given the entire cohort remained within the intermediate burnout category (score 25–50), our findings support the programme’s success in protecting against worsening burnout during the pandemic.

The results of this study are limited by the small sample size. Participation in the workshop component of the programme was only 30.9% (n=17), despite the workshops being delivered during dedicated teaching time. This highlights the demands being placed on trainee doctors over this time and reflects the practice of clinical responsibilities taking priority over educational opportunities. The P2P and codesigned structuring of the programme as a ‘toolkit’ enabled the workshop content and initiatives to reach the non-attending DiT, affecting a rapid execution of changes and success of the programme.

We also self-imposed limitations on the collection of participant characteristics to preserve anonymity. A key component of the MSC technique is sharing personal stories. This presents challenges for maintaining confidentiality. The sample size and close working relationships meant maintaining anonymity could not be guaranteed. We addressed this by disaggregating stories and by ensuring participants were informed that they may be identified.

Given the collection of data was dependent on the voluntary completion of interviews and surveys, our analysis is subject to non-response bias. Raising the possibility that those who responded were more motivated and healthier, and people with more burnout or depression did not respond. Our small numbers limited the ability to undertake inverse probability weighting or multiple imputation to address this. The mixed-methods design strengthened the findings of the evaluation, providing insight and breadth to inform future implementations.

Over the last few years, there has been a greater focus on the nurturing of emotional intelligence. Many of these efforts have involved self-directed learning, action coaching and formalised mentoring programmes.33 These attempts have faced substantial system-based obstacles and have, at times, paradoxically penalised the individual doctor for failing to self-care. DiT are a highly goal-motivated group, yet messages that a lack of resilience, weakness and laziness typify those who succumb to these stressors is enduring.3 7 33 The harm from erroneous messages sent through labelling doctors as superhumans is also well documented, often as a gesture of thanks this culture lends itself to messages of the need to ‘tough it out’.34 Together, these run the risk of adding to the misconception that seeking help is a sign of weakness, failure and not having what it takes to survive the rigours of medical training.

In codesigning our programme, we were able to create a system of resources, which were meaningful and useful. In combination with a solid commitment from the organisation to support and see through the delivery of the programme, we effectively created a cultural shift and built capacity with a lasting impact for our team. The provision of programmes with a directive to protect and prevent healthcare workers from burnout is desperately needed. Interventions must be directed and targeted, recognising time constraints, transient working locations and competing demands. Additionally, their evaluation is imperative.7 It is vital that doctors feel safe to seek help, and more importantly administrators need to identify, track and monitor the well-being of employees and act well before crisis point is reached.35

We have successfully used a mixed-methods approach to contextualise a productive programme to improve the well-being of COVID-19 front-line healthcare workers. We hope the evidence generated from our programme contributes to informing the implementation of future programmes within other healthcare groups and settings.

Data availability statement

Data are available on reasonable request. We commit to undertaking all reasonable requests to share relevant data. Deidentified raw data are securely stored within Qualtrics and with the research team. Open access evaluation reports are available at: https://leadingkindnesscovid19toolkit.wordpress.com/

Ethics approval

This study involves human participants and was approved by Monash Health Human Research Ethics Committee (QA/68545/MonH-2020-230841(v2)). Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The Pandemic Kindness Movement was a vital resource for inspiration and implementation of the programme. We would like to thank the Associate Professor Jane Munro along with all the clinicians and experts who contributed and curated this wonderful collection of information in such a user-friendly and available format. We would like to acknowledge the overarching guidance provided by Associate Professor Jacqueline Boyle and Associate Professor Arunaz Kumar who were instrumental in the planning of the project and evaluation processes. We would also like to extend our gratitude to Professor Beverly Vollenhoven, Dr Mark Tarrant, Dr Risha Bhatia and Associate Professor Arunaz Kumar whose participation and feedback were crucial to the success of the evaluation. We would also like to thank the Monash Women’s O&G DiT who codesigned this pilot project alongside us. Especially to those who took the time to participate, present and provide vital feedback during a time in which they were most under the pump. Without your efforts there would be no programme to report on.

References

Supplementary materials

Footnotes

  • Contributors Contributorship statement: MW, RM, JA and DR contributed to the study design and implementation. Quantitative data was collected by JA, RM and MW, and analysed by DR, RM and MW. Qualitative data collection and analysis was conducted by KC and WC. MW oversaw the compilation of the first draft of the manuscript. CL and DQ contributed to the programme design and development. RH acted as a senior supervisor and led programme resources allocation. All listed authors contributed to the manuscript and meet authorship criteria, without the exclusion of others. Guarantor: DR.

  • Funding The ‘Monash Women’s leading kindness COVID-19 toolkit’ well-being programme and its evaluation was funded by Monash Health COVID-19 Research Foundation grant. The programme and analysis were undertaken independently and without influence from the funding body.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were involved in the design, conduct, reporting or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.