Article Text

Original research
Critical realist exploration of long-term outcomes, impacts and skill development from an Australian Rural Research Capacity Building Programme: a qualitative study
  1. David Schmidt1,2,
  2. Kerith Duncanson1,
  3. Emma Webster3,
  4. Emily Saurman4,
  5. David Lyle4
  1. 1Rural and Remote Portfolio, NSW Health Education and Training Institute, Gladesville, New South Wales, Australia
  2. 2School of Public Health, The University of Sydney, Sydney, New South Wales, Australia
  3. 3School of Rural Health, The University of Sydney, Dubbo, New South Wales, Australia
  4. 4Department of Rural Health, The University of Sydney, Broken Hill, New South Wales, Australia
  1. Correspondence to David Schmidt; David.Schmidt{at}health.nsw.gov.au

Abstract

Objectives Research capacity building programmes usually only examine short-term outcomes, following up participants after 1 or 2 years. Capacity building in health research requires a long-term view to understand the influence and impact of capacity building endeavours. This study examined long-term outcomes for individuals regarding the maintenance and use of research skills and the conduct of real-world research in a rural area. We also explored the changes individuals had seen in their career, work team or organisation as a result of this training.

Design A qualitative study underpinned by critical realism and based on interviews and focus groups with graduates of the Rural Research Capacity Building Programme (RRCBP), a researcher development programme that has been delivered since 2006.

Setting Rural and remote areas of New South Wales, Australia.

Participants 22 graduates of the RRCBP from the 2006 to 2015 cohorts (20 female, 2 male). All were experienced rural-based health workers at the time of training.

Results Focus groups and interviews yielded three themes about capacity building outcomes: (1) developed research capable individuals; (2) embedded research capability into teams and (3) real-world research that makes a difference within an organisation.

Conclusions Research training improved graduates’ skill, experience, confidence and employability. Research capable individuals enabled others, enhancing team research capacity and raising the profile of research within their organisation.

Training in research, alongside tangible organisational support for research activity, creates real-world impacts for policy and clinical practice. Providing ongoing opportunities for researchers to undertake research would enhance return on investment and assist with retention of experienced staff.

  • medical education & training
  • education & training (see medical education & training)
  • organisational development

Data availability statement

Data are available on reasonable request. Data are available on reasonable request and with approval of authorising Human Research Ethics Committee.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This is the first study to examine long-term outcomes of a clinician researcher development programme in a rural area.

  • A critical realist framework allowed explanation of the underlying mechanisms that led to change for the individual, their team or organisational.

  • Stratified sampling was used to ensure participants with a range of experiences were included, however, it is possible that those who have been less connected with programme peers postgraduation were less likely to participate.

  • There is a potential lack of generalisability to other settings, but the rigour in analysis and theory along with existing literature indicate the findings may be applicable in other settings.

Background

The health of rural people is characterised by poorer health outcomes, associated with limited access to health services.1 Rural research capacity building is seen as one way for rural health organisations to understand and remediate this disparity.2 Research in rural areas presents its own challenges and a need for specific rural research training has been identified.3 4 Numerous workplace-based programmes have been introduced to address rural health research capacity including scholarships, training in place, fellowships, participatory approaches and other capacity building endeavours.5–12

There is a call for researchers to be embedded within rural health organisations.12 13 While this is consistent with known research capacity development principles,14 information on long-term outcomes of research capacity building endeavours is scant, with most programmes typically funded short term and with limited follow-up.15 Measuring research capacity building is complex16 and success in health services research development is often under-reported.17

The Rural Research Capacity Building Programme (RRCBP) was established in 2006 to improve research capability and capacity in rural New South Wales, Australia. The programme was intended as a response to the low levels of research activity and research skill in rural areas,11 and supports rural clinicians to undertake a self-selected research project with an accompanying education programme aimed at building research experience.18 Clinicians enrolled in the RRCBP are based in rural or remote areas of New South Wales, Australia, and receive project funding and mentoring in addition to education support.11 18 Research is not routinely conducted by rural clinicians19 and in some cases these clinician–researchers may be the only person in their facility with research experience.20

The RRCBP, unlike similar programmes,21 22 has received ongoing funding. The RRCBP and its adjunct educational strategies have demonstrated efficacy in increasing self-reported research experience,18 improving publication rates,23 retaining researchers to complete the programme20 and building research capacity.11 Internal evaluation findings indicate that RRCBP candidature can be transformative, personally or professionally, beyond the field of research.

Since 2006, 245 rural health workers have received research methods education and training, with 137 completing their research project and submitting a report detailing their findings (graduates) at the time of this study. While research experience and some capacity has been built,11 18 23 long-term outcomes for programme candidates or the rural health organisations they work for has not yet been examined.

The aim of this study was to understand the impact of undertaking research training for graduates, their careers and their organisations.

Methods

Study design

This qualitative study was underpinned by critical realism. Realist approaches uncover what works for whom, and in what circumstance.14 24 25 Critical realism facilitated the exploration of the context of research skill development and the mechanisms underlying these changes.

Sampling and recruitment

One hundred and thirty-seven graduates from the 2006 to 2015 RRCBP cohorts were eligible to participate. Of these, 113 had remained in contact and were stratified by training cohort as ‘early’ (2006–2010) or ‘recent’ (2011–2015), and then by known research activity since graduation; either research active, not research active, or those whose research activities were unknown. Research activity was determined by recent contact from graduates and confirmed by a search of Google Scholar for postgraduation publications. Graduates who had not made recent contact with programme coordinators and who had not published in peer-reviewed publication were designated as research status unknown. From these six subgroups, individuals were randomly selected by an independent administration officer and invited via email. Individuals who were unavailable, uncontactable or who declined were replaced by another randomly selected person from that subgroup. Potential participants were sent a reminder email 2 weeks after the initial invitation if no response was received. Recruitment continued until the subgroup was represented by five participants or the subgroup list contained no other eligible participants. A recruitment flow chart is seen in figure 1.

Figure 1

Recruitment flow chart. RRCBP, Rural Research Capacity Building Programme.

Data collection

Individuals were invited to an online focus group to stimulate discussion and allow ideas to develop among participants. Those who wished to participate via individual interview were accommodated. Focus groups and interviews were conducted by an independent facilitator (ES) and were digitally recorded. All recordings were transcribed by a professional transcription company and deidentified by the facilitator.

After the first focus group, a debrief was undertaken, with the facilitator and lead researcher refining the focus group and interview strategy.

Four focus groups were held, with 4 participants per group, and 6 individual interviews conducted; a total of 22 participants sampled from the 6 subgroups.

Data analysis

Qualitative analysis was completed by DS using the cut-and-paste method of coding and thematic development.26 Three coresearchers (ES, KD and EW) independently analysed randomly selected transcripts (one focus group and two interviews each) to ensure veracity of themes and corresponding subthemes. Theme refinement occurred through an iterative process of discussion and rereading transcripts. This staged inductive process was then considered deductively against Cooke’s framework for capacity building, using the structural levels of the individual, their immediate work team and the hosting organisation.27 Factors described as ‘superorganisational’ or outside the individual’s organisation24 were deemed unlikely to be affected by individual training and were therefore not included in the analysis.

The lead researcher (DS) read all transcripts with a view to assessing data saturation. After three focus groups and four interviews, no new ideas emerged and the remaining focus group and interviews ensured confidence in concept identification. When all team members reached agreement on the final iteration, analysis was deemed complete.

Reflexivity and rigour

Two of the researchers (DS and KD) are coordinators of the RRCBP and a third (EW) is a former coordinator. The use of an arms-length recruitment and independent facilitator was accompanied by a process of bracketing preconceptions28 and oversight of theme development by independent researchers (ES and DL).

Patient and public involvement

No patient involved.

The original protocol for the study has been uploaded as online supplemental file along with a copy of the interview/focus group schedule.

Results

The demographic characteristics of study participants are outlined in table 1. All graduates were rurally based at the time of their training. To maintain participant anonymity, professions were classified as nursing, allied health or other profession which may include Aboriginal health worker, doctor, health promotion officer, health manager, paramedic or project officer.

Table 1

Demographic details of graduates participating in the study (n=22)

Three themes and their relationship to the structural levels of individual, team and organisational levels proposed by Cooke27 emerged.

Developed research capable individuals

The programme successfully developed research capable individuals. Graduates identified (1) Growth in knowledge, skills, authority and confidence, (2) Ongoing research activity and (3) Personal and professional transformation.

Growth in knowledge, skills, authority and confidence

Learning about and undertaking research was a means for graduates to extend themselves, both from a personal sense but also providing new career directions. Participants reported their understanding of research had improved, building on existing skill sets and knowledge.

What the program allowed me to do was to understand a much broader range of research methodologies and approaches. Focus Group 2

Knowledge and expertise development was unsurprising, given the educational goals of the programme.18 However, undertaking a research project was pivotal to the development of a range of transferable or ‘soft’ skills including project management, communication techniques and critical thinking; strengthening attributes that were useful in other roles or settings. Critical thinking in action is a known indicator of research capacity building at an individual level.27

Maybe I can’t articulate it perfectly, but the ability to uncomfortably question, to sort of reflect in that space and sit in a space of discomfort to evaluate something surrounding yourself… so those kind of critical thinking skills and evaluation skills go with you in every aspect of your work. Focus Group 2

Ongoing research activity

One result of developing research capable individuals was ongoing research activity; this included using their expertise to attract grant funding and extend their research activity.

There were opportunities to apply for sort of grant money to do other kinds of [research] projects and that then created new relationships and added to the body of evidence around that. Focus Group 1

Ongoing research activity did not always equate to independence or leading their own research agenda. This highlights the foundational nature of the training.

The project that I did with [a non-government organisation] recently… I actually did it all by myself, which I found I could do and I knew how to do… I think the project that I did in 2006 really taught me those skills and I know those skills but I just didn’t feel confident to do that and the next one on, like totally on my own. Focus Group 1

Having research experience through the RRCBP enabled some graduates to take up new roles to pursue research opportunities.

I think everybody has moved onwards and upwards in roles, and I’m sure that, personally, the program helped me with a promotion upwards and, you know, it’s good in that sense. Focus Group 2

Other graduates progressed to further formal study via higher degree research, often an extension of their research project with the programme.

I was very fortunate I was able to springboard my research into a master’s degree. Focus Group 4

Not all graduates remained active in research, however the programme had value as a learning experience and their interest remained.

I’m not doing research, and I haven’t done any research, but what I am surprised about is how much interest I maintained in research, and other people’s research. Focus Group 2

A lack of research activity was not always due to individual factors. Organisational factors also influenced their ability to undertake research, including fitting research into a busy caseload. A lack of research opportunity impeded ongoing research activity for some graduates and for others around them.

With opportunity, I believe I've got the skills that I could contribute to further research in related areas … but there’s so little research happening that there aren’t any people to supervise. Focus Group 4

Personal and professional transformation

Graduates described transformative aspects of learning including a sense of belonging in the world of research, with connections to others outside their immediate workspace. Transformational learning can refer to a process that encourages the learner to question and potentially change their worldview.29 The sense of becoming part of the world of research extended to having and owning specialist knowledge of research tools, language and business processes. Similarly, transformation occurred when graduates were challenged in their ways of thinking and the assumptions underlying these thought processes.

It sort of taught me that you can reach out to a lot of different people and that they’re more than willing to help you …it made me more empowered to look beyond my own patch. Focus Group 3

The sense of becoming part of the world of research extended to having and owning specialist knowledge of research tools, language and business processes. Similarly, transformation occurred when graduates were challenged on their ways of thinking and the assumptions underlying these thought processes.

I was mentored by an Aboriginal man from [the university] which was really helpful. He would challenge me on a regular basis about my way of thinking. Focus Group 4

Embedded research capability into teams

The graduates’ research experience led to changes within their immediate work teams. They became ‘resource people’ within the workplace for research or evaluation activities; a role that included providing feedback on research and evaluation proposals, providing guidance and adding rigour to existing activities.

I give feedback to peoples’ quality improvement projects and I actually really like doing that. I wouldn’t say it’s pure academic research, but it’s using the methodology of research, of a research project and applying that to smaller projects that are more practical. Focus Group 4

There’s a project that was happening …[and] I was able to sort of help them with [the ethics application], and they’re really happy that they’ve done it. So that’s as a direct result of my being in the program. Focus Group 2

This contribution to the skill development of others was a way to develop local research capability and embed local researchers into projects. The process of building research activity and capability were part of creating a local culture of research.

We can actually influence other people either informally by—with their work or encouraging them to do research or to encourage it within a discipline or a department, or other people to do the [RRCBP], or even just set a culture of research around where we work. Focus Group 2

Moving beyond the role as resource person, graduates became active collaborators or facilitators of research. Those in leadership positions engaged with and facilitated research activity within their local teams. Others began collaborating with fellow RRCBP graduates or current RRCBP trainees.

I’ve had a couple of staff members that are that are currently enrolled in the same program and we’ve sort of been involved in a number of orthopaedic and other projects as well in the department. So I think it really fuelled my already existing interest and passion in research and maybe gave me a little bit of confidence to continue to support people to do that. Focus Group 1

Graduates were able to use their interest in research to promote and facilitate research across departments and districts. They were now setting the agenda and making research part of it.

So the department didn’t have a strong research focus before, but we’re really developing that now. It’s on every agenda, we’ve got a team looking at it, we’re constantly looking for grants to field how we can build research around what we’re doing. Focus Group 2

Real world research that makes a difference within an organisation

Participants wanted their research to be of value to the organisation and to make a difference in care for consumers. Making a difference was seen through: (1) research that influenced policy, practice and culture, (2) organisational support for research education and activity, and (3) retaining skilled workers within the organisation.

Research that influences policy, practice and culture

Graduates wanted to lead and engage with research that had impact and importance for clinical practice and underlying policy. They made a clear distinction between real-world, clinician-led research and theoretical research led by university academics, which was seen to be disconnected from clinical practice.

I think that’s probably what we as rural researchers, we’re going to research something that’s really practical, something that’s really doable, not something that’s sort of very theoretical… We need stuff that we can actually put into action now. Focus Group 2

Using research to improve services justified the effort required to undertake clinical research, even when those changes were not what was expected or hoped for. The quality of the graduates’ research was important for establishing credibility, which was necessary for practice change to be embraced. For some, the fact that the research was conducted by a clinician added to that perceived credibility.

I think once you can see that it can translate to service enhancements, and those sorts of things that everyone’s talking about then you realise that there’s such a huge benefit. Focus Group 3

I think that because [the research] had been done in quite a robust way and we were reporting it very clearly and very open with ‘this is exactly what happened and what we found’ and gave people kind of confidence to work with [the findings]. Focus Group 1

While changes that resulted from the graduates’ research included system level and local changes, not all projects had a demonstrable impact on policy or practice.

It really sort of started a little spark in my brain about being interested in how to drive sort of large system level change… really trying to better create evidence that helps us know whether what we’re doing is making a difference, and particularly models of care. Focus Group 1

I still have a clinical role and research—particularly the research that… led from my Rural Research Capacity Building Project has changed the way that I see my clients… there’s been a big difference in the way I treat clients. Focus Group 3

I think from my research project, it hasn’t, unfortunately, sort of been translated into practice much…. Interview 6

Some felt that their research contributed to change in clinical practice without change necessarily being the key driving factor.

I think that the practice for the cohort of patients that I wrote about has changed, but I don’t think it was as a result of my research paper, but I think it contributed to the body of knowledge around the area. Focus Group 2

Others felt that their research was controversial or unpopular. While these projects may not have led to a policy or process change, the research activity and distribution of findings created discussion. There was also demonstrable resilience developed in response to these challenges.

I got attacked by people sometimes about my project. I thought, really, is it that polarising? But ….it started a conversation at least. So there’s still no resolution necessarily but people in [my profession] are talking about it. Interview 1

Graduates felt that research influenced the culture of the hosting organisation where recommendations were embraced and implemented. When recommendations were not implemented, the impact of the research was perceived to be minimal. In some workplaces research activity led to research becoming integrated into everyday work.

I worked with the [local] Aboriginal Medical Centre, but we had all the recommendations implemented that were in the report, and so that was a big change and there was a big cultural shift… it had a huge impact I think on our service. Focus Group 1

[The program] is bringing research into the workplace so influencing other people, encouraging other people, making research part of the language of the workplace. …I think you’ll find that every person that’s been on the program, the area that they work in has – research has a higher profile in some shape or form than if they hadn’t been on the program. Focus Group 2

Support for research education and activity

Support for research education and activity within the workplace was mixed. When support was lacking, high degrees of autonomy were required by the researcher.

I can't say I was totally surprised, but I didn’t get any support from the health service. I did it all in my own time, basically. Focus Group 4

Other participants were well supported locally but that wasn’t necessarily matched at higher organisational levels. Engaging leaders within the organisation was critical for obtaining dedicated time and resources for research learning and activity. Support for research activity was perceived as being linked to how closely the proposed topic aligned with organisational goals, or at least did not interfere with organisational functioning.

I don’t think there’s any particular encouragement. I haven’t seen any managers who are resistant, but I wouldn’t say any of them see it as any sort of priority. If they can do it with only moderate inconvenience to them, they’ll support it. But, if not, it’s not something that’s on their radar. Focus Group 4

Graduates identified a gap between the rhetoric of research being valued and the reality when conducting research activities in the workplace. Engagement and endorsement from senior management was required to conduct the research and for research to enable change within the organisation. Graduates indicated the competing priorities faced by rural health organisations and the impact that this may have on support for research.

You can have all the structure in the world; if research is not a priority to senior management it ain’t going to happen. Focus Group 4

Retaining skilled workers in the organisation

For some participants, the programme reinvigorated an interest in their work and an associated sense of ownership and control, encouraging them to continue in their role. This retention is important given the typical profile of an RRCBP candidate is an experienced rural health worker with clinical, corporate and community knowledge.20

I was bored to death at work. It kept me in that same job for a little bit longer, because it was something interesting… I was allowed this little bit of space where I could just find my own train of thought, and it was something where, yeah, it kept me there. Interview 1

For clinical roles, having research experience was not seen as a priority. With limited options for research-specific or research-compatible roles within the health system. Organisations risk losing the graduates if they are unable to use their newfound skills. However, there are opportunities for organisational recognition of research expertise within clinical roles via promotion or ‘regrades’ within existing roles, which may in turn assist with staff retention.

It’s a bit of a dead end though in the sense that I've got these skills and there’s not really much infrastructure to actually use these skills. Focus Group 4

Discussion

This study examined the long-term outcomes of a research training programme from the perspective of graduates with up to 16 years’ experience post-training. These outcomes and the proposed underlying generative mechanisms are summarised in table 2.

Table 2

Context, mechanisms and long-term outcomes of rural research training

Graduates of the programme saw themselves and their research peers as possessing new and transferable skills, improved work performance and enhanced employability prospects. Undertaking research education through a supported research project developed knowledge in research and non-research related spheres, including critical thinking and project management. While this positive change was noted at the individual level, having a more competent, confident and skilled worker has obvious team and organisational benefits such as producing research which addresses local practice and policy needs, strengthening team evaluation activities and retaining a skilled workforce.

For individuals in this study there were improvements in the individual’s knowledge and skills to undertake research, which is consistent with real-world research capacity building.30 This individual capability was accompanied by changes in research capacity within the individual’s immediate work team and their broader organisation.27 While the programme has previously been shown to build research expertise, contribute to the research evidence base in rural health, and produce early capacity building outcomes,11 18 23 this study provides new insights into the long-term value of training in research for the individual and beyond.

Some graduates remained research active, either via higher degree research or through workplace-based projects. The level of research engagement for graduates varied and included being a research participant, research advocate, researcher in a team, or independent researcher. Transitioning from a foundational experiential research project to independent researcher depends on the individual agency of the worker, with some graduates actively seeking or creating research opportunities. However, as figure 1 indicates, there does not appear to be a progressive decline in research active individuals over time (early vs recent), raising the hypothesis that if individuals become research active as a result of the programme that this is a sustainable behaviour. A mismatch between the graduates’ desired level of research and limitations in their ability to reach that level led to feelings of discontent.

For novice clinician researchers who report limited growth in research agency, ongoing structural supports such as a second-stage research education stream or supported research mentorship could increase levels of research activity. Careful consideration of structural design in a research training programme has been shown to link with successful training outcomes.21 Further exploration of university partnerships would be a logical progression, such as host health organisations supporting doctoral students or post-doctoral researchers to undertake relevant research while embedded within the organisation.31

In teams where a rural health worker has research experience, that individual enabled, encouraged or enhanced research activity in others. This role went beyond enabling or encouraging others to access formal training programmes to active engagement, adding rigour to existing research or quality improvement activities. Embedding researchers in a health team raised the profile of research across the team and more broadly within the organisation. Future qualitative ethnographic study would provide valuable insights into the development of a research culture, with previous studies largely focusing on culture from a quantitative perspective.17 32 33

The organisational return on investing in research training was enhanced by high organisational commitment to research in practical terms such as funding or dedicated roles. It is a recognised challenge for organisations to create structures in which these skills can be utilised, either within the person’s existing role or potentially moving to roles such as an embedded researcher position.13 Providing opportunities for research was seen as a tool for retaining experienced staff, and can maximise the return on investment for organisations that support novice researchers.

Close-to-practice research, such as that completed in the programme, is a key element of capacity building.11 27 34 While graduates saw their research making a difference to practice and policy, this was associated with the graduates’ perceptions of their organisation’s support and willingness to embrace change. Clear strategic goals for research, matched with operational commitment, is necessary for close-to-practice research in rural areas.19 35

It is important to note that the RRCBP is conducted within rural health workplaces that have seen attitudes and commitment to research fluctuate over time. Research capacity development strategies that target individual skill development will continue to lead to limited levels of research capacity development within and across organisations. There is limited ability for educating institutions to influence these supraorganisational factors, however, this serves as a reminder of the importance of understanding the context in which training occurs.19 35 While attitudes towards and support for research may have changed in the time since graduates were trained, a coordinated approach to research investment and policy is needed to move research capacity development beyond the level of the individual.

These findings have been drawn from the experiences of programme graduates and further researcher on broader organisational perspective would be helpful in understanding organisation-level changes.

To our knowledge, this study is the first study to examine long-term outcomes of a clinician researcher development programme in a rural area. While a stratified sampling frame was used to ensure participants with a range of RRCBP experiences were included, those who have been less connected to programme peers and programme coordinators may have been less likely to participate (figure 1).

As this is a single educational programme held in rural areas of one public health system in Australia, there is a potential lack of generalisability to other countries or health systems. However, the use of critical realism to provide practical recommendations, the procedural rigour of the study and alignment with existing research capacity development literature indicate the findings may be applicable in other settings.

Conclusions

This study demonstrates that investment in research training leads to long-term improvement in skills, confidence and employability of individual workers, enhances team research capacity, provides immediate answers to local clinical and policy priorities and raises the profile of research within rural health services.

Training individuals in research, in conjunction with organisational support for ongoing research activity, led to changes in policy and clinical practice. Creating ongoing opportunities for researchers to use and advance their newfound skills, along with continued research support, will maximise the individual and organisational benefits of research training.

Data availability statement

Data are available on reasonable request. Data are available on reasonable request and with approval of authorising Human Research Ethics Committee.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and was approved by Hunter New England Heath Research Ethics CommitteeRef 2020/ETH00835 Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The authors wish to acknowledge Lynette Gillies for her assistance in the recruitment stage of the study.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @clin_researcher

  • Contributors DS designed the study with contributions from KD, EW and DL. All authors contributed to interview/focus group question development. ES collected all data. DS undertook primary data analysis with verification from KD, ES and EW. DS wrote the manuscript with contributions from KD, EW, ES and DL. All authors reviewed the study findings and read and approved the final version before submission.DS is responsible for the overall content as guarantor and accepts full responsibility for the finished work and/or the conduct of the study, had access to the data, and controlled the decision to publish. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding This work was supported by New South Wales Health. Award/Grant numbers are not applicable.

  • Competing interests DS and KD are programme managers for the Health Education and Training Institute (HETI), NSW Ministry of Health. HETI is the organisation that conducts and manages the program that was researched in this study. Emma Webster established the programme and is a former programme manager of the programme from 2006 to 2015.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.