Article Text
PDF
Abstract
Introduction Patients are a valuable source of information about ways to prevent harm in healthcare, and can provide feedback about the factors that contribute to safety incidents. The Primary Care Patient Measure of Safety (PC PMOS) is a novel and validated tool that captures patient feedback on safety and can be used by primary care practice teams to identify and prevent safety incidents. The aim of this study is to assess the feasibility of PC PMOS as a tool for data-driven safety improvement and monitoring in Australian primary care.
Methods and analysis Feasibility will be assessed using a mixed-methods approach to understand the enablers, barriers, acceptability, practicability, intervention fidelity and scalability of C PMOS as a tool for safety improvement across six primary care practices in the south-west region of Victoria. Patients over the age of 18 years attending their primary care practice will be invited to complete the PC PMOS when presenting for an appointment. Staff members at each practice will form a safety improvement team. Staff will then use the patient feedback to develop and implement specific safety interventions over a 6-month period. Data collection methods during the intervention period includes audio recordings of staff meetings, overt observations at training and education workshops, reflexive researcher insights, document collection and review. Data collection postintervention includes patient completion of the PC PMOS and semistructured interviews with staff. Triangulation and thematic analysis techniques will be employed to analyse the qualitative and content data. Analysis methods will use current evidence and models of healthcare culture, safety improvement and patient involvement in safety to inform the findings.
Ethics and dissemination Ethics approval was granted by Deakin University Human Ethics Advisory Group, Faculty of Health (HEAG-H 175_2017). Study results will be disseminated through local and international conferences and peer-reviewed publications.
- primary care
- quality in health care
- qualitative research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study in primary care to collect and use patient feedback on safety for safety improvement activities.
A mixed-methods approach will allow triangulation of qualitative and quantitative data sources.
Understanding the enablers, barriers, acceptability, practicability, intervention fidelity and scalability of the study will provide an insight into the future efficacy and the effectiveness of trials.
A limitation is that the data collected will be mostly descriptive, and, therefore, the generalisability of the findings may be limited to only one geographical area.
Introduction
For the majority of patients, journeys through the healthcare system are dominated and guided by the care received in the primary care setting. It is therefore essential to ensure that the care received in primary care is safe. The WHO defines safety as ‘the absence of preventable harm to a patient during the process of healthcare’.1
The rate at which patient safety incidents occur in primary care varies,2–4 but according to a recent meta-analysis of patient safety incidents in primary care, it may be as high as 24 per 100 consultations. Of these safety incidents, it is estimated that 4% will result in patient harm.5 The root causes of error in primary care are often referred to as the contributory factors to safety incidents.6 7 These underlying causes of events are important to understand and reflect where prevention of error efforts could be aimed.
The attempts to understand the magnitude of safety incidents in primary care have largely centred on data collection strategies, such as retrospective review of individual patient records, developing formal incident reporting systems and real-time staff feedback.2 Interventions to improve safety in primary care are concentrated on implementing clinical risk management standards for practice, encouraging service accreditation or participation in quality improvement activities, such as the Institute for Healthcare Improvement Breakthrough Collaboratives series8 and the Australian Primary Care Collaboratives programme.9 10 Although interventions and data collection methods for patient safety are improving, their impact on the reduction of safety incidents in primary care is unclear.11
Given these shortcomings with data availability and improvement efforts, patients’ and carers’ roles in preventing errors and reducing harm has been considered an important piece of the safety intelligence ‘jigsaw’.12 13 Their firsthand experience of care has been linked to a patient’s ability to provide detailed information about the processes, systems and structures that have led to the occurrence of a safety incident.14–16 Patients’ perspectives of safety are often different to health professionals and they can offer insights into their experiences along their care journey that impact on safety.15 17 Patient insights often focus on conditions in the latent environment that influence safety, such as access to care, communication systems, information and care planning, and transitions between care settings.18 Capturing patient feedback about these contributory factors to safety incidents in primary care is a developing and novel field of research,19 20 with most research coming from the hospital setting.21 22
Only one validated, prospective and theory-derived patient feedback tool for the assessment of contributing factors to safety in primary care is currently available—the Primary Care Patient Measure of Safety (PC PMOS).23 24 The PC PMOS was developed using both inductive and deductive methods, and has a direct lineage from the validated hospital PMOS tool.21 22 The PC PMOS is an anonymous 28-item survey covering nine latent conditions in the primary care environment influencing safety incidents including access to care, communication, the external policy environment, information flow, organisation and care planning, patient-related factors, the physical environment, referral systems and task performance.23 24 This tool is different from other patient feedback on safety surveys, which aim to capture information about safety incidents after the event has occurred.25
The aim of the PC PMOS is to enhance or complement current data collection methods to identify and prevent safety incidents,23 24 and is a practical response to finding appropriate and effective ways of involving patients in improving patient safety.12 15 18 The PC PMOS also provides a way for primary care professionals and organisations to learn about safety from the patient perspective, and then make real-time service improvements using patient feedback. There is an emerging evidence base exploring the use of patient measures of health system performance26–29; investigating the enablers and barriers of patient feedback as a mechanism for safety improvement30 31 or identifying particular patient or practice characteristics that are associated with patient-reported safety incidents.32 33 These studies contribute to the literature in this area but further research is needed to understand the use of patient feedback on safety specifically within the primary care setting.
The feasibility of implementing the PC PMOS in primary care as a tool for data-driven improvement and ongoing monitoring of safety in primary care remains unexplored. Studies that have evaluated the use of PMOS tool (hospital version of the PC PMOS tool) on safety improvement have shown mixed results.34 The cluster randomised control trial of the Patient Reporting and Action for a Safe Environment (PRASE) intervention demonstrated good feasibility and acceptability results among patients and hospital staff,35 as well as high intervention retention and uptake on the wards.34 However, no significant improvements in safety outcomes were found.34 The authors propose that while uptake and completion of the PMOS was high, adherence to the interventions undertaken by staff to address safety problems identified using the PMOS were poor, and that the safety outcome measurements were considered too blunt to obtain significant findings.36 37 These findings are likely to be applicable to the primary care setting for various reasons. Lack of formal incident reporting and learning systems limit the availability of data needed to develop and use robust measures of safety in primary care research.11 Often, data triangulation or case note review is needed to determine the frequency and nature of safety incidents, both of which have their weaknesses.2 Case note review is a time-intense and labour-intense activity and requires highly skilled clinicians to undertake this task,38 and composite measures of safety which use data triangulation techniques have demonstrated flaws mostly relating to transparency and uncertainty of primary data sources.39
Given these varied results in the hospital setting and the challenges with safety outcomes measures, it is important to explore the feasibility issues of implementing PC PMOS as a tool for safety improvement in primary care, prior to investigating its potential effectiveness in reducing safety incidents. Therefore, the aim of this feasibility study is to understand the enablers, barriers, acceptability, practicability, intervention fidelity and scalability of implementing PC PMOS as a tool for safety improvement in Australian primary care settings.
Methods and analysis
Study design
A feasibility research design will be employed to address the aim of this study. The aim of feasibility studies is to help determine whether an intervention should be recommended for future efficacy and effectiveness evaluations.40 Some of the approaches and methods employed in this feasibility study have been adapted from the PRASE intervention conducted in secondary care settings.34 35
Sampling
Sampling frame
A purposive sampling approach will be employed to identify potential primary care practices to be recruited into the study via the investigators’ known networks in the south-west region of Victoria, Australia. A mix of small and large practices will be sampled from the region. The population characteristics across the sampling region is reasonably homogeneous.41
Sample size
Primary care practices
A total of six primary care practices will be recruited to participate in the study. The number of practices selected is based on what is feasible with the research funding.
Patients
Given that the primary aim of this study is to test the feasibility of implementing the PC PMOS in primary care practice, a sample size calculation is not necessary for ensuring confidence around outcome measurements.
Instead, in order to confirm implementation feasibility and future capacity to integrate the PC PMOS tool into existing safety processes and systems for learning, sample sizes for the patient survey recruitment are aligned with accepted best practice, as directed by practice accreditation requirements. The Royal Australian College of General Practitioners (RACGP) recommend that practices undertake 30 patient surveys per one full-time equivalent (FTE) general practitioner (GP) for accreditation purposes.9 Given that the average size of primary care practices in Australia comprises 5.2FTE,42 a minimum 156 PC PMOS surveys at each of the practices is a suggested guideline for the baseline (T1) and 6 month postintervention period (T2) data collection time points. However, this will be negotiable depending on the number of FTE GPs employed at the participating practices.
Inclusion criteria
There is no inclusion criteria for primary care practice participation.
The inclusion criteria for patients to complete the PC PMOS are adults 18 years and over who are able to complete the questionnaire, and those who have attended the practice for 12 months or longer.
Recruitment and informed consent
Primary care practices
Primary care practices within the south-west region of Victoria, Australia will be approached by the lead investigator and other investigators to determine their interest in the project. A list of practices in the region that may be willing to participate in the project will be created by the investigating team using their known networks. The list of practices will include a mix of small and large practices to reduce potential generalisability limitations. Practices on the list will be approached sequentially until six agree to participate. Initial contact with the practice will be made by phoning the practice manager, and inviting them to participate in the study.
Practices who are interested and willing to participate in the project will receive a study information pack that details the study aims, methods and expectations of both the practice and the research teams. This pack outlines the time, resources, commitment, data and follow-up required for participation. Practices are required to sign an organisational consent form to participate in the study.
Practice staff members directly involved in the research project will be invited to participate in a semi-structured interview with the researchers at the conclusion of the intervention period. Staff who will be interviewed will be provided with a Plain Language Statement outlining the study requirement, and will be required to sign an individual consent form.
All staff from each practice are invited to complete an anonymous safety culture questionnaire at the beginning and end of the study.43 This survey will be circulated to all staff by practice managers at each practice and returned to researchers via a provided reply-paid envelope. Completion and return of the safety culture questionnaire implies consent.
Patients
Every adult (over 18 years) attending their practice over a 3-week period will be invited by the practice receptionist to complete the PC PMOS on presenting for their appointment, if they have visited the practice in the previous 12 months. The same recruitment procedure will occur at both baseline (T1) and post-intervention period (T2) data collection time points.
If practices have not collected the minimum number of patient surveys during this time, then the data collection time frame will be extended to 4 or 5 weeks. Completed PC PMOS surveys will be returned to the researcher via a secure questionnaire return box located in the practice waiting room or reception area.
Patient consent is implied by the completion and return of the PC PMOS questionnaire. A plain language statement will act as the coversheet for the PC PMOS questionnaire. This is removable for the patient to retain for their own records.
Intervention
Intervention comprises an iterative process with a cycle of measurement, learning, feedback, action planning and implementation, lasting for a period of 6 months. Briefly, patient feedback about the safety of their care is measured using the PC PMOS tool.24 44 Primary care teams then use this patient feedback to develop and implement specific safety interventions. Development and implementation of safety interventions mirrors the Plan-Do-Study-Act (PDSA) method of quality improvement in healthcare.8 PDSA methodology is familiar to some participating primary care staff as the RACGP endorse this approach to quality improvement activities.45 The PDSA cycle involves practice teams developing a plan to test the change (Plan), carrying out the test (Do), observing and learning from the consequences (Study) and determining what modifications should be made to the test (Act).45
The key phases and timeline of the intervention are described in more detail below, and can be viewed in figure 1.
Key phases and timeline of the intervention. MO, month.
Measurement
Primary care practices will collect patient feedback on safety through distribution of the PC PMOS survey at two time points—baseline (T1) and post-interventionperiod of 6 months (T2).
Anonymous patient demographic data on PC PMOS will also be collected and includes information on gender, age, postcode and frequency of attendance to the primary care practice in the previous 12 months (the PC PMOS tool is available on request to the authors).
The PC PMOS also captures patient-reported safety incident data. The questions used to elicit patient-reported safety incidents are based on the ‘Patient Incident Reporting Tool’ used in the PRASE evaluation (online supplementary file 1).46 47
Supplemental material
Learning
Primary care practices will be asked to elect a ‘safety improvement team’ (SIT). This team is suggested to comprise a minimum of three people who work at the practice, and range in levels of experience and backgrounds. Recommended SIT members can include any of the following representatives: practice manager, GP, GP registrar, practice nurse/s, allied health professionals (if located within the practice) or administrative staff members. The SIT are responsible for developing and implementing the intervention.
SIT members will participate in two training and education workshops during the intervention period. The first workshop will take place after baseline patient data collection has commenced. The purpose of the first training and education session is to provide a more detailed briefing about the feasibility study, its conceptual basis, how it will work and what is required of the SIT. SIT members will also receive training and education in teamwork, communication, implementation planning, and the model for improvement8 and PDSA methodology.45 48 This workshop will be delivered by a clinical educator/research fellow with extensive experience and knowledge in quality improvement in primary care. SIT members will receive a participant manual and practice workbook that contains study information and resources to assist with developing and implementing safety improvement interventions. The manual and resources were developed by the research team using best practice guidelines and evidence from the literature (a copy of the manual and workbook is available on request to the authors).8 48–54
Feedback
PC PMOS data from each practice will be collated and presented to the practice team in the form of a ‘feedback report’ (online supplementary file 2). The structure of the report was adapted from the PRASE intervention materials.35 The report provides a summary of the scores for each PC PMOS item and domain, qualitative comments (positive or negative) and any patient-reported safety incidents. The feedback report reflects the patient’s perspective of the safety of their care. Primary care teams then use this information to target areas for improvement in phase 4 of the intervention.
Supplemental material
Action Planning
Action planning will take place during a meeting with SIT members. This meeting will be facilitated by the research team members. In this meeting, the SIT members will receive the feedback report, discuss the data, consider which area(s) of safety improvement should be targeted and develop appropriate PDSA cycles. The research team members will facilitate the development of tangible PDSA cycles as this is an important outcome of the action planning phase.
Implementation
SIT members will be responsible for implementing and monitoring their specific safety intervention/s by developing and applying multiple PDSA cycles over the 6-month implementation period.
The second workshop will occur halfway through the 6-month intervention period (approximately 2–3 months after phase 4). The purpose of this workshop is to provide SIT members with support during the implementation of PDSA cycles. The workshop will bring together all of the SIT members from each primary care practice. In the workshop, teams will identify potential barriers to the PDSA process and share ideas about how best to manage them, receive updates from each of the SIT members about their progress, troubleshoot any problems and generally assist with maintaining motivation for the intervention and the study.
Outcome measurement
Primary outcome
The primary outcome of this study is the feasibility of implementing PC PMOS as a tool for safety improvement in Australian primary care settings. This will be measured by exploring the acceptability, practicability, fidelity, scalability, and barriers and enablers of implementing the PC PMOS in practice.
The above dimensions of intervention feasibility will be assessed using three qualitative data collection methods:
Recordings and observations of SIT members at workshops and action planning meetings.
Semistructured interviews with SIT members and other relevant practice staff members at each practice.
Researcher reflection and assessment of intervention component implementation.
Recordings and observations of SIT members
Action planning meetings comprise one of the most important components of the study. Understanding how SIT members consider and respond to patient feedback collected on the PC PMOS may provide vital details to explain their approach for development and implementation of safety interventions. Furthermore, team dynamics, practice culture and other contextual information that is disclosed in the workshops are essential to understanding the broader environment that influences the study.
These data will be collected using digital audio recordings of the action planning meetings and workshops, and an in-depth overt participant observation by the research team. The data collected from the action planning meetings and workshops will be transcribed verbatim, and participant observation will be recorded using detailed field note diaries and regular researcher discussion and reflection.
Semistructured interviews with SIT members
The research team will conduct semistructured interviews with SIT members after the intervention period has finished. These interviews will explore staff attitudes toward the intervention, engagement with the intervention, processes and support required for the intervention, the enablers and barriers for implementing the intervention, the wider practice context of the intervention and the general experiences of participating in the study. Furthermore, interview participants will be asked a specific question relating to intervention fidelity and whether each action plan had been implemented. It has a three choice response—yes, no or partially.
The data collected from these interviews will be transcribed verbatim.
Researcher reflection and assessment of intervention component implementation
Research team members will regularly reflect on their interactions with study participants via discussions or field note entries. These discussions will focus on what action plans and subsequent PDSA cycles practice teams decide to address, why they chose that particular safety domain, and what context the decisions were made in (ie, small scale or system wide changes, practical or cultural constraints).
The research team will monitor the attendance at action planning meetings and workshops, action plans made by staff, documented evidence of action plans and participation in semistructured interviews following the intervention period. Adherence, content, reach, context, responsiveness and quality of delivery will be scored by researchers using data collected from semistructured interviews and researcher observation to determine a total intervention fidelity score for each practice.
Secondary outcomes
PC PMOS
An overall mean PC PMOS score, together with nine domain scores will be calculated using the mean of two or more responses, at baseline (T1) and post intervention period (T2). Change in PC PMOS mean score overall (T2−T1), change in each of the PC PMOS domain mean scores (T2−T1) and change in the PC PMOS domain selected by each practice relating to their safety intervention will also be calculated.
Staff safety culture
Staff safety culture has been shown to influence a variety of patient safety outcomes, including reduction of safety incidents55 and improvements in incident reporting.56 57 Therefore, a secondary outcome of this study is to assess the extent of staff safety culture change through participation in this intervention. The validated Agency for Healthcare Research and Quality’s Medical Office Survey (AHRQ MOS) on patient safety43 will be used to obtain data about staff perceptions of safety culture at baseline (T1) (prior to patient data collection) and after the intervention (T2). All staff will be invited to complete the survey. A minimum of five staff per practice are required to complete the survey. Surveys are anonymous and completion is voluntary.
The overall mean AHRQ MOS score, together with five culture specific AHRQ MOS domain scores, will be calculated using composite scores from the multiple responses within each domain.
Safety incident reports
Practice managers will be asked to provide a copy of their practice’s clinical risk management incident report register at the beginning of the study and after the intervention period. The number/type of patient-reported safety concerns will be compared with the number and type of safety incidents before and after the intervention.
Patient-reported safety concerns on the PC PMOS will be examined by calculating for each practice: (1) the total number and type of concerns; (2) the number of patients reporting one or more concerns; (3) the mean number of reports per participant; (4) the mean (patient-assessed) severity for reported concerns; (5) the range of the severity (patient-assessed) of reported safety concerns and (6) the average level of patient-assessed preventability of reported safety concerns (expressed as the median due to it being an ordinal variable).
Response rate
The age, gender and postcode of all patients presenting to each practice for appointments during the PC PMOS data collection periods will be extracted by the practice manager from the practice database. This information will be used to calculate a response rate and provide information about the types of patients who do and do not participate in safety improvement activities.
Practice context information
Practice managers will be asked to provide information about their primary care practice profile and demographics using a provided questionnaire at the beginning of the study. Questions on the survey include patient profile (number, age, gender, top health conditions treated, number of patients seen per week/month), practice profile (number and gender of GPs, practice nurses, administration staff), types of services provided, average consultation time, accreditation history and previous or current quality improvement work.
Data analysis
Triangulation and thematic analysis techniques will be employed to analyse the qualitative and content data collected from semistructured interviews, recordings of action meetings and workshops, and researcher reflection and observation. Both inductive and deductive approaches will be used to undertake the analysis. Inductive coding of qualitative and content data will be employed in addition to deductive approaches,58 which will extend or complement the current evidence and models of healthcare culture and safety improvement, patient feedback and response theory, health service implementation science, and engagement and adaptation theory.35–37 NVivo (QSR International Pty Ltd) will be used to support this analysis.
Quantitative data collected on the PC PMOS and secondary outcome questionnaires, will be analysed using SPSS Statistics V.24 (IBM). Continuous variables will be compared pre- and post-intervention using t-tests. Where the assumptions of a parametric test are violated (departure from normality, heterogeneous variance), the non-parametric Mann-Whitney U test will be used. Categorical variables will be compared using χ2 tests. Results will be reported as statistically significant where p≤0.05.
Patient and public involvement
Patients and consumers were directly involved in the development and validation testing of the PC PMOS tool. Their involvement included item generation, face validity and readability assessment. During this research process, patients and consumers also provided feedback about preferences for translation and implementation of the PC PMOS into primary care, such as the format (paper-based or electronic), and time and place for survey completion.24 These findings informed the design of this feasibility study.
Study timeline
The study is expected to take place over an 18-month period. Primary care practice recruitment and baseline data collection are expected to occur in early 2018. Final patient and practice data collection is expected to occur in late 2018 and early 2019.
Ethics and dissemination
No identifying information will be disclosed in any dissemination of research findings as per the ethical requirements of this study.
Study results will be disseminated through local and international conferences and peer-reviewed publications. The results from this study will also be disseminated to the participating primary care practices, and other local or nationally relevant stakeholder groups via circulation of peer-reviewed publications.
Discussion
The PC PMOS is an innovative and novel mechanism for collaborating with and engaging patients and primary care practices in safety improvement. The results from this study will identify if patient-reported feedback on safety is a valid tool for data-driven improvement and ongoing monitoring of safety in primary care. The findings may also assist with expanding and integrating PC PMOS into policy and practice on a larger scale, and will contribute to expanding knowledge in this under-researched area. Furthermore, the feasibility data obtained from this study will inform the development of future efficacy and effectiveness evaluations.40
Acknowledgments
We would like to extend our thanks to the primary care organisations and patients who agreed to participate in this study.
References
Footnotes
Twitter @andreahernan11
Contributors ALH conceived and designed the study, and is the principal investigator for the study. ALH drafted the first version of the manuscript and was responsible for its revisions. KK, SJG, KMN and VV contributed to the study design and writing specific sections of the manuscript. HB and MJB contributed to specific sections of the manuscript. All authors read and approved the final manuscript.
Funding This work was supported by the Western Alliance (grant number WA-733721).
Competing interests None declared.
Ethics approval Ethics approval was granted by the Deakin University Human Ethics Advisory Group, Faculty of Health (HEAG-H 175_2017).
Provenance and peer review Not commissioned; externally peer reviewed.
Patient consent for publication Not required.