Article Text

Download PDFPDF

Using patients’ experiences of adverse events to improve health service delivery and practice: protocol of a data linkage study of Australian adults age 45 and above
  1. Merrilyn Walton1,
  2. Jennifer Smith-Merry2,
  3. Reema Harrison1,
  4. Elizabeth Manias3,
  5. Rick Iedema4,
  6. Patrick Kelly1
  1. 1School of Public Health, University of Sydney, Sydney, New South Wales, Australia
  2. 2Faculty of Health Sciences, University of Sydney, Sydney, Australia
  3. 3Department of Medicine & School of Health Sciences, University of Melbourne & School of Nursing and Midwifery, Deakin University, Melbourne, Australia
  4. 4Agency for Clinical Innovation and University of Tasmania, Sydney, Australia
  1. Correspondence to Professor Merrilyn Walton; merrilyn.walton{at}


Introduction Evidence of patients’ experiences is fundamental to creating effective health policy and service responses, yet is missing from our knowledge of adverse events. This protocol describes explorative research redressing this significant deficit; investigating the experiences of a large cohort of recently hospitalised patients aged 45 years and above in hospitals in New South Wales (NSW), Australia.

Methods and analysis The 45 and Up Study is a cohort of 265 000 adults aged 45 years and above in NSW. Patients who were hospitalised between 1 January and 30 June 2014 will be identified from this cohort using data linkage and a random sample of 20 000 invited to participate. A cross-sectional survey (including qualitative and quantitative components) will capture patients’ experiences in hospital and specifically of adverse events. Approximately 25% of respondents are likely to report experiencing an adverse event. Quantitative components will capture the nature and type of events as well as common features of patients’ experiences. Qualitative data provide contextual knowledge of their condition and care and the impact of the event on individuals. Respondents who do not report an adverse event will report their experience in hospital and be the control group. Statistical and thematic analysis will be used to present a patient perspective of their experiences in hospital; the characteristics of patients experiencing an adverse event; experiences of information sharing after an event (open disclosure) and the other avenues of redress pursued. Interviews with key policymakers and a document analysis will be used to create a map of the current practice.

Ethics and dissemination Dissemination via a one-day workshop, peer-reviewed publications and conference presentations will enable effective clinical responses and service provision and policy responses to adverse events to be developed.

  • adverse event
  • patient-centred care

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • The use of data linkage is a novel strategy that will facilitate identification of a large number of recently hospitalised patients; patient experience studies to date consistently suffer from small and unrepresentative patient samples.

  • Linkage with admitted patient data allows us to triangulate patient reported experiences with information in their medical record for example, the health problem they presented with, how long they were in hospital and the treatments received; this information validates the self-reported patient data.

  • The 45 and Up Study cohort does not include a representative sample of culturally and linguistically diverse (CALD) participants. We address this weakness by analysing a subsample of data of CALD participants to explore whether their experiences differ from the wider sample.

  • The patient sample only includes those who are 45 years or older therefore is not a representative population sample. Adults aged 45 and above are more likely to be hospitalised and have more frequent contact with the health system; therefore knowledge of the experiences of this group is valuable. Data from younger patients may be explored in future work using this study as a model.


Adverse events are a significant problem

Preventable harm in hospitals due to healthcare activities is one of the top six health problems in the developed world.1 At least one in nine hospitalised patients suffer an adverse event (AE) which may require extracare or cause permanent harm or even death.2 It is estimated that AEs add 13–16% to hospital costs alone—at least one dollar in every seven spent on healthcare.3 This figure does not include human costs such as pain and suffering or loss of independence and productivity for patients and their carers, or costs of litigation and settlement of medical negligence claims. Measurable and sustainable improvements in quality and safety are yet to be realised.4

Patient symptoms are often indicators of the amount of harm arising from healthcare,1 with retrospective medical record review studies detailing a large rate of errors of commission and omission; many causing AEs.5 ,6 While a very low percentage of patients with AEs initiate a formal complaint, significant numbers (23–50%) report concerns related to an AE or undesirable events.7–11 Complaints from patients may have prompted the initial scrutiny of safety in healthcare, but patients have been largely ignored as a source of safety evidence or measurement and are rarely involved in reporting AEs outside symptoms.4 ,5 ,12

Little is known about the impact of AEs on patients

Health systems rely on patient experiences as a central and integral source of knowledge of health issues, policy development and service planning for almost all aspects of healthcare, yet we know little of their experiences, which remain largely absent from any input into better understanding the nature of AEs. Models of the impact of AEs on patients are lacking due to lack of research. Evidence-based models generated from patient reported data regarding the types of harm that patients experience is urgently required as these are the patient groups most vulnerable to harm.

Knowing and understanding patients’ experiences of AEs is crucial for creating and maintaining trust in their health providers and in the health system.13–16 Patient experience data of AEs will also contribute to understanding the patient populations such as older patients who may be at a greater risk of AEs or of negative sequelae from AEs.17–19 These data will provide insight into AEs that occur after discharge (eg, infection); some postdischarge AEs are also believed to be due to deficient handover of information but data are limited.20–22 Finally, patient experience data will contribute to understanding the association between an AE and patients’ subsequent health needs (eg, requiring hospitalisation), which has not been explored.23

AE reporting in Australia

All Australian states and territories have active incident reporting systems; in 2007 the Australian state of New South Wales (NSW) implemented the Incident Information Management System (IIMS). This system records data reports on ‘incidents’ (including AEs along with other untoward incidents such as accidents or thefts) and after analysis are reported annually. The relationship between error and harm in healthcare is complex and the focus in these systems on encouraging staff to report errors or ‘incidents’ is limiting.24 Most reported errors are errors of commission, although chart reviews suggest that acts of omission are implicated in twice as many AEs.9 ,25 There are significant levels of under-reporting in current voluntary reporting systems.(ref. 26, p.56) Doctors, for example, rarely report using incident reporting systems resulting in particular events dominating such as procedures by nursing staff or nurse witnessed AEs.27 ,28 Incident reports have thus been critiqued as ‘a non-random sample of identified hazards from a larger unknown universe of hazards’.29

Another limitation to fulfilling the benefits of AEs using incident reporting alone is ‘the limited amount and variable quality of the information found within individual incident reports’.30 A perception also exists among health professionals that only AEs with serious outcomes should be reported. There is usually no opportunity for patients to contribute to information conveyed through incident reporting thereby excluding their knowledge and experience in models of routine incident reporting. Our research will provide an understanding of AEs through the patient's own experience of AEs. This focus will add an important dimension to current clinical, health service and policy responses to AEs.

Open disclosure

Open disclosure is the requirement to provide honest explanations to patients and families who have been impacted by AEs. The requirement for open disclosure has been formalised in Australian healthcare settings, predominantly through endorsement of the National Open Disclosure (OD) Standard in the different states and territories.9

There is a paucity of evidence regarding the number of AEs that are disclosed to patients; one study of mistakes by junior doctors reported a low 12% disclosure rate of AEs to patients.31 A US survey found that only one-third of patients who had experienced an AE had been informed; but in another US study eliciting patient-identified AEs, 40% reported disclosure (defined as a positive answer to the question: “Did anyone from the hospital explain why the negative effects occurred?”).32 ,33 Evidence suggests that disclosure of AEs doubled the odds that patients would give higher ratings to the quality of their care.20


The aim of this research is to investigate their experiences of AEs and to identify data that can be used to create more effective service and policy responses to such events.


  1. To determine the patient experience of AEs, including patients’ experience of information sharing (disclosure) after an AE and the role of patients in reporting AEs.

  2. To ascertain the frequency and characteristics of AEs experienced by patients in hospitals located in the state of NSW, Australia.

  3. To describe the characteristics of patients who experience AEs, in comparison to patients who do not experience AEs.

  4. To carry out a detailed examination of present service and policy structures designed to deal with AEs and make recommendations for change based on our findings.

  5. To undertake community consultation about the findings and transfer project findings to the community, policymakers, health practitioners and service managers.


Ethics approval

Ethics approval was granted by the NSW Population and Health Services Research Ethics Committee (HREC/13/CIPHS/66 &2013/12/496). It was agreed between the two ethics committees that the latter ethics committee would be the single committee managing and monitoring the research.


This is a mixed methods study involving data collection via cross-sectional survey from a large research cohort, interviews with high-level policymakers, and data linkage between The Centre for Health Record Linkage (CHeReL), the Admitted Patient Data Collection (APDC), the Register of Births, Deaths and Marriages (RBDM) and the 45 and Up Study databases.

Setting and participants

Study population -survey

The sample will be drawn from the 45 and Up Study cohort. The 45 and Up Study is the largest cohort study in the Southern Hemisphere, including 265 000 participants across NSW who are aged 45 and above. The study completed recruitment in 2008 and all recruits have consented to the use of their data in research. A wide range of data is available on the 265 000 participants in the 45 and Up Study cohort, including age, postcode, education, ethnicity, lifestyle and habits, current medications, history of disease, surgical procedures and employment status.34 The choice to use an existing collection of participants was made because the difficulty of recruiting respondents through their health services is well established.35 Advertising through the public media does not necessarily attract a representative patient population. Nor do hospital selected patients meet criteria for an unbiased study cohort.

A primary benefit of the 45 and Up sample is that it will improve access to a general and guaranteed sample population. Past efforts to research this topic have been hampered by the difficulty of accessing an unbiased sample population. Accessing the appropriate survey population is an important consideration for this research. Because the majority of patients hospitalised in Australia are over 45 the use of this study population is ideal.36 Furthermore, AEs tend to occur in older people. In one Australian study (N=979 834), patients with an AE were older (M=62.5, 95% CI 62.7 to 63 years) than those without an AE (M=48.2, 95% CI 48.1 to 48.3 years).37 In another Australian study (N=1177), age >70 years was found to be a strong predictor (OR 1.9, 95% CI 1.3 to 2.6) of AEs following surgery.38 Our study participants will be a random sample of 20 000 from the 45 and Up Study cohort who were hospitalised at any hospital in NSW over the most recent 6 month period to the time of data collection. These participants will be identified using data linkage provided by the Centre for Health record Linkage (CHeReL) with the Admitted Patient Data Collection, which is administered by NSW Health.

Limitations of survey sample

A recognised limitation of the 45 and Up Study is that it is not representative with respect to individuals from culturally and linguistically diverse (CALD) backgrounds. For example, while only 25% of the 45 and Up Study were born outside of Australia, 2006 census data puts this figure at 39% for those aged 45 and above in NSW.36 In order to address this limitation we will analyse a subset of the surveys we receive from CALD participants to compare the experiences of those CALD participants who experienced an AE with those who did not experience an AE. We will also compare the CALD participants in the 45 and Up cohort to the non-CALD members of the cohort to see if there are specific differences or variations in the characteristics associated with the AE.

Survey sample size

We anticipate there to be 20 000 participants in the cohort eligible to take part in the study based on the number of participants within the cohort who have been hospitalised. In a 6-month period (July–December) in 2007, there were 18 460 public hospitalisations among 45 and Up Study participants. Hospitalisation rates will rise every year as the cohort ages; thus, we estimate 20 000 public hospitalisations (our study population) for the 6-month period January 2014–June 2014. We anticipate a 60% response rate to our questionnaire based on the response rates observed in other substudies of 45 and Up participants, yielding an estimated 14 000 respondents. We expect approximately 3500 respondents (25%) will have experienced an AE based on patient AE self-reporting figures reported elsewhere; the remaining 10 500 respondents are expected not to have experienced an AE and will be the control group.

Study population—interviews

We will conduct interviews with 30 individuals who have a direct role in managing AEs and thereby holding embodied knowledge about practice in the field. Sampling will be purposive, based on the roles that individuals fulfil. Participants will be initially identified by the project reference group associated with the project and subsequently through Clinical Governance Units of Local Health Networks. We expect to conduct around 30 interviews, including 10 working in policy settings (with clinical governance managers working in government policy and within key bodies such as the Australian Commission on Safety and Quality in Health Care, NSW Clinical Excellence Commission and the NSW Health Care Complaints Commission) and 20 with individuals working in practice settings (service managers, practitioners working as complaints managers). We will purposively seek to include a sample from two Metropolitan Local Health Districts (Sydney, Western Sydney) and two Rural and Regional NSW Local Health Districts (Far West, Hunter New England). The final number of interviews will be determined as the interviews progress. Interviewing will be halted at the point of data saturation once no new data are emerging.


The study will run from July 2013 to July 2016 with data collection starting November 2014. This study comprises five phases:

  1. Questionnaire development;

  2. Completion of contractual arrangements with data linkage partners;

  3. Administration of questionnaire;

  4. Analysis of survey responses;

  5. Mapping of existing formal service and policy responses to AEs;

  6. Transfer of project findings to the policy and practice community.39

Phase one—questionnaire development

In phase one, the survey for patients was developed and refined in a series of stages. Past patient experience research was used to develop items to determine whether patients experienced an AE and opportunities for patients to report AEs (during care, discharge, posthospitalisation). Iedema et al11 (chief investigator B) and Weingart et al's40 surveys of patient experiences of open disclosure were a significant foundation of data for the development of these items.5 We included the Picker Adult Inpatient Questionnaire (PAIQ) which is a validated questionnaire designed to elicit patients’ overall hospital experiences about their care and treatment.41 It has a high degree of face validity, construct validity and internal reliability consistency.41 Items about open disclosure and subsequent decision making about complaint or litigation were also incorporated. A validated instrument for assessing open disclosure by CI Iedema was used to assess the quality of formal open disclosure against new national indicators. Items from this instrument along with that used by Weingart in the USA were used to identify what information was provided about the AE; what action was taken to remedy the situation; what did not happen; intention of making a formal complaint or of seeking compensation; areas not addressed that the patient wanted addressed. We also included items to determine where and when the AE(s) occurred (ie, public/private hospitals, in/out NSW, within correct time-frame parameters etc in order to identify possible confounding variables). An expert panel reviewed several iterations of the survey to attend to issues such as length, content validity, scaling responses and potential for response bias.

The resulting survey comprises five parts A–E. Part A asks patients for details of their hospitalisation followed by an additional four further validated sections. Part B incorporates questions about care and treatment from the validated Picker questionnaire.41 Part C includes a series of items regarding the healthcare incident based on Weingart et al.5 ,40 Part D asks about experiences of disclosure based on work by Iedema et al.11 Part E explores patient reports based on items developed by Daniels and also work by Walton et al (chief investigator A).42 The survey was reviewed by the Project Reference Group for suitability and face validity and approved by the 45 and Up Study Management Committee.

Phase two—questionnaire administration

Phase 2 will include administration of the questionnaire to a sample of recently hospitalised patients from the 45 and Up data bank. The Centre for Health Record Linkage (CHeReL) is a NSW Ministry of Health Agency.43 CHeReL will link data from the Admitted Patient Data Collection with the 45 and Up Study Database to identify participants who have been hospitalised in the 6-month period between January and June 2014 using admission data. A random sample of 20 000 of the eligible 45 and Up participants will be sent a survey pack by the mailing house including an invitation letter, a participant information leaflet and the survey with a perforated consent form attached to it. In keeping with other studies of the 45 and Up participants, CALD or any other participants who have difficulty reading in English are advised to seek help from a friend or relative to understand the study materials and to complete the survey. Those who wish to participate are required to complete the consent form and return with their completed survey to the 45 and Up Study using the reply-paid envelope included in the pack. Participants who have not experienced an AE will only complete parts A and B of the survey, providing details about their hospital stay and their experience of this stay. Participants who have experienced an AE will complete a further three parts (C, D and E) in which they will answer questions about the AE, the Open Disclosure process and whether they made a complaint. Returned surveys will be screened by the 45 and Up Study team to ensure the consent form is complete and then sent on to the research team in a de-identified form for analysis. Any surveys with no signed consent form will not be included.

Phase three—analysis of questionnaire data

Phase 3 will involve qualitative and quantitative data analysis of completed questionnaires to obtain comprehensive information on patient characteristics, the nature and frequency of any AEs experienced and the impact of AEs on patient outcomes, whether the patient experienced an Open Disclosure process (formal or informal) and whether the patient made a complaint or initiated legal action.

Qualitative analysis

Two analyses of qualitative questionnaire data will be performed:

The conceptual framework of the International Classification for Patient Safety will be employed.42 The International Classification for Patient Safety categorises patient safety information using standardised sets of concepts to facilitate ‘description, comparison, measurement, monitoring, analysis and interpretation of information to improve patient care, and for epidemiological and health policy planning purposes’ (ref. 44, p.1). Each AE incident will be independently classified by two safety experts. Using the method devised by Runciman et al45 ‘natural categories’ will be determined and contributing factors, detection, mitigating factors, patient and organisational outcomes and ameliorating factors will be sought. The comments that patients make about their AE will be examined and thematically coded using discourse analysis as this has been effectively used for analysing patient experience in previous research.46 The analysis will be structured around the research questions. NVivo will be utilised to manage and track the discourse analytical process that is applied to the data. Patient identified opportunities for reporting AEs will also be examined and thematically coded, paying special attention to the type of AE, location in the hospital, age group, cultural background and the opportunities to report.47

Quantitative analysis

Estimates of the frequency and type of AE identified from questionnaire results will be conducted. The use of the WHO International Classification for Patient Safety to categorise patients’ descriptions of their AEs will ensure that foreseeable and known complications of conditions and treatments are separately identified and described. Patient experiences that fall outside the classification system will be identified and explored. Descriptive statistics (eg, means, SDs, frequencies and percentages) will be calculated to summarise data collected. In particular, estimates of the frequency and type of AE will be estimated with 95% CIs. Logistic regression will be used to determine if patients who experience AEs differ from patients who do not experience AEs. Potential risk factors will focus on patient characteristics such as age; gender; region; chronic disease(s), length of stay, transfer to other healthcare services. Other factors may be identified through the conceptual framework noted above. Appropriate model building strategies and model checking will be employed.48

Phase 4—mapping of existing service and policy responses

In phase 4 we will analyse existing service and policy responses to AEs by undertaking a mapping exercise, combining data from a detailed document mapping exercise and interviews with key actors in the field. The results of this analysis to make detailed recommendations about how these service and policy structures can best make use of the patient experience data. This mapping is an important component of the research project because government organisations are mandated to implement processes related to AEs. An understanding of current practice will allow our recommendations to fit into and build on existing practice.

Stakeholder interviews: Interviews will be semistructured and open-ended. Questioning will focus on present service and policy structures designed to deal with AEs and the extent to which patient experiences are utilised as a source of knowledge in service and policy design. Participants will also be asked which documents they draw on in their work. They will be contacted directly via email and asked to participate in an interview which will take place at their place of work, or by phone if they prefer. Interviews will be recorded with the participant's permission and transcribed.

Interview Data Analysis: Data will be hand coded through the qualitative data analysis software NVivo. Thematic analysis will be conducted based around the project research aims. Coding reliability will be improved through independent coding of a subsample of the interviews by different members of the research team. Initial thematic codes from all interviews will be brought together and recoded to draw out subcodes. Individual sections of the data will be freshly coded to ensure an alignment between initial and later coding. Single instances of codes will be removed and those with close similarity to other codes will be merged. A narrative of the themes derived from the coding will be created. Representative quotations will be selected to demonstrate the themes where necessary.

Document Analysis: The aim of the documentary analysis will be to find and summarise the formal documents which structure practice around AEs. Health department policies, Guidelines distributed by Local Health Districts, quality and safety protocols published by the NSW Clinical Excellence Commission and the Australian Commission on Quality and Safety will be identified through a range of approaches: (1) literature review—by locating any formal documents cited by academic sources; (2) reference group—through suggestions made by the project research group; (3) interview respondents—through locating any documents spoken about in the interviews. Documents will be summarised according to purpose and audience.

Phase 5—transfer of research findings.

Phase 5 will involve transfer of findings to the community, policymakers, service managers and agencies. We will seek feedback from the community about opportunities to contribute to AE reporting.

The analysed results from phases 1 to 3 that relate to patients’ perceptions and suggestions will form the basis of the community consultation. We will hold a 1-day workshop based on the WHO Patients for Patient Safety Workshop model. This model brings together committed patients and family members from rural and metropolitan areas to establish a network of champions, many of whom were nominated by health regions or other healthcare organisations who see them as partners in improving patient safety.

Project advisory committee

A reference group of key stakeholders including representatives from patient safety agencies, consumers, policymakers and service providers has been convened to assist the research team from the inception of the project. The group includes representatives from the Sax Institute, the Clinical Excellence Commission, the Australian Commission for Safety and Quality in Health Care, Health Consumers NSW and the North Sydney Local Health District. This spectrum of members provides a diverse range of perspectives on the project topic and proposed methods to be employed which are discussed at bi-annual reference group meetings.


Project outcomes

The results of this study will have a profound impact on the management of AEs in Australian hospitals and add to international knowledge on AEs. For the first time we will have comprehensive evidence to enable health departments, practitioners, health educators and the community to understand the impact of AEs on patients.

Achieving the five study objectives will result in a number of important outcomes. Evidence of the patient experience of their hospitalisation will be useful in the development of policy and will assist practitioners to change practice or introduce new activities to reduce AEs. We will obtain a profile of patients who self-report an AE. This profile will enable future identification of groups at risk of AEs (such as age, gender, time of admission, cultural and language background, comorbidities) for the development of targeted safety strategies, including health service design, education and training of health clinicians. A map of the incidence and characteristics of AEs as reported by patients will assist the development of targeted programmes to reduce AEs.

We will provide insight into patient reactions to the process of information sharing that will be highly influential for reviews of communication processes and open disclosure guidelines across Australia. This is highly relevant information for practising clinicians in the health system as well as governments and indemnity insurers and those engaged in curriculum development and community education. Detailed policy and service recommendations will use our study data to enhance existing governance structures around AEs. Our work will also facilitate assessment of whether there are opportunities for patients to play a greater role in reporting AEs and how this could be realised. Finally, this work will produce a greater awareness within the health system and the community more broadly about the impact of AEs on patients. This project is also the first that we are aware of to use data from patient medical records to explore AEs. Using linked data, we will be able to triangulate patient reports with the content of their medical records and identify the extent of congruence between these information sources.

Ethics and dissemination

The 45 and Up Study cohort consent to their 45 and Up Study baseline data being linked with the Admitted Patient Data Collection; using linked data in this way to identify recently hospitalisation 45 and Up Study participants does not pose any ethical concerns. The cohort also consents to being contacted for health-related research and appropriate ethical approval has been obtained to approach these participants. Dissemination of findings to key stakeholders is a critical feature of the study protocol in order to support the community, health service providers and policymakers to develop more effective responses to AEs in patient care. In phase 5 of the study we will transfer findings to the community, policymakers, service managers and relevant agencies. This process includes a 1-day workshop based on the WHO Patients for Patient Safety Workshop model. We will also produce peer-reviewed publications and present findings at national and international conferences.



  • Contributors All the authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; drafting the article or revising it critically for important intellectual content; and to the final approval of the version to be published as set out in the Uniform Requirements for Manuscripts Submitted to Biomedical Journals. MW, RI, EM, PK and JS-M conceived and designed the research idea. MW and RH are primarily responsible for the acquisition of the data. Each author is responsible for the analysis and interpretation of the data. JS-M and RH were primarily responsible for drafting this protocol. Each author has been involved in drafting the manuscript or revising it critically for important intellectual content; have given final approval of the version to be published; and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Funding This project is funded by a National Health and Medical Research Council (NHMRC) Project Grant No. 1049703.

  • Competing interests None.

  • Ethics approval The study has received ethics approval from the Health Research Ethics Committee at the University of Sydney and the NSW Population and Health Services Research Ethics Committee.

  • Provenance and peer review Not commissioned; internally peer reviewed.

Linked Articles