Article Text
PDF
Abstract
Introduction Despite the growing body of research on quality and safety in healthcare, there is little evidence of the association between the way hospitals are organised for quality and patient factors, limiting our understanding of how to effect large-scale change. The ‘Deepening our Understanding of Quality in Australia’ (DUQuA) study aims to measure and examine relationships between (1) organisation and department-level quality management systems (QMS), clinician leadership and culture, and (2) clinical treatment processes, clinical outcomes and patient-reported perceptions of care within Australian hospitals.
Methods and analysis The DUQuA project is a national, multilevel, cross-sectional study with data collection at organisation (hospital), department, professional and patient levels. Sample size calculations indicate a minimum of 43 hospitals are required to adequately power the study. To allow for rejection and attrition, 70 hospitals across all Australian jurisdictions that meet the inclusion criteria will be invited to participate. Participants will consist of hospital quality management professionals; clinicians; and patients with stroke, acute myocardial infarction and hip fracture. Organisation and department-level QMS, clinician leadership and culture, patient perceptions of safety, clinical treatment processes, and patient outcomes will be assessed using validated, evidence-based or consensus-based measurement tools. Data analysis will consist of simple correlations, linear and logistic regression and multilevel modelling. Multilevel modelling methods will enable identification of the amount of variation in outcomes attributed to the hospital and department levels, and the factors contributing to this variation.
Ethics and dissemination Ethical approval has been obtained. Results will be disseminated to individual hospitals in de-identified national and international benchmarking reports with data-driven recommendations. This ground-breaking national study has the potential to influence decision-making on the implementation of quality and safety systems and processes in Australian and international hospitals.
- hospital quality management systems
- patient level factors
- patient safety
- hospital performance
- quality improvement
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- hospital quality management systems
- patient level factors
- patient safety
- hospital performance
- quality improvement
Strengths and limitations of this study
This is an in-depth study examining quality management systems and patient factors in a large sample of hospitals, substantially advancing earlier work.
We consulted widely with Australian and international stakeholders in order to operationalise definitions and measures in acute myocardial infarction, stroke and hip fracture.
The study informs the design of future interventions and quality management systems in Australia and internationally.
Key challenges include sufficient sample sizes, the recruitment of hospitals (n=∼60), complex ethical, governance and site approval processes and attrition of recruited hospitals during the approval process.
Introduction
Despite decades-long efforts to improve healthcare delivery systems worldwide, there remain deficiencies in the quality and safety of care patients receive.1–4 Hospitals are continuously assessed for levels of quality, safety and performance using a range of methods, bolstered by policies and regulatory activities at national, state or regional, local, and hospital levels.5 Assessments can be resource-intensive, and are often conducted with invalidated tools, or in a piecemeal manner.6 Results are typically used to demonstrate policy compliance, rather than to stimulate improvement.
Few studies in healthcare research systematically examine relationships between quality interventions and patient outcomes.7 Despite the growing body of research, there is little evidence of the association between the way hospitals are organised for quality and patient outcomes, limiting our understanding of how to effect large-scale change.8 ,9 In recognition of this, a large multinational study ‘Deepening our Understanding of Quality Improvement in Europe’ (DUQuE) was undertaken between 2009 and 2014.10 This landmark study, undertaken in 188 hospitals across seven countries (Czech Republic, France, Germany, Poland, Portugal, Spain and Turkey), examined the effectiveness of quality management systems (QMS) and strategies at organisation, department, professional and patient levels for stroke, acute myocardial infarction (AMI), hip fracture and obstetrics. DUQuE aimed to improve the way quality strategies and their effects on performance and outcomes were measured in European hospitals11–14 by uncovering the relationships between them. This study also aimed to identify factors influencing the uptake of quality management activities by hospitals, and establish the feasibility of using routinely collected data to compare hospital performance.10
DUQuE developed and validated several tools for European acute care hospitals, assessing the implementation of QMS,15 compliance with quality processes and the extent of implementation of clinical quality strategies at hospital level,16 departmental performance and the implementation of quality management activities across specific departments and care pathways,17 and levels of clinical management provided by doctors and nurses.18 Key published findings from DUQuE include:
Low baseline performance and high variations were found on a wide range of evidence-based quality and safety indicators; levels of patient involvement in quality management were found to be low and appeared tokenistic;19 ,20
Quality management was associated with clinical effectiveness of care, and to a lesser extent for patient safety culture; however, no link with patient-reported experience of care was established;19 ,21
Greater variation of evidence-based organisation and patient safety strategies was found within, rather than between, countries;22
An organisational management structure which supports innovation in care was associated with a more developed QMS;23
Discussions of quality performance at executive board-level meetings were associated with higher levels of quality management;24 and
Accreditation and certification were associated with clinical leadership, systems for patient safety and clinical review, but not with clinical practice.25
DUQuE produced an evidence-based guide on effective quality and safety strategies to assist purchasing agencies and hospital managers with decisions on implementation and improvement.26 However, despite providing evidence of associations between hospital quality and a range of processes and outcomes, generalisability of DUQuE findings are limited to Europe. Extending this work to other healthcare systems will provide in-depth understanding of the relationships that exist within these systems, and will enable hospital benchmarking using standardised measures both within and between countries—an international first.
While DUQuE took a pathway analysis approach, it did not distinguish between QMS and clinical treatment processes delivered in emergency departments (EDs) and those of inpatient departments treating patients with stroke, AMI and hip fracture. Given that the ED is a common point of entry for all three conditions, inclusion of ED factors in the data for each condition is likely to bias the study towards ED performance. It is also likely that differences will exist between the quality of care delivered in the ED and other hospital departments providing care for stroke, AMI and hip fracture following admission, thereby biasing the findings for those departments.27 Although there is work distinguishing the quality of care provided in the ED, and other inpatient department care,28 the relationship between quality strategies used in the ED, and quality of care provided in the ED, has not been assessed. In addition, no distinction has been made between quality strategies and quality of care provided in the ED and quality strategies and quality of care provided in subsequent inpatient departments for any acute medical condition. Understanding the components of ED care in relation to specific care pathways can help identify which aspects of care are attributable to the ED versus other wards or departments. It is important to monitor the critical ED segment of care and distinguish this from care provided elsewhere,28 so that barriers to the delivery of guideline-concordant, evidence-based care can be identified and addressed using relevant, context-specific interventions.29
The Deepening our Understanding of Quality in Australia (DUQuA) study involves substantial modification to the DUQuE study design, to advance the international evidence base. DUQuA aims to measure and examine relationships between organisation, department, and patient level factors within Australian hospitals. Specific research questions are:
What department-level factors (ie, QMS, leadership, culture) are associated with patient-level factors (ie, clinical treatment processes, patient perceptions of care and nationally recorded clinical outcomes)?
What organisation-level factors (ie, QMS, size, location) including ED factors (ie, QMS, leadership, culture) are associated with patient-level factors (ie, clinical treatment processes, patient perceptions of care and nationally recorded clinical outcomes)? How much does each factor contribute to the total variation in patient factors?
Methods and analysis
Conceptual framework
Working with DUQuE research leads (RS and OG), members of the DUQuA team (RC-W, NT and JB) assessed the evidence from DUQuE to refine the conceptual model. The DUQuA team (RC-W, NT, VP, EH and JB) developed the analytical framework iteratively at strategic and methodological meetings, streamlined the original model, and built the final main constructs and the relationships we aim to test.
In line with DUQuE, we aim to further our understanding of the characteristics of Australian hospitals and hospital departments, and examine links to the quality of care provided to patients.9 ,10 The DUQuA conceptual model (figure 1) outlines the expected relationships between organisation (hospital) and department-level constructs with patient-level factors (ie, clinical treatment processes, patient perceptions of care and nationally recorded outcomes) in three care pathways: stroke, AMI and hip fracture. Selection of these conditions is addressed below.
DUQuA conceptual model (DUQuA, Deepening our Understanding of Quality in Australia; QMS, quality management systems).
The specific objectives of DUQuA are to assess relationships between: (1) department-level QMS, leadership, culture and patient-level factors; (2) organisation-level QMS and patient-level factors; and (3) ED QMS, leadership, culture and patient-level factors.
Research design
The DUQuA project is a multilevel, cross-sectional study with data collection at organisation, department, professional and patient levels in hospitals across Australia. Organisation-level measures will consist of external assessment and self-reported QMS, and department-level measures will consist of external assessment of QMS and self-reported perceptions of leadership and culture. Patient-level measurements based on clinical treatment processes, patient perceptions of care and nationally recorded clinical outcomes are nested in hospital departments, which are in turn nested in hospitals in each of the six Australian states and two territories.
Sample size
Given our research design requires multilevel analysis, we used the following equation30 to determine the design effect of the study and inflate the sample size accordingly:
where nh is the number of hospitals, mp the number of departments needed in a one-level regression according to a standard sample size calculation, np,h the number of departments per hospital (in this case three), and ICC the intraclass correlation coefficient.
The sample size for the DUQuA multilevel model is the design effect multiplied by the sample size required for a standard linear regression. To identify which ICC to use to calculate our sample size, we used an α of 0.05 and power of 80%, in line with convention, and undertook a literature search of similar studies and ICCs. Taljaard et al31 published a set of ICCs for maternal health outcomes from the 2005 WHO Global Survey on Maternal and Perinatal Health. The ICC for the only continuous process variable analysed was 0.081, the median ICC for all process measures was 0.161, and the median for all continuous measures was 0.05. The same study indicated that ICCs for institutions tended to be larger for clinical treatment processes than patient outcomes. Similarly, for process variables in primary care settings, Campbell et al32 found that the ICC tended to range between 0.05 and 0.15, compared with outcome variables which were generally less than 0.05. James33 reported a median ICC of 0.12 for measures of organisational climate, and Forbes and Taunton34 reported a hospital-level ICC of 0.03–0.07 for three organisational phenomena. Smits et al35 also published hospital-level ICC for patient safety culture-related outcomes, all of which are under 0.07.
Taking this into account, an ICC of 0.16, higher than most ICCs in the literature, was used to calculate a conservative sample size. It reflects the median from the maternal health process measures, the most similar group of available reported ICCs. For this ICC, the minimum sample size required to answer our primary research question is 43 hospitals. However, in order to increase the power of the study, to be in line with simulation papers (which have shown that standard errors are estimated to be inadequate by 15% with <50 clusters),36 and to allow for hospital refusal to participate or subsequent withdrawal, a decision to oversample at the organisation level was made.
Selection of clinical conditions
DUQuE originally tested relationships between QMS and four conditions (AMI, stroke, hip fracture and obstetrics). These conditions were selected because they cover a broad range of hospital care, there are evidence-based standards for clinical treatment processes against which compliance could be assessed, and there is demonstrated variability in both compliance with clinical treatment process measures and nationally reported outcomes of care, such as complications or mortality, allowing for analysis of associations between these measured constructs.10 DUQuE conditions were retained except obstetrics; in Australia, maternity care is often provided in dedicated hospitals. Replacement conditions were considered but none were deemed suitable due to a lack of evidence-based standards for clinical treatment processes, or a lack of variability in compliance with processes or outcomes.
Selection and recruitment of hospitals
Hospitals will be approached in all jurisdictions: the Australian Capital Territory (ACT), New South Wales (NSW), the Northern Territory (NT), Queensland (QLD), South Australia (SA), Tasmania (TAS), Victoria (VIC) and Western Australia (WA). To guarantee adequate data for collection, the inclusion criteria are (1) general public hospitals with approximately 200 beds or more, (2) with an ED, and (3) that regularly admit more than 30 each of stroke, AMI and hip fracture patients over a 3-month period. Preliminary work indicated that 70 hospitals meet these criteria and will be invited to participate. Table 1 lists the number of hospitals identified as eligible to participate by state.
Number of eligible hospitals by state or territory
The Director of Medical Services (DMS), or an equivalent senior hospital leader, will be the initial point of approach for recruiting hospitals. Following discussion with the DMS and other relevant hospital stakeholders, a letter will be sent to the hospital executive team of eligible hospitals inviting their participation. The DMS or equivalent will be recruited as a ‘Local Principal Investigator’ (LPI) at each hospital that accepts this invitation, acting as a hospital coordinator. For each hospital, one or more healthcare professionals will be recruited to support the LPI and coordinate data collection. The Royal Australasian College of Medical Administrators (RACMA) has agreed to support the DUQuA team during recruitment and data collection. Other recruitment methods, including engaging a hospital executive network, or personal invitation, will be employed as necessary. LPIs and healthcare professionals will assist in operationalising the standardised recruitment strategy for staff and patients.
Selection and recruitment of participants
The source population for the study includes the Quality Manager at each participating hospital; all clinicians treating patients with stroke, AMI and hip fracture; and all patients with stroke, AMI and hip fracture meeting the study inclusion criteria. Where possible, we have adopted the same inclusion/exclusion criteria as DUQuE (The DUQuE Project team. Deepening our Understanding of Quality Improvement in Europe: Manual for Hospital Coordinators [unpublished material], 2011.). Specific details on selection of participants follows.
Quality Manager
Description
This is the person responsible for the coordination of quality improvement activities in the hospital. He or she should have a good overview of all activities supporting quality improvement. This may be a formally appointed Quality Manager, or another professional in charge of managing aspects of quality in the hospital.
Recruitment
The Quality Manager will be invited by the DUQuA team to complete the Quality Management Systems Index (QMSI) questionnaire (see ‘Organisation-level measures’). If more than one professional meets the description, then the most experienced person will be invited to participate until one questionnaire is completed.
Clinicians
Description
Clinicians will be invited to complete leadership and culture questionnaires. Clinician inclusion criteria are doctors, nurses and allied health professionals practising in the clinical area of the ED or a particular pathway (stroke, AMI, hip fracture) for at least 50% of their work time.
Recruitment
All clinicians that meet the inclusion criteria will be invited to participate in the leadership and culture questionnaire (see ‘Department-level measures’) until a minimum of 20 questionnaires for each department are completed at each hospital. The DUQuA team will invite clinicians to complete the questionnaire, by coordinating the dissemination of a web link to the online questionnaire.
Patients
Description
There are two patient-level measures that involve the collection of patient data. The inclusion criteria for each condition are listed in table 2. Patients who are admitted to the hospital with a primary diagnosis other than those listed in table 2 will be excluded.
Patient inclusion criteria
Recruitment for patient perceptions of care measure
All patients meeting the inclusion criteria who present to the hospital during the study period will be invited to complete the questionnaire. Additional criteria and screening will be applied to ensure that participation is voluntary and that participants are able to give informed consent. Healthcare professionals at each hospital will be trained to invite patients to complete the questionnaire. Recruitment will continue until a minimum of 30 questionnaires are completed for each condition at each hospital.
Clinical treatment processes indicator audit
The inclusion criteria (table 2) will be applied to records of patients admitted to the hospital between September 2014 and February 2015. Records of patients admitted to the hospital with a primary diagnosis other than those listed in table 2 will be excluded. For each condition in each hospital, 60 consecutive patient medical records will be extracted and reviewed until 30 audits of patients meeting the inclusion criteria have been completed. Healthcare professionals trained by the DUQuA team will undertake the medical record reviews.
Development, selection and collection of measures
Measures developed and validated for DUQuE9 ,10,15–17 ,22 ,37 were initially considered for DUQuA. During a 3-day consultation with the DUQuE lead researchers (RS and OG), we agreed on which constructs should be excluded from the DUQuA study based on a lack of relevance to the Australian healthcare system, poor variance demonstrated by a particular measure, an absence of a relationship with any other variables measured in DUQuE, or misalignment of the construct with our revised theoretical framework. Excluded constructs are: external pressure, hospital governance, hospital culture, hospital professional involvement, patient involvement in quality management and professionalism.
To establish new constructs not assessed as part of DUQuE (ie, department-level leadership and culture), literature reviews were performed to identify existing measures. Explicit criteria were used to assess and select measures, including psychometric properties, level of evidence and appropriateness for the Australian healthcare system. In particular, the significance of interhospital transfers in the care of patients with stroke, AMI and hip fracture was identified. Australian studies indicate that 14% of patients with hip fracture and a quarter of patients with AMI are transferred between hospitals, and 63% of Australian hospitals have ED protocols for interhospital transfers of patients with stroke.38–40 Consequently, variables regarding the potential impact of interhospital transfer were incorporated to allow for their measurement.
Where the DUQuE measures or items were unsuitable for the Australian context, or where it was felt that newer evidence-based measures were available, we reviewed the literature for new measures, consulted with expert clinicians and refined specific items to ensure measures were appropriate and relevant. Stakeholder groups of healthcare professionals and researchers with expertise in clinical practice, quality or both were established to inform revisions to DUQuA measures and the introduction of new measures. New measures have been piloted in at least one hospital (June 2015 and September 2015) to assess suitability and feasibility of use. The final list of DUQuA measures, content descriptions, evidence of reliability and validity, and data collection methods, is provided in table 3.
DUQuA measures: content, evidence of reliability and validity, and collection methods
Organisation-level measures
Development and selection
The three measures of organisation-level QMS are the QMSI, the Quality Management Compliance Index (QMCI) and the Clinical Quality Implementation Index (CQII). These measures, validated by DUQuE, have been modified by the DUQuA team in consultation with national quality assessment experts (eg, the Australian Council on Healthcare Standards) for the Australian context and to reflect changes made to the condition-specific indicators. The adapted QMSI is a seven-subscale survey which provides a proxy measure for the managerial aspects of quality management that might influence the implementation of quality systems in hospitals.15 The adapted QMCI is a four-subscale proxy measure for compliance with procedures used to plan, monitor and improve quality of care.16 The adapted CQII is a seven-subscale proxy measuring implementation of QMS, including whether specialised quality management committees and hospital policies exist, and the level of compliance monitoring, system sustainability and improvement focus.16
Collection
The Quality Manager will complete the self-report paper QMSI questionnaire. The QMCI and CQII will be completed in a quality assessment site visit by experienced external hospital surveyors who will be trained by the DUQuA team.
Department-level measures
Quality management systems
Development and selection
A modified version of a checklist developed and validated by DUQuE17 will assess department-level QMS. The four measures (see table 3) are Specialized Expertise and Responsibility (SER), Evidence-Based Organisation of Pathways (EBOP), Patient Safety Strategies (PSS) and Clinical Review (CR).17
Collection
Department-level QMS assessment will be completed in a quality assessment site visit by experienced external hospital assessors drawn from a pool of Australian Council on Healthcare Standards surveyors who will be trained by the DUQuA team.
Culture and leadership
Development and selection
The culture measure consists of two domain subscales from the Australian version44 of the validated Safety Attitudes Questionnaire (SAQ)41 ‘Teamwork climate’ and ‘Safety climate’. Two scales, each previously validated and measured on a four-point Likert scale, were combined to form the leadership measure: the ‘Perceptions of management’ domain subscale from the SAQ,41 and the Leadership Effectiveness Scale.42 These measures are combined in one questionnaire.
Collection
Clinicians will complete the self-report questionnaire online.
Patient-level measures
Clinical treatment processes indicators
Selection
The DUQuA team started with indicators for each condition used in the DUQuE study,10 and refined these following consultation with expert stakeholder groups for each condition and the Australasian College for Emergency Medicine (ACEM), and examination of indicators used by condition-specific Australian national registries. Box 1 presents the final list of DUQuA clinical treatment process indicators.
Deepening our Understanding of Quality in Australia (DUQuA) clinical treatment process indicators for stroke, acute myocardial infarction (AMI) and hip fracture
▸ Clinical treatment process indicators (acute ischaemic stroke OR not specified stroke)
1. Patient screened for stroke according to validated stroke screening tool (eg, FAST or ROSIER scale)
2. Patient screened for eligibility* for thrombolysis (intravenous recombinant tissue plasminogen activator (Rt-PA))
3. Patient received a brain scan within 60 min of arrival to the hospital
4. Eligible* patient received intravenous rt-PA†
5. Eligible* patient received intravenous rt-PA† within 60 min of arrival to the hospital
6. Swallow screen or swallow assessment performed before the patient was given oral food or fluids, or oral medication
7. Patient admitted to stroke unit
8. Patient spent at least 90% of their acute hospital admission on a stroke unit
9. Patient discharged on statin therapy or lipid-lowering medication unless contraindicated‡
10. Patient without atrial fibrillation discharged on antiplatelet therapy unless contraindicated‡
11. Patient with atrial fibrillation discharged on oral anticoagulants unless contraindicated‡
12. Patient discharged on an antihypertensive agent unless contraindicated‡
13. Patient received risk factor modification advice before leaving the hospital unless patient had severe cognitive impairment or communication impairment, or otherwise contraindicated‡
14. Ongoing care plan developed with and provided to the patient and/or family prior to discharge
▸ Clinical treatment process indicators (acute ST-segment elevation myocardial infarction)
1. ECG performed and interpreted before or within 10 min of arrival to the hospital for patient with chest pain or other acute coronary syndrome symptoms
2. Patient received aspirin prior to arrival at the hospital
3. Patient received aspirin prior to being sent to the catheterisation laboratory
4. Eligible§ patient received fibrinolysis or percutaneous coronary intervention (PCI)¶
5. Eligible§ patient received fibrinolysis before or within 30 min of hospital arrival
6. Eligible§ patient, treated with PCI,¶ had a door-to-device time of 90 min or less for PCI-capable hospital, or 120 min or less if transferred from a non PCI-capable hospital
7. Patient discharged with a written, individualised care plan (such as graded physical activity, smoking cessation therapies and addressing psychosocial needs) unless discharged on palliative care pathway or where adherence to a secondary prevention plan is not indicated
8. Patient discharged on aspirin or dual antiplatelet therapy unless contraindicated‡
9. Patient discharged on a statin or lipid-lowering therapy unless contraindicated‡
10. Patient referred to cardiac rehabilitation or other secondary prevention programme unless contraindicated‡
11. Discharge summary provided to general practitioner or ongoing clinical provider of patient within 48 h of discharge
▸ Clinical treatment process indicators (hip fracture)
1. Patient initial pain score recorded within 30 min of arrival to the hospital
2. Patient received analgesia or nerve blocks within 30 min of arrival to the hospital, unless patient declined pain relief
3. Patient pain reassessed within 60 min of arrival to the hospital
4. Orthopaedic team notified within 60 min of patient arrival to the hospital
5. Surgery performed within 48 h of arrival to the hospital
6. Patient received prophylactic antibiotic treatment within 60 min prior to surgical incision
7. Patient received prophylactic thrombolytic treatment within 48 h of arrival to the hospital
8. Surgery performed with the aim of allowing patient to fully weight bear without restriction in the immediate postoperative period
9. Patient mobilisation started day after surgery unless contraindicated‡ or patient declined
10. Patient offered a dedicated mobilisation session to regain function at least once per day until discharge
11. Patient received a specialist falls assessment (including falls history, cause of fall index, and risk factors for falling and injury) from a trained clinician prior to discharge
12. Patient received bone protection medication for secondary fracture prevention at discharge
*Eligibility for intravenous rt-PA: exclude patients who present at first emergency clinical contact more than 4.5 h after symptom onset, patients with unknown time of onset or patients where thrombolysis is contraindicated.
†Intravenous rt-PA should only be given in hospitals with appropriately qualified clinicians and adequate infrastructure and facilities.48 Evidence of transfer policies and procedures will be sought in hospitals without the capability to perform intravenous rt-PA.
‡Contraindications include advanced care directives, being on a palliative care pathway and clinical judgement. Contraindication must be documented.
§Eligibility for reperfusion: exclude patients who present at first emergency clinical contact more than 12 h after symptom onset, or where reperfusion is contraindicated.
¶PCI should only be given in hospitals with appropriately qualified clinicians and adequate infrastructure and facilities.49 Evidence of transfer policies and procedures will be sought in non PCI-capable hospitals.
Development of stroke indicators
DUQuE clinical treatment process indicators for stroke were presented to an expert stakeholder group that included members of the National Stroke Foundation (NSF). The NSF collects cross-sectional data in a biennial review of 40 consecutive eligible patient medical records by trained auditors of stroke clinical treatment processes over a 6-month period, from approximately 177 hospitals across Australia. Indicators developed for this registry align with the ‘Indicator Specification: Acute Stroke Clinical Care Standard’, developed by the Australian Commission on Safety and Quality in Health Care (ACSQHC) and certified by the Federal Minister for Health.45 Therefore, the indicators were amended to align with the NSF Clinical Audit and the national ACSQHC standards.
Development of AMI indicators
DUQuE AMI clinical indicators were compared against the ‘Indicator Specification: Acute Coronary Syndromes Clinical Care Standard’, developed by the ACSQHC and certified by the Federal Minister for Health.46 Final indicators chosen for DUQuA align with the relevant indicators in this document.
Development of hip fracture indicators
DUQuE hip fracture clinical indicators were modified for DUQuA in consultation with the Co-Chair of the Australian and New Zealand Hip Fracture Registry (ANZHFR), and the guidelines produced by the ANZHFR.47
Collection
An audit of clinical treatment process indicators will be undertaken. For hospitals that submit data to the NSF Clinical Audit, data will be directly retrieved from the most recent audit (September 2014 to February 2015) within the NSF database.i In hospitals not contributing to the NSF Clinical Audit, data will be collected via medical record review. As no national AMI or hip fracture registries were active during the NSF data collection period, AMI and hip fracture clinical treatment process data will also be collected via medical record review.
Patient perceptions of care
Development and selection
A modified version of the eight-scale, self-report UK-validated Patient Measure of Safety (PMOS) questionnaire,43 ,50 which assesses patients’ perceptions of the factors contributing to patient safety, will measure subjective patient factors. The instrument presents 43 items on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).
The PMOS has been piloted in one hospital (ethically approved, HREC reference LNR/14/157) with stroke, AMI, and hip fracture patients. Stage 1 involved four patients who participated in a ‘think aloud’ study to provide an indication of the suitability of the content, length, and presentation of the measure. Stage 2 tested the refined measure with 31 patients to establish the suitability of the PMOS for use with a cohort of patients similar to patients likely to participate in DUQuA, and demonstrate the level of variability produced by the measure.
Collection
Patients will complete the self-report paper questionnaire. Patients may request assistance to complete the questionnaire from a family member or the trained clinician responsible for administering the questionnaire.
Nationally recorded clinical outcomes
Development and selection
Clinical outcomes will be determined by analysing routine data collected by hospitals and submitted to the Australian Institute of Health and Welfare (AIHW), a national Australian Government data collection agency. Data will include the number of cases of each condition discharged by diagnosis, (including stratification by age and gender), in-hospital mortality, length of hospital stay, readmission, and mortality. Data will also be collected for the number of: percutaneous coronary interventions (AMI), thrombolysis procedures (stroke), and hip fracture operations performed in each hospital over the study period.
Collection
The DUQuA team will retrieve the required data from the AIHW.
Data analysis
The analysis plan for each research question is presented in table 4. Descriptive statistics will summarise the spread, frequencies or distributions of the variables of interest. Simple correlations, linear and logistic regression and multilevel modelling will be undertaken. The exact approach to the statistical analysis will depend on the distribution of the data, the variables that will need to be adjusted for (eg, case mix), and the final sample size achieved.
DUQuA statistical analysis plan
Multivariate modelling will take the form of a multilevel or hierarchical model in the first instance; with hospitals at the top level.ii This type of regression will adjust for both hospital-level and department-level covariates. Subject to the results of this modelling, the levels may be adjusted to select individual patient impact measures as level 1 or, alternatively, a single-level model utilised. Employing multilevel methods will allow for the amount of variation in patient-level factors attributed to the hospital and department level, as well as which factors contribute to this variation, to be identified.
Ethics and dissemination
Regionally based Human Research Ethics Committees (HRECs) in Australia have approved the DUQuA project for NSW (14/206)iii, VIC (15/36)iii, ACT (15/131)iii, SA (15/260)iii, WA (15/118)iii and QLD (15/361)iii. Remaining regionally based ethical applications are under review, and governance requests for participating hospitals will be submitted following confirmation of the final sample.
Results of the DUQuA project will be disseminated to participating hospitals in the form of confidential national and international benchmarking reports with a set of data-driven recommendations. An extensive publication plan will support the dissemination of results including reporting on the international validation of measurement tools, associations between measures using linear and multilevel models, the extent to which hospital benchmarking can identify consistent high and low performers, and the impact of the ED and patient transfers on the delivery of care for the three selected conditions. This ground-breaking study, with international implications, has the potential to build a coalition of hospitals for subsequent interventional work, influence policymaker decisions on the implementation of quality and safety systems and processes, and ultimately improve the delivery of patient care in Australian hospitals.
Acknowledgments
The authors would like to acknowledge Dr Annette Pantle for her expert advice on revision and development of DUQuA measures, and thank Professor Sandy Middleton, Associate Professor Dominique Cadillac, Kelvin Hill, Dr Carmel Crock, and Professor Jacqueline Close for their input into the development of our stroke, AMI, and hip fracture clinical process indicator lists, and Nicole Mealing for her statistical advice to the team during the initial phase of the project.
References
Footnotes
↵i The NSF provides training to internal clinicians for its biennial audit. The NSF reviews 40 consecutive medical records for each hospital. A random sample of 30 records will be selected for each participating DUQuA hospital.
↵ii Sample size was calculated based on a two level hierarchical model, with hospital as the higher level. This model structure was selected as it will likely be the model requiring the largest sample, and which answers the primary research questions. Additional models that are tested will give more precise estimates as their sample size requirements will likely be less.
↵iii This is the reference number supplied by the HREC. The reference numbers have been abbreviated (by removing the HREC name) to maintain the confidentiality of participating hospitals.
Contributors The research team consists of experienced researchers, clinicians, biostatisticians and project managers with expertise in health services research, survey design, and validation, large-scale research and project management, sophisticated statistical analysis, quality improvement and assessment, accreditation, clinical indicators, and patient experience. NT and RC-W led the study design and the development of the manuscript. VP provided statistical advice for the study design and developed the analysis plan for the manuscript. EH and ZL contributed to the refinement of measures and the development of the manuscript. Members of the original DUQuE team (OG and RS) were consulted in the early DUQuA stages, providing assurance for the design modifications made and the planned study approach. They also contributed to the development of the manuscript. JB conceived the idea, led the research grant to fund the project, chairs the steering committee, and contributed to the development of the manuscript.
Funding This work was funded by National Health and Medical Research Council (NHMRC) Program Grant APP1054146 (CI Braithwaite).
Competing interests None declared.
Ethics approval Several Human Research Ethics Committees have approved this study. Evidence of these approvals has been included as a supplementary file for editors only.
Provenance and peer review Not commissioned; peer reviewed for ethical and funding approval prior to submission.