Introduction 30-day readmission rate is considered an adverse outcome reflecting suboptimal quality of care during index hospitalisation for community-acquired pneumonia (CAP). However, potentially avoidable readmission would be a more relevant metric than all-cause readmission for tracking quality of hospital care for CAP. The objectives of this study are (1) to estimate potentially avoidable 30-day readmission rate and (2) to develop a risk prediction model intended to identify potentially avoidable readmissions for CAP.
Methods and analysis The study population consists of consecutive patients admitted in two hospitals from the community or nursing home setting with pneumonia. To qualify for inclusion, patients must have a primary or secondary discharge diagnosis code of pneumonia. Data sources include routinely collected administrative claims data as part of diagnosis-related group prospective payment system and structured chart reviews. The main outcome measure is potentially avoidable readmission within 30 days of discharge from index hospitalisation. The likelihood that a readmission is potentially avoidable will be quantified using latent class analysis based on independent structured reviews performed by four panellists. We will use a two-stage approach to develop a claims data-based model intended to identify potentially avoidable readmissions. The first stage implies deriving a clinical model based on data collected through retrospective chart review only. In the second stage, the predictors comprising the medical record model will be translated into International Classification of Diseases, 10th revision discharge diagnosis codes in order to obtain a claim data-based risk model.
The study sample consists of 1150 hospital stays with a diagnosis of CAP. 30-day index hospital readmission rate is 17.5%.
Ethics and dissemination The protocol was reviewed by the Comité de Protection des Personnes Sud Est V (IRB#6705). Efforts will be made to release the primary study results within 6 months of data collection completion.
Trial registration number ClinicalTrials.gov Registry (NCT02833259).
- hospital readmission
- prediction model
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
Potentially avoidable readmission within 30 days of index hospitalisation is a more relevant quality metric than all-cause readmission for patients with community-acquired pneumonia.
Yet, implicit assessment of readmission avoidability is highly subjective and unreliable.
In this study, the likelihood that a readmission is potentially avoidable will be quantified using latent class analysis based on independent structured reviews by four panellists.
This study does not track readmissions that occur at non-index hospital.
Community-acquired pneumonia (CAP) is a leading infectious cause of adult hospitalisation in Europe and North America,1 2 contributing to 250 000 hospital admissions in France each year.3 CAP is also a serious and potentially life-threatening illness: it ranks as the first cause of death from infectious diseases in Western countries, with reported short-term mortality rates ranging from 5% to 14% for adult patients hospitalised with this illness.4
Because of the associated adverse outcomes and related costs, CAP has been a focus for quality improvement efforts for the past two decades. From 10% to 21% of adult patients hospitalised with CAP are readmitted within 30 days of discharge.5 6 Short-term readmission following CAP-related hospitalisation poses significant problems for the patient and hospital. First, unplanned readmission is an undesirable outcome which matters for patients and families and negatively alters patient quality of life. Second, readmission exposes patient to unnecessary risk for hospital-acquired infections and venous thromboembolism. Third, readmission is associated with increased costs and resource utilisation.
Readmission rates can be easily computed and tracked from computerised hospital discharge data. As part of the Hospital Readmission Reduction Programme (HRRP) effective in fiscal year 2013, US hospitals with higher than expected 30-day readmission rates after pneumonia hospitalisation have been subject to financial penalties from the Center for Medicare and Medicaid Services.7 8 The underlying logic of the HRRP is based on the notion that short-term readmission is often a preventable adverse outcome, reflecting suboptimal quality of care during index hospitalisation. Yet, published evidence suggests that less than one in four all-cause readmissions is deemed avoidable.9 10 Because only avoidable readmissions can be influenced by interventions designed to decrease readmission rates, avoidable readmission is a more relevant metric than all-cause readmission for tracking quality of hospital care for pneumonia.
The suboptimal quality of care may relate to the management of pneumonia, the management of comorbid conditions present at the time of the index hospitalisation, the continuity of care after the discharge or ambulatory care after discharge. Causes and factors that contribute to avoidable readmission can be classified into four categories, including social context, patient health status, care organisation and patient behaviour.11 Socioeconomic features include lower education level, lower income, the lack of occupational activity12–15 and health insurance status.16 Markers of the patient’s health status include age greater than 65 years,17–19 multiple hospitalisations within the previous year,20 frailty, sensory deficiencies and the presence of comorbidities with higher Charlson Index.12 17 21 Care organisation-related factors include early discharge,22 23 clinical instability on discharge24 25 and poor discharge processes (eg, lack of medication reconciliation, patient education regarding continuity of care and follow-up processes).17 22 24–28 Patient behavioural risk factors for readmission are poor adherence to treatment, alcoholism, drug addiction,29 30 psychosocial problems (eg, housing instability, homeless), psychiatric disorders and depressive states.17 24 26 29 31
Although numerous risk prediction models of hospital readmission for patients with CAP have been developed,15 32–35 only few focused on potentially avoidable readmission. A systematic review of 11 models found moderate predictive accuracy in terms of discrimination (C statistic ranging from 0.59 to 0.77).34 More recently published models included various risk factors for readmission including comorbidities, pneumonia severity, clinical instability on discharge, number of previous hospitalisations, index length of stay, and various clinical and biological data.15 20 24 32 35
The broad objective of this study is to develop an administrative claims-based risk prediction model for identifying readmissions that are potentially avoidable within 30 days of index hospitalisation for patients with CAP. The specific aims of this project are:
To assess the positive predictive value of International Classification of Diseases, 10th revision (ICD-10) discharge diagnosis codes for CAP using a retrospective structured chart review as the reference method.
To estimate the rate of all-cause readmissions in the same hospital within 30 days and 1 year of discharge for patients.
To estimate the percentage of unplanned readmissions for patients hospitalised with CAP using a retrospective structured chart review.
To describe pneumonia-related and pneumonia-unrelated reasons for readmissions for patients hospitalised with CAP using a retrospective structured chart review.
To quantify the probability that an unplanned readmission is avoidable using latent class analysis based on independent chart reviews performed by four medical panellists.
To identify the characteristics abstracted from medical record that are independently associated with potentially avoidable readmission.
To derive and internally validate a medical record-based risk prediction model for identifying potentially avoidable 30-day readmission of patients hospitalised with CAP.
To identify variables from administrative claims data that are independently associated with potentially avoidable readmission.
To derive and internally validate an administrative claims-based risk prediction model for identifying potentially avoidable 30-day readmission of patients hospitalised with CAP.
To compare the overall accuracy, discrimination and calibration for the administrative claims data-based versus medical record data-based risk prediction model for identifying potentially avoidable 30-day readmission of patients hospitalised with CAP.
This risk prediction model development study will be conducted according to current guidelines.36–38 The present protocol describes the inclusion criteria, explains how data collection is undertaken, data will be analysed and findings will be interpreted.
Participating study centres and setting
The study will be conducted in a university-affiliated hospital and a general hospital in France. With a capacity of 1362 acute care beds, Grenoble Alpes University Hospital (GUH) serves a predominantly urban population of 675 000 inhabitants and reported 135 999 stays in 2014. Annecy Genevois General Hospital has a capacity of 896 acute care beds and reported 70 651 stays in 2014.
The study population consists of consecutive patients admitted from the community or nursing home setting with pneumonia. To qualify for inclusion, patients must have a primary discharge diagnosis code of pneumonia or a secondary discharge diagnosis code of pneumonia with a primary diagnosis code of respiratory failure, sepsis or pneumonia related-symptoms. The specific ICD-10 codes used to define the study cohort are listed in table 1.
Although nursing home-acquired pneumonia has been termed ‘healthcare-associated pneumonia’,39 it remains controversial whether nursing home-acquired pneumonia more closely resembles hospital-acquired pneumonia than CAP. Because nursing home-acquired pneumonia accounts for a limited proportion of CAP-related hospitalisations,40 it will not be an exclusion criterion for this study. In contrast, patients with hospital-acquired or ventilator-associated pneumonia will be excluded. Hospital-acquired pneumonia is defined as pneumonia not incubating at the time of hospital admission and occurring 48 hours or more after admission. Ventilator-associated pneumonia is defined as pneumonia occurring more than 48 hours after endotracheal intubation.
Patients will be excluded if they are admitted from another acute care facility, subsequently transferred to another acute care facility or admitted in a day care unit. Death during index hospitalisation will be collected and analysed, but these patients will not be eligible for readmission analysis.
Consistent with Lindenauer et al,33 additional pneumonia admissions within 1 year of discharge from an index pneumonia hospitalisation will be considered as readmissions and excluded as index admissions: a single admission cannot be counted both as an index admission and as a readmission for another index admission.
Patient and public involvement
Patients are not involved in the design or conduct of the study.
Data sources include routinely collected hospital administrative claims data and retrospective structured chart reviews.
Administrative claims data
As part of the French diagnosis-related group-based prospective payment system, computerised hospital discharge data include patient and hospital stay identifiers, admission and discharge dates, age, gender, length of stay, discharge location, primary and secondary ICD-10 discharge diagnosis codes for both index admission and readmission. ICD-10 coding complies with national guidelines and is done by trained technicians or physicians, depending on the hospital. Coders usually abstract diagnoses from physician notes, admission notes, daily progress notes, consultation reports, diagnostic imaging and treatments that are routinely recorded in the medical chart. Discharge diagnosis data are externally audited by reabstracting a random sample of hospital stays every year.
Structured chart review
Two clinical research assistants will perform structured retrospective chart review using a computerised data collection instrument. The following variables are recorded for index hospitalisations: patient and hospital stay identifiers; baseline patient characteristics, including demographics, pre-existing comorbid condition, Pneumonia Severity Index (PSI) risk class, physical examination and laboratory findings on admission, X-ray or CT-scan findings within 48 hours of admission, initial microbiological work-up; in-hospital antibiotic therapy and associated treatments, index hospital admission course, intensive care unit (ICU) admission, pneumonia-related and pneumonia-unrelated complications, physical examination and laboratory findings at discharge; discharge plan and treatments.
The following variables are recorded for the first hospital readmission within 1 year of discharge: patient and hospital stay identifiers, time from discharge to readmission, length of stay, physical examination and laboratory findings on readmission, X-ray or CT-scan findings within 48 hours of readmission, hospital readmission course (ICU admission, pneumonia-related and pneumonia-unrelated complications, in-hospital mortality), and primary and secondary reasons for readmissions.
Emergency department (ED) visits that do not result in hospital readmission within 30 days after discharge will be recorded. Similar to hospital readmission measure, only the first post-discharge ED visit will be counted in patients with multiple ED visits.43
To ensure optimal quality, all data collected retrospectively by chart review will be entered electronically by clinical research assistants using a personal identification code and a password-protected web-based data collection system. The clinical research assistants received formal training in the methods of data abstraction and recording. An operation manual that includes definitions and acceptable data sources for all variables have been distributed. Reliability of data abstraction will be assessed by randomly selecting cases for independent collection by a practicing physician.
Positive predictive value of ICD-10 discharge diagnosis codes for CAP
The discharge diagnosis codes used in claims databases do not distinguish between community-acquired and hospital-acquired pneumonia, two distinct clinical entities.44 Consistent with previous studies, the positive predictive value of ICD-10 discharge diagnosis codes will be assessed using three reference methods:
Medical record and/or discharge letter notation of CAP diagnosis.
Medical record notation of ≥1 respiratory symptom (cough, sputum production, dyspnoea, tachypnoea or pleuritic pain), and ≥1 auscultation finding (rales or crepitations), and ≥1 sign of infection (temperature >38°C, shivering, or white cell count >10 x109/L or <4 x109/L), and a new infiltrate on chest radiography or CT scan performed within 48 hours of admission.
A composite of #1 and/or #2.
Positive predictive value point estimate along with 95% CI will be reported for the three reference methods, separately.
A convenience sample of nine board-certified physicians with clinical experience in managing CAP was recruited, including three infectious disease specialists, three pulmonologists and three clinical epidemiology specialists. All readmission cases will be reviewed by four panellists, including at least one infectious disease specialist, one pulmonologist and one clinical epidemiologist (ie, the fourth panellist will be either an infectious disease specialist, a pulmonologist or an epidemiologist). The panellists will independently review medical records for both index hospitalisation and readmission.
Consistent with Jasti et al,12 each panellist will use predefined criteria to categorise the primary reason for rehospitalisation as:
Pneumonia-related worsening of signs or symptoms.
New or worsening comorbid condition(s) independent of pneumonia.
Any combination of pneumonia-related and comorbidity-related reasons.
The panellists will assign the primary reason for readmission, using 11 mutually exclusive categories45 : (1) unforeseen readmission for a new affection, (2) complication of surgical care, (3) complication of non-surgical care, (4) drug-related adverse event, (5) premature discharge, (6) discharge with a missing or erroneous diagnosis or therapy, (7) other inadequate discharge, (8) failure of post-discharge follow-up care, (9) inadequate patient behaviour, (10) relapse or aggravation of a previously known condition, (11) social readmission.
Consistent with van Walraven et al,46 the panellists will use a 6-point ordinal scale to rate whether the readmission is an adverse event and whether the readmission could be avoided. A readmission with a rating above 3 in both domains will be classified as potentially avoidable by that panellist. The panellists will indicate the factors contributing to the readmission among seven non-exclusive categories: medication-related readmission, procedure-related readmission, nosocomial infection, diagnostic error, management error, system error, surgical complication.
The primary outcome measure is potentially avoidable readmission within 30 days of discharge from index hospitalisation. The likelihood that a readmission is potentially avoidable will be quantified using latent class analysis based on the independent reviews by four panellists. A readmission will be considered potentially avoidable if the Bayes’ posterior probability exceeds 0.50.46
Descriptive summary statistics will be used for reporting continuous (mean and SD or median and 25th–75th percentiles) and categorical (numbers and percentages) variables. Patient stay characteristics will be compared between study subgroups using the χ² test or Fisher exact test where appropriate for categorical variables and the Student’s t-test or non-parametric Wilcoxon test for continuous variables.
Latent class analysis
We will perform latent class analysis to quantify the probability that a readmission is avoidable, based on the independent classification by four panellists. This is the same approach as previously used by others.46 Briefly, latent class analysis is a statistical approach that assigns individuals in two or more latent classes based on a set of observed categorical variables. The latent variable cannot be observed directly; instead it is measured indirectly by using multiple observed variables. We will specify a two-class model, reflecting the dichotomy of avoidable versus unavoidable readmission. The independent classification of readmission by each of the four panellists will be entered as observed categorical variables.46 We will derive from the latent class model the Bayes’ posterior probability of avoidability for each individual case of readmission. Finally, we will report the model-based sensitivity and specificity of each panellist in classifying readmission as avoidable.
Model development overview
The model development sample will consist of eligible patients who have been readmitted within 30 days of discharge. A flow chart will present graphically patient flow throughout the study. We will use a two-stage approach to develop an administrative claims-based model intended to identify potentially avoidable readmissions. The first stage implies deriving a medical record-based model with the use of data collected through retrospective chart review only. In the second stage, the predictors included in the resulting medical record-based model will be translated into ICD-10 discharge diagnosis codes in order to obtain an administrative claims-based model.
Medical record-based model development
Derivation and internal validation will be conducted according to current standards.38 The medical record-based model will be derived using multivariable logistic regression for binary dependent variable. Candidate predictors will be identified among both hospital index admission and readmission variables based on the findings from a systematic review and significant relationship with avoidability. We will assess the log-linearity assumption for continuous variables using fractional polynomial regression. Missing values will be replaced by multiple imputation. In internal validation, the potential for statistical overfitting will be quantified using bootstrapping. All 30-day readmission cases from the study sample will be used for both derivation and internal validation of the prediction model.
The resulting medical record-based model predictive performance will be evaluated using overall, calibration and discrimination measures.36 Overall model performance will be quantified using pseudo-R² and Brier score. Discrimination, which refers to the ability of the model to distinguish individuals with and without potentially avoidable readmission, will be quantified by the concordance C statistic. Calibration, which refers to the agreement between avoidability likelihood predicted by the model and observed avoidability frequency, will be assessed by calibration slope.
Administrative claims-based model development
Two physicians with expertise in discharge diagnosis coding will independently translate relevant predictors comprising the resulting medical record-based model into ICD-10 diagnosis codes. A single model will be obtained after a reconciliation meeting of the two physicians. The candidate variables include age, sex, diagnosis codes, Charlson’s Comorbidity Index, length of stay, ED readmission and time from discharge to readmission. The resulting administrative claims-based model predictive performance will be evaluated using overall, calibration and discrimination measures.
Competing prediction models
External validation of competing prediction models will consist in applying their inherent predictors and parameter coefficients on our study dataset. The predictive performance of the models will be evaluated in terms of both calibration and discrimination.
All statistical analyses will be performed using Stata Special Edition V.16 or higher (Stata Corporation, College Station, Texas, USA). Additional software may be used for the production of graphics and for statistical methodology not provided by this software package.
Ethics and dissemination
The protocol for this study was approved by the Comité de Protection des Personnes Sud-Est V, Grenoble, France (IRB#6705). The consent for data collection through chart review and the use of corresponding administrative claims data is sought under a regime of ‘non-opposition’ (opt-out): after appropriate written information is delivered by regular mail, data are collected unless the patient opposes. Computerised study data will be processed at GUH, in compliance with French data protection regulations.
Efforts will be made to reduce the interval between the completion of data collection and the release of the primary study results. It is expected that 6 months will be necessary for the writing committee to compile the primary study results before manuscript submission to an appropriate journal. No later than 3 years after final acceptance of the primary study paper, de-identified data will be available on request from the corresponding author for sharing purpose.
From 1 January 2014 to 31 December 2014, 1523 hospital stays with an ICD-10 diagnosis code of pneumonia were identified (figure 1). After excluding 186 hospital stays because of the discovery of an exclusion criterion and 187 hospital stays with a diagnosis other than CAP, our analytical sample consists of 1150 index hospital stays. Overall, 98 (8.5%) patients died in hospital and 184 were readmitted within 30 days of discharge, representing an early readmission rate of 17.5% (ie, 184/1052, 95% CI, 15.2% to 19.9%). The medical records for both index hospitalisations and readmissions of these 184 patients with CAP will be independently reviewed by the panellists for assigning the primary reason for readmission and rating the avoidability of readmission. The median age for all patients was 78 years, 56% were of male gender and 15% were nursing home residents (table 2). All patients had clinical or biological signs of infection. Median C reactive protein was 114 mg/L. Hypoxemia was common and more than one-third of patients (41%) required oxygen supplementation. Overall, 63.5% of patients were in PSI risk classes IV–V.
Most patients had pre-existing comorbidities, including arterial hypertension (49.8%), neurological conditions (36.9%), underlying respiratory disease (24.5%), cardiac arrhythmia (23.3%), diabetes mellitus (22.5%), coronary artery disease (20.1%) and kidney failure (15.8%) (table 3). Fifteen per cent of patients had one or more causes of immune depression. Charlson’s Comorbidity Index ranged from 0 to 12 with a median of 2 (25th–75th percentile, 1–3). The median duration of the index stay was 8 days. A total of 168 (14.6%) patients were admitted to the ICU, 42 (3.6%) underwent invasive mechanical ventilation and 50 (4.3%) received inotropic or vasopressor support.
Blood was obtained for culturing from 817 patients (71.0%), and a urine sample for urinary antigen detection from 583 patients (50.7%). A pathogen was detected in 311 patients, including one or more viruses in 57 (5.0%) patients, one or more bacteria in 252 (21.9%), both bacterial and viral pathogens in 15 (1.3%), and fungi or mycobacteria in 22 (1.9%). The most common bacteria detected were Streptococcus pneumoniae (6.9%), Haemophilus influenzae (2.8%) and Legionella pneumophila (2.3%).
In this study, the 30-day readmission rate is 17.5% for patients hospitalised with CAP. This result is consistent with previous estimates, ranging from 7.3% to 25% across studies. Thus, the rate of readmission varied from 13.6% to 25% in the studies of Makam et al35 and of Hatipoglu et al,32 from 7.3% to 20.1% in the study of Prescott et al6 and from 11.8% to 20.8% with a median of 17.3% for Weinreich et al34
Compared with previous reports, the patients enrolled in our study are older (median age, 78 years), more likely to present with severe pneumonia (prevalence of PSI risk classes IV–V, 63.5%) and have more comorbidities (prevalence of Charlson’s Comorbidity Index ≥2, 53%). The 24% rate of patients with identified pathogens is lower than previously reported.2 The potential explanations for such a low microorganism detection rate include the lack of testing for known pathogens, antibiotic use before specimen collection and non-infectious causes. Of notice, our study was observational in design and therefore microbiological diagnostic test ordering was left at the discretion of admitting physicians. PCR assay was rarely performed for the detection of respiratory viruses, which could have led to underestimating the prevalence of viral aetiologies. Consistent with previous reports, the most commonly detected bacteria was S. pneumoniae. With the exception of viruses, the distribution of other microorganisms was the same as reported in the literature.1 47 48
Various approaches have been used for assessing whether a readmission is potentially avoidable, including implicit assessment, explicit assessment and latent class analysis.
Implicit assessment based on unstructured chart review is the most common approach although its validity and reliability are questioned.49 The lack of standardisation in criteria might explain variations in percentages of potentially avoidable readmissions across primary studies.49 Yet, implicit assessment may be improved by the use of two independent reviewers, resolution of disagreement by discussion between the two reviewers or by a third reviewer, and by interviews with the physicians in charge of the patient and with the patient.50 51
Explicit assessment based on structured chart review by one or more reviewers has been used in various contexts. Halfon et al45 categorised readmissions as planned, unplanned for a new condition and unplanned for a condition known at index hospitalisation. Then, reviewers are asked to assign a root cause for readmission using 11 exclusive categories. This approach has been implemented throughout the «Striving for Quality Level and Analysing of Patient Expense» algorithm for use with administrative claims data.52
Van Walraven et al have refined the explicit assessment approach by quantifying the likelihood that a readmission is potentially avoidable using a latent class analysis based on independent reviews by multiple panellists. We are planning to use the same approach, which is a strength of the present study.
The limitations of our study deserve mention. First, our study tracks index hospital readmissions only. Indeed, previous studies reported that one in five 30-day readmissions may occur at non-index hospitals.53 Second, the effective sample size for model development is relatively limited with the potential for overfitting. As a rule of thumb, there will be a minimum of five potentially avoidable readmission cases per candidate predictor considered for inclusion in our multivariable logistic regression model. Third, our study is conducted in two hospitals in France and our findings may not apply to other settings or regions.
To conclude, we will develop an administrative claims-based model for identifying potentially avoidable 30-day readmissions of patients with CAP, using latent class analysis of explicit assessment by independent panellists as the reference method. Our study will also provide the unique opportunity to estimate the accuracy of competing models in predicting potentially avoidable readmission in an external validation sample.
Contributors All authors reviewed the manuscript and contributed to its improvement. All authors were involved in study population development. A-LM—literature review and manuscript writing. PF—study conceptualisation, protocol development, literature review and manuscript writing. MB—statistical analysis. XC—sampling and data collection. JL—statistical expertise and critical revision of the manuscript. AS—protocol development and statistical analysis. PP, ES, JG, BC, BD and MM—structured medical chart review for avoidability assessment.
Funding This work was supported by the French Ministry of Health, (PREPS-IQ 2013) grant number: PREPS1300302.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.