Overview

This is a multicentre, retrospective cohort study of children 6 months to less than 18 years of age presenting to 30 participating North American EDs from October 2015 to December 2019. The study was endorsed by and is being conducted through the PEMCRC of the American Academy of Pediatrics, a voluntary network of over 60 institutions in the USA and Canada. Cincinnati Children’s Hospital Medical Center (CCHMC) will serve as the data coordinating centre.

Patient and public involvement

No patient involved: this research was conducted without patient involvement; patients were not involved in study design or manuscript preparation.

Inclusion criteria

Anaphylaxis diagnosis:

1. Encounters that fulfil standard National Institute of Allergy and Infectious Disease/Food Allergy and Anaphylaxis Network (NIAID/FAAN) anaphylaxis criteria (box 1)15 based on documentation of symptoms from the pre-ED and ED settings.

2. ED encounters for allergic reactions that do not fulfil NIAID/FAAN anaphylaxis criteria but for which patients receive epinephrine (intravenous, intramuscular and subcutaneous (subQ)) in the pre-ED or ED settings. We are including these patients because administration of epinephrine may mitigate initial reaction severity and progression to anaphylaxis criteria.

Exclusion criteria

1. No receipt of epinephrine (intravenous, intramuscular and subQ) in the pre-ED or ED settings: although patients may have anaphylaxis symptoms that resolve by the time of ED arrival or there may be under-recognition or underdocumentation of anaphylaxis by the treating clinician, receipt of epinephrine is an objective and well-documented data point. We also seek to identify more severe cases for which there is no clinical uncertainty.

2. Transfer from another healthcare facility: it is difficult to accurately assess symptoms present at the time of initial outside healthcare facility evaluation. For the predictive models, we plan to include initial ED symptoms, and thus, it is important to ensure accuracy of data elements.

3. Anaphylactic reactions secondary to ED treatments: for anaphylactic reactions secondary to ED treatments (eg, anaphylaxis from ceftriaxone given to treat pneumonia), it is difficult to assess whether symptoms as well as disposition decisions are related to the primary diagnosis or anaphylaxis:

4. Significant comorbidities including hereditary angioedema, mast cell activation disorders and patients with tracheostomies or requiring home positive pressure ventilation: patients with these conditions may require specific management and disposition decisions and are thus not generalisable to the larger population of children with anaphylaxis.

5. Multiple ED encounters for anaphylaxis: for patients with multiple ED encounters for anaphylaxis, only data from the most recent ED encounter will be included in the data analysis to avoid including data from the same patient when deriving and validating the prediction model for safe ED discharge (objective 2).

Case ascertainment

The electronic medical record (EMR) will be queried for ED International Classification of Diseases, Tenth Revision (ICD-10) discharge codes and all subcodes starting from 2019 and then working backwards until reaching the target number of records as outlined in the sample size section of this protocol. We used ICD-10 codes for case ascertainment given they were adopted in the USA in 2015 and were used in clinical practice through the conclusion of the study period (December 2019).16 The following codes will be queried: ‘anaphylactic shock due to shell fish (crustaceans)’ (T78.01), ‘anaphylaxis to drug’ (T88.6), ‘anaphylaxis to serum’ (T80.5), ‘anaphylactic shock due to other fish’ (T78.02), ‘anaphylactic shock due to fruits and vegetables’ (T78.03), ‘anaphylactic shock due to tree nuts and seeds’ (T78.04), ‘anaphylactic shock due to milk and dairy products’ (T78.06), ‘anaphylactic shock due to eggs’ (T78.07), ‘anaphylactic shock due to other food products’ (T78.08), ‘anaphylactic shock due to unspecified food products’ (T78.09) and ‘anaphylactic shock, unspecified’ (T78.2) and subcodes (eg, T78.2XXA).5 Although anaphylaxis encounters for which there are no ICD anaphylaxis codes may be omitted, it is not feasible to perform manual reviews of encounters with non-anaphylactic diagnostic codes given it would triple the number of encounters to review5 and because our findings are not intended to describe population-level epidemiological trends in the prevalence of anaphylaxis-related ED visits.

Data abstraction

Trained coinvestigators will abstract variables from the EMR to a secure, Health Insurance Portability and Accountability Act (HIPAA)-compliant REDCap database hosted at CCHMC. Investigators will review each encounter to assess whether it fulfils inclusion criteria and does not collect any other data for encounters that do not fulfil these criteria or meet exclusion criteria. Variables include demographics, medical history (anaphylaxis, asthma and non-atopic comorbid conditions (eg, congenital heart disease)), time and type of allergen exposure and examination findings from the pre-ED and ED settings. Symptoms will be categorised by organ system involvement in accordance with NIAID/FAAN criteria (box 1).15 Additional variables include therapies received in the pre-ED, ED and inpatient settings such inhaled beta agonists, epinephrine (intramuscular, intravenous and subQ), racemic epinephrine, intravenous fluid boluses, vasopressors (dopamine, epinephrine, norepinephrine and vasopressin), positive pressure ventilation (high-flow nasal cannula, continuous positive airway pressure, bilevel positive airway pressure and intubation/mechanical ventilation), defibrillation/cardioversion, cardiopulmonary resuscitation (CPR) and extracorporeal membrane oxygenation (ECMO).

Demographic information, ED, inpatient admission and discharge times and vital signs will be abstracted through an automated query of the EMR or manually at sites that cannot feasibly perform the query. Each site will then manually review 10% of encounters to ensure accuracy of these data elements. Medical history, timing of allergen exposure and symptom onset, administered therapies and symptoms/exam findings (from the pre-ED, ED and inpatient settings) will be obtained through manual review of documentation in the medication administration record, ED and admission and discharge summary notes. The same process will be conducted for ED revisits within 72 hours of the index ED visit. Historical and physical examination findings, as well as pre-ED treatments, will be categorised as present/received or absent/not received based on documentation in the EMR. Variables not recorded will be categorised as not documented or missing based on criteria in the manual of operations (MOO).

Compliance

All data abstractors will be trained, and a detailed MOO will be available at each site. Teleconferences will be organised to discuss study progress and answer questions during the chart review process. The REDCap database will be created using best practices (data validation and minimisation of missing data) to accurately capture data. Routine data monitoring and compliance checks of the database will be performed to ensure data accuracy throughout the conduct of the study.

Outcome definitions

Outcome definitions were developed by group consensus among the authors, based on commonly cited definitions in the literature,5 17 18 biological plausibility and the feasibility/reliability of extracting data elements from the EMR to evaluate for the presence/absence of the outcomes. Data abstractors will not categorise patients as having anaphylaxis or specific anaphylaxis outcomes (severe, persistent or biphasic anaphylaxis, as well as persistent or biphasic non-anaphylactic reactions) other than that of refractory anaphylaxis; instead, they will be responsible for extracting data elements from the EMR (symptoms and the clinical course following epinephrine administration), which will be used to generate the composite outcome definitions described below.

Objective 1

1. Severe anaphylaxis: presence of at least one of the following two criteria:

   1. Hypotension, shock, respiratory distress, respiratory failure, life-threatening arrhythmias, coma or death.

   2. Receipt of any of the following therapies in the pre-ED, ED or inpatient settings: ≥3 doses of epinephrine (intravenous, intramuscular and subQ), ≥3 fluid boluses, vasopressors, antiarrhythmics, every 1 or 2 hours inhaled beta agonists, continuous albuterol, magnesium sulfate, intravenous terbutaline, positive pressure ventilation and life-saving interventions (tracheostomy, CPR, defibrillation/cardioversion and ECMO).

2. Persistent anaphylaxis18:

   1. Persistent symptoms/examination findings after treatment with epinephrine (intravenous, intramuscular and subQ) that fulfil the 2006 NIAID/FAAN anaphylaxis criteria.15

   2. Following treatment with each epinephrine dose, persistent symptoms will be assessed by asking, ‘Did symptoms completely resolve after treatment with epinephrine?’ Symptoms will be categorised as persistent if this variable is marked as ‘no,’ and all persistent symptoms will be recorded. Although the definition of persistent anaphylaxis would ideally include the duration of persistent symptoms, this time interval cannot be reliably assessed in the EMR.

3. Refractory anaphylaxis18: presence of bothof the following two criteria:

   1. Presence of anaphylaxis following appropriate epinephrine dosing and symptom-directed medical management (eg, intravenous fluid bolus for hypotension and inhaled beta agonists for wheezing).

   2. The initial reaction must be treated with three or more appropriate doses of epinephrine or initiation of an intravenous epinephrine infusion.

   3. Reviewers will be asked to determine whether patients had refractory anaphylaxis at any point in the course of the reaction (no, yes and inconclusive), including in the pre-ED, ED or inpatient settings (among hospitalised patients), as well as for patients who have ED revisits.

4. Biphasic anaphylaxis18: presence of both of the following criteria:

   1. Recurrent or new allergic symptoms after initial symptom resolution without allergen re-exposure.

   2. Recurrent or new symptoms must fulfil the 2006 NIAID/FAAN anaphylaxis criteria and occur within 48 hours from initial symptom onset (eg, symptoms that occur beyond 48 hours will not be categorised as biphasic anaphylaxis).15

   3. Symptoms will be categorised as resolved if this variable is marked as ‘yes’ or ‘not documented’. Although the definition of biphasic anaphylaxis would ideally include a minimum asymptomatic period, these time intervals cannot be reliably assessed in the EMR. To determine whether children had recurrent/new symptoms, we will ask, ‘Did the patient have recurrent or new symptoms after treatment with epinephrine?’ and all recurrent symptoms will be recorded. The upper time threshold for recurrent/new symptoms (48 hours) will be conditional on the onset, instead of resolution, of initial symptoms given the timing of the former is not reliably documented in the EMR. ED triage will serve as a proxy for symptom onset if the timing of which is not documented. Given patients transferred from outside healthcare facilities are excluded from the study, the time from symptom onset to ED triage is relatively brief and unlikely to change the outcome.

5. Biphasic and persistent non-anaphylactic reactions18: The only difference between the definitions of biphasic and persistent non-anaphylactic reactions and biphasic and persistent anaphylaxis is that the former definitions do not fulfil NIAID/FAAN anaphylaxis criteria.15

   1. Patients who do not fulfil severe, persistent, refractory or biphasic anaphylaxis criteria, nor persistent or biphasic non-anaphylaxis criteria, will be categorised as having anaphylaxis not otherwise specified.18

Objective 2: safe ED discharge

Safe ED discharge is defined as no receipt of acute anaphylaxis medications and no hypotension beyond 4 hours from first administered dose of epinephrine (intravenous, intramuscular and subQ). Patients may have the outcome for safe ED discharge despite fulfilling criteria for severe, persistent or biphasic anaphylaxis. For example, a child may have persistent or biphasic anaphylaxis following first administered dose of epinephrine but not receive acute anaphylaxis medications (or have hypotension) beyond 4 hours from the initial epinephrine dose. Four hours was used as the time interval to evaluate for safe ED discharge based on guidelines that recommend observation periods of 4 hours to monitor for biphasic anaphylaxis.8

Acute medications are defined as epinephrine, vasopressors, antiarrhythmics, racemic epinephrine, receipt of inhaled beta agonists every 1–2 hours, continuous albuterol, magnesium sulfate or terbutaline. We did not include therapies such as non-invasive ventilation, intubation or CPR as the timing of these interventions is not reliably documented in the EMR and because it would be exceedingly rare for patients to receive these therapies without first receiving an acute medication. We did not include fluid boluses or receipt of inhaled beta agonists ≥every 4 hours as acute medications given there may be subjectivity around use of these therapies (eg, fluid bolus given for tachycardia without cardiovascular compromise). To account for the potential of children receiving medications such as epinephrine for mild, non-life-threatening symptoms (eg, subjective throat discomfort), we will record the severity of persistent and recurrent symptoms documented in the EMR following all epinephrine doses.

Objective 3: Time to last dose of epinephrine and vasopressor cessation following initial epinephrine dose.

These data will inform clinicians on data-driven observation periods in the ED and inpatient setting within predefined clinical subgroups (see the Predictors and subgroups section).

Secondary outcomes

Secondary outcomes include (1) ED and inpatient lengths of stay, (2) admission rates, (3) ED revisits related to the index encounter within 72 hours of ED or inpatient discharge, (4) the frequency of use of adjunctive anaphylaxis medications (antihistamines, antacids and corticosteroids) and (5) the frequency at which providers prescribe epinephrine auto-injectors and adjunctive medications and refer patients to allergists at the time of ED/inpatient discharge.

Predictors and subgroups

Candidate predictors by objective are outlined in table 1. Predictors were developed based on group consensus; biological plausibility; and, if applicable, previously cited risk factors for the outcome of interest. For each predictor, we sought to ensure data elements could be reliably and accurately extracted from the EMR. We will not include anaphylaxis biomarkers (eg, tryptase) as candidate predictors given they are not routinely obtained in clinical practice in the USA and Canada.8

Objective 1: Predictors of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions. We will only include variables present prior to first administered dose of epinephrine. Thus, we will not include symptoms present at the time of initial ED evaluation for patients who received pre-ED epinephrine.

Objective 2: Safe ED discharge. Given the outcome is time dependent (not receiving acute anaphylaxis medications and no hypotension beyond 4 hours from first administered dose of epinephrine), symptoms present after initial ED evaluation will not be included as predictors given there is unreliable documentation as to the timing of these variables. However, we will include the number of administered doses of epinephrine (intravenous, intramuscular and subQ) received within 4 hours of the first administered dose of epinephrine because this is an objective and frequently used criterion for admission. We will not include other medications because there may be an association between symptoms and therapies (eg, wheezing and receipt of albuterol). Additionally, we will not include intravenous fluid boluses as a predictor because there is often subjectivity around administration and because wide pulse pressure and hypotension will be included as predictors, and we feel they are more specific markers of cardiovascular involvement.

Objective 3: Time to last dose of epinephrine and vasopressor cessation following initial epinephrine dose. Observation periods likely do not need to be uniform for all children with anaphylaxis given patients with less severe reactions may not need to be observed as long as those with more severe reactions. Thus, we will assess the timing from initial epinephrine administration to receipt of last dose of epinephrine and vasopressor cessation, which may be a proxy for observation periods, by clinical subgroups based on initial reaction severity. Anaphylaxis severity will be categorised into three subgroups based on symptoms at the time of initial ED evaluation: (1) absence of hypotension and respiratory involvement, (2) respiratory involvement without hypotension and (3) presence of hypotension.15

Analyses

Objective 1: Prevalence of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions. We will estimate the prevalence of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions using proportions with 95% confidence intervals. Similar analyses will be performed for hospitalisation rate and rate of 72-hour ED revisits. To assess risk factors and their association with these outcomes, we will use GEEs to model each of the outcomes, on the risk factors in table 1, accounting for the between-hospital variations. The quasi-likelihood under the independence model criterion will be used for model selection.19

Objective 2: Safe ED discharge. We will derive and validate a predictive model and identify patients who do not need to be hospitalised, that is, at very low risk of receiving acute anaphylaxis medications or being hypotensive beyond 4 hours from first administered dose of epinephrine. The database will be randomly divided into a derivation set comprising 50% of encounters and a validation set including the remaining 50% encounters. The LASSO regression will be used to reduce the number of candidate predictors and build the predictive model.20–22 The LASSO method improves prediction accuracy and performs covariate selection to make the model more interpretable, by forcing the sum of the absolute values of the regression coefficients to be less than a fixed value, and, therefore, forces certain coefficients to be set to zero. This is achieved by penalising models that have more and larger coefficients. The penalty parameter will be chosen by 10-fold cross-validation. We will assess model ability to discriminate between those who received acute medications beyond 4 hours from first administered dose of epinephrine and those who did not use the area under the receiver operating characteristic (AUROC), using the validation dataset. A model with AUROC between 0.7 and 0.8 indicates fair model discrimination, and above 0.8 implies good model discrimination.23 Model calibration will be assessed using the Hosmer-Lemeshow test, which assesses whether the observed rates of the outcome match the expected rate in subgroups at the deciles of fitted risk values.24 We will aim to maximise the sensitivity and negative predictive value of the model to avoid miscategorising patients who receive acute medications or are hypotensive beyond 4 hours from first administered dose of epinephrine.

We will also use classification and regression tree (CART) analysis to develop models to predict patients who are at very low risk of receiving acute medications or being hypotensive beyond 4 hours from first administered dose of epinephrine. The CART analysis involves the segregation of different values of classification variables through a decision tree composed of progressive binary split, and the optimal split is selected based on the complexity parameter that corresponds with the lowest cross-validation error. As a result, CART analysis produces decision trees that are simple to interpret and readily used for risk stratification.25 26

Objective 3: Time to last dose of epinephrine and vasopressor cessation following initial epinephrine dose. We will assess (1) timing from first to second administered doses of epinephrine, as well as (2) timing from first to last administered doses of epinephrine and cessation of vasopressors depicted using the KM method in each subgroup: (1) absence of hypotension and respiratory involvement, (2) respiratory involvement without hypotension and (3) presence of hypotension. We will perform log-rank test to test whether the KM curves are different based on the severity of initial reaction. We will evaluate the cumulative risk of receiving the last dose of epinephrine and vasopressor cessation at different time intervals within the first 24 hours from first administered dose of epinephrine. The optimal length of ED and inpatient observation will be determined as <2% increases in cumulative risk of receiving additional epinephrine (intravenous, intramuscular and subQ) as the observation time increases by 1 hour.

Statistical analyses will be conducted using SAS V.9.4 (SAS Institute); the ‘rpart’ package in R will be used to fit the CART model.25

Data validity and reliability

Missing data

As with any retrospective chart review, there is potential for missing or inaccurate documentation. These limitations will be mitigated by providing clear definitions and instructions regarding data extraction and interpretation, especially regarding subjective data points. We will also record reasons for missing data and examine factors related to missingness. This will strengthen the missing at random assumption. In the event of excessive missing data (>10% of units with missing predictors or outcomes), we will perform multiple imputation or conduct sensitivity analyses using pattern-mixture models.

Sample size

We based our sample size on the ability to provide precise estimates of the sensitivities for the model to predict patients at very low risk of receiving acute medications or being hypotensive beyond 4 hours from first administered dose of epinephrine (objective 2). To be clinically useful, the clinical prediction rule must effectively identify a group of patients at near zero risk of receiving acute medications or being hypotensive beyond 4 hours of initial epinephrine. Therefore, the rule must have a sensitivity of near 100% to predict those who do receive acute medications beyond 4 hours of initial epinephrine administration. When applied to a validation sample, the lower end of the 95% CI for the sensitivity must be no <95% for the risk of receiving acute medications beyond 4 hours of initial epinephrine. If the rule is 100% sensitive, with a sample size of 150 patients receiving acute medications beyond 4 hours of initial epinephrine, the lower limit of the 95% CI will be 97.6%. If the rule is 98% sensitive, the lower end of the CI will be 94.3%.

It is estimated that 5%–15% of children will receive acute medications beyond 4 hours of initial epinephrine.4 To be conservative, we use 5% for sample size calculation. This would necessitate ~30 institutions each reviewing 400 records, of which approximately 200 would fulfil inclusion criteria, which we believe is feasible in the PEMCRC. We will ask each participating institution to review anaphylaxis encounters between January 2018 and December 2019. In case the target sample size is not reached, we will request participating sites to extend their review 12–36 months before January 2018, until the target sample size is reached. Half the sample will be used for model derivation, and the other half will be used for validation.

Ethics, reporting adverse events and dissemination

Cincinnati Children’s Hospital IRB has approved the study; for a complete list of sites that have received IRB approval at the time of article publication please, see online supplemental table 1. Not all ethical approvals are in place at the time of publication of the protocol; however, all participating sites will obtain local IRB approval prior to study initiation. Adverse events including breach of confidentiality will be reported to local IRBs and CCHMC. The results of the study will be published in peer-reviewed journals and shared via social media, traditional media, blogs and online education platforms. We anticipate the main study findings to be submitted for publication within 6 months of data analysis. Additionally, successful dissemination and application of our findings in clinical care require a diverse team of researchers, clinicians and public health professionals with expertise in clinical research, healthcare research, allergy/immunology and quality improvement. The primary investigators have broad expertise in these fields, and initial site recruitment has included coinvestigators and clinicians with extensive research expertise. Dissemination efforts at the study’s conclusion will include broad participation of key stakeholders in the areas of allergy, emergency care, healthcare policy, quality improvement and dissemination/implementation science.