Article Text
Abstract
Introduction Newborns are at risk for early-onset sepsis (EOS). In the Netherlands, EOS affects less than 0.2% of newborns, but approximately 5% are treated with empirical antibiotics. These numbers form an example of overtreatment in countries using risk-factor based guidelines for administrating antibiotics. An alternative to these guidelines is the EOS calculator, a tool that calculates an individual EOS risk and provides management recommendation. However, validation outside the North-American setting is limited, especially for safety outcomes. We aim to investigate whether EOS calculator use can safely reduce antibiotic exposure in newborns with suspected EOS compared with the Dutch guideline.
Methods and analysis This protocol describes a cluster randomised controlled trial assessing whether EOS calculator use is non-inferior regarding safety, and superior regarding limiting overtreatment, compared with the Dutch guideline. We will include newborns born at ≥34 weeks’ gestation, with at least one risk factor consistent with EOS within 24 hours after birth. After 1:1 randomisation, the 10 participating Dutch hospitals will use either the Dutch guideline or the EOS calculator as standard of care for all newborns at risk for EOS. In total, 1830 newborns will be recruited. The coprimary non-inferiority outcome will be the presence of at least one of four predefined safety criteria. The coprimary superiority outcome will be the proportion of participants starting antibiotic therapy for suspected and, or proven EOS within 24 hours after birth. Secondary outcomes will be the total duration of antibiotic therapy, the percentage of antibiotic therapy started between 24 and 72 hours after birth, and parent-reported quality of life. Analyses will be performed both as intention to treat and per protocol.
Ethics and dissemination This trial has been approved by the Medical Ethics Committee of the Amsterdam UMC (NL78203.018.21). Results will be presented in peer-reviewed journals and at international conferences.
Trial registration number NCT05274776.
- NEONATOLOGY
- Paediatric infectious disease & immunisation
- PAEDIATRICS
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This trial addresses an important clinical question regarding the safety of early-onset sepsis calculator use.
Results of this trial are likely to directly impact neonatal clinical care and inform antibiotic stewardship programmes.
A coprimary superiority and non-inferiority trial design is used.
The trial includes at-risk newborns rather than only proven infected newborns.
Introduction
Newborns are at risk for early-onset sepsis (EOS). The incidence of proven EOS is 0.5–2.0 per 1000 live births. For the Netherlands, based on 170 000 births per year, this equals approximately 85–340 newborns yearly. However, in compliance with the current Dutch guideline, approximately 5% of late preterm and term newborns are given antibiotics for suspected EOS, equivalent to a mere 8500 newborns per year.1–3 Meanwhile, adverse short and long-term consequences of antibiotic overtreatment are increasingly recognised.1 4–7
The current standard of care in the Netherlands is the Dutch Society of Paediatrics guideline ‘Prevention and treatment of early-onset neonatal infections’,8 an adaptation of the 2012 version of the National Institute for Health and Care Excellence (NICE) guideline ‘Antibiotics for early-onset neonatal infection: antibiotics for the prevention and treatment of early-onset neonatal infection’.9 Evaluating its use in nine Dutch hospitals, we found limited adherence, especially when antibiotic therapy was recommended by the guideline.10 This limited adherence may suggest a clinical need to amend the Dutch guideline with use of alternative strategies to guide antibiotic therapy. A relatively new strategy is the neonatal ‘EOS calculator’.11 12 This EOS prediction tool calculates the risk for an individual newborn using five maternal risk factors combined with the newborn’s clinical condition after birth with concrete treatment advice. The EOS calculator was developed and validated using data of over 600 000 newborns born with a gestational age of 34 weeks or more. It has now been evaluated in more than 50 studies, of which over 20 with actual implementation. Meta-analysis showed up to an average of 44% less antibiotic use, without indications of negative consequences.13 There are no known implementation studies in which EOS calculator use was ineffective or proven unsafe. Previous, observational evaluations of the EOS calculator have shown potential benefits for the Dutch population.10 14
For the Dutch situation, current equipoise between the Dutch guideline and the EOS calculator can be established given the balance of risk for both overtreatment and undertreatment: the Dutch guideline is likely to result in more overtreatment with adverse consequences yet potentially carries a smaller risk for a delay in treatment of unclear clinical significance. Validation outside the North-American setting is limited, especially for safety outcomes. This warrants a randomised controlled trial to confirm that adoption of the EOS calculator outside of the United States is safe and effective. Therefore, we present a protocol for a trial to determine whether the use of the EOS calculator reduces antibiotic exposure in newborns with suspected EOS compared with the Dutch guideline without compromising on safety. The protocol adheres to the Standard Protocol Items: Recommendations for Interventional Trials (online supplemental file 1).
Supplemental material
Objectives
Primary objectives
To investigate whether the use of the EOS calculator, compared with the Dutch guideline, is non-inferior regarding safety.
To investigate whether the use of the EOS calculator, compared with the Dutch guideline, reduces antibiotic exposure in newborns with suspected EOS in the first 24 hours after birth.
Secondary objectives
To investigate if the use of the EOS calculator, compared with the Dutch guideline, decreases the total duration of antibiotic therapy in newborns with suspected EOS.
To investigate if the use of the EOS calculator, compared with the Dutch guideline, decreases the percentage of antibiotic therapy started for suspected and, or proven EOS if symptoms started between 24 and 72 hours after birth.
To evaluate the impact of managing EOS risk using observation compared with using empirical antibiotics on quality of life reported by parents.
Methods and analysis
Design
A prospective, cluster randomised trial will include 1830 newborns from 10 hospitals (listed in online supplemental file 2) in the Netherlands during an 18-month period. Cluster randomisation occurs at the hospital level. A cluster will continue to participate until a total of 183 participants are included. Participating hospitals provide care up to level II or III (1 hospital),15 with annual birth rates between 1200 and 4000 births per year. Before randomisation, all clusters use the Dutch national guideline to guide antibiotic use in newborns at risk for EOS.
As a result of cluster randomisation, the strategy assigned via randomisation (EOS calculator or Dutch guideline) is considered standard of care for newborns with suspected EOS within 0–24 hours after birth during the trial period in that hospital. This means that the Dutch guideline or the EOS calculator (depending on randomisation) will be used for all newborns at risk for EOS (within 0–24 hours after birth) in the particular hospital, independent of study participation. Study participation will entail prospective data collection, subject to informed consent. Participating hospitals will be required to commit to adherence to the assigned strategy prior to randomisation.
Eligibility
Newborns are eligible for participation if they have a postmenstrual age of 34 weeks or more and in case of suspected EOS, defined as if one or more criteria for elevated maternal EOS risk or neonatal clinical signs of EOS are present within the first 24 hours of life (in accordance with the Dutch guideline,8 table 1). The presence of major congenital anomalies16 or a significant language barrier will lead to exclusion from participation.
Control and intervention
Control: Dutch guideline
The Dutch guideline uses 8 maternal and 15 neonatal risk factors, each categorised as either red flag or non-red flag (table 1, online supplemental file 3). These criteria guide clinicians on the management in case of suspected EOS. Briefly, antibiotic treatment is recommended in the presence of at least one red flag and, or, two or more non-red flags. The guideline recommends to obtain a blood culture before the start of antibiotic treatment, as well as to consider determining the serum level of C-reactive protein.
An observation period of 12–24 hours is recommended in the presence of one non-red flag. This could be a maternal risk factor or a clinical symptom of the neonate. Antibiotic treatment is recommended when an infection is suspected during this observation. Newborns will be discharged from the hospital if there are no maternal risk factors, the newborn is in good clinical condition, and the gestational age is at least 36 weeks. If the guideline recommends an observation period, the newborn is discharged after repeating physical examination that concludes that the newborn is in good clinical condition. In case antibiotic treatment is started, discharge depends on the duration of treatment and the clinical course.
Intervention: EOS calculator
Using the EOS calculator application, between 0 and 24 hours after birth, maternal EOS risk factors combined with the results of physical examination of the newborn are used to assign a risk category and accompanying clinical recommendation based on estimated EOS risk for each newborn at risk for an infection. The EOS calculator results are used to guide a clinical management decision on performing either a diagnostic work-up and start of empiric intravenous antibiotics for suspected or proven EOS, or a conservative approach with routine controls of vital parameters (heart rate, respiratory rate, and temperature every 3 hours) by the nurse. In case of routine controls, re-evaluation of physical appearance by a paediatric resident or paediatrician will take place within 24 hours post partum. The original EOS calculator recommends obtaining a blood culture without the start of empiric antibiotics, a strategy seldom practised in The Netherlands. This recommendation was therefore replaced with clinical observation with a blood culture in case of start of antibiotics during observation. As an additional safety precaution, newborns will be observed for 24 hours using vital parameters. In case antibiotic therapy is started, the need for further treatment is assessed after 24–36 hours of treatment depending on blood culture results, infection parameters and clinical condition of the newborn. Discontinuation of antibiotic therapy and discharge is at the discretion of the treating physician.
In order to meet requirements of the European Regulations for Medical Devices, the EOS calculator has been developed as a smartphone application specifically for this trial, and CE marking has been obtained. Figure 1 shows a screenshot of the smartphone application. Two more screenshots are available in online supplemental file 4. Only in the hospitals randomised for the EOS calculator, this application will be available for physicians in charge of newborns at risk for EOS.
At discharge, parents/guardians receive information when to contact the hospital in case of signs of infection within the first 14 days of life.
Randomisation and blinding
Randomisation for either the EOS calculator or Dutch guideline will take place at cluster level (randomisation per hospital) using a 1:1 randomisation scheme using a computer-generated algorithm and will be executed by an independent methodologist. Given the nature of this trial, hospitals and physicians could not be blinded for the assigned trial protocol.
Informed consent
The trial will be conducted according to the ‘Code of conduct relating to expressions of objection by minors participating in medical research’ approved by the Board of the Dutch Society of Paediatrics on 21 May 2001. Parents or guardians are free to decide to withdraw from the trial at any stage, and for any reason, without prejudicing their child’s further treatment. In this cluster randomised trial, two forms of usual care are compared under equipoise. If inclusion criteria are met, parents/guardians will receive a parent information letter with extensive trial information after birth. Separate consent for data collection regarding mother and newborn will be gained by the investigator/research nurse. Consent forms are included in online supplemental file 5.
Data collection, management and follow-up
Data will be collected by research nurses at participating hospitals and stored anonymously in a digital database (Castor EDC, Ciwit B.V.). Independent monitors will perform source data verification and assess performance of trial procedures at least once a year at each site.
Figure 2 outlines the trial procedures. Laboratory analysis will happen according to local protocols of each participating hospital, and be interpreted by the local clinical care team. To investigate the impact of suspected or proven EOS on the quality of life of parents/guardians and their child, a survey will be conducted on day 14 after birth, using a questionnaire based on preceding relevant studies.17 18 Information on side effects, number of medical visits, readmission, medication use, sleep quality of the participant, (breast)feeding success rate, subjective parental/guardian evaluation of the impact of newborns’ admission, and parental/guardian projection of future quality of life of their newborn will be collected. The questionnaire is included in online supplemental file 6. To enhance follow-up, parents/guardians will be called when they have not yet completed the questionnaire after a reminder by email.
Outcome measures
Coprimary non-inferiority outcome
A composite non-inferiority outcome for this trial is defined as the presence of one or more of four predefined safety criteria: the need for any respiratory support, the need for an intravascular fluid bolus for haemodynamic instability due to sepsis, referral to a Neonatal Intensive Care Unit for sepsis treatment, and the incidence of proven EOS. Respiratory support is defined as any form of respiratory support (invasive ventilation, continuous positive airway pressure, high flow nasal cannula, low flow oxygen) during the first week of life. Intravascular fluid bolus is defined as the intravenous administration of a fluid bolus of 10 mL/kg within 15–30 min. It is the first step in the treatment of hemodynamically unstable newborns due to sepsis. Proven EOS is defined as a blood or cerebrospinal fluid culture obtained within 72 hours after birth growing a pathogenic bacterial species. Cultures will be processed and analysed according to local protocols of the participating hospital.
Coprimary superiority outcome
The coprimary superiority outcome is the proportion of participants that started antibiotic therapy for suspected and, or proven EOS in the first 24 hours after birth.
Secondary trial parameters/endpoints
We defined three secondary endpoints: the total duration of antibiotic therapy, the proportion of antibiotic therapy started for suspected and, or proven EOS if symptoms started between 24–72 hours after birth, and quality of life as measured by the survey at day 14 after birth.
Statistical analysis
Power calculation
The sample size is calculated for both the superiority and non-inferiority outcome. For superiority, we expect 40% antibiotic therapy in newborns in the control group at risk for EOS. To reduce this to 25% (reduction 15%) by use of the EOS calculator (80% power, ICC 0.0025, alpha 0.05, 10 hospitals), a total sample size of 330 is required. For non-inferiority, we expect in the control group and the intervention group that 10% of newborns will fulfil one of the predefined safety criteria. The non-inferiority margin is set at 15% (absolute difference 5%). The required sample size is 1640 (80% power, ICC 0.0025, alpha 0.025, 10 hospitals). Because both hypotheses regarding superiority and non-inferiority have to be met, the trial sample size equals the largest of the two sample sizes. The potential drop-out or withdrawal is estimated at 10%. We will include 1830 (= 2 × 915) newborns, which means 183 per cluster. With an average number of circa 2000 eligible newborns per hospital, a baseline suspected EOS of 20% (based on experience in previous observational studies10 14), and at least 50% participation rate of eligible participants, the average number of possible participants per hospital in a 1-year period equals 200. In the Netherlands, there are currently no competing trials in this population.
Primary analysis
Analyses will be performed both as intention-to-treat and per protocol; any protocol violations will be documented. The primary analysis will estimate the relative risk with 95% CIs and p values for the primary outcomes, using a generalised linear mixed model with log link and binomial distribution, with random intercepts and random slopes per cluster as appropriate. A significance level of 5% will be used. Both non-inferiority for adverse clinical outcome and superiority for the proportion of participants that started antibiotic therapy must be shown to conclude the effectiveness of the EOS calculator. Use of the EOS calculator will be considered non-inferior when the upper bounds of a 95% CI of the relative risk of complications does not exceed 1.5 (comparable to a 15% complication rate). Use of the EOS calculator will be considered superior when it leads to at least a 15% absolute reduction in the proportion of participants who started antibiotic therapy. We expect 10% neonatal complications in the control group.
Dichotomous secondary outcomes will be analysed using the same method as the primary outcome. Continuous data of secondary outcomes will be assessed using a linear mixed model to estimate mean differences, with random intercepts and slopes as appropriate. Median differences will be calculated as appropriate.
Outcome reporting of the quality of life questionnaire will be mainly descriptive. Mixed models can be used to further explore the effect of treatment and hospital if a trend is observed for some of the questions. Mean scores and SD are calculated for composite indicators. We use t-tests to evaluate differences in mean scores between groups for normally distributed data. The significance level is set at an alpha value of <0.05.
Interim analysis
Interim analyses for safety are planned after outcome data for the first 200 participants in either arm (in total 400) have come available. This will be repeated at a total of 900 participants. The data safety monitoring board may advise to terminate the trial prematurely in case an interim analysis shows clear harm of either one of the interventions, or due to external evidence. The data safety monitoring board will not be blinded when first assessing the data. A formal interim analysis for efficacy will not be conducted.
Ethics and dissemination
Safety consideration
All adverse events will be recorded up until and including day 14 after birth. This period is consistent with the follow-up time and reasonably includes complications that may be linked to initial EOS management, such as a different course of EOS or readmission. In the unusual case that a newborn still receives antibiotic therapy after day 14 of life because of EOS (or a serious adverse event), adverse events will be reported until the end of therapy.
Newborns are vulnerable research participants. To ensure safety of trial participants an independent data safety monitoring board will monitor the trial and consists of several members with expertise in the relevant fields of paediatrics, epidemiology and statistics.
Patient and public involvement
The society Care4Neo is involved in the conception of the design of the trial. Care4Neo has approved the execution of this trial. Furthermore, a summary of findings will be published on the website of Care4Neo (https://www.care4neo.nl/) and on the website of Zorgevaluatie Nederland (https://zorgevaluatienederland.nl/eos-calculator-rct).
Ethics approval
According to Dutch law, this trial was centrally approved by the Medical Ethics Committee of the Amsterdam UMC (NL78203.018.21. 2022, 14 January). The trial will be conducted in accordance with the principles of the declaration of Helsinki (October 2008), the ICH GCP guidelines (CPMP/ICH/135/95), the Regulations on Medical Research involving Human subjects (Medical Research involving Human subjects Act, 1999), and the Medical Device Regulation (MDR/2017/745).19
Provenance and peer review
Not commissioned; externally peer reviewed by an independent committee of the Dutch Society of Paediatrics.
Dissemination
The trial will be considered for publication and presentation at (scientific) symposia or congresses. Authorship will follow the guidelines defined by the International Committee of Medical Journal Editors (http://www.icmje.org). Since participant data are recorded anonymously, participant privacy will be guaranteed. The results will be used to improve and publish new guidelines. Access to the final trial dataset, the full protocol, and statistical code will be managed according to Dutch regulatory instructions.
Discussion
This paper presents a protocol for a pragmatic cluster randomised clinical trial comparing use of the EOS calculator to use of the Dutch national guideline for managing risk for EOS at birth. Several issues regarding safety, ethics, study design, statistical power and clinical applicability and urgency were considered and will be discussed here.
Compared with conventional strategies like the Dutch guideline,8 EOS calculator use is associated with less antibiotic prescription.13 This can be interpreted as better specificity, reducing the risk of overtreatment. However, sensitivity of the EOS calculator may be significantly lower in identifying EOS, especially directly after birth.13 19 20 Lower sensitivity may increase the risk of delayed and thus possibly less adequate treatment in initially well-appearing newborns. Differences in EOS sensitivity and specificity between the EOS calculator and Dutch guidelines potentially may thus carry risks of undertreatment or overtreatment of newborns included in this study. The risk of undertreatment was considered minimal due to added safety with close clinical observation in case of withholding antibiotic treatment. The risk of overtreatment is a given of today’s clinical practice and Dutch guideline, and thus not increased by inclusion in the study.
Current equipoise between the Dutch guideline and the EOS calculator was based on findings of previous studies, and established in consensus with participating hospitals. Equipoise between a risk-factor based approach like the Dutch guideline and the EOS calculator is confirmed by adoption of the EOS calculator in both the recently updated version of the NICE guideline,21 and recommendations of the American Academy of Pediatrics.22 These guidance documents thus consider both a risk-factor based approach and the EOS calculator as valid strategies, although the NICE guideline stipulates a prospective audit to accompany implementation of the EOS calculator.
Equipoise between the Dutch guideline and EOS calculator enabled adoption of these strategies as standard of care and a cluster randomised design: a participating hospital will commit to the protocol for which it is randomised as standard of care. Although a standard of care is used, the attending physician will always have the option to divert clinical management from the assigned protocol if deemed appropriate. Our preceding observational study of the Dutch guideline showed this common practice,10 a finding that provides rationale to use both intention-to-treat and per protocol analysis in our trial.
Our study uses a cluster randomised trial instead of an individual randomisation design, mainly because time for randomisation can be very restricted due to the immediate need to start antibiotic therapy in newborns that appear clinically sick.23–25 Furthermore, the clustered approach prevents confusion and contamination between the different protocols being compared, thereby limiting contamination bias.25 26 Finally, the probability of participation in the trial increases as a result of better timing of recruitment.27
The trial has been codesigned by Care4Neo, the Dutch neonatal-patient organisation. In particular, the cluster randomised design was considered beneficial from a participant and parental perspective: informed consent will not be asked directly after birth, which is associated with less stress.27–29
Due to the low incidence of proven EOS, a study with adequate statistical power to research consequences of potential delayed treatment of proven EOS as a result of EOS calculator implementation would require an unrealistically large sample size. Therefore, the power calculations for our study are based on suspected EOS cases, with incidence of proven EOS included as a safety outcome.
To adhere to national regulations regarding medical device and software regulations, our study uses a dedicated EOS calculator smartphone application. It received applicable CE marking as a medical device for healthcare professionals, by Dutch regulators.
A pragmatic non-inferiority trial addressing safety outside of the USA is needed to improve care for newborns at risk for EOS, and urgently called for by both clinicians and patient organisations. Our trial answers calls to address the pressing clinical issue of antibiotic overtreatment in newborns. A survey and prioritisation project of the Dutch Society of Paediatrics put better identification of newborns at risk for EOS in the top three of the current knowledge gaps in paediatrics.20 Similarly, the British National Institute for Health and Care Research recently published a call to fund high-quality evidence which aims to determine the validity of the EOS calculator.21
Limitations of this trial will include limited generalisability due to a limited number of clusters in a single country, the inclusion at-risk newborns rather than only proven infected newborns, and the use of a composite safety outcome. Despite these limitations, results of this trial could inform and facilitate implementation of the EOS calculator. As such, the trial may lead to reduced antibiotic exposure in newborns at risk for EOS and improved quality of life of both newborns and parents.
Trial status
This manuscript is based on the trial protocol ‘Safely reduce newborn antibiotic exposure with the early-onset sepsis calculator: a cluster randomized study (EOS Calculator RCT)’ version 4.0, 1 March 2022. After approval by the Medical Ethics Committee of the Amsterdam UMC protocol modifications will be communicated with the local investigators and ClinicalTrials.gov. The trial is recruiting since 12 April 2022. The approximate date of completing recruitment is the end of September 2023.
Ethics statements
Patient consent for publication
Acknowledgments
We thank Dr. Sylvia Obermann, prof. Dr. Annemarie M.C. van Rossum, Dr. Merijn W. Bijlsma, Dr. Douwe H. Visser, Dr. Lisanne van Leeuwen, and Dr. René F. Kornelisse as co-writers of the grant application.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors NBA and FBP conceived the original idea for this trial. All authors were involved in further conception and trial development. BMvdW wrote the first draft. All authors rewrote the manuscript. All authors approved the final version of the manuscript.
Funding This work is supported by the Dutch Society of Paediatrics.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.