Article Text

Download PDFPDF

Protocol
Validity of algorithms for identifying mild traumatic brain injury in the French national emergency department database OSCOUR: a retrospective multicentre validation study protocol
  1. Louis-Marie Paget1,
  2. Cécile Forgeot2,
  3. Fleur Lorton3,4,
  4. Nozar Aghakhani5,6,
  5. Maurice Raphael7,
  6. Anne Gallay1,
  7. Nathalie Beltzer1
  1. 1Direction des maladies non transmissibles et traumatismes, Santé publique France, Saint-Maurice, Île-de-France, France
  2. 2Direction Appui, Traitements et Analyses de données, Santé publique France, Saint-Maurice, Île-de-France, France
  3. 3Nantes Université, CHU Nantes, INSERM, CIC 1413, F-44000 Nantes, France
  4. 4Department of Pediatric Emergency, Hôpital Femme Enfant Adolescent, CHU de Nantes, Nantes, Pays de la Loire, France
  5. 5Department of Neurosurgery, Hôpital Bicêtre, Assistance Publique - Hôpitaux de Paris, Le Kremlin-Bicêtre, Île-de-France, France
  6. 6Center for Evaluation and Multidisciplinary Care of the Mild Traumatic Brain Injury, Hôpital Bicêtre, Assistance Publique - Hôpitaux de Paris, Le Kremlin-Bicêtre, Île-de-France, France
  7. 7Urgences Hirslanden, Lausanne, Switzerland
  1. Correspondence to Louis-Marie Paget; Louis-Marie.paget{at}santepubliquefrance.fr

Abstract

Introduction The French emergency department (ED) surveillance network OSCOUR transmits data on ED visits to Santé publique France (the national public health agency). As these data are collected daily and are almost exhaustive at a national level, it would seem relevant to use them for national epidemiological surveillance of mild traumatic brain injury (mTBI). This article presents the protocol of a planned study to validate algorithms for identifying mTBI in the OSCOUR database. Algorithms to be tested will be based on International Classification of Diseases (ICD)-10 codes.

Methods and analysis We will perform a multicentre validation study of algorithms for identifying mTBI in OSCOUR. Different combinations of ICD-10 codes will be used to identify cases of mTBI in the OSCOUR database. A random sample of mTBI cases and non-cases will be selected from four EDs. Medical charts will serve as the reference standard to validate the algorithms. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the different algorithms, as well as their 95% CIs, will be calculated and compared.

Ethics and dissemination The ethics committee of the French National Data Protection Authority (CNIL) approved this study (n° 921152, 1 August 2021). Results will be submitted to national and international peer-reviewed journals and presented at conferences dedicated to trauma and to methodologies for the construction and validation of algorithms.

  • EPIDEMIOLOGY
  • PUBLIC HEALTH
  • ACCIDENT & EMERGENCY MEDICINE
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • This is a multicentre study conducted in four emergency departments in France; it will be the first in France to develop algorithms to identify cases of mild traumatic brain injury (mTBI) at a national level.

  • The review of patients’ medical charts will be used as the reference standard to validate the accuracy of the algorithms.

  • A wide variety of algorithms will be evaluated, combining International Classification of Diseases-10 codes.

  • In some cases, difficulties in diagnosing mTBI may complicate case identification in patient charts.

Introduction

Between 50 and 60 million new cases of traumatic brain injury (TBI) of all levels of severity are recorded worldwide each year.1 Of these, over 90% are mild TBI (mTBI),2 defined as ‘an acute brain injury resulting from mechanical energy to the head from external physical forces’. Operational criteria for clinical identification include: (1) one or more of the following: confusion or disorientation, loss of consciousness for 30 min or less, post-traumatic amnesia for less than 24 hours and/or other transient neurological abnormalities such as focal signs, seizure and intracranial lesions not requiring surgery; and (2) Glasgow Coma Scale score of 13–15 after 30 min post-injury or later on presentation for healthcare.3

An increase in mTBI incidence has been described in several recent international studies.4–7 These very common traumas mainly affect men. All age groups are concerned but particularly young children under 5 years of age, people aged 15–24 years old and those aged 75 years and over. The most common causes are falls and road accidents.8 Although mTBI are classified as mild, they are not benign. While rarely life-threatening, the literature shows that a significant proportion of patients (20–36%) continue to have symptoms months and even years after the trauma occurs.9 These symptoms, which are not specific to mTBI, are physical (headaches, fatigue, balance or hearing disorders, neck pain, etc) and intellectual (attention, concentration, memory disorders, etc) in nature; some patients have mood and behavioural disorders (impatience, anxiety, irritability, depression, guilt, etc). All these symptoms cause personal and family suffering and can lead to situations of social withdrawal. They sometimes also affect employment or schooling opportunities in younger people. Moreover, several studies have shown that mTBI may be a risk factor for several neurodegenerative diseases.10

In France, despite the severity of this issue, no recent epidemiological data exist to quantify and characterise the victims of mTBI at the national level. However, such data are essential to estimate the burden of mTBI, with a view to better adapting the offer of care—in particular for patients suffering from complications—and to implementing prevention measures. Using data from the French emergency department (ED) visits database (OSCOUR network data) could be relevant for national epidemiological surveillance of mTBI, since these data are collected daily and are almost exhaustive at the national level. Moreover, a significant proportion of patients with mTBI who seek medical care are diagnosed and managed in ED.6 However, the accuracy of potential algorithms which could be used in OSCOUR is not as yet known.

A study conducted in the USA which evaluated the accuracy of a proposed algorithm based on the ICD-9 (ICD, International Classification of Diseases) codes proposed by the Centers for Disease Control and Prevention to detect mTBI cases from medical and administrative databases,11 and which used the clinical examination of ED patients as the reference standard, showed that the algorithm had low sensitivity (45.9%) but high specificity (97.8%). While these results are informative, they do not predict the accuracy of algorithms based on ICD codes that could be used to detect cases of mTBI in the OSCOUR database as each database has specific features and coding practices vary within and between countries. Accordingly, before OSCOUR data can be used to monitor mTBI in France, a multicentre validation study of potential algorithms is essential.

The study protocol presented here aims to measure the accuracy of potential algorithms in the OSCOUR database, which combine ICD-10 codes to identify visits for mTBI in France.

Methods and analysis

Study population selection

The validation study we will conduct is a retrospective multicentre study. Initially, three EDs have been randomly selected from the OSCOUR network and in addition, we also included the ED in which we conducted a pilot study. Of the four centres included for this study, two were teaching hospitals and two were non-teaching hospitals. These centres were located in different regions in France: Auvergne-Rhône-Alpes region, Centre-Val de Loire region, Grand-Est region, Ile-de France region (Paris region).

In each selected ED, a sample of visits (all ages) among all-cause visits occurring between 1 January 2019 and 31 December 2019 will be selected. This period reflects the most recent year before the COVID-19 pandemic for which we have consolidated data on ED visits which are representative of ED activity before the pandemic started.

Setting and data source

Administrative database

In France, data on ED visits are collected on a daily basis by ED participating in the OSCOUR network.

In 2019, the network included 680 EDs and covered 93% of all ED visits, including French overseas regions (except Martinique). An average of 56 700 ED visits per day were recorded in the OSCOUR database.

For each ED visit, an emergency visit report (EVR) is systematically produced. EVR contains medical information such as the primary diagnosis (PD), up to 10 associated diagnoses (AD) coded according to the ICD-1012 (in 2019, the PD was recorded in 77% of the EVRs while AD were recorded in less than 10%) and chief issues. EVR also contains demographic (sex, age, residence area code) and administrative (ED structural information, release date and time from ED and orientation on discharge from ED (ie, home, hospital ward, etc)) data. Data for each visit are in the OSCOUR database pseudonymised; accordingly individual patients cannot be directly identified.

Case and non-case identification in OSCOUR

The EVR from the four selected EDs will be classified as mTBI ‘cases’ and ‘non-cases’ by the different algorithms tested. In our article, the term algorithm is used simply to refer to a list of ICD-10 codes.

The first algorithm, we plan to test is based on the S06.0 ‘concussion’ ICD-10 code alone. The S06.0 code is the only code which specifically describes mTBI in ICD-10 classification. In addition, a literature review we conducted earlier pointed out that the most frequently found mTBI identification algorithm included only the code of concussion.

Next, we plan to develop other algorithms in order to explore the feasibility of better accuracy in identifying mTBI cases compared with the algorithm based on the S06.0 code. These ‘broad algorithms’ will be based on a list of ICD-10 codes which will include the S06.0 and all clinically relevant ICD-10 codes found in EVRs which correspond to medical charts of cases that we will have identified during the review of the medical charts we will carry out in the four centres. From this list, different ‘broad algorithms’ (ie, combinations of ICD-10 codes) will be developed. Box 1 shows, for information purposes, the list of all clinically relevant ICD-10 codes identified during the pilot study. This list includes most of the codes identified in the literature previously13–16 but also other codes never used such as open wound codes.

Box 1

List of International Classification of Diseases (ICD)-10 codes of mild traumatic brain injury cases identified during the pilot study

ICD-10 code: Code title

  • S00.0: Superficial injury of scalp.

  • S00.3: Superficial injury of nose.

  • S00.7: Multiple superficial injuries of head.

  • S00.9: Superficial injury of unspecified part of head.

  • S01.0: Open wound of scalp.

  • S01.1: Open wound of eyelid and periocular area.

  • S01.4: Open wound of cheek and temporomandibular area.

  • S01.5: Open wound of lip and oral cavity.

  • S01.8: Open wound of other parts of head.

  • S01.9: Open wound of unspecified part of head.

  • S02.80: Fracture of other specified skull and facial bones.

  • S06.0: Concussion

All the algorithms (‘S06.0 algorithm’ and ‘broad algorithms’) will be tested as such and then with the addition of exclusion criteria to try to exclude some severe forms of TBI: moderate and severe TBI that would have been wrongly selected by our algorithms.

The management of patients with moderate or severe TBI are quite different from that of patients with mTBI. Patients with moderate or severe TBI are much more frequently referred to intensive care units or neurosurgery than patients with mTBI.17 Using the information coded in OSCOUR that describes the patient’s orientation after ED visits, it should be possible to exclude some cases of moderate-to-severe TBI.

Thus we will use the following two exclusion criteria:

  • Patient referred after ED visits to an intensive care unit.

  • Patient referred after ED visits to a surgical service (The type of surgical service is not specified in OSCOUR. It will therefore not be possible to identify patients referred to neurosurgery).

These two exclusion criteria will be added to the ‘S06.0 algorithm’ and to the ‘broad algorithms’ first in isolation and then in combination.

EVR with at least one medical diagnosis (PD or AD) ICD-10 code included in the algorithm will be considered ‘cases’ in OSCOUR. EVR containing no ICD-10 codes included in the algorithm in diagnostic will be considered ‘non-cases’.

Medical chart abstraction and case ascertainment

The computerised medical charts produced for each ED visit and stored in the hospital servers which the ED use will be used as the reference standard to validate the mTBI case identification algorithms in the OSCOUR database. We will identify the medical charts through the computerised medical charting systems (DMU, Cristal-Net, DxCare, and Crossway) of the ED selected for the study.

To identify medical charts which correspond to selected EVR, we will cross-link various variables (sex, residential postal code, date of birth, date and time of entry to ED, ED exit date and time, and orientation on discharge from ED).

The medical charts will be read independently by two epidemiologists who will then categorise medical charts as an mTBI ‘certain case’, ‘probable case’, ‘possible case’ or ‘non-case’ using the validation criteria presented below. File classification will be blinded: the epidemiologists will not know the ICD-10 codes in the EVR corresponding to the medical chart analysed.

In order to ensure a good agreement between the two epidemiologists in the classification of medical charts, we will implement the following process:

Initially, the two epidemiologists will analyse, in one centre, 100 identical medical charts, independently. The Kappa statistic will be used in order to quantify the agreement between the two epidemiologists in the classification of medical charts. Disagreements will be analysed and resolved by consensus between the two epidemiologists and by an expert physician if needed through telephone exchanges. This process will be repeated (review of 100 additional medical charts by the two epidemiologists) until the kappa statistic is greater than 0.8, meaning an almost perfect agreement.18 At the end of this process, the review of the medical charts will continue to be carried out independently by the two epidemiologists but without measuring the agreement in the classification of medical charts. During the medical charts review, problems with classifying medical charts will be resolved through telephone exchanges with mTBI expert physicians who participate in the project.

For feasibility reasons, we will not review all medical charts. We will use an approach that allows us to select all medical charts of mTBI cases while minimising the number of medical charts of non-case. First, we will identify all the ICD-10 codes corresponding to medical charts of mTBI cases. ICD-10 case codes will be searched in each of the four centres independently to take into account the coding specificities of each centre. To identify, in each of the four centres, the list of ICD-10 codes corresponding to the medical charts of cases, a ‘saturation process’ will be used. This saturation process will be implemented in the following way. At the end of each day of medical charts review, ICD-10 codes associated with the medical charts of mTBI cases will be listed. Based on the results of a pilot study we conducted previously, about 10 medical charts of mTBI cases should be retrieved each day of medical charts review.

Then, the list of ICD-10 codes identified at the end of each day will be compared with that of the previous days (the list identified on the second day with that identified on the first day, the list identified on the third day with that of the two previous days and so on). When for one day no new codes will be identified compared with those identified the previous days, this could mean that saturation has been reached. To ensure that saturation has been reached, we will analyse the medical charts of a new day. At this stage, we will stop the process of saturation if no new codes are identified. At this stage, if new codes are identified the iterative process will continue as described above until reaching saturation (no new ICD-10 codes of mTBI cases are found for two consecutive days of medical charts review)

At the end of the saturation process we will have for each of the four centres a comprehensive list of ICD-10 codes of mTBI cases. Thus, in order to reduce the number of medical charts to be reviewed we will select and classify in each hospital only the medical charts corresponding to EVR with codes of mTBI cases identified with the saturation process. Medical charts corresponding to EVR without mTBI cases codes will be classified directly as non-cases without being reviewed. We cannot exclude that in some centres the saturation process does not succeed. If this scenario occurs, all the medical charts will have to be reviewed.

Criteria for the classification of medical charts

No biological or radiological examination exists to help definitively diagnose an mTBI. Diagnosis of an mTBI is based solely on the search for symptoms and clinical signs reported by the patient or his/her family and on the physician’s clinical examination. Accordingly, the diagnosis of mTBI is sometimes uncertain. In order to take into account this uncertainty, we will distinguish ‘certain cases’ cases of mTBI, ‘probable cases’ and ‘possible’ cases based on the elements found in the medical charts.

Certain (ie, conclusive) mTBI cases will be defined according to the most widely accepted criteria established by the WHO3:

(1) An explicit statement of head trauma:

  • A direct or indirect blow to the head.

Or

  • A whiplash-like mechanism involving a violent head acceleration/deceleration movement.

AND

(2) A Glasgow Coma Scale (GCS) score between 13 and 15, 30 min post-injury or later on presentation for healthcare

And

(3) At least one of the following: post-traumatic amnesia of less than 24 hours, confusion or disorientation, loss of consciousness for 30 min or less and/or other transient neurological abnormalities such as focal neurological deficits, seizures and intracranial lesions found by CT scans that do not require neurosurgical intervention.

The medical charts, without GCS and with no evidence describing moderate or severe TBI but which reflect other criteria of the WHO definition will be considered as certain cases. Indeed, we assume that if the GCS is not indicated in the file, it is normal (GCS=15).

The elements describing moderate-to-severe TBI to look for in the medical charts when the GCS is not mentioned are the following:

  • Items describing intracranial lesions associated with items describing a neurosurgical management.

  • Items describing a coma or signs or symptoms specific to moderate and severe TBI.

  • Items describing the three components assessed in the GCS (eye opening, verbal response and motor response) and indicative of moderate or severe TBI.

Examples of words to look for in the medical charts to identify cases of moderate and severe TBI when GCS is not mentioned are listed in box 2.

Box 2

Examples of words to look for in the medical charts to identify cases of moderate and severe traumatic brain injury (TBI) when the Glasgow Coma Scale (GCS) score is not mentioned

Items describing intracranial lesions associated with items describing a neurosurgical management

Items describing intracranial lesions:

  • Epidural haemorrhage.

  • Subdural haemorrhage.

  • Intracerebral haemorrhage.

  • Subarachnoid haemorrhage.

  • Brain oedema.

  • Ischaemic brain damage.

Items describing a neurosurgical management:

  • Neurosurgical management.

  • Neurosurgical intervention.

Items describing a coma or signs or symptoms specific to moderate and severe TBI:

  • Coma.

  • An inability to wake up from sleep.

  • Increased confusion, nervousness or agitation.

  • Prolonged loss of consciousness (>30 min).

  • Prolonged amnesia (>24 hours).

Items describing the three components assessed in the GCS (eye opening, verbal response and motor response) and indicative of moderate or severe TBI:

  • The patient does not open eyes to a painful stimuli or open eyes only to a painful stimuli.

  • The patient does not answer simple questions or is making incomprehensible sounds to answer simple questions.

  • The patient has no motor answer, or abnormal extension to pain or abnormal flexion to pain.

Then, we hypothesised that if the duration of loss of consciousness or amnesia is not indicated in the medical charts, it is because it is not significant (loss of consciousness <30 min and amnesia <24 hours). Thus, in the same way as for the medical charts without GCS mentioned, we will considered as certain cases, the medical charts with loss of consciousness or amnesia mentioned but without duration of loss of consciousness and/or post-traumatic amnesia and without a GCS strictly inferior to 13 and/or without other evidence describing moderate or severe TBI (see above in the definition of certain cases).

Then, in the category of ‘probable cases’, all uncertain cases will be included.

Probable mTBI cases will be defined based on the following criteria :

(1) An explicit statement of head trauma:

  • A direct or indirect blow to the head.

Or

  • A whiplash-like mechanism involving a violent head acceleration/deceleration movement.

AND

(2) At least one of the following criteria:

  • Suspected loss of consciousness or amnesia.

  • At least one post-concussion symptom (symptoms frequently found in victims of mTBI but which are not specific to mTBI) of the Sport Concussion Assessment Tool Fifth Edition19 (box 3) .

  • Criteria specific to <2 years old : scalp haematoma, abnormal behaviour according to parents.

Box 3

List of symptoms of the Sport Concussion Assessment Tool Fifth Edition checklist

  • Headache.

  • ‘Pressure in head’.

  • Neck pain.

  • Nausea or vomiting.

  • Dizziness.

  • Blurred vision.

  • Balance problems.

  • Sensitivity to light.

  • Sensitivity to noise.

  • Feeling slowed down.

  • Feeling like ‘in a fog’.

  • ‘Don’t feel right’.

  • Difficulty concentrating.

  • Difficulty remembering.

  • Fatigue or low energy.

  • ConfusionA.

  • Drowsiness.

  • More emotional.

  • Irritability.

  • Sadness.

  • Nervous or anxious.

  • Trouble falling asleep (if applicable).

  • APatients with symptoms of confusion will be classified as certain cases because confusion is also part of the criteria of the WHO definition.

(3) No evidence describing moderate or severe TBI: no mention of a GCS strictly lower than 13 and/or no mentions of other elements describing moderate or severe TBI (see above)

Possible mTBI cases will be defined based on the first WHO criteria only:

(1) An explicit statement of head trauma:

  • A direct or indirect blow to the head.

Or

  • A whiplash-like mechanism involving a violent head acceleration/deceleration movement.

(2) No evidence describing moderate or severe TBI: no mention of a GCS strictly lower than 13 and/or no mentions of other elements describing moderate or severe TBI (see above in the definition of certain cases).

Medical charts that are not classified in certain cases (no criteria of certain cases), probable cases (no criteria of probable cases) or possible cases (no criteria of possible cases) will be considered as ‘non-cases’.

Analysis of medical charts corresponding to EVR without an ICD-10 code

As previously mentioned, not all visits recorded in the OSCOUR database contain an ICD-10 code. To ensure that there is no systematic bias in the coding of mTBI cases (lower coding of mTBI cases, lower coding of non-hospitalised mTBI cases, etc) we will review a sample of medical charts corresponding to visits without coded ICD-10 codes. In one of the four centres randomly selected (for feasibility reasons) participating in the study, 200 medical charts will be analysed and classified using the criteria presented above. This additional analysis conducted in a single centre will allow us to discuss the generalisability of the results obtained from our algorithms for identifying mTBI cases.

Statistical analysis

Sample size

We estimate that a minimum sample size of 100 certain mTBI cases (our most restrictive definition) in patients’ medical charts is needed to achieve a sensitivity and specificity of the algorithms of 50% (worst case scenario knowing that we do not know, a priori, the sensitivity and specificity of our algorithms applied to data in the OSCOUR database) with a precision of 10% and a 5% α risk. Accordingly, assuming that mTBI in certain cases accounts for 1% of ED visits,20 we would need a sample of at least 10 000 patients for our validation study. As it may not be possible to link or analyse some files (because of a lack of information to classify the patient) 12 000 files will be selected. An equal number of files will be randomly selected in each of the four ED sites (ie, 3000 cases per site).

If saturation is not reached in any centre, all the 12 000 medical charts (6000 medical charts per epidemiologist) will need to be reviewed. If the saturation process is reached in one or more centres, the total number of medical charts that will be analysed is difficult to predict. Because we do not know in advance at what stage saturation will be reached. Finally, we do not know in advance the list of ICD-10 codes of mTBI cases that will be found in the centres where saturation will be reached.

Accuracy of algorithms

Patients with medical charts that are not linkable or not analysable will be excluded from the analysis.

The sensitivity, specificity, PPV and NPV of the different algorithms tested in the OSCOUR database will be calculated, respectively, for ‘certain’ cases, for ‘certain’ and ‘probable’ cases and for ‘total’ cases (ie, certain, probable and possible cases combined). The calculation formulas used to calculate these different indicators are presented in table 1. Calculation of 95% CIs will be made for each of these four metrological qualities. All four qualities of the various algorithms will be measured globally (ie, for all four EDs) in the study . Statistical analyses will be performed using SAS Enterprise Guide V.7.4.

Table 1

Formulas for calculating sensitivity, specificity, PPV and NPV of mTBI case selection algorithms in OSCOUR

Patient and public involvement

No patients were involved in the design, or conduct, or reporting, or dissemination plans of the research.

Reporting

We will ensure that we present the methodology and results of our study in a transparent and accurate manner. We will follow guidelines proposed by Benchimol et al in 2011 regarding the presentation of the methodology and results of validation studies.21

Discussion

This protocol outlines the approach we will follow to study the accuracy of potential algorithms for identifying mTBI in the OSCOUR database. The method we will use is based on the methodological framework and recommendations proposed by Widdifield et al regarding the implementation of validation studies.22

ED data (OSCOUR database) are a particularly relevant source of data for national-level mTBI surveillance in France, and can be used to produce regularly updated information, for victims of all ages. More generally, the use of the OSCOUR database for national surveillance of mTBI could help highlight the importance of this public health issue in France.

Before this database can be used for epidemiological monitoring of mTBI, it is essential to conduct a validation study to ensure that cases can be accurately defined using algorithms based on ICD-10 codes. We will seek to identify which of the algorithms tested has the highest sensitivity and specificity. If no algorithm is found to be effective in identifying mTBI cases, the results of the validation study will nevertheless be useful in making recommendations to improve the coding of mTBI in OSCOUR.

Our study has several limitations. First, we cannot exclude that some cases of mTBI in our study will not be identified. The mTBI is complicated to identify and diagnose. The diagnosis of mTBI is based solely on the signs or symptoms reported by the patient or his or her family or highlighted by the physician during the clinical examination: there are no biological or radiological examinations that allow a diagnosis of certainty. To try to take into account this limitation inherent to all studies on mTBI, the epidemiologists who will classify the medical charts will be helped during the medical charts review by the mTBI expert physicians who participate in the project. Then, apart from the difficulties related to the identification of mTBI as such, another limitation of our study is related to the use of patient medical charts as a gold standard to validate our algorithms. The information written in the medical charts is not systematically complete and accurate and some records could be complicated to classify. In order to anticipate this difficulty, we determined precise criteria for the classification of cases with the mTBI expert physicians before the study. Moreover, the relevance of the criteria we had determined was checked thanks to a pilot study we conducted. Following this pilot study, some criteria were refined or adapted. Finally, the generalisability of our study could be questioned. For feasibility reasons, our study involved a limited number of centres (4 out of 700 EDs in France). Nevertheless, among the four centres selected for this study, there were two university hospital emergency departments and two hospital emergency departments. Thus, the two main types of French emergency departments are represented in our study. Moreover, among the four selected centres, three were randomly selected in order to avoid potential biases inherent to volunteering (over-representation of ‘good coders’ centres).

Ethics and dissemination

The ethics committee of the French National Data Protection Authority (CNIL) approved this study (n° 921152, 1 August 2021). Results will be submitted to national and international peer-reviewed journals and presented at conferences dedicated to trauma and to methodologies for the construction and validation of algorithms.

Ethics statements

Patient consent for publication

References

Footnotes

  • Contributors L-MP, AG, NB, FL, NA, MR and CF designed the study. L-MP performed the review of literature. L-MP drafted the first version of the manuscript. L-MP, AG, NB, FL, NA, MR and CF critically revised the manuscript. All authors approved the final version.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests FL has a scientific expert contract without grants with the Laboratoire GlaxoSmithKline. The authors report no other conflict of interest in this work.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.