Inclusion criteria

Participants presenting to one of the study hospital EDs after sustaining a head injury will be eligible if they

1. Are aged 5–17 years;

2. Have a concussion, defined by Zurich consensus statement3:

   A direct blow to the head, face, neck or elsewhere on the body with an impulsive force transmitted to the head, resulting in one or more of the symptoms in one or more of the following clinical domains (which may or may not have involved loss of consciousness):

   • Somatic symptoms (eg, headache, nausea, loss of balance, dizziness, sensation to light or noise, visual problems and clumsiness)

   • Cognitive symptoms (eg, feeling like in a fog, difficulty concentrating or remembering, answering questions more slowly and confused with directions/task)

   • Emotional/behavioural symptoms (eg, irritable, sad, nervous and emotional lability), physical signs (eg, loss of consciousness and amnesia)

   • Sleep disturbance (eg, sleeping more, fatigue, drowsiness and insomnia);

3. Suffered the initial injury in the previous 48 h;

4. Are proficient in English or French.

Exclusion criteria

Patients will be excluded if they present with traumatic head injuries with any of the following: (1) Glasgow Coma Scale ≤13; (2) any abnormality on standard neuroimaging studies,3 including any positive head CT findings (note: neuroimaging is not required, but may be performed if believed to be clinically indicated); (3) neurosurgical operative intervention, intubation or intensive care required; (4) multisystem injuries with treatment requiring admission to hospital, operating room or procedural sedation in ED (note: admission to hospital for observation or management of ongoing concussion symptoms is not an exclusion criteria); (5) severe chronic neurological developmental delay resulting in communication difficulties; (6) intoxication at the time of ED presentation as per clinician judgment; (7) no clear history of trauma as primary event (eg, seizure, syncope or migraine as primary event); or (8) previously enrolled in this same study.

Patient selection

A consecutive sample of patients with possible concussion arriving between the hours of 12:00 and 22:00 will be screened for the study. Census data from the sites reveal that less than one-fifth of paediatric concussion patients present outside of these study hours and, therefore, do not justify the additional expense of research personnel coverage. For comparative purposes, demographic and outcome data will be collected from the ED record of treatment for eligible patients who are not enrolled into the study.

Study setting

The patients will be enrolled from the EDs of nine Canadian paediatric hospitals: IWK Health Sciences Centre (Nova Scotia), CHU Sainte-Justine (Quebec), Montreal Children's Hospital (Quebec), Children's Hospital of Eastern Ontario (Ontario), The Hospital for Sick Children (Ontario), Children's Hospital of Western Ontario (Ontario), Children's Hospital of Winnipeg (Manitoba), Stollery Children's Hospital (Alberta) and Alberta Children's Hospital (Alberta). These centres constitute 9 of the 12 paediatric hospitals in Canada and have a combined annual ED census of approximately 500 000 patient-visits. All sites are active members of Pediatric Emergency Research Canada (PERC) and have successfully collaborated in many multicentre prospective studies. Formed in 1995, PERC is an established network of healthcare researchers dedicated in improving care in paediatric emergency medicine through multicentre research. The PERC network received a 2011 CIHR-CMAJ Top Achievements in Health Research Award32 and has significant experience in decision rule derivation and validation studies including the validation of decision rules for the use of radiography for children with ankle and knee injuries, and recently in the derivation and validation of a clinical decision rule regarding the need for a CT scan in children with minor head injuries.33

Patient recruitment

When a child aged between 5 and 17 years presents to triage/registration with a head injury, the patient's chart and/or electronic patient tracking system will be flagged as a potentially study-eligible patient. Research assistants (RAs) located in the department will work collaboratively with the ED clinical team to also identify potential study participants. Prior to assessment by the physician, RAs will screen these patients for eligibility using brief screening questionnaire. If eligible, RAs will discuss the study with the patient and families. Eligible and willing parents, along with children and adolescents, capable of consenting on their own behalf will be asked for written informed consent, and those children aged 7 years or older who are unable to consent on their own behalf will be asked for assent. RAs will maintain a screening log of all potentially eligible patients during the study enrolment hours complete with reasons for exclusion. A second log will also be maintained of those potentially eligible patients who present outside of the study enrolment hours to confirm that patient characteristics are similar. These recruitment and tracking strategies have been pilot tested in our previous work and were found to be highly successful.

Standardised patient assessment of concussion using a validated tool

With the guidance of an RA, parent/guardian/child will answer a series of questions in electronic survey format in their first language (English or French) using a portable computer tablet. In addition to questions regarding patient demographics, both parents and children will also complete the validated Acute Concussion Evaluation (ACE),34 which is embedded into the Child Sports Concussion Assessment Tool 3.3 The ACE is a 22-item dichotomous evidence-based inventory for the initial evaluation and diagnosis of concussion and has been validated as an initial concussion assessment tool for parental informants of children aged 3–18 years. Data are collected on injury characteristics, symptoms checklist (physical, emotional, cognitive and sleep) and history (prior concussion, headache, developmental or psychiatric). This is to ensure that the child does have a concussion, rather than rely solely on physician judgement. By using a validated standardised assessment, we will ensure uniform application of the definition of concussion.

Physician management and prognostication

The treating physician will assess the child as per the normal operating procedures of ED. Once a patient has given consent to participate in the study, the RA will liaise with the treating physician to notify them that the patient is enrolled in the concussion study. The child will remain in the normal treatment queue, and participation in this study is not anticipated to lengthen their ED visit. The physician will complete a brief electronic survey on the same tablet computer. This survey will collect data about medical history, comorbidities, therapies received during the ED visit, discharge instructions and prognostication regarding symptom duration.

Quality assurance

Since data will be collected electronically via portable tablet, embedded logic safeguards will ensure that variables are entered within predetermined ranges. Warning messages will be programmed to alert for incomplete data fields. There will also be an ongoing evaluation of the quality of the patient assessments judged by compliance in enrolling eligible patients. During our pilot study, 80% of eligible patients were enrolled and, to ensure similar numbers recruited, research coordinators at each site will provide clinicians with monthly feedback regarding recruitment data as well as specific review of any individual problems that may arise. Clinicians will not, however, be given any indication of the preliminary accuracy or reliability of individual variables.

Variables from history and physical examination

The 46 potential variables selected for assessment in the study were chosen based on expert discussions during the planning meeting, our recent systematic review, previous clinical studies by our team and clinical experience. Variables selected are listed in box 1.

These variables were felt to be most useful in predicting whether or not patients with concussion may be at risk for developing PCS. Further, to ensure that our results will be effectively compared across studies and aggregated into future meta-analyses, we will incorporate the standardised dataset recommendations from the internationally recognised National Institute of Neurological Disorders and Stroke (NINDS) Common Data Element Project.35 This National Institute for Health (NIH)-funded project established standards for clinical research within the neurological community on traumatic brain injury. This dataset utilises expert consensus approved data dictionary instructions, including the preferred format for recording.

Inter-rater reliability

All ED physicians and research staff at each participating site will be trained on data collection methods using a standardised lecture/video and training session. A study manual including a data dictionary will be available at each site. To measure inter-rater reliability, 5% of patients will be assessed for clinical variables by a second physician who will be blinded to the results of the first assessment. These second assessments will be performed in all centres on a feasibility basis whenever two physicians are available in ED.

Follow-up

Families will be asked to provide contact information prior to discharge. Depending on their preferences, enrolled patients will either be entered into the automated follow-up web survey using REDCap36 ,37 or the telephone follow-up survey. For those who would prefer electronic follow-up, a link to a secure web-based questionnaire will be sent to the parental email address on day 7 following the initial injury and then 2, 4, 8 and 12 weeks postenrolment. The same schedule and questionnaires will be used for the families that opt for telephone follow-up. In the event that patients do not complete the electronic survey within 24 h of receipt, a second email will be sent. If there is still no response to the electronic survey, the family will be contacted by telephone for a phone interview. The follow-up questionnaires employ the validated Post-Concussion Symptom Inventory (PCSI). In addition, the follow-up survey will query symptom management, follow-up with health providers, school absenteeism, changes in academic performance, date of return to sport and school and patient/family quality of life. Our pilot established success with web-based follow-up with 91% (89/98) of patients having requested web follow-up; only two families changed from web to telephone follow-up. Patient-level assessment of longer term outcomes (beyond 3 months) is felt to be beyond the scope of this study.

Neuropsychological follow-up (optional)

All children from four centres (CHU Sainte-Justine, Montreal Children's Hospital, Children's Hospital of Eastern Ontario, the Hospital for Sick Children) will be offered enrolment in an optional neuropsychological arm of the study at the time of their initial consent by the onsite trained RA. This will involve two return visits to the centre, at 4 and 12 weeks postconcussion. Each visit will be approximately 2.5 h in length, during which we will measure the mental and social capacities of children and adolescents who have suffered a concussion. These data will be gathered using questionnaires and tests in their first language (English or French) designed to measure mental skills and behaviour in two ways: (1) indicators of behaviour and functional abilities and (2) measurements of mental and social skills. These measures will be administered by an RA who is fully trained in psychometry. The results of these assessments will be reviewed and interpreted by a licensed neuropsychologist, who will follow-up with any significant clinical concerns and refer appropriately. In cases of significant clinical concerns arising at the 4-week assessment, the patient and parents will be notified. A summary written report will be issued after the completion of the 12-week assessment, again, with appropriate follow-up indicated if necessary.

Primary outcome measure

The primary outcome is the proportion of children aged 5–17 years who have PCS at 1-month follow-up. A PCS case is defined as an increase from preconcussion baseline of three or more symptoms on the validated PCSI at 1 month (consistent with the International Classification of Diseases, 10th Revision (ICD-10) definition of PCS).

Post-Concussion Symptom Inventory

PCSI is a set of symptom scales for parents (26-item, 7-point Likert scale) and developmentally specific self-report forms for children aged 5–7 years (13 items, 3-point Likert scale), aged 8–12 years (25-item, 3-point Likert scale) and 13-year-olds to 18-year-olds (26-item, 7-point Likert scale). PCSI is the only one of the two measures applicable to younger children who have both validity and reliability data published in the literature.34 ,38 ,39 The PCSI queries symptoms reflecting physical, cognitive, emotional and sleep domains. Even with younger children, there is excellent internal consistency with PCSI (aged 5–7 years, r=0.76; aged 8–12 years, r=0.87).38 Inter-rater agreement has also been examined comparing symptoms ratings from the child and parent. At enrolment, patients and parents will complete an inventory of symptoms that were present prior to the injury (72 h prior to enrolment) to establish a patient baseline. Since each case of PCS is defined as a change from an individual's baseline in three or more symptoms, total summation of subunits is not required for PCSI scoring. Therefore, it is not problematic that there are different item values or totals across age groups. Symptom severity (mild –moderate –severe) at the time of ED presentation will also be analysed as a potential predictor of symptom duration.

Paediatric quality-of-life inventory (PedsQL)

Standard paediatric practice requires the involvement of parents. The role of the family is crucial to the recovery process, and delayed recovery from concussion has been shown to impact the quality of life of the patient and the family.9 ,16 ,17 The PedsQL is a reliable and valid measure of health-related quality of life in healthy children and adolescents and those with acute and/or chronic health conditions.35 Parent versions exist for children aged 2–18 years (in 4 age groups) and child versions for those aged 5 years and above. The inventory covers four domains: physical, emotional, social and school (8, 5, 5 and 5 items, respectively) and takes approximately 4 min to complete. This secondary outcome measure will be used to determine the impact of PCS on the quality of life of patients and families.

Neuropsychological evaluation

A separate battery of neuropsychological assessment measures will be administered to those children who choose to participate in this arm of the study (see box 1). This battery includes measures of intelligence, language, visual-spatial/motor functions, attention, memory/working memory, executive functioning, academic achievement as well as behavioural/socioemotional functioning.

Wechsler Abbreviated Scale of Intelligence

The Wechsler Abbreviated Scale of Intelligence is a reliable and valid measure of intellectual functioning in children and adults.40 The Wechsler Abbreviated Scale of Intelligence consists of four subtests, which provide estimates of the Full Scale IQ, the Verbal IQ (based on Vocabulary and Similarities subtests) and Performance IQ (based on Block Design and Matrix Reasoning subtests). It takes approximately 30 min to administer this measure.

Delis-Kaplan Executive Functioning System

The Delis-Kaplan Executive Functioning System is a reliable and valid measure of executive functioning in children and adults. It consists of many subtests, of which we will use three.41 The Trails subtest measures the ability to complete several visual spatial sequences, including numbers and letters in a mixed array as quickly as possible. The Verbal Fluency subtest asks the participant to generate as many words as possible in 1 min that either begin with a specific letter of the alphabet, fit in a specific category, or that switches between two categories. The Color-Word Interference Test measures the participant's ability to inhibit a dominant and automatic verbal response when reading the names of colours that are printed in a variety of ink colours. It takes approximately 20 min to administer these subtests.

Digit Span and Coding subtest of the Wechsler Intelligence Scale for children (fourth edition)/Wechsler Adult Intelligence Scale (fourth edition)

The Digit Span and Coding subtests are reliable and valid measures of short-term memory/working memory and speed of information processing in children and adults.42–,44 The Digit Span subtest measures the participant's ability to repeat numbers read aloud by the examiner in a forward and backward order (for participants aged 16 years or older, the ability to sequence a series of numbers will also be assessed). The Coding subtest requires the participant to reproduce symbols in a sequence as quickly as possible. It takes approximately 10 min to administer these subtests.

Rey-Osterrieth Complex Figure Test

The Rey-Osterrieth Complex Figure Test is a reliable and valid measure of visual spatial and executive functioning in children and adults.45 The participant is asked to reproduce the complex figure with it initially in front of her/him. The participant is then asked to reproduce the figure from memory both immediately after and 30 min after the initial reproduction. It takes approximately 10 min to administer this test.

Dynamometer (Grip Strength Test)

The dynamometer test is a reliable and valid measure of the grip strength in children and adults.46 The participant is asked to squeeze the dynamometer with the dominant hand, followed by the non-dominant hand. It takes approximately 5 min to administer this test.

Grooved Pegboard Test

The Grooved Pegboard Test is a reliable and valid measure of speeded eye–hand dexterity in children and adults.47 The participant is asked to place key-shaped pegs in a pegboard as quickly as possible with the dominant hand, followed by the non-dominant hand. It takes approximately 10 min to administer this test.

Conner's Continuous Performance Test—second edition, version 5

The Conner's Continuous Performance Test is a task-oriented computerised assessment of attentional functioning in children and adults.48 The participant is asked to respond to some characters on the computer screen and not to respond to other characters. It takes approximately 15 min to administer this test.

California Verbal Learning Test for Children/California Verbal Learning Test—second edition

The California Verbal Learning Test is a reliable and valid measure of verbal learning and memory in children and adults.49 ,50 A lengthy list of words is read aloud to the participant for a total of five times, with a recollection after each presentation. Long-term recollection is also assessed. It takes approximately 20 min to administer this test.

Wechsler Individual Achievement Test—second edition

The Wechsler Individual Achievement Test is a reliable and valid measure of academic achievement in children and adults, normed in both official languages of Canada.44 The participant is asked to read a series of words, to respond to questions assessing reading comprehension and to complete a series of mechanical arithmetic. It takes approximately 30 min to administer this test.

Child Behavior Checklist

The Child Behavior Checklist is a reliable and valid parent questionnaire assessing their child's functioning in emotional, social, cognitive and behavioural domains.51 It takes approximately 20 min to complete this test.

Conner’s Rating Scale—third edition

The Conner's Rating Scale is a reliable and valid parent questionnaire assessing their child's functioning in emotional, social, cognitive and behavioural domains.52 The emphasis of this measure is attentional in nature. It takes approximately 15 min to complete this test.

Behavioral Rating Inventory of Executive Function

The Behavioral Rating Inventory of Executive Function is a reliable and valid parent questionnaire assessing their child's executive functioning.53 It takes approximately 15 min to complete this test.

Strength and Difficulties Questionnaire

The Strength and Difficulties Questionnaire is a reliable and valid parent questionnaire assessing their child's emotional symptoms, conduct problems, hyperactivity/inattention, peer relationships and prosocial behaviour.54 It takes approximately 10 min to complete this test.

Inter-rater agreement

The inter-rater agreement for each variable will be assessed using a κ coefficient, the proportion of potential agreement beyond chance, along with 95% CIs.55 ,56 For variables with three or more ordered categories, a weighted κ measure of inter-rater agreement will be calculated.57 A variable will be deemed to have an acceptable agreement if the κ coefficient has a value of at least 0.6.55 ,56

Univariate analysis

Univariate analyses will be used to determine the strength of association between each variable and the primary outcome, PCS. This process will aid in the selection of the best variables for the multivariate analyses. The appropriate univariate technique will be chosen according to the type of data: for nominal data, a χ² test with continuity correction; for ordinal variables, a Mann-Whitney U test; and, for continuous variables, an unpaired two-tailed t test, using pooled or separate variance estimates, as appropriate.

Multivariate analysis

Multivariate analysis will be used to derive a predictive model for PCS. Variables found to be both reliable (κ>0.6) and independently associated with the outcome (p<0.2) will be used as input to a multivariate logistic regression. Model fit will be assessed using the Hosmer-Lemeshow test and Nagelkerke's pseudo -R².

An alternate strategy for deriving a predictive model is recursive partitioning. This approach has been recommended for deriving highly sensitive predictive models, while logistic regression may produce better predictive accuracy.58 However, high sensitivity is not the primary goal in the proposed study. Further, a fitted logistic regression model has the advantage that it can be used to derive a risk score using a points system.59 For these reasons, logistic regression was chosen as the primary analytic approach.

The objective of the logistic regression analysis will be to find the best combinations of variables for accurately predicting PCS. Receiver operating characteristic (ROC) curves will be used to assess the trade-off between sensitivity and specificity based on applying different probability thresholds to the fitted model. The derived models must be easy to use by clinicians and therefore should contain as few variables as possible. As described in a recent systematic review of paediatric clinical prediction rules, ‘Seeking 95% sensitivity for clinical prediction rules for child health conditions may be an elusive and counterproductive goal, especially when the sensitivity of a less-than-perfect clinical prediction rule is superior to a clinician's judgement alone (Ref. 60, p.e671)'. Assuming more than one model meets the minimum acceptable criteria, the best model will be the one which has the highest positive predictive value/sensitivity and the fewest number of component variables.

Classification performance

The derived prediction rule will be cross validated by comparing the classification of each patient to their actual status for the primary outcome. This will allow an estimate, with 95% CIs computed using the Wilson score method, of the sensitivity and specificity of the rule. A more robust validation will be carried out prospectively on a new set of patients in phase II.

Physician judgement

Physicians will be asked to predict likelihood of PCS using a Likert scale. Data from those predictions will be tabulated in descriptive format. Information on the predicted probabilities will be used to calculate ROC curves and likelihood ratios for determining the duration of symptom persistence. The accuracy of the physicians’ predictions will be compared to that of the derived prediction rule by the ROC curve analysis.

Sample size

A well-supported recommendation for the development of a predictive model is that 10 events are required for each candidate variable.61 ,62 Further, a recent systematic review examining the methodology of clinical prediction rule studies endorses this recommendation.63 On the basis of preliminary work, we identified 46 potential predictor variables (see box 1). Our pilot demonstrated that 25% of patients had PCS (as per ICD-10 definition of 3 or more symptoms at 1 month). After screening for inter-rater agreement (we assumed a 25% dropout of variables based on our CT Head Rule derivation33), we would require 345 PCS cases (10 cases per predictor variable after κ screening). In order to obtain 345 PCS cases, we would need to enrol 1380 new concussions. Assuming a comparable 23% loss to follow-up at 1 month, we would require a sample size of 1792 patients.

Alternate definition of PCS case considered

The timing discrepancy between Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) and ICD-10 definitions of PCS is conceded to be a shortcoming in the literature.12 While DSM-IV criteria define a case as persistent symptoms at 3-month duration, experts at our 2-day planning meeting determined that 1 month of persistent symptomatology was the most clinically relevant outcome (ICS-10 criteria). Both definitions require the presence of multiple symptoms as compared to baseline. Preliminary data from our pilot study confirmed that most children with PCS have changes from baseline for multiple symptoms rather than for just one or two symptoms.

Alternate analysis considered

Recursive partitioning has been widely used in the development of clinical prediction rules and recursive partitioning may be more suitable than logistic regression when the objective is to correctly classify one outcome group at the expense of the other.33 Recursive partitioning creates a binary decision tree by automatically splitting on the values of input variables so as to achieve optimal classification of an outcome variable.64 However, predictive models are likely better developed as risk scales with points based on logistic regression analyses. The use of risk scales gives the probability of outcomes; clinicians are then able to choose their own cut points for action.