Article Text

Protocol for a living systematic review for the management of concussion in adults
  1. Alexander Lithopoulos1,2,
  2. Mark Bayley3,4,
  3. Dorothyann Curran1,2,
  4. Lisa Fischer5,
  5. Christopher Knee1,2,
  6. Julia Lauzon1,2,
  7. Margaret Nevison1,2,
  8. Diana Velikonja6,
  9. Shawn Marshall2,7
  1. 1Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  2. 2Ottawa Hospital Rehabilitation Centre, Ottawa Hospital, Ottawa, Ontario, Canada
  3. 3Hull-Ellis Concussion Research Center, Toronto Rehabilitation Institute, Toronto, Ontario, Canada
  4. 4Physical Medicine and Rehabilitation, University of Toronto, Toronto, Ontario, Canada
  5. 5Department of Family Practice, The University of British Columbia, Vancouver, British Columbia, Canada
  6. 6Psychiatry & Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
  7. 7Physical Medicine and Rehabilitation, Bruyere Continuing Care, Ottawa, Ontario, Canada
  1. Correspondence to Dr Alexander Lithopoulos; allithopoulos{at}


Introduction Concussion/mild traumatic brain injury (mTBI) often presents initially with disabling symptoms that resolve, but for an unfortunate minority some of these symptoms may become prolonged. Although research into diagnosis and interventions for concussion is increasing, study quality overall remains low. A living systematic review that is updated as evidence becomes available is the ideal research activity to inform a living guideline targeting clinicians and patients. The purpose of this paper is to present the protocol of an ongoing living systematic review for the management of adult concussion that will inform living guidelines building off the Guideline for Concussion/Mild Traumatic Brain Injury and Persistent Symptoms: third Edition.

Methods and analysis The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol guidelines were followed in the reporting of this systematic review protocol. We are including English peer-reviewed observational studies, trials, qualitative studies, systematic reviews and clinical practice guidelines related to diagnosis/assessment or treatment of adult concussion. Future searches will be conducted at minimum every 6 months using the following databases: MEDLINE ALL, EMBASE, Cochrane, PsycInfo and CINAHL. The data are managed in the Covidence website. Screening, data extraction and risk-of-bias assessments are being done through multiple raters working independently. Multiple validated tools are being used to assess risk of bias, and the tool applied matches the document or study design (eg, Downs and Black Scale for healthcare interventions). Many concussion experts in various clinical disciplines from across North America have volunteered to examine the evidence in order to make recommendations for the living guidelines.

Ethics and dissemination No ethical approval is necessary because primary data are not collected. The results will be disseminated through peer-reviewed publications and on the living guidelines website once built.

PROSPERO registration number CRD42022301786.

  • Protocols & guidelines
  • Quality in health care
  • Adult neurology
  • Head & neck imaging

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Frequent searches will ensure the accompanying adult concussion living guidelines are up to date.

  • There is a large multidisciplinary concussion expert team who have volunteered to interpret the evidence.

  • The review focuses only on adults while excluding the paediatric population, which is a limitation.

  • The review is limited to documents published in the English language.

  • Perspectives of the expert team are geographically limited to North America and may not reflect the full global perspective.


Concussion/mild traumatic brain injury (mTBI) describes an acute neurophysiological event related to a mechanical energy applied to the head, neck or body (with transmitting forces to the brain), such as from sudden acceleration, deceleration, rotational forces or repetitive subconcussive hits.1 All concussions are considered to be an mTBI; however, mTBI can differ from concussion when there is evidence of brain injury on conventional neuroimaging or there is persistent neurologic deficit.1 Concussion can cause meaningful morbidity, with many persons who have sustained a concussion suffering from prolonged symptoms for years post injury.2–4 Concussion is also among the most common neurological conditions with an estimated annual incidence of 503 per 100 000 in the USA based on emergency department data,5 and even higher estimates of up to 1153 per 100 000 if community-based concussions are taken into account.6 Therefore, effective diagnosis/assessment and treatment is critical.

Systematic reviews provide the best evidence available, and there are many that focus on concussion management.7–10 However, systematic review currency and accuracy is challenged by the increasing rate of research output.11 12 People might consider conducting a traditional systematic review update, but these updates tend to be inefficient because a new team often needs to be assembled for each update meaning the ‘institutional memory’ of the original team is lost.13 Living systematic reviews may be an effective solution. A living systematic review is defined as: ‘a systematic review that is continually updated, incorporating relevant new evidence as it becomes available’13 (p. 24). In addition to pushing the limits of currency and accuracy, living systematic reviews provide an a priori commitment to a frequency of review giving predictability to end users such as clinicians.13 Applying a living systematic review process to concussion diagnosis/assessment and treatment is appropriate given that research output in this particular field is increasing every year14 and certainty in much of the existing evidence is low,1 making frequent updates necessary.

A living systematic review is the ideal research activity to inform living guidelines. Guidelines are normally developed to support clinicians and their patients in making choices to optimise outcomes.15 Living guidelines are: ‘an optimisation of the guideline development process to allow updating of individual recommendations as soon as relevant new evidence becomes available’16 (p. 47). There have been a few guidelines published on adult concussion.1 17 18 However, no group or organisation has developed living guidelines to address all aspects of diagnosis/assessment and treatment of concussion in adults. The purpose of this paper is to present the protocol of an ongoing living systematic review for the management of adult concussion (eg, diagnosis, initial management, post-traumatic headache, return to activity) that will inform living guidelines building off the Guideline for Concussion/Mild Traumatic Brain Injury and Persistent Symptoms: third Edition.1

Methods and analysis

The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol guidelines19 (see online supplemental material 1) for the completed checklist) were followed in the reporting of this systematic review protocol.

Eligibility criteria

The inclusion criteria are: studies related to concussion diagnosis/assessment or treatment; at least 50% of sample has concussion (eg, a Glasgow Coma Scale score of 13–15,20 confirmation through a standardised concussion assessment tool, diagnosis by a physician or nurse practitioner) in cases where one group is analysed; at least 50% of the sample is 18 years of age or older; and the sample is human. Peer-reviewed observational studies (cross-sectional, cohort, case–control), clinical trials, qualitative studies, systematic reviews and clinical practice guidelines are included. Documents are limited to English language and publication from May 2017 because that covers the literature that did not inform the Guideline for Concussion/Mild Traumatic Brain Injury and Persistent Symptoms: third Edition.1

The exclusion criteria are: studies that focus on moderate or severe TBI (ie, a Glasgow Coma Scale Score of less than 13); more than 50% of sample has moderate or severe TBI in cases where one group is analysed; more than 50% of sample is under 18 years of age; and the sample is not human. Case reports/n of 1 studies, non-systematic reviews, conference abstracts/presentations, theses, non-peer-reviewed articles (eg, newspaper articles), letters or commentaries, addendums/erratums and book chapters are excluded. Documents not available in English, published before May 2017, or originating from the grey literature are also not included.

Information sources

The search strategy (see next section) was originally created in April 2020 for the MEDLINE ALL database in collaboration with a librarian at a research-intensive hospital. The strategy was then peer-reviewed by another librarian at a separate hospital according to the Peer Review of Electronic Search Strategies (PRESS) guideline.21 The PRESS guideline is a checklist of topics that information specialists should consider when evaluating an electronic search strategy. The strategy was approved with minor revisions. EMBASE, Cochrane, PsycInfo and CINAHL databases are also being searched using the strategy.

Search strategy

An initial search was completed at the beginning of April 2020 covering May 2017 to the end of March 2020. That search yielded 19 745 results. The search was updated to cover recent literature published April 2020 to the end of March 2021. The new search yielded 5071 results, meaning the total number of search results was 24 816. The full search strategy for the initial MEDLINE ALL search has been reported in online supplemental material 2 as an example. The next search will cover the literature published from April 2021 to the end of February 2022 (search to be conducted on 1 March 2022). After this next search, consistent with living systematic review recommendations,13 the search is planned to repeat every 6 months at minimum to capture the recent literature.

Data management

The search results are being imported into the Covidence systematic review website.22 This website automatically removes duplicates, and provides the opportunity for screening, data extraction and risk-of-bias assessments with multiple raters.

Selection process

After duplicates from the initial searches were removed by Covidence, 16 086 documents remained (11 916 from the initial search and 4170 from the updated search). At the title and abstract screening phase, raters select ‘yes’,‘no’, or ‘maybe’. A rating of ‘maybe’ is selected when there is not enough information to choose ‘yes’. However, a ‘maybe’ rating does allow the document to move to the full-text screening phase.

For this first phase of screening, a test set of 100 references was exported from Covidence into an Excel file. All raters independently provided a vote for each document as a calibration training exercise. All votes were compiled on the spreadsheet and discussions were held to determine a consensus vote (as required) for each study. The actual screening was then started in Covidence in dual-screen mode (ie, two votes were needed per document), until approximately 1200 documents were completed. There was less than a 10% conflict rate, and any conflicts were resolved through discussion with the final decision being made by a senior researcher. Since the team has demonstrated satisfactory inter-rater reliability, only one vote is now needed from raters to decide whether documents should move to the full-text screening phase.

Regarding full-text screening, a test set of documents (n=50) was first exported as a training exercise for the raters (conflict rate was less than 20%), followed by group discussion. Each document requires two independent votes of ‘yes’ to be included in data extraction. In the case of conflicts, the project leader (a physician with many years of clinical experience in the concussion field) or a third rater not involved in the conflict makes the final decision. Finally, each document has been given labels to reflect important themes in the research. Most documents have received at least one label that match the 12 sections appearing in the Guideline for Concussion/Mild Traumatic Brain Injury and Persistent Symptoms: third Edition.1 For example, a common label has been ‘diagnosis/assessment of concussion/mTBI’, which is the first section title of the current guideline. Other labels such as ‘biomarkers’ reflect other themes that may be added to the original 12 sections if the experts deem it appropriate.

Data collection process

A standardised data extraction form was created by the investigators in Covidence to ensure relevant data are collected (see next section for Data items). The raters completed extraction together for several articles per main study design or document type (eg, intervention, observational, systematic review, qualitative research, clinical practice guideline) in order to enhance inter-rater reliability. Two raters extract data from each included document independently. A third rater completes ‘consensus’ for each article. In Covidence, the consensus rater has the ability to view the original two extractions simultaneously and can then select the best response or can write their own based on the information provided by the raters.

Data items

The data extraction form has the following sections: document ID (assigned by Covidence); authors; year of publication; title of paper; country in which study was conducted; aim(s) of study related to assessment or treatment; study design; specific design information (eg, group information, intervention treatment, measurement time points, etc); relevant outcome measure information; study definition of concussion; number of participants for each group; gender frequencies and percentages for each group; average age and SD for each group; and findings related to assessment or treatment.

Outcomes and prioritisation

Due to the breadth of the present review, no specific outcomes are sought. Any outcome that contributes to understanding of diagnosis/assessment and treatment of adult concussion is considered relevant.

Risk of bias in individual studies

Risk-of-bias assessment is currently being conducted at the study or document level (not outcome level). Inter-rater reliability optimisation and the rating and consensus procedures are the same as those in data extraction. However, the consensus process only allows the third rater to select a final response due to the nature of the form.

A variety of validated tools have been included in the review. Each tool pertains to the study design or document type. The following tools were included with very minor modifications: Downs and Black Scale for healthcare interventions;23 an amalgamation of items adopted from the Joanna Briggs Institute (JBI) critical appraisal tools for observational studies;24 Critical Appraisal Skills Programme (CASP) for qualitative studies;25 A MeaSurement Tool to Assess Systematic Reviews (AMSTAR 2) for systematic reviews;26 and Appraisal of Guidelines for REsearch and Evaluation (AGREE II) for clinical practice guidelines.27 The scoring is as follows: Downs and Black (/28); JBI (/16); CASP (/9); AMSTAR 2 (/20); AGREE II (23 items each scored on a scale ranging from 1 (strongly disagree) to 7 (strongly agree)). The tools are provided in online supplemental material 3.

Data synthesis

In order for the findings to be translated to recommendations in the living guidelines, over 35 concussion and TBI experts from across North America have thus far volunteered to interpret the evidence. Each expert must have peer-reviewed publications about adult concussion and/or be recommended by a current expert panel member. All experts also must be approved by the project leader. Currently, 12 groups covering the sections appearing in the Guideline for Concussion/Mild Traumatic Brain Injury and Persistent Symptoms: third Edition1 have been created. A minimum of five experts have been assigned to domain areas that match their expertise. This number was deemed by consensus to be necessary to reduce bias in decision-making and to encourage discussion. Each expert is also required to declare any conflicts of interest.

The experts deal only with documents related to their domain area. The summarised information from the data extraction form (including risk-of-bias assessments and full-text copies of each document) is provided to the expert panels. The experts also receive documents/assessments of documents informing the third edition recommendations for that domain area, related guidelines since 2010 (with AGREE II ratings), and a list of relevant evidence for each individual recommendation within a domain area. Ratings for the overall quality of evidence and the strength of recommendation pertaining to only relevant evidence for each recommendation is based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach28 29 (see Confidence in cumulative evidence section below for more details). Finally, there are voting options to keep, modify or delete recommendations based on the new relevant evidence. Space to write a revised recommendation and to propose new recommendations is also provided. Several weeks later, the expert panel meets virtually 1–2 times with a group moderator through a video call to make decisions. Afterwards, results of the meeting(s) are circulated to the entire expert team for feedback. Based on this feedback, the project team makes the necessary revisions. Finally, a round of voting (and feedback) will occur with the entire expert team. See online supplemental material 4 for the full guideline domain update algorithm.


There are no planned assessments of meta-bias(es) (eg, publication bias, selective reporting within studies) due to the nature of this review.

Confidence in cumulative evidence

The GRADE approach is being used to rate the overall quality of relevant evidence informing each recommendation. This approach initially labels randomised controlled trial evidence as high-quality and observational study evidence as low quality. Ratings are lowered if there is risk of bias, inconsistency in results, indirectness (ie, studies not examining interventions, patients and outcomes of interest), imprecision (eg, large CIs) and publication bias. Ratings can be elevated if there are large effect sizes, evidence of a dose–response gradient, and if all possible confounding would reduce a demonstrated effect or would suggest a spurious effect if no effect was observed. The quality of evidence is rated as very low, low, moderate or high.28 We have made an amendment in cases where there is a mix of randomised controlled trials and observational studies informing a recommendation. In these cases, if at least 50% of the studies are randomised controlled trials, the grading will begin at high quality. Also, we have made the decision that in cases where a recommendation is based on expert opinion only, the quality of evidence will be ‘very low’ because it is based on anecdotal clinician observation.28 Each recommendation is also rated as being strong or weak based on a cost/benefit analysis. There are four specific factors that determine the strength of recommendation: magnitude of the difference between desirable and undesirable effects, quality of evidence, the values and preferences of patients, and the resources that need to be expended.29

Patient and public involvement

Patients or the public are not involved in the design or conduct of the research. However, these individuals will be involved in the drafting of guideline recommendations designed for clinicians, the production of the patient version of guideline recommendations and the review of resources that accompany the guidelines.


Concussion can lead to health issues acutely and in some cases may result in prolonged symptoms.2–4 There is continued need for up to date guidelines to assist clinicians in managing persons with concussion and prolonged symptoms where the complex presentation of symptoms can often be challenging for the primary care provider to manage. In addition to this, research output in this field is also increasing every year14 where study quality is typically low,1 making a living systematic review of diagnosis/assessment and treatment of adult concussion necessary.

Although this review has many strengths, it is not without limitations. First, only papers published in the English language are being included so other potentially valuable documents could be missed. Our multidisciplinary expert team, although large, is also geographically limited to North America. Therefore, recommendations included in the living guidelines may not reflect the full global perspective. Also, only a minimum of five content experts are involved in the initial examination of the literature and recommendations for each specific domain. Although it would be ideal to include the entire expert team at this phase, it is not feasible because of the workload. Finally, regarding demographics, the review focuses only on adults and on concussion while excluding the paediatric population. The guidelines could potentially be more comprehensive if paediatrics were included but it is not feasible given our infrastructure which is primarily adult expert focused and there are now available parallel paediatric concussion living guidelines30 using similar rigorous approaches which have formal ties to these guidelines.

This continuous review process will greatly benefit clinicians and patients by informing living guidelines that will lead to timely guideline recommendations that over time will have increasing certainty as the evidence improves.

Ethics and dissemination

No ethical approval is necessary because primary data are not collected. The review results will be published in peer-reviewed journals in addition to being on the guidelines website in order to enhance dissemination and implementation. Any important amendments to this protocol will be documented on the guidelines website.

Ethics statements

Patient consent for publication


Supplementary materials


  • Twitter @DocMarkBayley

  • Contributors AL (guarantor) and SM were involved in conceptual development, writing and editing. MB, DC, LF, CK, JL, MN and DV were involved in conceptual development and editing.

  • Funding This work was supported by the Ontario Neurotrauma Foundation (grant number: 1083) and the Ontario Ministry of Health and Long-Term Care (grant number: 721).

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods and analysis section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.