Article Text
PDF
Abstract
Objectives Rehabilitation clinical practice guidelines (CPGs) contain recommendation statements aimed at optimising care for adults with stroke and other brain injury. The aim of this study was to determine the quality, scope and consistency of CPG recommendations for rehabilitation covering the acquired brain injury populations.
Design Systematic review.
Interventions Included CPGs contained recommendations for inpatient rehabilitation or community rehabilitation for adults with an acquired brain injury diagnosis (stroke, traumatic or other non-progressive acquired brain impairments). Electronic databases (n=2), guideline organisations (n=4) and websites of professional societies (n=17) were searched up to November 2017. Two independent reviewers used the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, and textual syntheses were used to appraise and compare recommendations.
Results From 427 papers screened, 20 guidelines met the inclusion criteria. Only three guidelines were rated high (>75%) across all domains of AGREE-II; highest rated domains were ‘scope and purpose’ (85.1, SD 18.3) and ‘clarity’ (76.2%, SD 20.5). Recommendations for assessment and for motor therapies were most commonly reported, however, varied in the level of detail across guidelines.
Conclusion Rehabilitation CPGs were consistent in scope, suggesting little difference in rehabilitation approaches between vascular and traumatic brain injury. There was, however, variability in included studies and methodological quality.
PROSPERO registration number CRD42016026936.
- rehabilitation
- practice guideline
- stroke
- evaluation
- brain injury
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
A large comprehensive review of 20 clinical practice guidelines across all acquired brain injury conditions, which identified 2088 separate recommendations for best practice rehabilitation.
The first review to summarise evidence for individual rehabilitation interventions for acquired brain injury conditions—12 guidelines were related to stroke, 4 were related to traumatic brain injury, the remaining 4 guidelines were discipline specific (occupational therapy n=2, nursing n=1, pharmacological treatment n=1).
Low Appraisal of Guidelines for Research and Evaluation II applicability rating of included guidelines—poor identification of barriers/facilitators to guideline implementation and resource implications.
Guideline development groups applied different methods to generate recommendations which led to variability in both quality and scope; universal, international guideline may overcome such limitations.
Introduction
Acquired brain injury from both vascular and traumatic causes is a major health issue, being a leading cause of disability.1 Acquired brain injury (brain damage occurring after birth) is an umbrella term that encompasses many aetiologies and includes vascular causes (stroke) and traumatic causes.2 Within rehabilitation, clinicians commonly treat impairments and functional limitations rather than according to a specific diagnosis, with little observable difference in rehabilitation approaches between vascular versus traumatic brain injury. Provision of care based on evidence is known to improve patient outcomes3–6; however, there are documented gaps between the generation of stroke and other health research and its use in clinical practice.7 For example, a recent Australian audit of stroke rehabilitation services found that only 20% of patients are discharged without a care plan8 despite strong evidence for their routine use.9–11 Clinical practice guidelines (CPGs) aim to facilitate clinicians’ use of evidence.12 13
In addition to supporting proven interventions, CPGs also assist to raise awareness of ineffective practices.14 While CPGs are developed with the aim of bridging the research–clinical practice gap, issues regarding their use and implementation still remain. Many countries produce their own national guidelines, updates occur at varying intervals, and CPG content and scope differs with context (eg, country and guideline developer/sponsor). The level of evidence underpinning recommendation statements and the detail of these recommendations also differ across guidelines.15 16 Finally, despite rehabilitation approaches often being consistent clinically between vascular and traumatic brain injury, these diagnostic groups are separated in rehabilitation CPGs published to date. From clinicians’ perspective, having multiple guidelines that are inconsistent based on differences in assessments of evidence or scope may be overwhelming and confusing.
Therefore, the research questions for this study were to:
examine the methodological quality of rehabilitation CPGs for acquired brain injury (vascular and/or traumatic);
explore the scope of CPGs (ie, what do they include in terms of target population, clinical questions and topics covered);
examine the consistency of CPG recommendation across guidelines;
compare CPG recommendations across both diagnoses (vascular and/or traumatic);
present synthesised recommendations of the five guidelines rated as being of highest methodological quality.
Methods
Identification and selection of guidelines and their recommendations
Eligible guidelines focused on moderate to severe acquired brain injury rehabilitation (inpatient and community rehabilitation settings). The definition of acquired brain injury used “includes traumatic brain injuries, strokes, brain illness, and any other kind of brain injury acquired after birth. However, acquired brain injury does not include degenerative brain conditions such as Alzheimer’s disease or Parkinson’s disease”.17 Only recommendations pertaining to adults with a moderate or severe acquired brain injury, as defined by the source study’s authors, were included (ie, recommendations pertaining to transient ischaemic attack, mild stroke or brain injury were excluded). Guidelines not published in English were ineligible.
Search for guidelines
Medline and Embase databases were searched from the earliest record until November 2017; guideline repositories including Guidelines International Network, National Guideline Clearinghouse, Intercollegiate Guidelines Network (SIGN), National Collaborating Centre for Chronic Conditions18 and professional rehabilitation society websites were also searched. Search terms included words related to brain injury, stroke, rehabilitation, guidelines, therapy and practice guidelines. Reference lists of included articles were also reviewed. Titles and abstracts were screened (LJ) and full-text papers retrieved and reviewed independently by two reviewers (LJ and NAL) using predetermined criteria (box 1). Disagreements were adjudicated by an independent reviewer (TH). In instances where guideline development groups updated their guidelines in a modular format (ie, update of specific topic areas) and published these over separate papers, we recognise this as ‘one guideline’ (inclusive of update) and Appraisal of Guidelines for Research and Evaluation (AGREE) rated both papers as one. The search strategy is available in online supplementary appendix 1, and list of the excluded papers with reasons for exclusion is available in online supplementary appendix 2.
Guideline inclusion criteria
Systematic literature searches and review of existing scientific evidence published in peer-reviewed journals were performed during the guideline development or the guidelines were based on a systematic review published in 4 years preceding the publication of the guideline (PEDro, 2016).
The clinical practice guideline was produced under the support of a health professional association or society, public or private organisation, healthcare organisation or plan, or government agency (PEDro, 2016).
The clinical practice guideline contains systematically developed statements that include recommendations, strategies or information to guide decisions about appropriate healthcare.
Refer to inpatient rehabilitation and/or community rehabilitation of patients with acquired brain injury diagnosis.
Guidelines focus on more than one single component of rehabilitation (eg, memory and attention retaining).
Are published in English, from 1 January 2006 onwards.
Note: PEDro Physiotherapy Evidence Database. Criteria: PEDro, Criteria for inclusion of clinical trials, 2016, https://www.pedro.org.au/english/downloads/criteria/ (accessed Feb 2018).
Supplemental material
Supplemental material
Appraisal of guidelines
The AGREE-II instrument19 was used to assess the methodological quality of the included guidelines across six domains: scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability and editorial independence. Additionally, an overall guideline assessment score was assigned by the rater and recommendation decision made (options were yes, yes with modifications or no). The 23-item AGREE-II tool uses a 7-point agreement scale from 1 (strongly disagree) to 7 (strongly agree). Each guideline was independently rated by two authors (LJ and NAL). Major discrepancies in the scores (where assigned scores differed by more than two points) were discussed and independently reassessed by the third author (TH). Domain scores were calculated, whereby a total quality score was obtained for each domain by summing the score of each item.20 The mean domain score (between the two raters) was used to standardise the domain score as a percentage. To measure interobserver agreement across the ordinal categories of the AGREE-II ratings, a weighted kappa was calculated using SPSS V.24.0. This takes into account the degree of disagreement between assessors by assigning less weight to agreement as categories are further apart.21 22 An overall kappa was also calculated across all guidelines. A kappa value of <0.2 indicates poor agreement: 0.21–0.4 fair; 0.41–0.6 moderate; 0.61–0.8 good and 0.81–1.0 very good agreement.23
Synthesis of guideline recommendations
Textual descriptive synthesis was used to analyse the scope, context and consistency (ie, similar or conflicting messaging) of the CPG recommendations. Initially, each guideline was read to gain an overall knowledge of content, one author (LJ) then independently coded the CPG to identify domains covered by the guidelines. Initial codes were identified and refined through constant comparison of each CPG’s recommendations as data collection proceeded. For each domain, guideline recommendations were compared across CPGs to identify similarities and discrepancies. Within each theme, the recommendations were further coded into discrete categories where appropriate (eg, ‘motor therapy’, ‘patient/family education’).
Where a guideline had a generic recommendation without providing details on time frame, approach or assessment or discipline responsible, that is, ‘all patients should be assessed for pressure injury’, these were not included within the relevant category of the scope table. All included guidelines’ levels of evidence and grades have been converted to a unified level of evidence grading of National Health and Medical Research Council (NHMRC)24 for ease of comparison (indicated on table 1 by an double dagger symbol (‡)). Authors (LJ and NAL) compared guidelines for consistency (congruence in content and recommendations), scope (number of different categories of recommendations) and depth (number of recommendations per category). Finally, recommendations from the guidelines rated highest in quality (AGREE-II rating) were synthesised to provide an overview of all recommendations.
Characteristics of the included guidelines (n=20)
Results
Search and guideline characteristics
The electronic search strategy identified 427 publications with 48 duplicates. After screening and review, 23 documents containing 20 guidelines were included in the review (figure 1 shows Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart).25 Included guidelines covered stroke (n=12) and traumatic brain injury (n=4); and some were discipline specific (occupational therapy n=2, nursing n=1, pharmacological treatment=1).
Flow chart of papers through the review.
The characteristics and the development processes of each guideline are provided in table 1. Guideline development groups were from Australia/New Zealand (4), Europe (6), USA (6) and Canada (4). All guideline developers conducted a systematic literature search; however, methods used to extract the data and synthesise the evidence varied. Some guideline developers (n=7) graded the level of study evidence included for review, while most graded both the level of study evidence and strength of the recommendations (n=13).
Methodological quality
The AGREE-II domain scores for each guideline (n=20) are shown in table 2. The mean scores (range; SD) for the domains were: scope and purpose 85.1% (53%–100%; SD 18.3); stakeholder involvement 67.9% (14%–100%; SD 25.2); rigour of development 64.0% (9%–96%; SD 26); clarity of presentation 76.2% (22%–100%; 20.5); applicability 36.6% (0%–100%; SD 35.2) and editorial independence 57.9% (0%–100%; 37.2). The kappa values ranged from fair κw= 0.38 (95% CI 0.11 to 0.64) to very good 0.94 (95% CI 0.88 to 1.0). The overall inter-rater agreement was intraclass correlation=0.95 (95% CI 0.92 to 0.97), indicating very good strength of agreement.
Guideline assessment according to the AGREE-II instrument (n=20)
Fifteen (75%) guidelines were assessed as ‘recommended’ for use,2 9–11 18 26–37 since their quality scores ranged between 5 and 7, representing good-quality to high-quality guidelines. Four (20%) guidelines were ‘recommended for use after modification’, since they were given quality scores of 3 and 4.38–42 One guideline with an overall score of 2 was ‘not recommended’.43 Three of the 20 guidelines were rated as high (>75%) in all domains of AGREE-II.9 10 26 Guidelines updated more frequently were more often of higher quality (ie, had higher AGREE-II scores).
Synthesis of recommendations
The synthesis of clinical management themes and corresponding categories for each guideline are provided in table 3. Five major clinical management themes were identified within the eligible guidelines. These were: medical management (management of depression, pain, behaviour); organisation of services (composition of therapy teams, rehabilitation processes, discharge planning); rehabilitation therapies; managing complications and community management. The primary recommendations from the highest rated guidelines9–11 26–28 are synthesised in online supplementary table 1. Comparison of guideline recommendations between the top-rated stroke guideline and the top-rated guideline for traumatic injury32 (ie, where a recommendation is consistent across both aetiologies) has been made and is displayed in online supplementary table 1.
Guideline recommendation themes (five) and associated theme categories in acquired brain injury rehabilitation (n=20)
Medical management
Thirteen2 9–11 26–28 31 32 36 38–40 42 43 of the 20 guidelines (65%) included recommendations for medical management. Of these thirteen guidelines, the most common category was for spasticity management (85% provided recommendations), followed by depression management (77% provided recommendations), pain management (54% provided recommendations) and aggression management (46% provided recommendations). Few guidelines had recommendations for heterotopic ossification (8.3%), psychosis (8.3%), arousal/attention (17%) and memory (17%). Consistency of guideline recommendations were noted for: the use of botulinum toxin type A for the management of spasticity, minimising the use of benzodiazepines and neuroleptic antipsychotic medications in the management of aggression, not routinely prescribing antidepressants poststroke for the prevention of depression and use of selective serotonin reuptake inhibitors (SSRIs) as first line of drug treatment for depression postbrain injury.
Organisation of services
Eighteen of the included guidelines (90%) contained recommendations related to the organisation of rehabilitation services, which were grouped in the following categories: carer support, peer support, multidisciplinary service delivery, specialised rehabilitation unit of care (stroke/neurological ward) and process/delivery of service (table 3). Guideline recommendations within this theme were consistently reported across guidelines; with 511 18 26 32 40 of the 18 guidelines reporting at least one recommendation in all 5 categories. The most common categories of service organisation recommendations of these 18 guidelines were use of a multidisciplinary team model (88% provided recommendations), followed by processes/delivery of rehabilitation services (67% provided recommendations) and provision of carer support (56% provided recommendations). It is noted that guidelines that have been updated more recently (ie, Stroke Foundation9) are removing recommendations related to organisation of services from the guideline, instead referring readers to a national stroke services framework.
Rehabilitation therapies
Nineteen of the 20 guidelines (95%) had recommendations pertaining to rehabilitation therapies. There were 15 categories identified within this theme (table 3). The most common category of recommendations was for ‘motor function’ (95% of the 19 guidelines provided recommendations), ‘activities of daily living’ (89% provided recommendations) ‘cognition’ (84%), ‘upper limb management’ and ‘patient/family education’ (79% each) and ‘communication’ and ‘psychosocial’ (74% each). Few guidelines made recommendations for the categories of ‘sensation/sensorimotor’ rehabilitation (42%) and ‘home programme/self-practice’ (42%).
The guidelines with the broadest scope (ie, had at least one recommendation in most of the 15 categories) were the Stroke Foundation of New Zealand and New Zealand Guideline Group (SFNZ and NZGG),26 Stroke Foundation (Australia) guidelines9 and Intercollegiate Stroke Working Party of UK (ISWP)11 with recommendations in all categories (100%). Guidelines narrowest in scope (ie, recommendations in the fewest number of categories) were Khadilkar et al,37 SIGN2 and Registered Nurses’ Association of Ontario, Canada (RNAO)29 30 with recommendations in 13%, 33% and 33% of categories, respectively. Guideline recommendations were less consistent across categories in rehabilitation therapies, as shown in table 3.
Managing complications
Most (n=18, 90%) guidelines had recommendations for managing complications, which were grouped into: spasticity, contracture, subluxation, pain, oedema, fatigue, behaviour, pressure care, falls, nutrition, incontinence, deep vein thrombosis, swallowing (dysphagia), heterotopic ossification, seizure management and neurological nursing. The Stroke Foundation (Australia) guidelines9 was broadest in scope within this category, with complication recommendations in 12 of the 16 categories (75%), followed by SFNZ and NZGG26 and Weinstein,43 both with recommendations in 11 of the 16 categories (69%). It is important to note that while Weinstein43 had broad scope in this category, this guideline was not recommended for use according to the AGREE-II rating.
Community management
Sixteen guidelines (80%) included community management recommendations with the most common categories of recommendations being ‘driving’, ‘return to work/volunteer’ and ‘sexuality’ (11 of the 16 guidelines; 69% made recommendations in these categories). Recommendations in this category varied in terms of specificity; that is, some guidelines stated more general recommendations (ie, therapy should be provided), whereas other guidelines made specific recommendations about therapeutic interventions (ie, task-specific practice).
Overall, we found that the guidelines with the highest AGREE-II ratings of mean domain score percentage (ie, >75% in all six domains) were Stroke Foundation (Australia),9 SIGN10 and SFNZ and NZGG.26 The top four guidelines for breadth of scope and recommendation specificity are NZGG,32 Canadian Stroke Strategy27 28 and ISWP11 and for medical management, Acquired Brain Injury Knowledge Uptake Strategy.40
Discussion
This systematic review explores the quality and the scope of published CPGs for both vascular and traumatic acquired brain injury rehabilitation in a single, comprehensive review. The quality of the reviewed guidelines, as well as the scope and breadth of recommendations contained in these guidelines varied greatly, which has implications for the clinical use of each CPG. Research has demonstrated an association between stroke outcome and CPG compliance,44 thus, providing clinicians with this synthesised set of recommendations (from highly rated guidelines) is the first step in ensuring quality of care universally in rehabilitation, irrespective of type of acquired brain injury or of country of injury.
This review of 20 CPGs, containing more than 2088 recommendations, demonstrated differences between guidelines which could be expected to substantially influence clinical rehabilitation. The methodological quality of the reviewed guidelines varied, with only three guidelines achieving high ratings in all six AGREE-II domains. Across all the guidelines, the highest AGREE-II domain score was for ‘scope and purpose’ and the lowest was for ‘applicability’, suggesting that few guidelines provide information to clinicians for how to implement CPG recommendations into rehabilitation.
While the majority of CPGs were of sufficient quality according to AGREE-II ratings to be recommended, the scope of recommendations along with the depth of recommendations varied. For example, while Miller41 and RNAO29 30 made only one recommendation for incontinence management, NZGG32 provided 11 separate recommendations in the same category. Despite its recent publication (2016), one guideline was not recommended for use43 and contained multiple recommendation statements that were contradictory to the majority of the other guidelines. For example, in this guideline it was stated that ‘routine use of prophylactic antidepressant medications is unclear’ which contradicts recommendations in all five top-rated guidelines, whereby ‘routine use of antidepressants to prevent poststroke depression is not recommended’.9–11 26 27 Similarly, this guideline stated ‘acupuncture may be considered as an adjunct treatment for dysphagia’, which directly contradicts the Australian Stroke Foundation’s9 updated recommendation, whereby ‘acupuncture should not be used for treatment of dysphagia in routine practice’. Aside from this, there were recommendations which appeared to be universally agreed to by all guideline development groups. These were those specifically pertaining to ‘using a multidisciplinary approach for rehabilitation’, ‘the prescription of SSRIs for the management of poststroke depression’ and the use of ‘task-specific motor retraining’ for impaired movement. Recommendations in these categories were consistent in their clinical recommendations, the research evidence cited in support of the recommendations and the breadth of content summarised. Having such consistency suggests to clinicians that these areas of practice are universally held as representing ‘quality’ rehabilitation.
The differing methods used by each guideline development group may explain some of the observed variation between recommendations. Other explanations may include the year of guideline development (ie, availability of evidence for inclusion may have varied), date of search by guideline development group or the eligibility criteria and prioritisation process used when writing the guideline recommendations. Our findings support the importance of moving towards a universal, international guideline with pooled resources for funding adequate searching and appraisal (such as achieved by the international guidelines for the selection of lung transplant candidates).45
Separating out clinical conditions (ie, vascular from trauma) is likely inefficient in clinical practice, given that both conditions are treated consistently with common research evidence findings. Our synthesis found common recommendations across both vascular and trauma CPGs in the areas of organisation of services, rehabilitation therapies, managing complications and community management. We do acknowledge unique guidelines for each condition in the areas of ‘medication’ and ‘behaviour’ management; however, rehabilitation practice recommendations do not appear to differ outside these areas which suggests that a synthesised set of recommendations could substantially improve the quality of rehabilitation. Kirsner and Marston46 highlight that variability in guidelines and issues around applicability of recommendations to ‘real-life’ contexts can make the selection and use of guidelines challenging. The usefulness of CPGs rests on the reasonable assumption that following the recommendations will improve care, but having multiple guidelines to apply within a single neurorehabilitation setting is unlikely to achieve this. Factor such as 20 available guidelines, published across 23 separate documents, with updates occurring in a modular format and varying modes of access (online, freely available, paid access) hinder clinicians’ behaviours regarding guideline selection and implementation.
Pragmatically, rehabilitation clinicians are likely to work with mixed acquired brain injury patient populations. Synthesising recommendations of the guidelines with higher methodological quality, as in the present review may improve the future consistency of clinical rehabilitation guidelines and in turn influence the quality of care in this field. Further to this, having direct comparison within a single document between stroke and trauma brain injury recommendations may highlight where rehabilitation practices should differ. Our study has rated all rehabilitation CPGs across both clinical conditions and suggests that clinicians become familiar with those of both high quality and broad scope. While clinicians may be more familiar with their own national/local clinical practice guidelines, findings from our systematic review suggest that these may not always be of the most methodologically rigorous.
The main limitation of the present study is, perhaps also one of its strengths. That is, the use of a standardised method and rating tool. As previously discussed, the AGREE-II instrument assesses how well a CPG development process is reported but not the specific clinical content of the CPG recommendations. As we synthesised only the highest quality guidelines for this review, it must be acknowledged that a guideline could receive a high AGREE-II rating, yet contain low-quality recommendations based on the level of evidence accepted by the guideline development group. Our chosen review method may mean that additional and important aspects of a CPG and its ease of implementation were not rated. For example, since the rating tool selected (AGREE-II) does not rate the level of intervention detail provided in the recommendation statements, these aspects fell outside of the current systematic review findings. We have sought to capture this detail in our qualitative synthesis; however, we recommend that future discussions of CPG rating tools and systematic reviews of CPGs continue to explore this issue.
Summary
Multiple CPGs exist to guide rehabilitation for adults after acquiring a brain injury, reporting on either vascular (stroke) or traumatic literature, which makes selecting a high-quality guideline to implement overwhelming and difficult. Variability exists in guideline quality, breadth and detail of recommendations and availability of information on applicability of these guidelines. This is likely underpinned by the evidence included and method of evidence synthesis employed by each guideline development group. Clinicians need to be aware of quality differences between these guidelines and be prepared to look beyond their local guidelines to use the highest quality guidelines in the rehabilitation of adults with an acquired brain injury from stroke or traumatic causes.
Supplemental material
Acknowledgments
The authors thank Professor Maria Crotty for assistance with textual synthesis analysis.
References
Footnotes
Contributors All authors made substantial contribution to the design of the work. LJ, NAL and TH completed PROSPERO trial registration. NAL and LJ completed the search as per search strategy. LJ completed screening and full-text review. LJ and NAL completed AGREE-II ratings for included guidelines. All authors made significant contribution to the interpretation of data and drafting the work or revising it critically for important intellectual content. They provided final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding LJ was supported by a postgraduate fellowship from the National Health and Medical Research Council (NHMRC; GNT 1114522); NAL was supported by NHMRC Translating Research Into Practice fellowship (GNT 1112158); DAC was supported by a National Heart Foundation/NHMRC Future Leader fellowship (GNT 1063761).
Competing interests NAL, TH and DAC have been involved in the development of clinical practice guidelines referenced in this paper. They have also authored papers (randomised controlled trials) cited within the body of the guidelines that this systematic review appraised.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement There are no additional, unpublished data arising from this research.