Objective Clinical guidelines are designed to optimise patient care and provide efficient approaches for therapy. Epilepsy is a chronic brain disorder that continues to experience a considerable treatment gap due to non-standard recommendations. We assessed the reporting quality of clinical practice guidelines on epilepsy over the past 5 years to generate a reporting specification for this study.
Setting Seven databases were searched in May 2018 focusing on the period from 2013 to 2018. These included Medline, EMBASE, PubMed, Cumulative Index to Nursing and Allied Health Literature, China National Knowledge Infrastructure, Wanfang and Chinese Science and Technology Journal Database (VIP). Reporting quality of epilepsy guidelines was assessed by two independent authors using the Reporting Items for practice Guidelines in HealThcare (RIGHT) approach. Spearman’s correlation was used to assess inter-rater reliability.
Participants Participants with epilepsy or seizure, not limited by age, gender, course of disease or cause of epilepsy, were included.
Interventions There were no limitations with regard to intervention.
Primary and secondary outcome measures The outcome was the ability of the RIGHT tool to measure reporting quality.
Results Twelve relevant guidelines were included in this study. The reporting quality was not high in any of the included guidelines. The highest reporting quality included a ‘yes’ proportion of 77.1%, whereas the worst included a corresponding proportion of 37.1%. Overall evaluation results showed that 16.7% of the included guidelines were of high quality, 75% were of medium quality and 8.3% were of low quality. The correlation between the two estimators was credible (ρ>0.7).
Conclusions Appraisal of these guidelines using the RIGHT tool revealed that the quality of reporting varied among guidelines. Items that exhibited low quality in most included guidelines were healthcare questions, rationale/explanation for recommendations, quality assurance, funding source(s) and role(s) of the funder, and limitations of the guideline. Thus, these aspects should receive greater attention in future guideline reporting.
- clinical guidelines
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
To the best of our knowledge, no studies have assessed the quality of epilepsy guidelines using the Reporting Items for practice Guidelines in HealThcare (RIGHT) checklist.
The included guidelines were measured using the RIGHT tool, produced by the Practice Guidelines in Healthcare Group, a component of WHO.
Twelve relevant guidelines, involving six regions, were included in this study.
The study showed insufficient reporting quality in some areas.
This study indicates that greater attention is needed with regard to healthcare questions, rationale/explanation for recommendations, quality assurance, and funding source(s) and role(s) of the funder in future guidelines reporting.
Epilepsy is a chronic, repeating, relapsing neurological brain disorder with high incidence and mortality rates, and the disorder can affect any individual irrespective of age, region and ethnicity.1 2 According to WHO in 2014, there are approximately 50 million patients with epilepsy worldwide, with the disease morbidity rate being 4‰−7‰. Compared with adults, children and adolescents have a higher prevalence but a lower mortality rate. An earlier study reported that mortality from the disorder was up to 3% in American children, but more than 30% in American adults.3 Although the disorder can be controlled in most patients by appropriate therapy, some (especially those living in developing countries) are not able to receive appropriate treatment for reasons such as poor income, cognitive deficiencies and healthcare costs. In China, for example, the treatment gap is approximately 63%, implying that about four million patients do not receive the recommended treatment for epilepsy.4
Clinical guidelines based on high-quality systematic review evidence assessed the benefits and limitations of alternative care options for facilitating optimised patient care and effective therapy approaches.5 With increasing worldwide attention to epilepsy, a growing body of clinical guidelines are available. However, these guidelines are not standardised, varying in terms of the respective country’s definition of epilepsy. Although clinical guidelines allow for standardising and improving the quality of clinical practice, questions on guideline development and reporting remain unanswered.6 7 Ineffective treatment methods for epilepsy persist because of unclear pathogenesis and lack of quality, standardised clinical guidelines. An instructive clinical guideline should be based on high-quality evidence-based systematic reviews and reporting. However, the reporting quality of clinical guidelines seems low,8 and the currently used tools do not accurately address quality assessment and reporting in a single statement. There are two reporting checklists available for clinical guidelines: one is the Appraisal of Guidelines, Research and Evaluation instrument (AGREE) and another is the Reporting Items for practice Guidelines in HealThcare (RIGHT). AGREE, developed by a small group, was produced for use in both quality assessment and reporting, although it was limited to items derived from the tool itself (rather than other assessments).9 To construct a specific clinical guideline and fill the gap in current assessment approaches, Chen and colleagues10 from the WHO established the RIGHT tool.
In this review, we analyse the reporting quality of epilepsy practice guidelines based on the RIGHT tool. This will help identify insufficient reporting section to better guide clinical control of epilepsy.
This study comprised a review of epilepsy clinical practice guidelines (CPG) using the RIGHT tool.
This study was performed in accordance with the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses.11
Types of guidelines
Guidelines that focused on preventive and/or therapeutic intervention in epilepsy were included, whereas those solely describing epidemiology, training, research methods or legal issues regarding epilepsy were excluded. Furthermore, summarised organisational guidelines, comments or correspondence studies were excluded.
Types of participants and public involvement
There were no patients involved in this study. In this study, we focused on guidelines and not participants themselves, which needs no ‘Patient and Public Involvement’. In those included guidelines, participants with epilepsy or seizure, regardless of age, gender, course of disease or cause of epilepsy (eg, caused by pregnancy or trauma), were included.
Types of interventions
There were no limitations with regard to interventions. Drug therapies and non-drug therapies recommended in the guidelines were included.
Guidelines meeting the eligibility criteria were searched in English and Chinese using a computer program to avoid subjective interpretation. Seven databases, Medline, EMBASE, PubMed, Cumulative Index to Nursing and Allied Health Literature, China National Knowledge Infrastructure, Wanfang and Chinese Science and Technology Journal Database (VIP), were searched for articles published from January 2013 to December 2018. The search strategy used the terms ‘epilepsy’ or ‘seizure’ (‘癫痫’) AND ‘guideline’ or ‘guidance’ or ‘recommendation’ or ‘consensus’ or ‘policy’ (‘指南’ or ‘专家共识’). We also searched Medline, a publicly available repository of guidelines in China (http://guide.medlive.cn), using keyword searches based on the eligibility criterion of ‘epilepsy’ (‘癫痫’). Concomitantly, the National Guideline Clearinghouse (http://www.guideline.gov), National Institute for Health and Care Excellence (http://www.nice.org.uk), International League Against Epilepsy (ILAE) (http://www.ilae.org) and WHO (http://www.who.int) were searched using the terms ‘epilepsy’ or ‘seizure’ AND ‘guideline’. Two authors (ZW and YZ) screened the titles and abstracts independently to standardise screening, and selection differences, if any, were resolved through discussion. Full texts were screened to confirm eligibility.
The following information was extracted from each guideline: title of the guideline, region(s) of guideline development, year of publication, source of publication, organisation(s) responsible for the guideline, number of authors, target population, funding, guideline focus and whether it was an update of a previous edition. Other information pertaining to guideline format included basic information, background, evidence, recommendations, review and quality assurance, funding, declaration and management of interests, and other information (eg, suggestions for further research and limitations of the guideline).
Assessment of the quality of reporting
The RIGHT tool is a checklist that can be used to assess the reporting quality of CPG, and aids in understanding and implementation of clinical guidelines, serves as a standard for guideline reporting in peer reviews (ie, for reviewers and journal editors), and assists developers in guideline reporting.10 The RIGHT tool consists of seven sections: basic information (items 1–4), background (items 5–9), evidence (items 10–12), recommendation (items 13–15), review and quality assurance (items 16–17), funding and declaration and management of interests (items 18–19), and other information (items 20–22). Assessment was performed by two authors (ZW and YZ); ‘yes’ indicated full reporting of necessary information, whereas ‘no’ indicated partial or no reporting. We defined reporting to be of high quality if the ‘yes’ responses were >70%, medium quality if they were 40%–70% and low quality if they were <40%. Spearman’s correlation was used to assess inter-rater reliability (ρ>0.7 indicated good inter-rater reliability). If opinions differed, a third author (LL) made a final decision. The percentage of fully reported items was expressed to assess reporting quality of guidelines.
Data analysis and investigation of heterogeneity
Data were analysed using SPSS V.19.0 and Microsoft Excel (Microsoft, Redmond, Washington, USA). The descriptive results are shown in tables. Inter-rater reliability was calculated for each domain of the RIGHT instrument using intraclass correlation coefficient with a ρ value.
Of the total 938 potentially relevant articles identified, 883 were excluded after title and abstract screening. The remaining 55 were retrieved and full texts were read, resulting in 12 guidelines that met the inclusion criteria (figure 1).
The characteristics of the included guidelines12–23 are detailed in table 1. Of the included guidelines, one16 produced by the ILAE was an international guideline, one20 established by the ILAE-Commission on European Affairs focused on women and girls with epilepsy in Europe, one19 was published for neonatal seizure in Australia, two were published in the USA (one15 focused on vagus nerve stimulation and the other17 focused on convulsive status epilepticus), two were published in the UK (one14 was published in Scotland not focused on epilepsy in pregnancy and the other18 placed emphasis on epilepsy in pregnancy all over the UK), and four were published in China (three12 21 22 were produced in mainland China and one14 was produced in Hong Kong). One22 guideline referred to treatments involving traditional Chinese medicine (TCM), but was limited to a summary of this approach; another was a guideline established for TCM clinical diagnosis and treatment of paediatric patients, including a detailed description of TCM treatments for children with epilepsy; and one23 guideline focused on presurgical epilepsy work-up. The correlation between the two estimators is shown in table 2, and the ρ of each item was >0.7, indicating the robustness of this study results.
Quality of reporting evaluation by RIGHT
The quality of guideline reporting was evaluated using the RIGHT tool; notably, most included guidelines did not show high reporting quality (table 3). We assessed each item in strict accordance with the standard and calculated the percentage of fully reported items. The best reporting quality achieved a score of 77.1%,18 whereas the worst achieved a score of 37.1%.21 Overall evaluation results showed that 16.7% of included guidelines were of high quality, 75% were of medium quality and 8.3% were of low quality (figure 2).
In general, all included guidelines showed sufficient reporting of basic information. Only two12 14 did not report any corresponding developers or authors, whereas others fully reported the title, executive summary, abbreviations and corresponding information.
Background was not adequately reported in any of the included guidelines; notably, two16 22 did not describe the epidemiology of epilepsy, two19 22 did not clearly describe the aim of the guidelines, three12 18 21 had no subgroups, one19 did not explicitly describe the intended primary and potential users, two13 21 did not describe the specific roles of authors who contributed to guideline development, and four12 15 16 21 did not list the identity information of authors. Furthermore, only two12 13 guidelines were intended to focus on low-income regions.
Reporting of evidence was inadequate. Although most included guidelines stated that they were developed based on randomised controlled trials or meta-analyses, none included regular reporting of population, intervention, comparator and outcome. Four guidelines13 14 21 23 did not indicate the manner in which outcomes were selected, and it was unclear whether those guidelines were the first version or an updated version.21 Five16–19 22 included guidelines assessed evidence in accordance with the Grading of Recommendations Assessment, Development and Evaluation approach.
Clear and actionable recommendations were provided in all guidelines; however, four12 19–21 did not clearly indicate the strength of each recommendation, only one22 considered feedback from children with epilepsy and their parents (ie, through follow-up and online survey), two16 22 emphasised that guideline development groups had made decisions through repeated discussions, and only two guidelines18 23 included equity when formulating its recommendations.
Review and quality assurance
Funding and declaration and management of interests
Three guidelines12 14 21 did not describe the funding sources, whereas two15 16 precisely declared that the stakeholder did not participate in the development of the guidelines at any stage. Four15 18 20 22 had detailed a description of the conflict of interest of the authors.
This study identified 12 guidelines on epilepsy published in the past 5 years, including both Western medicine and TCM. Appraisal of these guidelines with the RIGHT tool revealed that the quality of reporting varied among guidelines. Some of the included guidelines showed relatively higher quality and favourable overall recommendations; these could be used by healthcare providers and patients as the basis for discussion on management of epilepsy. However, considering individual differences among patients, each recommendation should be used with caution. Moreover, some items in these guidelines were of low quality (eg, healthcare-related questions, rationale/explanation for recommendations, quality assurance, funding source(s) and role(s) of the funder, and limitations of the guideline); thus, these aspects should receive greater attention in future guideline reporting.
Notable strengths of this study include its use of comprehensive systematic review to identify eligible clinical guidelines and its assessment of quality using the RIGHT tool, which is published by the WHO and is an internationally accepted standard for appraisal of guidelines. This study found that CPGs for epilepsy are wide-ranging and have been published in many regions in the past 5 years.
However, according to the appraisal of each item based on the RIGHT tool, all included guidelines were designated as ‘recommended with provisions or modifications’ due to inadequate reporting quality. A previous research drew a conclusion of a heterogeneity in methodological quality and great gaps in topics of epilepsy guidelines as assessed by AGREE II tool.24 Although different tools were used to assess epilepsy guidelines, and both assessment tools as mentioned above focus on different aspects, similar findings were concluded in this study, showing that the included epilepsy guidelines are of poor reporting quality. The findings in the present study are consistent with the assessments of guidelines in multiple regions, as well as patients and clinical medicine interventions: the included guidelines did not adhere to established reporting quality standards, and further improvement in guideline development is needed. Although there were limitations in the included guidelines according to assessment via the RIGHT tool, only four of the included guidelines provided suggestions for future guideline developers to avoid such limitations in future work. The greatest limitation in the included guidelines was a lack of consideration of the requirements and recommendations for patients and their relatives with regard to psychological or economic burden. An ideal clinical guideline should enable improvement for patients and their disorders, thereby guiding clinical doctors and better serving the patients. Furthermore, few or the included guidelines reported how stakeholders influenced the development of the guidelines, which suggests that the guidelines may exhibit low credibility, especially those targeting a specific treatment recommendation. Only one guideline focused on TCM interventions for epilepsy; reporting of this guideline was not of high quality and solely targeted children with epilepsy. It is well known that antiepileptic drugs perform irreversible harm to the liver and kidneys; epilepsy in 25.3% of adult patients and 13.4% of paediatric patients was reported to have led to intractable epilepsy due to antiepileptic resistance and long-term treatment.25 Thus, to reduce the adverse effects of antiepileptic drugs, TCM or other complementary alternative therapies should be recommended for use in clinical applications according to a systematic review. The findings of this study may serve as an alert for epilepsy guidelines development and reporting in the future as poor reporting quality of CPGs could mislead clinical care for patients with epilepsy.
There were some limitations to this study. The main limitation was that only papers published in English and Chinese were searched for this study, leading to some guidelines published in Japanese, Russian, French, Spanish, German, Italian or other languages being missed. Second, the inclusion of six regions limited the findings to those regions; however, the regions were chosen because they were expected to be representative. We consider epilepsy interventions and other healthcare practices to have a particular impact in the USA, Europe, UK and China, where higher morbidity is reported. Third, a robust series of eligibility criteria were formulated and tested before these guidelines were identified; however, some guidelines might be missed by computerised searches. Finally, the RIGHT tool has a broad range of assessment of individual guideline components for specialists across medical specialties and levels of seniority, which might have led to subjective estimation during the decision-making process. However, the correlation between the estimators suggested that the RIGHT tool overcomes this potential bias.
In future updates, guidelines that achieved higher quality of reporting and overall recommendations could be improved based on the RIGHT tool specifications, as well as with insight from a large number of resources that are available to support guideline development and implementation.26 27 Future research should identify patients with epilepsy and interventions other than those reviewed here in a manner supported by sufficient evidence to facilitate guideline development.
Appraisal of these guidelines using the RIGHT tool revealed that the quality of reporting varied among guidelines. Items that exhibited low quality in most included guidelines were healthcare questions, rationale/explanation for recommendations, quality assurance, funding source(s) and role(s) of the funder, and limitations of the guideline. Thus, these aspects should receive greater attention in future guideline reporting.
ZW and YZ contributed equally.
Contributors ZW and YZ contributed equally to this work. This review was drafted by ZW and YZ, and revised by XG and LL. The search strategy was addressed by XMH and WG, and updated by ZW. ZW and XMH screened potential trials, extracted the data and completed the data synthesis independently. XG and LL arbitrated in cases of disagreement and ensured the absence of errors. All authors gave final approval for the version to be published. We confirm that we have read the journal’s position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.
Funding This study is supported by the International Program for Postgraduates, Guangzhou University of Chinese Medicine, and Construction of High Level University, Guangzhou University of Chinese Medicine.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.