Article Text
PDF
Abstract
Objectives We systematically analysed recommendations from gout guidelines as an example, to provide a basis for developing a reporting standard of recommendations in clinical practice guidelines (CPGs).
Design Systematic review without meta-analysis.
Methods We systematically searched MEDLINE and all relevant guideline websites (National Institute for Health and Care Excellence, National Guideline Clearinghouse, Scottish Intercollegiate Guidelines Network, WHO, Guidelines International Network, DynaMed, UpTodate, Best Practice) from their inception to January 2017 to identify and select gout CPGs. We used search terms such as ‘gout’, ‘hyperuricemia’ and ‘guideline’. We included the eligible CPGs of gout according to the predefined inclusion and exclusion criteria after screening titles, abstracts and full texts. The characteristics of recommendations reported in the included guidelines were extracted and analysed.
Results A total of 15 gout guidelines with a range of 5–80 recommendations were retrieved. Several indicators were used in the gout guidelines to facilitate identification of recommendations, including grouping all recommendations in a summary section, formatting recommendations in a particular or special way, using locating words for recommendations and indicating the strength of recommendation and quality of evidence. We found some components commonly used in the recommendations. The wording of recommendations varied across guidelines. Recommendations were detailed and explained in the section of rationale and explanation of recommendations. In some guidelines, recommendations were accompanied with other material to assist their reporting.
Conclusions Variability and inconsistency were found on the reporting and presentation of recommendations in gout guidelines. Several points for reporting recommendation can be summarised. First, we suggested summarising and highlighting the core recommendations in a guideline. Second, guideline developers should try to structure and write recommendations reasonably. Third, it was necessary to detail and explain the recommendations and their rationale. Finally, describing and providing other potential useful contents was also a helpful way for clear reporting.
- clinical practice guidelines
- recommendation
- gout
- RIGHT
- checklist
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The study is the first to have fully analysed each area of reporting and presentation of recommendations in gout clinical practice guidelines.
The findings of the study can serve as a framework for Reporting Items for Practice Guidelines in Healthcare for reporting recommendations and provide strong support to form the final checklist.
The representativeness of the sample used in this article may be limited.
Background
Clinical practice guidelines (CPGs) are statements that include recommendations intended to optimise patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.1 Recommendations are the core components of CPGs and should be presented as clear, specific and actionable statements.2 Ambiguous, inconsistent or unnormative recommendations may confuse the audience and hamper the uptake of guidelines.
Studies systematically analysing the reporting condition of recommendations are currently lacking, although some papers revealed various deficiencies on the reporting and presentation of recommendations. Two studies3 4 by Hussain et al found that in many guidelines of different conditions, recommendations were not easily identifiable or executable in the way they were presented and lacked essential information such as the strength of recommendations (SOR). WHO and National Institute for Health and Care Excellence (NICE) have released guidance on how to report and present recommendations for their own guidelines,5 6 and the Appraisal of Guidelines for Research & Evaluation (AGREE) instrument7 has also tailored a special domain for this issue. These tools are however not comprehensive or systematic enough, and there is no international standard for reporting recommendations and their underlying rationale at present.
The Reporting Items for Practice Guidelines in Healthcare (RIGHT) working group8 was established in 2013, aiming to provide a reporting checklist for guidelines on health policy and systems, public health and clinical medicine. The working group is developing several extensions for RIGHT, of which one is RIGHT for reporting recommendations (RIGHT-R). The RIGHT working group established a special research team and systematically analysed the recommendations taking the gout guidelines as an example, in order to provide an initial and important insight into the items to be considered for the eventual checklist of reporting recommendations.
Methods
Information sources
Researchers of our team have participated in the development of 2016 China gout CPG,9 and we chose CPGs on gout worldwide as the sample for this study. We systematically searched MEDLINE and all relevant guideline websites (NICE, National Guideline Clearinghouse(NGC), Scottish Intercollegiate Guidelines Network(SIGN), WHO, Guidelines International Network (GIN), DynaMed, UpTodate, Best Practice). All searches of these database and guideline websites were conducted from inception to January 2017. A manual search in Google Scholar was also performed to identify potential additional CPGs. We used the ‘gout’, ‘hyperuricemia’, ‘guideline’ as keywords. The full search strategy is presented in Additional file 1.
Inclusion and exclusion criteria
We included CPGs for diagnosis and treatment of gout published in Chinese or English. Former editions of the guidelines were excluded if a later version was available. Translations into languages other than the original were also excluded.
Screening and extraction of literature
Two researchers (YY, AZ) independently screened titles and abstracts of all literature identified in the initial search, and then checked the full texts of the selected articles. Disagreements were resolved by face-to-face discussion, or in case of persistent disagreement, by consultation with a third researcher (YC). We used a predeveloped information extraction table to extract the general characteristics of included guidelines, including region, year of publication, developer and update status. We also retrieved the supplementary material of the guidelines for further analysis if available.
Data analysis
Our team discussed and designed the data extraction table based on the original RIGHT checklist and performed a preliminary study to improve and modify the table. We analysed the aspects of identification of recommendations, structure and content of recommendations, rationale and explanation of recommendations and other information in the included guidelines (box 1). All data were retrieved from the PDF or printer-friendly version of the guidelines. We calculated descriptive statistics, such as the total number of guidelines or recommendations where each aspect was used.
Key points included for analysis of recommendations
Identification of recommendations
Analyse how the core recommendations were indicated and highlighted:
Whether all recommendations are presented together in a summary section?
Whether recommendations are formatted in a particular or special way in the full text?
Whether a locating word for recommendations was used (ie, a search for this word can identify recommendations in the full text)?
Whether the strength of recommendations and quality of evidence were specifically indicated?
Structure and content of recommendations
Analyse how the recommendations were designed and written:
Whether recommendations were structured with essential and specific components?
Whether recommendations were worded with appropriate language?
Rationale and explanation of recommendations
Analyse how rationale and explanation of recommendations were reported and structured:
Whether recommendations were extended in more detail?
Whether the evidence supporting the recommendations was provided?
Whether the factors that influence recommendations were described?
Other information
Analyse how the reporting of recommendations should be assisted, including algorithm map and summary table of information on the usage, indication, adverse reactions and cost of the drugs.
Patient and public involvement
Patients and the public were not involved in this research.
Results
Basic information
Our search revealed 15 gout CPGs with a range of 5–80 recommendations per guideline (figure 1). The guidelines were developed by five kinds of organisations or institutions based in 12 countries or regions, and were published over decade period of 10 years (2008–2017) (Table 1). Ten guidelines were first versions and five were updates. For one guideline (2016, Taiwan),10 we were not able to extract the recommendations separately because they were embedded within long paragraphs, and we therefore excluded this guideline from the analyses investigating recommendations.
Flow diagram of the search and selection of the guidelines. CPG, clinical practice guideline; GIN, Guidelines International Network; NICE, National Institute for Health and Care Excellence; NGC, National Guideline Clearinghouse; SIGN, Scottish Intercollegiate Guidelines Network.
Characteristics of included gout CPGs
Identification of recommendations
Summary of recommendations
Among the 14 guidelines included for analysis, 7 guidelines11–18 used a table to group the core recommendations together in a summary section. Five guidelines12 14–18 listed their summary tables of recommendations under the result section, one guideline11 on the title page and one13 in the discussion section. Recommendations were accompanied by an indicator of quality of evidence (QOE) or SOR in the summary table of six guidelines.11–15 17 18
Format of recommendations
Several formatting indicators were used to distinguish the recommendations from the main body text in eight guidelines, including boldface (three guidelines9 13 19), italics (two guidelines16–18), frames (two guidelines),19 20 colouring (two guidelines11 19) and fine print (one guideline15). The recommendations were numbered in the full text of nine guidelines,9 12 14–21 and the total number of recommendations in the guidelines was clearly reported in the main text in five guidelines.12 14–16 21
Locating words for recommendations
An obvious locating word for recommendations could be found in 12 guidelines. Examples include ‘recommendation’, ‘statements’ and ‘推荐意见’ (‘recommendation’ in Chinese). Formatting indicators, such as boldface, colouring and large font, were also used to highlight the locating words (Additional file 2, online supplementary table 1).
Supplemental material
Indicator of SOR and QOE
Table 2 shows the different grading systems and indicators that were used in the guidelines (including letters, numbers, words and their combination). Recommendations in the text were accompanied by an indicator of SOR in two guidelines,11 13 an indicator of QOE in one guideline22 23 and both of them in seven guidelines.9 17–21 24 25
Grading systems for evidence and recommendations in gout CPGs
Structure and content of recommendations
The drug intervention recommendations in the included guidelines involved mainly seven medicines or drug classes or categories: three (colchicine, non-steroidal anti-inflammatory drugs (NSAIDs) and corticosteroids) for the treatment of acute gouty arthritis and four (allopurinol, febuxostat, benzbromarone and probenecid) for the urate-lowering therapy of chronic gout. The structures of the recommendations were not consistent. A total of nine components were used across all guidelines: drug dosing, target population, administration route, therapeutic goal (outcome), line of treatment, usage, adverse reaction, contraindication and timeframe. The number of included components ranged from one to eight across the recommendations. The word counts of the recommendations for drugs varied from <1010 to several hundred. Twenty-one per cent of the recommendations for drugs had 0–20 words, 41% 21–40 words, 22% 41–60 words, 7% 61–80 words, 3% 81–100 words and 7% over 100 words (Additional file 2, online supplementary tables 2 and 3).
Drug dosing, target population, administration route and therapeutic goal (outcome) were the four most common components included in the recommendations for drugs for acute gouty arthritis. Recommendations on colchicine involved on average more components than the recommendations on NSAIDs or corticosteroids. For recommendations on drugs for chronic gout, the most common four components were line of treatment, drug dosing, target population and therapeutic goal (outcome).
The preference of wording for recommendations varied across guidelines, but can be generalised into three types (Additional file 2, online supplementary table 4). One was the use of a modal verb (such as should, can, may, will, must, used by 12 guidelines), for example, ‘probenecid, benzbromarone or febuxostat can be used as second-line urate-lowering therapy according to clinical context’. The second type was the use of the verb ‘be’ (used by 10 guidelines), for example, ‘allopurinol is the first-line urate-lowering therapy’. The third type was the use of a notional verb (used by six guidelines), for example, ‘consider the use of systemic glucocorticoids or IL-1 inhibitors'. Active voice was used to write recommendations in 9 guidelines and passive voice was used in 12 guidelines (Additional file 2, online supplementary table 5). The six most frequently used expressions used in writing recommendations in the included English-language guidelines were the following (in a descending order): should be done, be followed by an adjective, be followed by a noun, is/are recommended, is/are indicated, can be done (Additional file 2 online supplementary table 6). Due to the small number of included Chinese-language guidelines, such common expressions in Chinese could not be identified.
Rationale and explanation of recommendations
All guidelines reported the rationale in the formulation of recommendations and the explanation for recommendations, but the presenting style and contents differed considerably.
Details of recommendations were presented in 11 guidelines.9 11–13 15–23 These included specific descriptions of the dosage, usage of a specific drug or the preconditions in which the drug can be used. Four guidelines9 12 13 15 presented these extensions in a separate paragraph while other seven guidelines embedded them within the whole section of rationale and explanation of recommendations.
All guidelines described the supporting evidence together with the sources for recommendations. Three guidelines13 17 18 21 described the type of evidence, such as a systematic review or randomised controlled trial, eight11–14 17 18 22–25 noted the QOE and five13–15 21 24 reported the approach used to generate recommendations when no evidence was available, such as based on expert advice.
The guidelines also reported the factors taken into consideration when generating the recommendations. The most commonly reported factors were the following: adverse reaction to drugs (safety, 13 guidelines), feasibility (10 guidelines), magnitude of the effects and relative importance of the outcomes (6 guidelines), cost and resource implications (6 guidelines), values and preferences of the target population (6 guidelines), equity (5 guidelines) and acceptability (5 guidelines).
Of the four guidelines that were updates of earlier versions, two16 21 listed the differences between old and new versions in this section.
Assisted production for recommendations
Other materials found in the included guidelines that can be used to assist the reporting of recommendations were algorithm map and a summary table of drug information. Three kinds of algorithm maps were found in the guidelines: an algorithm map containing only treatment,11 16 21–23 an algorithm map containing both diagnosis and treatment20 and an algorithm map involving prophylaxis.22 23 The summary table of drug information was found in three guidelines,13 17–19 comparing the drugs recommended in the text from the aspects of usage, indication, adverse reaction and cost.
Discussion
We found that a remarkable inconsistency existed across institutions and organisations involved in guideline development concerning the styles and the contents of recommendations, which poses a huge obstacle to the correct understanding and use of the guideline. The clarity and adequacy of recommendations reporting contribute greatly to the implementation and dissemination of guidelines. Recommendations should be reported in a comprehensible and visible way.26 We will summarise several points requiring attention when reporting recommendation according to the finding from gout CPGs.
Summarising and highlighting the core recommendations
Key recommendations should be easily identifiable, and not embedded within long paragraphs.7 26 This ensures that the audience can efficiently obtain and use the core of the guidelines. We found that nearly half of the gout guidelines grouped recommendations together in a summary table, which was very helpful for identifying and finding the recommendations. However, the place of the summary table in the guideline document may also be important. For example, the Italian Society of Rheumatology guideline13 placed the summary table in the Discussion section, which means users may need to read the whole text to find recommendations. We suggest placing the summary of recommendations in an eye-catching part of guidelines, for example, the first page of the guideline.
Other methods used to indicate recommendations were using a distinctive format, numbering or specific locating word for recommendations. We found a variety of formatting indicators which were used for highlighting recommendations. Selecting one of them, or a combination, according to specific circumstances can obtain a good effect. Numbering for recommendations, an effective and simple method, facilitates identification of recommendations and helps the locating of recommendations. It is of great help for audience who use electronic documents to add a locating word (eg, ‘recommendation: XXX’) for recommendations. In general, recommendations can be identified in many different ways, and no matter which way we used, it was significant to make the users find the most relevant recommendations easily.7
In addition, clear identification of the QOE helps, and SOR increases the trustworthiness and improves the implementation of clinical guidelines.27 We found that because of the use of different grading systems, the reporting of SOR and QOE varied across guidelines and multiple different methods were used by guideline developers for demarcating SOR and QOE. A randomised trial28 found that for presentation of SOR and QOE, symbols may be easier for the audience to understand than letters or numbers. However, considering that there is no commonly agreed grading system for guideline development at present, we believe the most important measure is to explain the definition of SOR and QOE clearly when reporting and presenting recommendations.
In summary, we suggest that recommendations in guidelines should be listed in tables, and each recommendation should be companied by SOR and QOE using clear and accessible forms. If conditions allow, guideline developers should try to use a distinctive format to highlight the core recommendations in their guidelines.
Structuring and writing recommendations reasonably
Recommendations should contain enough information to be understood without referring to the evidence or other supporting materials.6 26 Most CPGs on gout included target population, drug dosing and therapeutic goal (outcome) in their recommendations. The structure of recommendations can be affected by multiple factors. For example, the content of recommendations for a specific medication (colchicine) was obviously more specific than for a drug class (NSAIDs and corticosteroids). Obviously, the structure of the recommendations depends on the category of the medication. As another example, some characteristic of a drug may also change the structure of recommendations. There are several different routes of administration for acute gouty arthritis drugs, and the route may significantly influence the therapeutic effect or relevant adverse reaction. Therefore, administration route was much more often reported in recommendations involving acute gouty arthritis drugs than in recommendations involving drugs on urate-lowering therapy of chronic gout.
Guideline developers need to consider a variety of factors when structuring recommendations. According to the NICE guideline development manual,6 the target population (eg, patients) and the setting should be clearly reported in recommendations when applicable and the target audience (eg, physicians) should also be reported in some special conditions. According to the WHO guideline development handbook,5 recommendations need to reflect the Population, Intervention, Comparator and Outcome format. And according to the AGREE II instrument,7 a recommendation should provide a concrete and precise description of which option is appropriate in which situation and in what population group.
Word counts also varied across the recommendations. Word count is related both to the adequacy of recommendations, and the conciseness of the language. Studies have found that the clarity and specificity of recommendation could significantly influence decision making among physicians.29 30 The presentation of recommendations should thus be simple, unambiguous and actionable on the premise that sufficient information is included.
Recommendations have a deontic meaning. Some special terms such as ‘should’ and ‘may consider’ appear frequently in recommendations. These expressions are related to deontic logic, the branch of logic that concerns notions of obligation and permission. Several different deontic terms were used in the same guideline. We believe it is a result of the guideline authors intending to show variable levels of obligation. A study investigated the level of obligation conveyed by deontic terms commonly found in CPGs, and found that ‘must’ conveys the highest level of obligation, while ‘may’ and ‘may consider’ convey lower levels of obligation and ‘should’ and all other deontic terms convey intermediate levels of obligation.31 It is thus clear that wording of recommendations is closely related to the SOR. We think the type of wording chosen for drafting recommendations can directly affect the SOR, for example, starting with a verb indicates a strong recommendation, or using modal verbs like ’may', ’can' indicate that the recommendation is made with less certainty. GIN suggested that recommendations related to interventions should use unambiguous language that reflects the SOR.27 The variable interpretation of expressions used in medicine has been well documented, most notably with regard to interpretation of probabilities by physicians.32–35 Some individual gout guidelines however used different deontic terms intending to convey the same SOR. We think guideline developers should make explicit the connection between deontic terms chosen and their intended level of obligation, for example, to link deontic terms to grades of recommendation strength, to avoid subjectivity of the audience in interpreting recommendations. The number of deontic terms used would thus depend on the particular grading system applied by the guideline developers.
In summary, the structure and content of recommendations depend on the clinical questions, but recommendations should be clear, understandable and actionable by the intended audience. The wording must be concise, unambiguous and easy to translate into practice when writing recommendations, and consistent language is necessary to avoid misunderstanding.
Detailing and explaining the recommendations and their rationale
Recommendation statement (ie, rationale and explanation of the recommendations), a necessary supplement for recommendations, can extend and explain the recommendations in detail and make the guideline audience better understand and implement the recommendations. According to the GIN-McMaster Guideline Development checklist, it was important to include remarks that describe the context, feasibility and applicability of the recommendation and highlight key considerations such as equity issues and specific conditions that might apply to the recommendation.26 We gained the following enlightenment from the recommendation statement in the gout guidelines we reviewed.
First, guidelines should extend and explain the recommendations. The details of recommendations should be fully described, such as the intended population, dose of drugs and other precautions. Second, guidelines should provide and list the evidence supporting the recommendations. Each recommendation should be linked to a summary of the evidence,36 which we suggest to summarise and report with tables, including the review results (type and quality of studies) and main findings (relative and absolute effects) of evidence. The rationale and approach used to generate recommendation (eg, based on expert advice) need to be clearly reported when no evidence is available. Third, guidelines should describe the factors taken into consideration when generating the recommendations. A systematic and normative reporting of this part was lacking in gout guidelines, which affects the transparency of the developing process of the recommendations. We therefore suggest that guideline developers clearly report which factors are taken into consideration to generate recommendations, and how these factors contribute to the process of recommendation development. Finally, we also advise to report each section aforementioned in a separate paragraph and avoid embedding all of them within a long paragraph for an easy identification and distinction.
In summary, we suggest that each recommendation should be followed by a clear and detailed statement, which should include a clear explanation for recommendations, the evidence supporting recommendations and any factors that influence recommendations.
Describing and providing other potential useful contents
Assisted production may also be an important part when reporting recommendations. For example, an algorithm map can clarify and visualise the specific implementing procedure of recommendations.7 The algorithm maps should include SOR, QOE and other matters needing attention, and they should be able to provide basic execution steps for clinical practice without the aid of recommendations in the full text. Making a list to compare the strengths and weaknesses between relevant interventions is another example, which can provide direct reference for the audiences to assist in decision making. Details regarding the drugs such as their usage and cost, adverse reactions, and the ways to how to access to them, can be listed in such table. Guideline developers certainly can also design and use other appropriate materials based on their own guidelines’ characteristic to improve the effect of reporting recommendations.
There are some specific points that need to be considered when reporting and presenting updated versions of guidelines. Guideline developers should clearly present and label each recommendation as new, modified or unchanged, and deleted recommendations should be also mentioned. Differences between old and new recommendations should be reported and justified in the section of recommendation statement.37
In summary, algorithm map, summary tables of drugs and any other potential materials should be designed based on the guidelines to facilitate understanding of recommendations.
Strengths and limitations
We systematically searched and assessed eligible CPGs for gout. The study is the first to have fully analysed each area of recommendation reporting and presentation in gout CPGs. The findings of our study can draw a framework for RIGHT-R and provide strong support to form the final checklist.
Our study also had some limitations. First, we included only CPGs on the theme of gout and written in Chinese or English, but CPGs in other languages or other fields were not assessed. Second, we only analysed the reporting and presentation of drug intervention recommendations and more studies are expected for other types of recommendations (eg, recommendations about diagnosis). However, we will continue taking different samples to extend and strengthen our findings.
Conclusions
Variability and inconsistency were found on the reporting and presentation of recommendations in gout CPGs. The RIGHT-R working group is developing a checklist on how to report recommendations in an attempt to promote standardisation and adequacy of reporting recommendations in CPGs.
Supplemental material
References
Footnotes
NY and YY contributed equally.
Contributors YTY, NY and YLC designed the study. CLQ, QZ and JYZ retrieved the literature. YY and AQZ screened the literature and extracted data. YY, NY, AQZ, JE, XQW, MFZ, YFM and QW conducted data analysis and interpreted the results. YY and NY drafted the manuscript. YTY, YLC and XFL provided methodological support. All authors critically reviewed and revised it for important intellectual content. YY and NY are the guarantors of the manuscript and affirm that it is an honest, accurate and transparent account of the study being reported. All authors approved the final version of this article.
Funding This work was supported by Research Funds for Evaluation and Transformation of Clinical Evidence from Dominant Diseases of Traditional Chinese Medicine in Gansu Province (grant 2016-RC-1; Dr. Yang), the Translation and Training Cooperation Base for Standards of Traditional Chinese Medicine of Belt and Road Initiative (grant GZYYGJ2018042; Mr. Chen). All these funding had no role in the design of the study and collection, analysis and interpretation of data and in writing the manuscript.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.