Article Text
PDF
Abstract
Introduction Peripheral arterial disease (PAD) is the third leading atherosclerotic arterial disease. There is evidence that there is a high variation in the quality and recommendations of clinical practice guidelines for PAD, leading to the possibility of confusion among clinicians and patients. This study aims to conduct a quality assessment and comparative analysis of the clinical practice guidelines on PAD written between 2010 and 2020.
Method and analysis We aim to perform a systematic review of clinical practice guidelines written between 2010 and 2020. A search for guidelines will be conducted through medical databases Scope, Pubmed, TRIP, Guideline Clearinghouses and specialist international organisations’ specific websites. Guidelines that meet the inclusion criteria will be extracted from the search result. The Appraisal of Guidelines for Research and Evaluation II (AGREE-II instrument) will assess the quality of the selected guidelines. The recommendations, level of evidence and other relevant information will be extracted in a datasheet for qualitative analysis. The score for each guideline’s quality will be represented using charts and central tendency measures for comparison. The summary of recommendations will also be represented in tables for easy comparison for similarities and variations across sections. Finally, the level of evidence on which the recommendations are based will also be noted along with other significant characteristics such as the authors’ financial relationship to the biomedical community. We aim to point out deficiencies present in current guidelines and elucidate areas where recommendations are made with low-level evidence. The results will enable the scientific community to design future research to fill in PAD management knowledge gaps.
Ethics and dissemination No ethical approval was sought. Dissemination will be via journal articles and conference presentations.
PROSPERO registration number CRD42020219176.
- adult cardiology
- protocols & guidelines
- vascular medicine
- vascular surgery
- pharmacology
- vascular medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- adult cardiology
- protocols & guidelines
- vascular medicine
- vascular surgery
- pharmacology
- vascular medicine
Strengths and limitations of this study
This review will be exhaustive, critically appraising all aspects of clinical practice guidelines (CPGs) for peripheral arterial disease (PAD), from the quality, through screening and diagnosis, to all aspects of treatment (pharmacological and non-pharmacological), and to our knowledge, previous reviews have only reviewed aspects of the CPGs, so this will be the first all-encompassing review.
In the previous reviews, Ferket and colleagues focused on reviewing recommendations on screening across the PAD guidelines, Barriocanal and colleagues focused on reviewing the quality of PAD CPGs and, finally, Chen et al reviewed guideline quality and recommendations across screening and pharmacological aspects of PAD management; however, this review will not be limited in that regard and as such will include quality assessment, and recommendations across screening, diagnosis and investigative evaluations, pharmacological and non-pharmacological interventions.
Our review will focus on the most recently written CPGs, that is, those written within the last 10 years with an expectation that the more recent advancements revealed through clinical trials and improved standards for writing CPGs should reflect in this work when compared with the previous reviews.
This study, being a systematic review that will not involve patient recruitment and using a qualitative methodology, will be cost-effective, furthermore, since a qualitative approach is to be used for the CPG recommendation synthesis is that the underlying reasons for variations can be explored in detail with the potential of exposing knowledge gaps in PAD and atherosclerosis management.
Qualitative analysis is inherently difficult to analyse and summarise especially as there is so much information contained in the CPGs, as such, distilling all that information into useful summaries will be a daunting task, with the potential loss of vital information that may be difficult to prevent and also, given the large amount of information in the CPGs, data extraction will be a time-consuming process.
Introduction and background
Atherosclerotic vascular diseases remain the world’s leading cause of mortality today despite dramatic declines in trend over the last few decades.1 Epidemiological data show that behind ischaemic heart disease and stroke, peripheral arterial disease (PAD) is the third most common atherosclerotic arterial disease.2–5
Despite its significant contribution to morbidity and mortality globally, there had been a paucity in the number of randomised clinical trials and high-quality systematic reviews of randomised clinical trials on PAD with consequent low-quality recommendations in practical guidelines. Over time, the results of high-powered RCT have been published with others on the way, which we expect to influence the more recent clinical practice guidelines (CPGs). Also, several recommendations in available guidelines were reached via expert consensus.6 It is no surprise that reviews of existing guidelines have revealed variations in PAD treatment recommendations in the past.7 8
CPGs are methodically developed statements aimed at guiding physicians and patients in making safe healthcare decisions based on the best available evidence.9 10 The last 30 years have witnessed a skyrocketing in developing CPGs,11 calling into question quality issues; consequently, several reputable organisations have continued to improve the standards for CPG developments.12–14 Ideal CPG recommendations are based on strong evidence.15 However, high-level evidence is often unavailable for specific situations for several reasons, giving room for introducing various forms of bias with consequent variations in recommendations across various CPG developers for the same clinical scenario.16
Literature search reveals high interest among academics in reviewing CPG’s quality for their specialty areas with numerous studies on the topic. Interestingly, very few reviews have been conducted regarding the CPGs available for PAD. Also, to our knowledge, the available reviews focused on aspects of the PAD guidelines such as reviews on screening recommendations,17 reviewing the quality of the CPGs and reviewing the pharmacologic recommendations.18 19 In this study, we aim to conduct a more exhaustive review of the most recent guidelines (written in the last 10 years).
In 2012, a systemic review was conducted on eight guidelines published between 2003 and 2011, comparing their quality and recommendations for PAD screening. The study results revealed that the majority of the guidelines favoured screening for PAD. However, three guidelines did not advocate for PAD screening due to the absence of appropriate clinical trials. The studies were considered inappropriate because the available clinical trials were conducted on individuals with established PAD and were unsuitable to be the basis for clinical advice for the general population. The guidelines’ quality was also assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) tool, with results revealing a range between 33% and 81%.17
Further work was done on this topic by another team of researchers who reviewed seven guidelines written between 2006 and 2012. The study focused on the quality of the guidelines using the AGREE-II instrument. Their results revealed a significant variation similar to the 2012 study with a range of 45%–72%. The reviewed CPGs were found to have high scores in clarity and editorial independence but with low scores in applicability and rigour of development. In their recommendations, they stated that only one CPG could be recommended for use without modification.19
The most recent review assessed CPGs written between 2000 and 2017. This study was a more exhaustive review. They assessed the guideline quality using the AGREE-II instrument and the recommendations across screening and pharmacological management. The result revealed a quality range between 39% and 73%, similar to the earlier reviews. However, this work found the CPGs to have low scores in the rigour of development (similar to the previous study) and editorial independence (unlike the previous study where they scored high marks). This difference may be because Chen and colleagues reviewed more CPGs. It was also observed that just two of the CPGs reached the standard for conflict of interest from the Institute of medicine. Regarding the screening recommendations, 8 guidelines out of 14 recommended screening (at different strengths) while the others stated insufficient evidence or were against it. Treatment recommendations also showed conflicts concerning target values for lipid-lowering and antiplatelet therapy.18
In summarising these findings, the PAD guidelines show considerable variation in quality and variations in their recommendations. The paucity of high-quality research could explain these variations for the specific topics for which recommendations are needed, prompting the need for reliance on lower strengths of evidence such as expert consensus or research conducted on established disease participants. Clearly, there is a knowledge gap that can easily be filled with the right form of interest from the research community.
The rationale for the study
Systematic reviews of CPGs are used to systematically identify, assess and summarise the current state of guidance on a clinical topic. Well-written systemic reviews that adhere to a rigorous methodological approach and use transparent reporting to identify knowledge gaps where improvement in current recommendations can be achieved.20
The previous reviews on the CPGs for PAD have revealed a wide variation in the quality and variations in screening and pharmacologic management recommendations.17–19 However, these reviews were restricted in their comparators, focusing on aspects of the CPGs rather than performing a more holistic review.
Furthermore, the previous reviews included CPGs written over a wide range of time. Advancements in treatment options of atherosclerotic diseases have advanced considerably in the last decade, with consequent paradigm shift occurring after the results of relatively recent randomised clinical trials. We expect that this will be reflected in more recent CPGs compared with their older counterparts.
The findings of this review will be compared with those of the previous reviews. Significant areas of interest, such as changes in overall quality over time and changes in the strength of pharmacological management recommendations, will be made manifest. Also, a nouvelle comparison of non-pharmacological management will be conducted across the guidelines.
Aim
A quality assessment and comparative analysis of the CPGs on PAD written between 2010 and 2020 to assess the quality of the CPGs and identify the gaps in evidence as reflected by the nature of their recommendations.
Objectives
To compare the quality of the CPGs on PAD written between 2010 and 2020 using the AGREE-II instrument.
To compare the recommendations for screening for PAD across CPGs on PAD written between 2010 and 2020.
To summarise the recommendations for pharmacologic management across CPGs on PAD written between 2010 and 2020.
To critically appraise the non-pharmacologic recommendations across CPGs on PAD written between 2010 and 2020.
To collate and contrast the follow-up recommendations across CPGs on PADs written between 2010 and 2020.
Methodology
Patient and public involvement statement
Patients who are members of the Peripheral Arterial Diseases Support Group (https://www.facebook.com/groups/pad.pvd.support/members) were involved in this study’s design (in modelling the research objectives) and will be involved in the study when it commences. The Way to My Heart.org (https://www.thewaytomyheart.org/) founded this support group. The patient public involvement will be coordinated through the group’s leaders/founders (also patients themselves are actively involved in providing support to their fellow patients) who are advisory members to the research team. They have identified this research as a priority area for clinicians who provide care to patients living with PAD. The group members will be informed of this study’s results through their group page on Facebook in a newsletter suitable for a non-specialist audience. The patients and public will also be sought in the development of an appropriate method of dissemination.
Guideline identification
A systematic search will be conducted, and eligible guidelines selected based on the attributes listed in table 1. These selected guidelines will be comparatively assessed across quality and recommendations. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement will be used as a reference to report items and results in this review.21
Population, Clinical Indication, Comparators, Attributes of Eligible guidelines, Recommendation characteristics (PICAR) statement
One reviewer will perform the search and extraction for recommendations, which will be validated by another reviewer. A third reviewer will be consulted to resolve disagreements if they arise. The AGREE-II instrument will be used to assess the quality of the selected guidelines by four reviewers. One reviewer will extract the recommendations, and another reviewer will validate this.
Search strategy
A systematic search will be performed to identify relevant CPGs on PAD. A concept table will be used to generate appropriate search terms (MeSH, Free text vocabulary, Key Words) depending on the database’s peculiarities.
The searches will be conducted on the following databases:
Example: A draft of the search strategy for PubMed via MEDLINE.
Arterial Disease, Peripheral.
Arterial Diseases, Peripheral.
Disease, Peripheral Arterial.
Diseases, Peripheral Arterial.
Peripheral Arterial Diseases.
Peripheral Artery Disease.
Artery Disease, Peripheral.
Artery Diseases, Peripheral.
Disease, Peripheral Artery.
Diseases, Peripheral Artery.
Peripheral Artery Diseases.
Peripheral Arterial Disease [MeSH].
Intermittent Claudication [MeSH].
Limb Ischemia.
1 OR 2OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14.
Screening.
Treatment.
Management.
Diagnosis.
Pharmacological.
Diagnosis[MeSH Terms].
Therapy[MeSH Terms].
16 OR 17 OR 18 OR 19 OR 20 OR 21 OR 22.
Guidelines.
Guideline.
Standards.
Practice guideline[MeSH Major Topic]
24 OR 25 OR 26 OR 27.
Quality.
Recommendations.
Quality improvements[MeSH Terms]).
29 0R 30 OR 31.
15 AND 23 AND 28 AND 32.
The search was conducted on 11 December 2020. Result: 7014 references.
Guideline selection
The extracted references will be searched through the title and abstract for guidelines that meet the inclusion and exclusion criteria outlined below. This selection will be done by the lead researcher and verified by another researcher. Conflicts of ideas will be resolved by consensus by taking a third researcher’s opinion to minimise selection bias risk.
Inclusion criteria
The guideline is developed for people with PAD.
The guideline covers recommendations regarding screening, non-pharmacological and pharmacological interventions.
The guidelines were written between 2010 and 2020.
The guideline is the most recent version.
The guideline is available online.
Related or international academic organisations wrote the guideline.
Exclusion criteria
The topic is only mentioned in the guideline.
The guideline is limited to a specific aspect of PAD management, such as screening, pharmacologic management, etc.
Outcomes: the outcomes in this study are
Guideline quality.
Guideline recommendations.
Quality assessment
Instrument
The updated AGREE-II instrument (online supplemental appendix 1) will be used to assess the quality of the selected guidelines.22 The AGREE-II instrument is a 23-item tool with international certification that evaluates the six domains of methodological quality of a guideline, including scope and purpose, stakeholder involvement, rigour of development, clarity of presentation and applicability and editorial independence.23 The assessment will be conducted by four reviewers (as recommended by the developers of the tool to minimise bias) using the instrument to assess all selected guidelines. The reviewers will score each guideline across each domain on a Likert scale of 1 through 7 (from strongly disagree to strongly agree).
Supplemental material
In addition, the reviewers will give an overall score of the guidelines on a similar Likert scale. As such, each guideline will have two sets of scores: (1) the domain scores and (2) the overall score for the guideline.
Scoring
Domain scores are calculated by summing up all the individual items’ scores in a domain and scaling the total percentage of that domain’s maximum possible score. The example on scoring below was extracted from the user manual.
To give an example, if four appraisers give the following scores for domain 1 (dummy scores are generated for scope and purpose in table 2):
Example of domain scoring for scope and purpose
Maximum possible score=7 (strongly agree) × 3 (items) × 4 (appraisers)=84.
Minimum possible score=1 (strongly disagree) × 3 (items) × 4 (appraisers)=12.
The scaled domain score will be:
Interpreting domain scores
There are no fixed cut-offs for high-quality or low-quality guidelines set by the instrument developers. The scores of the domains will be compared against each other between the guidelines. The overall assessment will be arrived at using the domain scores, and for this purpose, we have decided to set out cut-offs in line with the study conducted by Chen and colleagues because of its practicability.18 If most (four or more) domains scored over 60%, a guideline would be regarded as ‘strongly recommended for use in practice’; if scores of most domains (four or more) ranged 30%–60%, the guideline would be regarded as ‘recommended for use with some modification’; if most of the domains (four or more) scored less than 30%, the guideline would be regarded as ‘not recommended for use in practice.’
Interpreting the overall guideline scores
This will be used as an additional matrix for assessing the guideline as a supporting statistic. It will not provide any direct contribution for the final assessment into high-quality or low-quality guidelines.
Data extraction and management for quality scores
The data from each appraiser for the AGREE instrument will be entered into an initial excel sheet for upload into SPSS V.22 for analysis. The four appraisers’ scores will be aggregated within the SPSS datasheet in line with the formula highlighted above. The final scores, which will be used to generate the recommendation for using the guidelines, will be presented in the Results sections. The preliminary datasheet templates are attached below (see online supplemental appendix 2).
Supplemental material
Guideline recommendations
Recommendations extraction
A recommendation matrix will be developed based on the focus areas of the data synthesis in line with the research objectives. The recommendations will be extracted across screening, pharmacological and non-pharmacological treatment modalities for comparative assessment. Systematic methodology will be employed to harmonise specific details of the guidelines, which may vary due to differences in terminology or differences in interventions/comparators. For example, recommendations will be harmonised into themes (thematic analysis), which can then be coded and entered into the software/datasheet.
Particular interest will be paid to the level of evidence on which the recommendations are based. A preliminary review of some guidelines shows variations in the grading system for the level of evidence. The evidence grading schemes for each guideline will be harmonised and standardised to enhance the data synthesis process. Evidence categories will be developed using an iterative process of refinement through discussions within the review team.
Other characteristics of interest, such as the data aimed at evaluating the financial relationship between guideline producers and the biomedical industry, and others outlined in the PICAR statement, will be extracted. A preliminary version of what the data extraction sheet will appear like is attached below, highlighting all the variables that will be extracted (see online supplemental appendix 3). As the study progresses, the datasheet is bound to evolve to fit the study’s objectives better.
Supplemental material
Recommendation data management
The recommendation data will be extracted using Nvivo software for qualitative data extraction and management. The extracted information will be summarised through qualitative/thematic analysis. The variables of interest are listed in table 3.
List of variables
Results
Flowchart of search strategy.
Results of quality assessment using AGREE-II represented by bar charts/histograms, also +overall recommendations.
A tabular summary of screening recommendations for PAD.
A tabular summary of non-pharmacological recommendations for PAD.
A tabular summary of pharmacological recommendations for PAD.
Additional relevant information on the guidelines.
The study is proposed to be completed within a period of 26 weeks, with dedicated attention from all participants. The activity breakdown and allotted time for each activity are shown in table 4.
Timeline
Significance of the study
This study’s significant finding will be identifying low-grade recommendations in the available guidelines (recommendations based on low-level evidence). The only way to remedy this situation is for researchers to conduct appropriate-sized randomised controlled trials tailored to answering the recommendations’ problems. These shortcomings will be highlighted in the results and discussions, paving the way for improved PAD CPGs in the future.
The results of this study will also serve as a guide for future CPG writers to pay attention to all aspects of CPG development, especially domains where they performed poorly in the quality assessment using the AGREE-II instrument.
Ethics and dissemination
Because this is a systematic review and no human subjects, we do not see the need to seek ethical approval.
We aim to disseminate this work through a journal publication and conference presentation. The work will also be disseminated through our Patient and Public Initiative Network.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors ODU conducted the initial concept and design, planning and conduct, the conceptualisation of data acquisition, management and analysis, literature search and initial draft write up, editing and funding. CO contributed to the concept and design, planning and conduct, the conceptualisation of data acquisition, management and analysis, literature search, editing and funding. JI contributed to the concept and design, planning and conduct (contributing expert insight—cardiology), writing and editing the draft, and funding. EC contributed to the concept and design, planning and conduct (contributing expert insight—haematology), writing and editing the draft, and funding. OE contributed to the concept and design, planning and conduct (contributing expert insight—clinical pharmacology), writing and editing the draft, and funding. OO contributed to the concept and design, planning and conduct, writing and editing the draft, and funding. AA contributed to the concept and design, planning and conduct (contributing expert insight—lifestyle medicine), writing and editing the draft and funding. All authors read the manuscripts and made important intellectual content contributions. All authors also approved the final manuscript before submission.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.