Article Text
PDF
Abstract
Objectives Evidence-based clinical guidelines play an important role in healthcare and can be a valuable source for quality indicators (QIs). However, the link between guidelines and QI is often neglected and methodological standards for the development of guideline-based QI are still lacking. The aim of this qualitative study was to get insights into experiences of international authors with developing and implementing guideline-based QI.
Setting We conducted semistructured interviews via phone or skype (September 2017–February 2018) with guideline authors developing guideline-based QI.
Participants 15 interview participants from eight organisations in six European and North American countries.
Methods Organisations were selected using purposive sampling with a maximum variation of healthcare settings. From each organisation a clinician and a methodologist were asked to participate. An interview guide was developed based on the QI development steps according to the ‘Reporting standards for guideline-based performance measures’ by the Guidelines International Network. Interviews were analysed using qualitative content analysis with deductive and inductive categories.
Results Interviewees deemed a programmatic approach, involvement of representative stakeholders with clinical and methodological knowledge and the connection to existing quality improvement strategies important factors for developing QI parallel to or after guideline development. Methodological training of the developing team and a shared understanding of the QI purpose were further seen conducive. Patient participation and direct patient relevance were inconsistently considered important, whereas a strong evidence base was seen essential. To assess measurement characteristics interviewees favoured piloting, but often missed implementation. Lack of measurability is still experienced a serious limitation, especially for qualitative aspects and individualised care.
Conclusion Our results suggest that developing guideline-based QI can succeed either parallel to or following the guideline process with careful planning and instruction. Strategic partnerships seem key for implementation. Patient participation and relevance, measurement of qualitative aspects and piloting are areas for further development.
Trial registration number German Clinical Trials Registry (DRKS00013006).
- quality in health care
- qualitative research
- clinical audit
- protocols & guidelines
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This qualitative study explores facilitating and hindering factors for the whole process of developing and implementing guideline-based quality indicator (QI) including context aspects of the guideline groups position and cooperation possibilities in the respective healthcare setting.
Following purposive sampling of European and North American guideline institutions in different settings, we interviewed guideline methodologists and clinical guideline authors covering a broad range of expertise.
We provide some practical solutions to overcome challenges reported by the interviewees.
However, the number of interviewees and guideline organisations were limited. Supplementary qualitative research with guideline groups is recommended, focusing on structural and methodological insights for realising patient relevant QI and considering qualitative aspects as shared decision making.
Background
Evidence-based clinical guidelines play an important role in healthcare and can be a valuable source for quality improvement ideally monitored by guideline-based quality indicators (QIs).1–3 An QI can be defined as a ‘measurable element of practice performance for which there is evidence or consensus that it can be used to assess the quality, and hence change in the quality, of care provided’.4 However, the link between guideline recommendations and QI is often neglected and international methodological standards for the development of guideline-based QI are still lacking.5–7 To enable an international comparison and harmonisation of approaches, the Performance Measures Working Group of the Guidelines International Network (G-I-N) published reporting standards for guideline-based performance measures in 2016.8
In the context of a broader research project aiming to guide the development of an evidence and consensus based methodological standard for the development of guideline-based QI in Germany (DFG Project No. 289625106; https://gepris.dfg.de/gepris/projekt/289625106),9 we here report an international qualitative study. Further subprojects included an analysis of existing German guideline-based QI,10 qualitative interviews with German authors from guidelines with and without development of guideline-based QI,11 a comparison of international and German guideline-based QI published on the same topic12 and a systematic compilation of existing assessment instruments for QI. The results of this study will be shared within the respective G-I-N working group with the aim to contribute to an international methodological standard.
A systematic search conducted in preparation of the study up to January 2017 yielded no qualitative publications reflecting on the whole methodological process for developing and implementing guideline-based QI.13
Aim
This study sought to contribute to the following research questions:
What are the implicit and explicit processes in the development of guideline-based QI?
What are the international experiences in the development of guideline-based QI?
Which factors hinder or facilitate the development of guideline-based QI from clinical guidelines internationally?
In addition, we explored whether and how guideline-based QI were implemented.
Methods
We chose a qualitative study design to capture experiences and reflections of clinical and methodological experts who have been involved in the development of guideline-based QI using qualitative content analysis, following Mayring.13 A study protocol was developed14 and the study was registered at the German clinical trials register. Study reporting was chosen according to the Consolidated Criteria for Reporting Qualitative Research developed by Tong et al15 (see online supplemental appendix 1).
Supplemental material
Researcher characteristics
The study was mainly carried out by three researchers. The senior researcher (MN) has a clinical and public health background with >10 years of experience in guideline and QI development and is first author of the G-I-N reporting standards. The second researcher (MB) has a social science and public health background with previous experience in qualitative research but not in QI development. The third researcher (MS) has a social science and healthcare management background (for other authors see the Declarations section).
Sampling of interview participants
We used a broad definition of ‘guideline-based’ QI, including any QI related to an evidence-based guideline. First, international guideline organisations who had developed guidelines with QI were identified based on guidelines from a systematic review comparing QI from German and corresponding international guidelines12 and a survey on guideline-based QI development among G-I-N members carried out November 2016. The respective guideline organisations were categorised according to
Country.
Operating level (national, regional and local).
Funding sources (public/private).
Interviewees were selected aiming at purposive maximum variation sampling in relation to guideline development organisations and healthcare settings to reflect a wide range of experiences.16 17 For each organisation, the aim was to interview both a clinician and a methodologist, who have worked on guideline-based QI, to consider possible different perspectives. The sample size was planned with 16 on the assumption to interview participants of at least two public and two private organisations acting at the local/regional or at the national level in different countries with the aim to expand the sample according to thematic saturation. Interview participants were identified either via guideline documents or through contact with research coordinators at the respective guideline organisations. Interviewees were contacted by MB via email and followed-up via e-mail and phone, interviews were conducted via phone or skype. MB did not have prior existing relationships with any participant. On first contact, possible interview participants were informed about the aims of the study and were sent a topics overview with the possibility to decline. All interview participants gave written informed consent.
Interview questions
An interview guide was developed by MS, MB and MN (see online supplemental appendix 2). Based on the G-I-N reporting criteria8 it comprised a mix of open and a few closed questions focussing on methods, experiences and facilitating/hindering factors in the different stages of QI development from guidelines and the role of the interviewee in QI development and QI implementation. The guide was adjusted after a test interview with an experienced guideline and QI methodologist from the UK and followed for all interviews with slight adaptations to the context and follow-up questions where appropriate. To prepare for the interviews MB reviewed the publicly available information on the selected QI and the development process. Interviews were conducted by MB.
Supplemental material
Patient and public involvement
Patients were not directly involved in this study, but the perception of patient involvement, patient contributions and patient relevance of the resulting QI were aspects addressed in each interview.
Data analysis
Interviews were audio-recorded and transcribed verbatim with pseudonymised analysis using qualitative content analysis. Figure 1 gives an overview of the different steps: first, the transcribed interviews were read by MB and MN to get familiar with the content. Unclear passages could be resolved by verifying the audio-records. Then, a coding grid was developed with deductive main codes using the G-I-N reporting criteria8 and ‘implementation (of guideline-based QI)’ as a framework for analysis. Coding was first done by MB using the qualitative research software MaxQDA.18 During the coding process the deductive codes were supplemented with inductive subcodes using phrases or little text segments as unit (see figure 2 as an example of a coding tree). New main codes were assigned for aspects not covered. Coding was reviewed by MN and discussed and agreed between MB and MN To ensure intercoder reliability, a sample of six interviews was coded by both researchers and compared for consistency with a resulting high level of concordance. The analysis followed next iterative categorisation according to Neale,19 which allows for a systematic summary and categorisation of themes directly linked to the original data source. Iterative categorisation was carried out by MB, reviewed by MN and discussed with final agreement. Coding grids were analysed per main categories and summarised including facilitating and hindering factors (see online supplemental appendix 3 as example). The main categories were finally grouped into main themes.
Supplemental material
Steps of analysis: coding with iterative categorisation. QI, quality indicator.
Example of coding tree (analysis step 1).
Results
Sample characteristics
Potential participants from 8 organisations of the 11 identified by Becker et al12 were initially contacted, three did not respond. For each non-responder, an alternative organisation was identified and could be recruited. Fifteen semistructured interviews were carried out with eight methodological and seven clinical experts of four organisations in European and four organisations in North American countries (see table 1 for characteristic of organisations and interviewees, for one organisation none of the contacted clinicians was available). All interviewees had experience in guideline development. Most of the methodologists and half of the clinicians had previous experience in QI development and were involved directly in QI development whereas some clinicians took a more consulting role. The interviewees from an organisation operating at the local level were not involved in QI development as the indicators selected were taken from a third party. Interviews lasted 25–60 min and were largely variable in the level of details describing the QI development processes.
Characteristics of guideline organisations and interviewees
Processes, experiences and factors hindering or facilitating the development of guideline-based QI
Following iterative categorisation, four main topics were found to be most important for QI development methods from guidelines: (i) organisation/context of guideline and QI development process, (ii) panel composition and decision making, (iii) QI selection criteria/attributes and (iv) intended use and implementation of QI. Table 2 provides an overview with representative quotations and suggestions for approaches according to facilitating factors outlined in the interviews as well as areas for further development. In the following we summarise processes reported as well as experiences of the interview participants for each main topic including facilitating and hindering factors.
Main topics of QI development processes with representative citations and suggested approaches as outlined by interviewees
1. Organisation/context of guideline and QI development process
The organisation/context of guideline and QI development emerged as one important theme in the interviews leading to an additional main code. Interviewees reported very different ways of organising the development of guideline-based QI. Three organisations developed guideline and QI in one process. The QI development team was identical to the guideline team or consisted of a subgroup or was led by methodologists with input from the guideline panel. Four organisations developed the guideline first and the QI in a later process with a different team, although individual experts may have been involved in both processes. In one case, a guideline organisation acting at the local level solely adopted existing QI developed by third party groups related to the guideline topic. Finally, one organisation developed informal audit measures with the guideline and national QI were developed in a separate process led by a national group. When guidelines and QI were developed simultaneously, the need to consider QI development early on from the beginning of the guideline process rather than towards the end was considered an important planning factor. To drive QI development forward in a guideline organisation, a dedicated person with QI experience was deemed crucial by several clinicians and methodologists.
Institutional leadership and capacities/resource aspects
A lack of interest and a disjunction between the clinical and political sphere were seen hindering the QI development, whereas strong institutional leadership and capacities were seen favourable. Interviewees from organisations with public and private funding sources stated that resource constraints hampered QI development in general or for certain aspects, for example, piloting. Many concluded that government funding (national or regional) was conducive to QI development. Interviewees judged the QI process to be time-consuming. Time constraints were a hindering factor identified by several interviewees, clinicians and methodologists.
2. Panel composition and decision making
Most interviewees—methodologists and clinicians—stressed the importance of selecting an QI panel that represents different professional groups and settings, is balanced in its representation of interest groups and has a high level of knowledge about QI.
Not including certain representatives was perceived negatively by several interviewees for the risk of missing perspectives and knowledge, whereas others stated, a small panel size was preferable for the consensus process. Clinicians and methodologists deemed both, clinical and methodological expertise as beneficial, especially for specifying QI in detail to ensure content validity and feasibility. Concerns were raised by some about personal biases of panel members.
Concerning patient participation, half the interviewees stated that patient involvement was crucial. On the other hand, it was argued that most patients on the panels were not ‘vocal about measurement’ and some careful selection or prior education on measurement was necessary. Clinicians and methodologists shared similar views.
Decision-making process
There was no agreement on the ideal decision-making process. Several interviewees were advocates of formalised forms of consensus, some favouring a written Delphi process, which was considered easy to understand and implement, allowing for consideration of all opinions before final consensus. However, an equal number of interviewees considered informal table discussions as sufficient, allowing for more active discussion and debate. Clinicians and methodologists reported both difficulties in reaching consensus predominantly concerning feasibility of guideline-based QI, for example, regarding measuring shared decision making as well as acceptance of QI due to potential economical judgements of payers or health authorities.
Collaborating with future implementers
A collaborative approach that involves feedback loops with clinical stakeholders in the field was emphasised by several interviewees as facilitating factor for QI development and implementation. Pilot testing was seen by clinicians and methodologists an essential opportunity for adapting QI based on feedback from implementers, although it was realised only in about half of the organisations due to time and resource constraints.
3. QI selection criteria/attributes
Methodologists and clinicians mentioned a wide range of QI selection criteria and attributes. Whereas interviewees of some organisations named criteria agreed by national institutions, others reported on criteria specific for their institution. Some clinicians referred to the methodologists for details.
Scientific soundness: evidence base as key
One main prerequisite for high quality QI seen by the interviewees was a strong evidence base. Clinicians and methodologists pointed out that up-to date and methodologically rigorous developed evidence-based guidelines as a basis increase QI acceptability and implementation. The GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach was deemed helpful to assess the certainty of the evidence and develop measures from recommendations with a robust evidence base. It was cautioned that gaps in current evidence as for rare diseases or weak evidence and weak recommendations themselves can hinder QI development and interpretation. In addition, some interviewees identified a time lag between the current evidence, the QI development and dissemination as a challenge which caused QI to be outdated quickly and not reflect current practice.
Relevance: need-based planning and relevance to patients
Several clinicians and methodologists considered carrying out a needs assessment prior to guideline and QI development crucial to identify key areas for improvement in care rather than focussing solely on literature driven recommendations.
QI were described mostly as structure or process related and perceived less directly relevant to patients than outcome measures. One interviewee argued that patient relevant QI would require a separate set of indicators as the physician or system perspective would differ a lot from the patient perspective. Clinicians and methodologist deemed the GRADE approach as helpful as it includes prioritising patient relevant outcomes from the beginning of guideline development. One interviewee identified a lack of resources as a hinderance to carrying out more patient oriented QI.
Measurability remains a challenge
A lack of measurability was raised by clinicians and methodologists as one of the main challenges in QI development. Over half the interviewees stated that important guideline recommendations, particularly related to qualitative and individualised aspects of care, were difficult to measure and often not reflected in current QI. Some interviewees, mostly methodologists, argued that it was difficult to develop measurable QI directly linked to patient relevant outcomes and judged the expense and effort of measuring patient reported outcomes as not feasible. A pivotal unsolved problem was seen by some clinicians and methodologists in measuring shared decision making. In line with this, several interviewees emphasised that measurable QI do not necessarily reflect the actual quality of care and should be cautiously interpreted.
In addition, a lack of infrastructure, related to IT-systems, data availability or collection capacities was reported as still hindering the development of measurable QI meeting core methodological criteria. Some interviewees also called attention to capacities of clinicians. Clinicians and methodologists argued that the use of preferably pretested existing measures can reduce the burden of data collection, save costs and guarantee that measures are feasible and valid.
Consideration of specification and pilot testing
Another challenge reported, was to develop QI that are specific but also applicable in different contexts. Clinicians and methodologists emphasised that guideline recommendations needed to be worded precisely and unambiguously to facilitate QI development. However, interviewees of one organisation decided not to determine numerator and denominator to encourage local adaptability and wide-reaching implementation whereas interviewees of other organisations argued that for an QI a clear idea of what the indicator should achieve, consideration of outliers/risk adjustment and pilot testing was needed to determine and refine the numerator and denominator.
Pilot tests were deemed by most interviewees essential for new measures or settings to improve clarity of definitions and test feasibility, validity and applicability, although only half of the interviewees stated that they were done.
4. Intended use and implementation of guideline-based QI
All interviewees named ‘quality improvement’ or ‘quality assurance’ as intended use of the QI including two mentioning ‘discouragement of overuse’. Other purposes stated were information of clinicians and/or policy, identifying gaps in current care or assessment systems, assessment of guideline implementation, accreditation and benchmarking. Additionally, interviewees of half of the organisations stated reimbursement as one QI purpose. A clinician of a privately funded guideline organisation stressed the fact that a motive to develop guideline-based QI was to get quality assurance more influenced by clinicians. He and other interviewees of public funded organisations highlighted that they could not be sure if their QI were used. One clinician did not agree with the intended use of public funded QI to centralise care by applying minimum operative volumes.
Most interviewees said that the intended use needed to be considered before indicator development. Some—clinicians and methodologists—stated, that the selection and the final use of QI should only be determined after testing, consultation and feasibility checks.
Experiences with implementation were heterogenous. Some interviewees did not see themselves responsible for implementation but deemed information of clinicians and other decision makers as their main contribution. Other interviewees were very eager to actively engage in implementation of their QI in existing systems. Those who worked in a context where QI were implemented were aware—both, methodologists and clinicians—that measurement characteristics could only be assessed after piloting.
Discussion
For this qualitative study, international guideline experts have been interviewed to reveal insights into all development steps of guideline-based QI. Following a strategy of purposive sampling and using a broad definition of guideline-based QI accepting every effort to connect QI and guidelines, we could include a variety of perspectives and experiences from experts of organisations with different scope, funding and role within different healthcare systems. This allowed for the identification of common patterns, feasible approaches and factors facilitating or hindering the development of guideline-based QI. Following qualitative analysis, we identified four main topics, for which we could describe how processes were organised and experienced: (i) organisation/context of guideline and QI development process, (ii) panel composition and decision making, (iii) QI selection criteria/attributes and (iv) intended use and implementation of QI.
The G-I-N reporting standards helped to address relevant QI development aspects for the interview guide.8 The categorisation according to Neale19 was expedient to structure and summarise the comprehensive data.
Interviewing a clinician and a methodologist from each organisation broadened the perspective. Although some clinicians did not remember every step of the QI development process in-depth, as the guideline they had worked on was completed up to 4 years ago, our interviews revealed that clinicians and methodologists shared predominantly same views, except that clinicians had less knowledge on QI attributes and assessed measuring of direct patient relevant aspects easier than methodologists. In one case a clinician did not accept the intended use of the QI.
Our results indicate that QI development connected to guidelines can be organised in different ways depending on the position that guideline developers have within a certain healthcare system as well as how they work together with other stakeholders. This is in line with the systematic review of Kötter et al who reported on different approaches published.7 Methodologists and clinicians from almost all organisations talked far more about organisational and context factors than we had expected. This underlines the importance of carefully setting up guideline-based QI development together with relevant stakeholder of the respective healthcare system.
Kötter et al found the reporting of the QI methodology often lacked transparency.7 The latter was still found by Becker et al.12 Our results suggest that methodological weaknesses in QI development may still be due to a lack of knowledge and expertise and limited financial resources.
Our interviews with international experts unanimously revealed a need to have a dedicated person instructing guideline groups about the QI development process. This was also stressed by German guideline authors.11 In contrast, Blozik et al had found in a survey on simultaneous development of guidelines and QI with 24 participating institutions 2012 that the initiative to develop QI was made predominantly by the guideline coordinator or group itself.20 Thus, our results might reflect an organisational change towards more professionalism in terms of building institutional capacities instead of leaving the initiative to single (guideline) groups.
Interviewees deemed clinical and methodological knowledge important for an adequate QI panel. But the fact that personal biases of panel members were mentioned, highlights the need for a conflict of interest management also for QI development which should be reported on.
The varying perception interviewees had on patient involvement and patient relevance of guideline-based QI warrants further consideration. Kötter et al found only few papers mentioning patients as part of QI-panels.7 21 The recognition of patient centredness as an important dimension of quality of healthcare led to more direct patient involvement in various processes,22 but the critical voices of some interviewees reveal that there is still a need to agree on the role of patients in QI development. When asked about guideline-based QI for chronic heart failure, neither patients nor professional experts deemed the QI elected by the guideline group very patient centred although patients had been involved.23 24 Interviewees in our study supported the importance of ‘relevance’ as a selection criterion, but some highlighted that relevance can be judged differently dependent on the target group (doctors, patients or people responsible for the system). Guideline groups should reflect this and achieve to an agreement about improvement needs, avoiding a mechanical process going from guideline recommendations to QI. As stressed by some, the prioritisation of patient relevant outcomes can help solving this issue.
Most interviewees emphasised a strong evidence base as the most important criterion for guideline-based QI. Reports about flawed QIs in use for public reporting and pay for performance25 indicate a need to establish a broad and common understanding of ‘evidence’ supporting an QI. Scientific soundness of an QI should not only refer to the direct evidence base of the intervention measured (eg, antibiotic therapy for pneumonia) but also to the evidence for further QI characteristics like a required cut-off (eg, ‘door to needle time’ for antibiotics) and consideration of potential undesirable effects when implementing an QI (eg, risk of inappropriate diagnosis of pneumonia and associated risk of overtreatment with antibiotics). Scientific soundness of an QI implies additional measurement characteristics like content validity (do implementers understand the intervention in the same way) or reliability (are there systematic measurement errors). The characteristics and effects of an QI can only be confirmed by a pilot test. Interviewees—methodologists and clinicians—were mostly aware of this, but only half the organisations realised piloting.
In general, putting new measures into practice and make them measurable was—still—seen a big challenge in our study and using existing QI to avoid additional burden of documentation and validation was emphasised. This confirms the need to first appraise measures already in use when starting to develop QI from guideline recommendations.
Especially patient reported outcomes were reported as not yet feasible to measure, even though quality assurance institutions like the US National Quality Forum call since years for measuring those.26
To determine the quality of the underlying evidence of a recommendation, the GRADE approach was deemed useful. Some interviewees though were uncertain how to deal with weak or conditional recommendations for which they assigned shared decision-making processes a greater weight than for strong recommendations, without satisfactory solutions to measure. Knowledge gaps in bringing together evidence-based medicine with awareness of shared decision-making processes and quality measurement were, for example, also reported in a project to implement shared decision making in the UK.27 There are some survey methods described for collecting information about shared decision-making processes, such as measuring, if decision aids are distributed or using patient questionnaires,28 29 which need certainly further scientific development. However, the most important aspect for implementation might be the strategical decision to include shared decision making into quality improvement on the organisational or systems level with the possibility of trialability.29 30
Concerning the intended use of QI, authors should agree for both, existing and new guideline-based QI, on their foreseen use and assessment should include if the QI is suitable for the intended purpose and this should be reported.8 QI improvement initiatives of guideline authors might be subverted, particularly when QI are used mainly to drive forward certain political aims, for example, to centralise care—a risk that may be dependent on the funding source of the respective organisation.
This study has some limitations: first, transcripts or results were not returned to participants for comment or correction. However, we had very clear transcriptions after reviewing doubtful passages with the recorded interviews. Another limitation is, that despite looking for maximum variation, most organisations identified and willing to participate were public funded and acted on the national level with location in Europe or North America. Due to resource constraints the number of interviews was limited, thus it cannot be ensured that we captured all important aspects for each theme, especially concerning the processes and experiences of private funded and local/regional organisations. Nevertheless, we have found an instructive sample concerning the range of views on guideline-based QI.
Conclusion
Our study indicates that a diversity of development methods, realisation and use of guideline-based QI exists and development of guideline-based QI can succeed either parallel to or following guideline development. The results suggest that success will need:
A programmatic/strategic decision to develop and implement guideline-based QI with a dedicated person to lead the process.
The establishment of an adequate panel including clinicians and methodologists to develop guideline-based QI.
The provision of methodological instruction on the development of guideline-based QI.
The need to consider methodological criteria for QI, such as relevance, evidence base and feasibility.
Collaboration and the establishment of strategic partnerships to implement guideline-based QI in a healthcare system.
Further exploration and development are needed concerning the role of patients, direct patient relevance of QI, conflict of interest management of QI panels as well as measurement of qualitative aspects as shared decision making.
Acknowledgments
We would like to thank all participants of the interviews for their time and valuable contributions. We would further like to thank Professor Dr Max Geraedts for methodological advice and supervision of the whole project and him and Dr rer. -medic. Susanne Blödt for the review of this publication.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors The study was mainly carried out by three researchers MN, MB and MS (MN, MS, MB: study planning, study protocol, interview guide; MB: leading interviews; MN, MB: analysis, manuscript preparation) see also the Researcher characteristics section. The other authors were involved in study planning (JS, SD), reviewed the study protocol (KA, MBe, SD, IK, JS), the interview guide and the manuscript (both all authors: MN, MB, MS, KA, MBe, SD, IK, JS).
Funding The study was funded by the German Research Foundation (DFG No. 289625106).
Competing interests MN and IK are employed by the AWMF as methodological advisers for guideline and guideline-based QI development. The AWMF receives a continuous grant from the German Cancer Aid for guideline and guideline-based QI development.
Patient consent for publication Not required.
Ethics approval Ethical approval for the study was given by the ethics commission of the Medical Chamber Berlin/Germany (Berlin Chamber of Physicians No. 31-17).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement The datasets used and analysed during the current study are available from the corresponding author on request.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.