Article Text
PDF
Abstract
Objectives In 2016, the SKINCATCH Trial, a clustered multi-centre randomised trial, was initiated to assess whether low-risk basal cell carcinomas (BCCs) can be treated by general practitioners (GPs) without loss of quality of care. The trial intervention consisted of a tailored 2-day educational course on skin cancer management. The aim of this process evaluation was to investigate GPs’ exposure to the intervention, implementation of the intervention and experiences with the intervention and trial.
Research design and methods Data on exposure to the intervention, implementation and experiences were obtained at several points during the trial. Complementary quantitative components (ie, surveys, database analysis, medical record analysis) and qualitative components (ie, interviews and focus groups) were used. Quantitative data were analysed using descriptive statistics; qualitative data were summarised (barrier interviews) or audiorecorded, transcribed verbatim and thematically analysed using Atlas.Ti (focus groups).
Results Following a 100% intervention exposure, results concerning the implementation of the trial showed that aside from the low inclusion rate of patients with low-risk BCCs (n=54), even less excisions of low-risk BCCs were performed (n=40). Although the intervention was experienced as highly positive, several barriers were mentioned regarding the trial including administrative challenges, lack of time and high workload of GPs, low volume of BCC patients and patients declining to participate or requesting a referral to a dermatologist.
Conclusions Although GPs’ participation in the highly valued training was optimal, several barriers may have contributed to the low inclusion and excision rate of low-risk BCCs. While some of the issues were trial-related, other barriers such as low patient-volume and patients requesting referrals are applicable outside the trial setting as well. This may question the feasibility of substitution of surgical excisions of low-risks BCCs from secondary to primary care in the current Dutch setting.
Trial registration number Trial NL5631 (NTR5746).
- dermatological tumours
- primary care
- dermatology
Data availability statement
Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. The data that supports the findings of this study are available from the medical ethics committee of the Erasmus University Medical Center in Rotterdam (Contact: Interview study reference number MEC-2016-204 and Focus group study reference number MEC-2015-492), but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the medical ethics committee of the Erasmus University Medical Center in Rotterdam.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A strength is that this process evaluation uses complementary descriptive quantitative measures as well as qualitative measures at different time points during the course of the trial.
It provides essential in-depth insight into general practitioners’ exposure to the intervention, implementation of the intervention, and their experiences with the intervention and trial.
A limitation of our study is the late conduction of a barrier analysis instead of addressing identified barriers prior to the onset of the trial.
Background
Healthcare is becoming increasingly expensive with rising percentages of the gross domestic product spent on healthcare.1–3 Since research has shown health systems with stronger primary care tend to have lower healthcare costs, initiatives such as substitution of hospital care towards primary care are increasingly developed and experimented with worldwide.4–13 The main goal of these initiatives is to maintain the affordability, and thus sustainability, of healthcare. Furthermore, it is a means to provide more easily accessible care closer to the patients’ home. However, not every type of care may be suitable for substitution towards primary care. Whether a particular type of care is deemed appropriate for substitution depends on various disease and care specific factors, such as high-volume and being low-complex care, and the support of different stakeholders including general practitioners (GPs), medical specialists and patients.5
One type of care that has been conceived as a potential candidate for substitution of hospital care towards primary care is low-risk skin cancer care.5 14 In the Netherlands, as in several other countries such as the UK and Australia, GPs have a gatekeeper function.5 15 16 Consultations are mainly patient driven, and GPs, who until recently did not have a related primary care guideline, determine whether patients need access to secondary and tertiary healthcare.17 A substantial proportion of patients with a basal cell carcinomas (BCC) (60% in a comprehensive Dutch primary care database analysis) are referred to the dermatologist.18–21 The idea of substituting low-risk skin cancer care to GPs is reflected in the recently published guideline ‘suspicious cutaneous lesions’ of the Dutch College of General Practitioners, which includes recommendations for GPs on the diagnosis and treatment of low-risk BCCs.17 Particularly, low-risk BCCs (ie, non-aggressive histological subtypes, low-risk locations and size <2 cm) are relatively easy to diagnose and treat. Minor surgery can be performed in primary care offices, and innovations such as teledermatology can support GPs.22 23
In 2016, the SKINCATCH Trial (SKIN Cancer And Tumour Healthcare) was initiated to assess whether low-risk BCCs can be treated by GPs without loss of quality of care. The study design was a multicentre cluster randomised non-inferiority trial, in which the intervention included a tailored 2-day educational course on skin cancer management. Participating GPs showed great enthusiasm and interest at the start of the trial,14 and although the patient inclusion rate of all skin tumours suspicious for skin cancer was consistent with the researchers’ expectations, the inclusion rate of low-risk BCCs (primary outcome) lagged far behind.
Therefore, a process evaluation was conducted alongside the trial. A process evaluation is crucial for providing insight in to what extent the trial intervention was actually implemented, how it was experienced by study participants and whether the intervention is feasible in daily practice.24 25 The results can be used to guide the implementation of similar care substitution initiatives.24 The aim of our process evaluation was, therefore, to assess GPs’ exposure to the intervention, implementation of the intervention and experiences with the intervention and trial.
Methods
Description of SKINCATCH trial
The SKINCATCH Trial (see figure 1) was initiated based on the hypothesis that conventional excision of low-risk BCC could be performed by GPs in a primary care setting while maintaining the same quality of care. The study design was a multicentre cluster randomised non-inferiority trial, with GP practices (including group practices) being included as clusters. These clusters were randomised into two parallel arms: the intervention group, which was trained before starting the trial, and the care-as-usual group. Main outcomes included the histological completeness rate of low-risk BCC excisions by GPs in the intervention group compared with dermatologist (primary outcome), diagnostic accuracy of GPs regarding skin tumours, cost-effectiveness of the intervention and treatment and patient reported outcomes regarding preferences and cosmetics (secondary outcomes) (see table 1).
Overview of SKINCATCH trial design. BCC, basal cell carcinoma; CEA, cost-effectiveness analysis; GP, general practitioner; PROMs, patient-reported outcome measures.
Interventions, recommendations and outcome measures of the SKINCATCH trial
The GPs in the intervention group were offered an extensive training in BCC (and skin tumour) management consisting of a tailored 2-day educational course including hands-on surgical training in cadaveric workshops. The GPs in the care-as-usual group did not receive the 2-day educational intervention and were asked to provide skin cancer care the way they were used to. As compensation, they were offered the same BCC management training after completion of the trial. Eligible patients (ie, all patients with a skin tumour suspicious for malignancy) were to be included in the trial during the period February 2016 to May 2018. The first patient was enrolled on Feb 23 2016. Included patients were asked to complete questionnaires at start of their treatment, and 3 and 6 months post-treatment.
The power analysis for the primary outcome was based on a t-test of the proportion of histological completeness of the physicians (GPs and dermatologists), where the physician is the unit of analysis. We expected five eligible patients in the non-inferiority part of the trial per GP per year, which was based on national incidence rates and a prior GP survey.26 27 Using a non-inferiority margin of 5% (based on a clinically accepted margin) and a one-sided significance level of 2.5%,28 a sample size of 45 GPs per group (90 GPs total) was required to obtain a power of 80%. This sample size was increased to 129 GPs to account for (1) the possibility of drop-outs of GPs, and (2) the effect of within-practice correlations of the GPs.
A total of 600 patients with a suspicious skin tumour were included in the trial; 316 patients were included by the GPs in the intervention group and contained 54 patients with a low-risk BCC (9% of the needed sample size for sufficient statistical power (n=600)). As recruitment of removed BCCs was so low, we are unable to report on the primary outcome of the trial (histological completeness rate of low-risk BCC excisions by GPs in the intervention group compared with dermatologists). The process evaluation presented in this paper was based on this low inclusion rate of low-risk BCCs.
Consent and permissions
As this process evaluation is an evaluation among trial participants, conducted as integral part of the trial, we did not obtain separate ethical approval, except for the focus groups. The Standards for Reporting Qualitative Research (SRQR) guidelines were applied, as far as applicable. These guidelines provide a tool for the transparent reporting of qualitative studies.29
Design process evaluation
In designing this process evaluation, we used the framework of Hulscher et al24 to gain insight into the processes responsible for the (variation in) results in the target group. Data on exposure to the intervention, implementation of the intervention and experiences with the intervention and trial were obtained. We used both quantitative and qualitative components, which are described in detail below.
Data collection, outcome measures and analyses
Surveys
Two types of surveys were conducted among participating GPs during the course of the trial to assess their exposure to the intervention and their experiences with the intervention and trial: a training evaluation survey and an online trial evaluation survey. Participation in each of the surveys was voluntary.
Training evaluation survey: After completing the prestudy training, all GPs were asked to complete a survey to evaluate the training. With this survey, both their exposure to and experiences with the training were assessed. The survey consisted of eight statements (seven statements on the content of the training, and one statement on the organisation of the training) using a five-point Likert-scale ranging from strongly disagree to strongly agree (online supplemental appendix A).
Supplemental material
Trial evaluation survey: Ten months after the start of the trial, an online survey was sent to all participating GPs to further explore their experiences with the trial. The survey consisted of four multiple-choice questions, focusing on experiences with the trial and assessing the perceived barriers (online supplemental appendix B).
Training and trial evaluation surveys were analysed separately using SPSS V.24.0 statistical software.
Database analysis
To gain insight into the implementation of the intervention and more specifically the low inclusion rate of BCC patients, a database analysis at the end of the inclusion period was performed investigating the number of inclusions for the primary outcome measure of the trial (ie, histological completeness of low-risk BCC excisions) based on the paper or digital case report forms (CRF) (ie, OpenClinica).30 The CRF included (among others) information on tumour characteristics (eg, size and location), the histopathological diagnosis of the skin tumour and whether or not the GP performed a surgical excision. The CRFs in OpenClinica were exported to and analysed with SPSS V.24.0 statistical software. Descriptive statistics were used to assess the number of performed low-risk BCC surgical excisions as compared with the number of included low-risk BCCs.
Medical record analysis
A medical record analysis was performed to further explore the implementation of the intervention by obtaining quantitative information regarding the number of potential eligible patients and potential eligible excisions. This analysis was performed among seven randomly selected GPs in two primary care practices, participating in the intervention group of the trial. All GP records from February 2016 to February 2017 were screened for eligible patients by a GP practice healthcare assistant using International Classification of Primary Care (ICPC) codes for skin tumours (online supplemental appendix C). Information was obtained on number of patients, clinical diagnosis of the GP, size of the tumour, localisation of the tumour and choice of treatment. In case of histopathological examination additional information was obtained on histopathological diagnosis from the biopsy and/or excision, and histological completeness in case of surgical excision. If the patient was referred to secondary care information was obtained on clinical or histopathological diagnosis. Descriptive statistics were used to assess the GPs’ management of eligible patients.
Telephonic ‘barrier’ interview
Six months after the initiation of the trial, telephonic interviews were conducted by one of the researchers (EN) to identify GPs’ experiences with the trial in terms of perceived barriers regarding the inclusion of patients. We invited GPs from both arms either with no inclusions or one or more inclusions to participate. After 12 interviews with GPs in the intervention group and 10 GPs in the care-as-usual group no new barriers emerged. The semistructured interviews were conducted between August and November 2016. The datawere analysed by the researcher conducting the telephonic interview (EN), noting reported elements during the interview and descriptively summarising the main barriers afterwards.
Focus groups
Three focus groups were conducted between December 2017 and March 2018 to gain an in-depth understanding of GPs’ experiences with the intervention and the trial. Focus groups were chosen as these facilitate interaction between participants, enabling us to identify the GPs’ views on substitution of care, and their experiences with the trial.31–33 All GPs participating in the trial were invited by email, containing an information leaflet about the qualitative evaluation study. GPs could register for one of the three organised focus groups by contacting one of the researchers.
The sessions were moderated by an experienced independent qualitative researcher (ML) and an assistant, both not being involved in the trial. One of the SKINCATCH Trial researchers (EN) was present during the focus groups, but only to answer substantive questions regarding the trial.
In each focus group, the discussion was semistructured using a predefined topic list consisting of two separate parts: general views on substitution of care (part 1) and GPs’ experiences with the trial (part 2). The current study focuses on the latter part (online supplemental appendix D). Results on their general views on substitution of care have been described elsewhere.14
All focus groups were audiorecorded with consent of participants. Subsequently, the audio tapes were transcribed verbatim and imported to Atlas.ti (V.8 for Windows) for analysis.
Two researchers (EN and ML) independently openly coded the first transcript after which the obtained codes were discussed and a preliminary coding scheme was developed. Next, all transcripts were coded by one researcher (EN or ML) and subsequently checked by a second researcher (EN or ML). Differences were discussed and refined until agreement was reached, and new codes were added when needed. The initial coding phase was followed by the phase of constant comparison.31 Different codes were compared and the relationship between codes were explored to detect emerging themes.
Results
Participants
A total of 128 GPs from 90 different primary care practices were included for randomisation (table 2). One GP in the intervention group, and 22 GPs in the care-as-usual group dropped out. Most drop outs occurred within 3 months after the start of the trial. Reported reasons mostly concerned lack of time and personal illness. All 128 GPs were included for the database analysis, and a subgroup of 7 GPs (12%) of the intervention group were included for the medical record analysis. See table 2 for more information on the participants of the different quantitative and qualitative components. For further details regarding the focus groups, see online supplemental table S1.
Participants (GPs) of the SKINCATCH trial and each of the components of the process evaluation
Exposure to the intervention
All GPs in the intervention group (n=58) completed the extensive 2-day training programme. Regarding the e-learning, it was not possible to measure the exposure quantitatively; it could be openly accessed by GPs at all times. The focus groups suggested that a wide variation existed regarding the exposure to the e-learning. Whereas some GPs stated to have gone through the files, others reported not remembering it have been offered or not to have opened it due to time restrictions.
Implementation of the intervention
Only 54 patients with low-risk BCC (9% of needed sample size) of the total of 600 patients with suspicious skin tumours were included in the trial. Furthermore, the GPs in the intervention group performed 95 surgical excisions of skin tumours in total, of which 40 concerned a low-risk BCC. In the care as usual group 29 of the 284 included patients concerned patients with histopathological confirmed low-risk BCCs.
The medical record analysis of potentially eligible BCCs patients in 1 year among 7 GPs resulted in 448 potential patients. After manual extraction by two of the authors (EN, KHRR), 35 confirmed BCC patients remained of which 16 were low-risk BCC. Three BCCs (19%) were excised by two of the seven GPs; the remaining 13 tumours were not excised by the GP. Reported reasons in the medical records were: preference for topical treatment (n=2), patient preference for dermatologist (n=1), referral due to melanoma in differential diagnosis (n=1), coinciding melanoma (n=1), not reported in medical record (n=8).
Experiences with the intervention and trial
Experiences with the intervention
Training evaluation survey: The training was generally evaluated positively by the GPs (figure 2); almost all (n=54) indicated to have found the training useful and almost all (n=53) indicated they would recommend the training among colleagues. All GPs (strongly) agreed with the statement the training would change the way they manage skin cancer, and the vast majority (n=47) confirmed that it was clear to them what was expected regarding their participation in the trial. For further details on the training evaluation survey see online supplemental figure 1.
Results from the training evaluation survey.
Focus groups: The focus groups confirmed that the GPs were highly positive about the training. Some reported it to be the best training they have ever had. According to the GPs it offered them guidance in managing skin tumours in general, and it was particularly useful to learn techniques for minor surgery hands-on. GPs indicated to feel more empowered to extend their services regarding skin tumour management in daily practice. However, some GPs did mention that with time passing they returned to old patterns. According to the GPs, the training may not have been enough for all GPs to change their role in the management of skin tumours. Furthermore, according to some GPs the participation in the trial caused them to diminish their role in skin cancer management as they were used to performing minor surgery on high(-er) risk skin cancers (eg, BCCs located in the face), which was restricted by the study protocol. Regarding the e-learning, the few GPs who used the e-learning were generally positive and reported it was fun to do.
Experiences with the trial
Trial evaluation survey: Reported reasons for the low number of included (BCC) patients in the trial concerned lack of time (n=34) and realising the patients’ eligibility afterwards (n=27), patients rejected participation (n=11), not understanding the different study forms (n=5), the trial restricts me on performing excisions due to trial recommendations (n=3), the GP being afraid to perform minor surgery (n=1) and having to treat the patient different from what they were used to (n=1). A smaller group of GPs (n=13) agreed with the statement that it would make it easier for them to only include patients with a low-risk BCC rather than all skin cancers, and the largest part (n=44) disagreed with the option of clustering consultation hours for skin cancer patients for GPs individually to make patient recruitment more easy.
Telephonic ‘barrier’ interview: During the telephonic interview six barriers were identified. Main perceived barriers reported by the GPs concerned ambiguity regarding eligibility criteria of patients, and lack of clarity regarding the trials’ CRFs. GPs indicated that they expected one of the researchers to visit their practices for one-on-one explanation on the forms. Further perceived barriers included the trial not being a priority, the inclusion process being too time-consuming, difficulty retaining information over time, and discouragement due to refusal of patients or skin tumours appearing high risk.
Focus groups: GPs’ experiences regarding the trial varied. Whereas some GPs were positive about the trial and managed to include patients (up to 53), others reported rather negative experiences. Several barriers were identified which may have contributed to the relatively low inclusion rate (both in general as well as concerning low-risk BCCs). First, administrative challenges related to the inclusion of patients to the trial were reported as a barrier. According to the GPs, the inclusion procedure (informed consent procedure and CRF) was difficult to integrate in daily practice with several study forms needed to be completed at different times during the treatment course of the patient. GPs reported this to be difficult and too time-consuming. However, GPs lacked suggestions on how to improve these administrative challenges as they know it is crucial for data collection. Some GPs reported to have experienced the start of the trial as rather confusing; they stated study forms were not immediately present, and that both the start-date for inclusion as well as the eligibility criteria were not clear. Others were more positive and reported to have found a way of structuring it for themselves, and commented that inaccuracies were picked up well by the researchers. The online CRF application (ie, OpenClinica) was variably received by the GPs, though it was specifically designed for the trial in an attempt to facilitate the GPs in data registration. Some GPs reported it to be not user-friendly and continued using the paper forms, while others stated it to be of great help. Suggestions on reducing the administrative challenges included having researchers collect the data themselves by visiting the GPs’ practices and using an automated digital data collection programme.
Another reported barrier related to the administrative barrier, was a perceived lack of time and high workload to include patients. According to the GPs, this was related to cramped consultation hours, being behind schedule and patients presenting multiple problems during consultation with their GP in which the skin tumour was not perceived as the main issue. As a result of the lack of time and high workload, GPs were more hesitant to recruit patients as this would consume additional time.
A third barrier as reported by the GPs was the low volume of eligible patients seen in practice. GPs reported to only see a small number of low-risk BCC annually. Some also stated to have seen less BCC patients during the course of the trial than anticipated, for reasons not clear.
A fourth barrier reported were patients declining or refusing to participate in the trial. According to the GPs, some patients did not want to participate due to the difficulty and large amount of information they had to read on participation request, and things needed from them after inclusion (ie, questionnaires). The GPs further mentioned that especially older patients and patients less intelligent often declined to participate.
In addition to the low inclusion rate, the GPs were also asked for possible explanations for the low rate of excisions performed by GPs during the trial. Whereas some GPs indeed reported to have only performed few excisions, others were rather surprised hearing this as it did not align with their own experiences. Reported reasons for the low number of excisions were the low number of BCC patients seen in daily practice, patients requesting a referral to the dermatologist, a lack of time and high workload, having a colleague who performs all the excisions, and the training course not being sufficient to change GPs’ behaviour, particularly considering the reported already high workload.
Discussion
This evaluation study showed that, although GPs initially showed great enthusiasm towards the concept of substitution,14 and all GPs participated in the highly valued training, several barriers may have contributed to the low inclusion and excision rate of low-risk BCC patients. Some of these barriers seem to be attributable to the trial setting (eg, administrative challenges, patient recruitment issues), complicating its implementation in daily practice. However, other reported barriers such as high workload, low volume of low-risk BCC patients and patients requesting a referral, apply outside the trial setting as well.
Although several trial-related barriers, such as clear study forms and inclusion criteria, should have been adequately addressed in the current trial, other practical issues such as patient recruitment challenges are commonly reported problems within (multicentre) randomised controlled trials and are difficult to prevent completely.34–38 Similarly, the reported barrier of lack of time/high workload of GPs seems to be inherently related to GP practices,38–40 and may have further impeded study implementation. To tackle these barriers, targeted interventions to enhance recruitments skills of GPs may be valuable to optimise the feasibility of trial interventions in clinical medical care.38
In addition to the trial-related barriers, other reported barriers also apply outside the trial setting and concern the topic of substituting low-risk BCC care towards primary care. Despite high and rising incidence rates of BCCs reported in the literature,27 41 we found that only a small proportion of BCCs can be considered ‘low-risk’ when taking into account body site, diameter and histological subtype,41–43 which was recently confirmed by Fremlin et al.42 Aside from the low volume, the number of excisions performed by GPs in the intervention group was even lower. According to the GPs this may have been partly related to the training being insufficient to change GPs’ practices. Also, GPs were less inclined to perform a surgical excision when patients requested a referral to a dermatologist, which has been found in previous studies as well.14 15 44–48 These barriers, related to feasibility, need to be addressed, where possible, before assessing whether low-risk BCCs can be treated by GPs without a loss of quality of care.
Indeed, with the patient volume being this low (based on the medical record analysis approximately 2 patients with low-risk BCC per GP per year), it will be challenging, if not impossible, for GPs to obtain and maintain their competencies in low-risk BCC management.14 42 Particularly in the context of this low patient volume, a 1-day training may not be sufficient to acquire the relevant competencies. Offering adequate training in a repetitive setting tailored to the specific needs of each GP may, therefore, contribute to a better integration of what is learnt into daily practice.49 50 Although this was attempted by offering an e-learning module, the uptake (although variable) seemed to be only minimal. Furthermore, the cost-effectiveness of such interventions may be questioned. Other solutions may focus on organisational changes in primary care such as concentrated substitution.14 Within this concept GPs refer patients to a colleague GP with noted interest, experience and competence in skin cancer care, thereby clustering these patients within or between practices.14
A limitation of our study includes the late conduction of a barrier analysis. Implementation of change is a complex process, and a preceding barrier analysis among all involved stakeholder groups is advocated to increase the success of interventions.51 By addressing identified barriers prior to the onset of this trial, failure may have been prevented. In addition, such input can serve to promote awareness and stimulate involvement among the target groups, incentivising more successful adoption at a later stage.52 However, it is also important to elicit views of stakeholders who already have some experience with the intervention at hand, as this often elicits different types of barriers.14 Performing a barrier analysis both before the onset of the trial as well as during the trial as part of a process evaluation is therefore advised.
A strength of this study is that we used several complementary evaluation methods, combining both quantitative and qualitative data at different time points during the course of the trial, focusing on both the intervention and care-as-usual group. Although only a low number of GPs was included in the medical record analysis and data on the use of the e-learning module was lacking, by using triangulation of data we were able to capture different dimensions of the observed phenomena. As such, our process evaluation provides essential in-depth insight into the trial and the observed outcomes.
Conclusions
This process evaluation has identified some trial-related as well as more general topic-related barriers that may be responsible for the low inclusion and excision rate of low-risk BCC patients by GPs within the trial. Based on the results of this study, without being able to measure the surgical effectiveness of GPs, the feasibility of substituting low-risk BCC care from secondary to primary care in the current setting should be questioned. Future trials on care substitution may benefit from thorough qualitative barrier analyses among all involved stakeholders, before onset as well as during the course of the trial, to increase the likelihood of successful implementation.
Data availability statement
Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. The data that supports the findings of this study are available from the medical ethics committee of the Erasmus University Medical Center in Rotterdam (Contact: Interview study reference number MEC-2016-204 and Focus group study reference number MEC-2015-492), but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the medical ethics committee of the Erasmus University Medical Center in Rotterdam.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by medical ethics committee of the Erasmus University Medical Center in Rotterdam (MEC-2016-204 and Focus group study reference number MEC-2015-492). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors would like to thank the patients and GPs who participated in the study. We thank Mirjam Droger and Sven van Egmond for their assistance in conducting the focus groups, and primary care practice assistants Claudia Baetsleer (GP practice Pallion) and Shirley Ipskamp (GP practice Doktershuis) for their indispensable and proficient help in conducting the Medical Record Analysis.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors EN, ML, MW, KHRR, RRvdB, PJEB and TN involved in the concept and design of the quantitative and qualitative study. EN and ML performed the focus groups. EN, ML and KHRR involved in quantitative and qualitative data acquisition. EN, ML, MW, RRvdB, PJEB and TN involved in data analysis and interpretation. The paper was written by EN and ML, and was critically revised by all authors. All authors read and approved the final manuscript. RRvdB is the responsible author acting as guarantor.
Funding Funding for this research was provided by the Foundation Achmea Healthcare (in Dutch: Stichting Achmea Gezondheidszorg) which had no role in the design, conduct, analysis or interpretation of the study. Award/grant number is not applicable.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.