Article Text
PDF
Abstract
Introduction An online interactive repository of available medication adherence technologies may facilitate their selection and adoption by different stakeholders. Developing a repository is among the main objectives of the European Network to Advance Best practices and technoLogy on medication adherencE (ENABLE) COST Action (CA19132). However, meeting the needs of diverse stakeholders requires careful consideration of the repository structure.
Methods and analysis A real-time online Delphi study by stakeholders from 39 countries with research, practice, policy, patient representation and technology development backgrounds will be conducted. Eleven ENABLE members from 9 European countries formed an interdisciplinary steering committee to develop the repository structure, prepare study protocol and perform it. Definitions of medication adherence technologies and their attributes were developed iteratively through literature review, discussions within the steering committee and ENABLE Action members, following ontology development recommendations. Three domains (product and provider information (D1), medication adherence descriptors (D2) and evaluation and implementation (D3)) branching in 13 attribute groups are proposed: product and provider information, target use scenarios, target health conditions, medication regimen, medication adherence management components, monitoring/measurement methods and targets, intervention modes of delivery, target behaviour determinants, behaviour change techniques, intervention providers, intervention settings, quality indicators and implementation indicators. Stakeholders will evaluate the proposed definition and attributes’ relevance, clarity and completeness and have multiple opportunities to reconsider their evaluations based on aggregated feedback in real-time. Data collection will stop when the predetermined response rate will be achieved. We will quantify agreement and perform analyses of process indicators on the whole sample and per stakeholder group.
Ethics and dissemination Ethical approval for the COST ENABLE activities was granted by the Malaga Regional Research Ethics Committee. The Delphi protocol was considered compliant regarding data protection and security by the Data Protection Officer from University of Basel. Findings from the Delphi study will form the basis for the ENABLE repository structure and related activities.
- health informatics
- public health
- social medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The diverse expertise and geographical spread of the European Network to Advance Best practices and technoLogy on medication adherencE COST Action members (39 European countries) and their wider professional network represents a unique and timely opportunity to develop a repository of medication adherence technologies that meets the needs of a diverse audience.
The scope and content of the Delphi survey represent the work of extensive literature review combined with multidisciplinary expertise of the steering committee.
The real-time Delphi approach provides improved efficiency of the process, shortens the time of study completion and is particularly suitable for managing larger groups and including people from different geographic locations.
The Delphi study will use state-of-the-art methodology to measure agreement and predetermine agreement/consensus criteria as well as stability of responses.
The real-time approach requires specialised software, which limits the range of possible survey configurations and raw data availability for detailed process analyses and requires relatively elaborate instructions for participants, which may increase participation burden.
Introduction
Taking medication as prescribed often proves difficult for people when managing their health, particularly in the long term.1 Medication adherence is suboptimal in numerous chronic conditions2 3 and has a negative impact on chronic disease management, patient’s general health status, quality of life, working ability and healthcare costs.2 4 5 Research on medication adherence has expanded and contributed to raised awareness of the prevalence of suboptimal adherence and how it affects health outcomes. Digital technologies have increasingly gained interest as new interventions for supporting medication adherence have been developed. A diversity of technologies has been proposed, from electronic monitoring devices to mobile applications, to support medication adherence measurements and empower patients with their disease management. However, the rapidly expanding offer of medication adherence technologies (MATech) makes it increasingly difficult to access, evaluate and compare different technologies to make informed decisions and select appropriate tools for specific clinical or research needs. In a 2018 review by Ahmed et al,6 5881 medication adherence apps were identified on Google Play and Apple App Stores. However, most of them lacked evidence of effectiveness and did not involve healthcare professionals (HCPs) during their development.6 Lack of collaboration between stakeholders results in a limited number of developed MATech actually being implemented into the healthcare systems and used daily by HCPs and/or patients.7 Furthermore, due to differences in healthcare systems across countries, healthcare organisations and reimbursement processes, harmonisation of implementation strategies are lagging behind, which further delays adoption of best practices across countries.4 7
The ENABLE COST Action (‘European Network to Advance Best practices and technoLogy on medication adherencE’, CA19132)8 was initiated by experts in medication adherence and digital technologies to fill these gaps regarding evidence and implementation of MATech within healthcare systems. ENABLE aims to raise awareness of available technologies, expand multidisciplinary knowledge on medication adherence at multiple levels, accelerate knowledge translation to clinical practice and collaborate towards economically viable implementation of best practices and technologies across European healthcare systems. These objectives are being pursued within a 4-year period (2020–2023), by three distinct and inter-related working groups (WGs) that map best practices available (WG1), identify and showcase adherence technologies (WG2) and identify suitable reimbursement strategies for implementation in healthcare systems (WG3), supported transversally by a WG4 coordinating communication and dissemination. At present, the ENABLE Action includes a large interdisciplinary network of experts in medication adherence from 39 European countries.8
Effective implementation of technology-supported healthcare has been facilitated by centralisation of information in public repositories or ‘solution showrooms’, where users can search for technologies that meet their specific requirements.9 Several such repositories already exist in the field of digital health, including medication adherence (eg., NHS app Library,10 MyHealthApps,11 InterventieNet,12 GGD AppStore,13 DIGA,14 Weisse Liste15), but are limited to single countries or types of technology and none represents a comprehensive resource to facilitate adoption of appropriate MATech across health systems. Therefore, ENABLE sets out to develop and maintain a public online repository of MATech where patients, HCPs, researchers and healthcare managers would be able to access and select technologies for adoption in their adherence management activities.8 For example, a patient may be interested more in the practical benefits of using a MATech in their daily lives, while a researcher may be keen to examine in detail the methodology theory and evidence base behind the MATech development. To meet this goal, the ENABLE repository would need to represent a flexible knowledge management system that would include information relevant to the needs of different stakeholders in a user-friendly format. In medical informatics, knowledge management relies on standardised terminologies, classifications and ontologies to record, share and use data on healthcare research and practice. These standards specify the types of information to encode in the form of distinct ‘entities’ representing objects or phenomena in the real world and their properties (‘attributes’), thus enabling knowledge generation through inference and learning.16 Adoption of evidence-based health innovations is also facilitated by these common standards, as new technologies need to interact with existing ecosystems in terms of both data interoperability and communicating with potential users in appropriate domain-specific language.17
The field of medication adherence is highly interdisciplinary, therefore a useful repository would cross multiple knowledge domains and align with several standards, whether medical (eg., WHO International Classification of Disease18), behavioural (eg., the Behaviour Change Intervention Ontology (BCIO)19 20) or technical (eg., WHO Classification of Digital Health Interventions21). Stakeholder involvement would need to be at the core of this development process, to ensure its content is relevant, clear and complete, and meets community needs.22 The diverse and geographically spread ENABLE membership and their wider professional network represents a unique and timely opportunity to conduct this work. Considering these quality standards and following methodological recommendations,22–24 the initial version of the repository structure was prepared. A stakeholder consultation process is proposed to explore their views and level of agreement on the relevance, clarity and completeness of the initial version.22 23 The resulting improved version would represent the structure of the ENABLE repository, which will be tested and populated in subsequent steps with users and developers of available technologies.
The present manuscript describes two elements:
The proposed structure for the repository.
The protocol of the real-time Delphi study to explore stakeholder views on this structure.
Methods and analysis
Steering committee
A steering committee (SC) was established within the COST ENABLE WG2 to coordinate and perform the work. The committee includes 11 ENABLE members from 9 countries in the following areas of expertise: adherence research and education, clinical practice, policy making and technology development. Members are responsible for: (i) determination of the repository scope and framework of attributes defining repository structure, (ii) preparation of the Delphi protocol, (iii) configuration and piloting the Delphi survey, (iv) selection and invitation of stakeholders to participate in the study, (v) moderating study performance via the online tool and (vi) analysis and interpretation of results.
Determining the repository scope and framework of attributes defining its structure
The determination of scope and development of the attributes’ labels with definitions aimed to align with ontology development procedures as described by Wright et al24 and follow a stakeholder engagement methodology as described by Norris et al22 and Khodyakov et al.25 The principles of ontology development, actions taken when generating the framework of attributes and examples of how these principles are applied in the ENABLE project are presented in table 1. The stakeholder engagement is primarily achieved through the proposed real-time Delphi study, which is described in more detail in the next sections.
Principles of ontology development after Wright et al24 and actions taken in the ENABLE project
Scope and definition of MATech
Four established definitions were used to define the scope of repository and set the framework of attributes: (i) WHO definition of health technologies 26; (ii) the ABC definition of medication adherence1; (iii) the WHO definition of adherence to long-term therapies2 to highlight the importance of shared decision-making between the patient and the healthcare team and (iv) the definition of best practice in healthcare proposed by the European Commission to guide improvements in European health systems.27 The information in this definition denotes evidence on safety, efficacy, effectiveness, cost-effectiveness, appropriateness, social and ethical values and quality of the healthcare interventions.
Therefore, we propose to define MATech as devices, procedures or systems developed based on evidence to support patients to take their medications as agreed with healthcare providers (ie, to initiate, implement and persist with the medication regimen).
Devices, procedures or systems emphasise the inclusion of all technologies, irrespective of their mode of delivery (whether based on electronic or printed supports, delivered through human interaction or a combination of these), with the aim to construct a comprehensive repository in which users can identify diverse technologies to fit their potentially diverse needs.
Developed based on evidence encompass the requirement of evidence/research that supports at least a potential contribution to either measurement or intervention on medication adherence (eg, validation or pilot studies). Thus, technologies that are not (yet) supported by evidence (eg., development and testing stages), or clinical practice protocols without an evidence base on at least one aspect (safety, efficacy, effectiveness, cost-effectiveness, appropriateness, social and ethical values or quality), will not be (yet) included in the repository until such evidence is produced and reported.
Support patients to take their medications as agreed with the healthcare providers (ie, to initiate, implement and persist with the medication regimen) encompass the contribution of the technology to medication adherence management—either directly in patients’ self-management, or by supporting professionals to offer such services to patients through all phases of medication adherence. Thus, technologies that focus on other medication management goals, but do not target adherence specifically would be out of scope for this repository.
Furthermore, the technologies included would need to be described in terms of their technical characteristics and validation, their behaviour change content, format and context, as well as the characteristics facilitating appropriate implementation in care processes. Hence, evidence from behaviour,19 28 implementation29 30 and computer sciences18 21 31 32 informed the initial scope and attributes framework to ensure key features, such as user-centeredness, trustworthiness/credibility, accuracy and relevance of the presented information, tailoring to the needs of different users and interoperability with existing evidence and other sources of information on healthcare technologies.
Framework of attributes
An initial list of attributes was developed based on a literature review and knowledge from the ENABLE members activities such as (i) an ongoing systematic review of e-health interventions on medication adherence for chronic conditions,33 (ii) a checklist of e-health quality criteria under development,34 (iii) Interventienet.nl—platform showcasing evidence-based medication adherence interventions in the Netherlands12 and (iv) the ABC taxonomy—consensus-based terminology and definitions of medication adherence.1
The initial list was presented to the SC and discussed via several videoconferences to generate a more detailed list of attributes grouped on several themes. Each theme was further elaborated by a subgroup of two SC members following a standard format including labels and adherence-related definitions. We adopted the approach from BCIO,19 where related attributes were searched in topic relevant ontologies/taxonomies/classifications and original definitions and codes were added. The reasons for the choice of certain attributes and labels were detailed for each attribute group. The proposed framework of attributes is graphically presented in figure 1 and online supplemental file 1, while rationale and sources used to define the labels for the MATech repository are presented in table 2 and online supplemental file 2.
Supplemental material
Supplemental material
The interactive graph showing the framework of attributes for medication adherence technologies (MATech) (‘the MATech tree’). The MATech tree is available as interactive feature in the online supplemental file 1.
The proposed framework of attributes used in the MATech repository
The final proposed framework consists of three domains: (i) product and provider information (D1), (ii) medication adherence descriptors (D2) and (iii) evaluation and implementation (D3) aligning with the three elements of the Donabedian healthcare model (i) structure, (ii) process and (iii) outcomes.35 The domains branch in 13 attributes groups, which then branch further to up to four sublevels of attributes. Each attribute is described with a label and related definition.
Choice and description of the study design
We will perform an online real-time Delphi (RT-Delphi) survey to explore the level of agreement on the MATech definition and relevance, clarity and completeness of the proposed framework of attributes defining the repository structure and gain a deeper insight into stakeholders’ distinct needs and requirements. The Delphi process is a flexible iterative process to consult and/or reach consensus among a group of people on a particular topic.36 37 The key characteristics of a Delphi study are anonymity, iteration, controlled feedback and statistical description of group response.38 The RT-Delphi approach was developed by Gordon and Pease to improve efficiency of the process and shorten the time of performance.39 Since then, several online tools have been developed to facilitate the RT-Delphi design40 and literature describing the use of RT-Delphi and comparison with the traditional multiround Delphi approach is growing.23 41–44 In contrast to the traditional Delphi, the real-time approach is round-less and offers a constant iteration by providing immediate (real-time) individual and aggregated feedback. Based on new information participants can rethink and modify their answers, which could lead to reconciliation of opinions and eventually to consensus. Participants are encouraged to revisit and engage in the survey several times during the study period.39 40 42 44 In comparison with the traditional approach, the real-time approach encompasses all key Delphi features43 and is similar from all key perspectives.23 41 43 44 Furthermore, the real-time approach is particularly suitable for managing larger groups, decreases moderators’ workload, simplifies inclusion of people from different geographic locations and can be leaner in costs.23 39 44 On the other hand, the approach requires specific software, which can sometimes be rigid in terms of survey configuration and analysis, contributes to increases study costs and requires specific instructions for participants.40 44 Acknowledging the potential challenges, the advantages of the approach outweighed them and supported a decision to adopt the real-time approach for our Delphi study.
Sampling and sample size
We aim to include stakeholders from all 39 countries, participating in the COST ENABLE, covering five different backgrounds per country: (i) adherence and eHealth research (measurement, intervention development, implementation science, health economics), (ii) clinical care (specialist and primary care practitioners providing medication adherence support), (iii) patient representation (age >18 years, active representative in patient associations or healthcare facilities), (iv) policy making and (v) technology development. Hence, the targeted sample size is at least 195 panellists to be invited in the study (39 countries×5 stakeholders).
Purposive sampling will be applied to identify potential panellists. First, requests will be sent through the ENABLE Cost Action membership list to representatives of all 39 countries, requesting them to identify suitable panellists from all five backgrounds. ENABLE members will provide the SC the name, background and email for every potential panellist. Participants’ emails will be entered in the online platform (eDelphi.org—Delphi method software45), which will enable anonymity in further steps, that is, individual’s activity and or/answers will not be linked to personal data. All communication with the panellists (invitation, reminders, etc) will be performed through the platform. If more candidates from the same background and country will be suggested, we will invite all candidates to increase the likelihood of achieving the planned sample size. If the expressed interest exceeds the planned sample size, purposeful sampling will be performed to ensure variation in expertise, country and balance other characteristics (eg, years of expertise, gender). To reach simple size and variation in sample characteristics, key international organisations from the field (eg., The International Society for Medication Adherence (ESPACOMP), Pharmaceutical Care Network Europe (PCNE), European Medicines Agency (EMA), European Patient Forum (EPF), etc.) will be contacted to fill any missing gaps, if needed.
Patient and public involvement
The goal of this Delphi consultation is to involve stakeholders (patient representatives among them) in decisions regarding the development of ENABLE repository and is part of the broader approach to patient and public involvement followed in the ENABLE Action. Results will be communicated to all stakeholders, and they will be listed and acknowledged among ENABLE collaborators.
Data collection
We will use an online platform, eDelphi.org (Metodix, Helsinki, Finland45) for data collection. All survey activities—distribution, reminders, communication with and between the panellists and interim analysis of the process will be performed through the tool. The survey will be conducted from 1 October 2021 to 15 January 2022 in three stages:
Pilot stage: at least 10 members of the COST ENABLE Action, specifically members of the WG2, will be asked to test the survey (including instructions for participants) and to provide feedback on face validity as well as user experience.
First stage phase: invitation of 20 purposefully selected stakeholders (aiming for variation in expertise, geographical location and gender) to create initial aggregated feedback of the RT-Delphi.
Full-scale RT-Delphi: all remaining stakeholders will be invited to participate in the study.
Stakeholders will receive an email invitation via the eDelphi platform with a personalised link to the survey. Detailed instructions describing survey aims, rules of engagement and how to use the platform will be available on the platform.
At the beginning of the survey, participants will be encouraged to think of a hypothetical situation in which they would search for MATech applicable to their own setting/role and to assess the proposed attributes from this perspective throughout the survey. First, panellists will be asked to familiarise with the proposed structure and provide general feedback on the completeness. Furthermore, they will be asked to rate agreement with and clarity of the MATech definition and relevance and clarity of each proposed attribute group on a 9-point Likert scale, where 1 represents extremely irrelevant/unclear and 9 represents extremely relevant/clear. We will use the Live 2D format,45 where each outcome represents one of the two dimensions, that is, the x-axis stands for relevance and the y-axis stands for clarity. Additionally, an open-text field will be provided for panellists to comment on completeness of each attribute group, that is, proposing additional attributes or revising definitions. We will moderate the discussion in the following ways: (i) address technical issues with the platform by responding to the comment when the issues will be solved or provide instructions how to manage the issue and (ii) outline the progress of the study and the most commented questions in bulletins send through the platform once a week. We considered these strategies to encourage panellists to participate, taking into account the length of the survey and the complexity of the concepts they are rating. Delphi survey materials (online supplemental file 3, online supplemental file 4 and online supplemental file 5), including all attributes labels and definitions (online supplemental file 1 and online supplemental file 2) as well as participant instructions (online supplemental file 6), are shown in the online supplemental materials.
Supplemental material
Supplemental material
Supplemental material
Supplemental material
For sample description purposes, participants will be requested to provide information on their expertise (profession, years of experience, relevant professional experiences) and demographic characteristics (age, gender, country of practice). This information will also be used to examine differences in participants’ ratings and comments depending on their background and location. These data will be presented in aggregated form and not linked to the individual’s activity or answers. Revisiting and rerating will be encouraged by weekly reminders.
Data collection will be stopped on reaching adequate sample size and characteristics to achieve sufficient representability and generalisability of the opinions gathered. Therefore, we propose stopping the Delphi when three criteria will be met: (i) the total response rate to the survey is ≥30% (number of participants completing the survey, of the total number of stakeholders invited)46; (ii) a minimum of 10 panellists in each stakeholder group completed the survey; (iii) a minimum of two stakeholders from at least 2/3 of the COST ENABLE countries has completed the survey. We will operationalise survey completion as providing background data and answering at least 75% of the repository structure questions.
Data analysis
Descriptive statistics will be used to characterise the sample of panellists and each stakeholder subgroup regarding profession, years of experience, age, gender and country.
Several measures can be used to determine when consensus is reached, with the percentage of agreement being the most common.47 Prespecification of the consensus measure and criteria for consensus increases trustworthiness of findings.48
Level of agreement on relevance, clarity and completeness
Stakeholder agreement on the proposed definition and attributes will guide decisions on the repository structure. Therefore, we selected a set of criteria representing different levels of agreement and consequently carrying different weights in these decisions. The level of agreement on every attribute for both outcomes (eg., relevance and clarity) will be quantified using the interpercentile range adjusted for symmetry (IPRAS) analysis technique from the RAND/UCLA Appropriateness Method (RAM).49 First, the disagreement index (DI) will be calculated as a ratio between the interpercentile range (IPR) and IPRAS. A DI >1 (ie, IPR >IPRAS) indicates disagreement exists. IPR is calculated using the 30th–70th percentile. IPRAS for the 9-point Likert scale is calculated according to the formula presented in the RAM User Manual.49
Second, the median and DI will define different levels of agreement and steer the decisions about the repository structure. For the relevance:
items with the median of 7–9 and no disagreement will be considered as relevant and mandatory;
items with the median of 4–6 or disagreement will be considered as optional;
items with the median of 1–3 and no disagreement will be considered not relevant and candidates for exclusion.
For an even number of participants, median ratings of, for example, 6.5 or 3.5 will be assigned to the higher level.49 Stakeholders’ responses per question will be summarised using descriptive statistics.
For clarity ratings, the above criteria will be applied as (i) sufficiently clear to remain unchanged; (ii) optional changes and (iii) candidates for rephrasing.
Panellist comments in the open-text fields will be analysed qualitatively, using content analysis. Findings will be used to rephrase and improve clarity of certain attributes or to add additional attributes proposed by stakeholders.
Subgroup analysis
Following the primary analysis on the whole sample, a subgroup analysis per stakeholder group will be conducted to examine variation in opinions and potential differences among subgroups. The same agreement criteria will be applied and descriptive statistics will be stratified by stakeholder group. In addition, we will determine the reliability of ratings per question within stakeholder group by calculating the intraclass correlation coefficient (ICC). The ICC calculation is based on the two-way random model, considering type (average measures) and definition of relationship (consistency) and is presented in Equation 1. ICC >0.70 will indicate moderate-to-good reliability.50 51
Equation 1. Calculation of the ICC expressed in %. MSR stands for mean square for rows and MSE stands for mean square for error.
Analysis of process indicators
By analysing process data from the online tool, we will describe in more detail how stakeholders’ responses evolved through iterations and how consensus or certain level of agreement has formed.25 52
Stability of response presents the consistency of responses within the study period and between respondent group stability, which is considered a necessary precondition for determining the level of agreement or if consensus was achieved.53–55 Different measures of dispersion (eg., median, IQR) and statistical approaches (eg., descriptive, inferential) can be used44 55 to measure stability, which can be calculated between rounds (traditional Delphi) or at the end of the study (RT-Delphi).41 44
We will use the coefficient of quartile variation (CQV) as a descriptive measure of response stability. CQV will be calculated over all participants (CQVtotal) and within the same stakeholder group (CQVsub) to account for expected higher variation in response between different stakeholder groups. A CQVtotal <30% and CQVsub <15% will be considered as stable response. CQV calculation is shown in Equation 2.54 56
Equation 2. Calculation of the CQV, expressed in %. Q3 stands for value of the third quartile and Q1 for first quartile.
Final repository structure
After conducting the analyses described above (planned to be finalised at the end of April 2022), results suggesting modifications to the proposed structure will be considered for adoption by the SC in a subsequent version, which will represent the final structure of the ENABLE repository implemented on the initial ENABLE repository version. Further work will be considered to address results that might suggest ongoing debates in the field about certain attribute groups or the need for more in-depth consultation and evidence generation. This work will accompany the iterative improvement of the repository during the ENABLE Action.
Ethics and dissemination
Ethical considerations and consent to publish
The study is designed to ensure participants’ anonymity and to manage personal data in line with EU regulation. Before starting the survey, every participant will provide an informed consent electronically on the study entry page. Participants will be asked to carefully read through the statement regarding the study aim and nature as well as the data handling procedures and to mark their understanding and agreement. The results will only be published in an aggregated form and no personal details will be revealed.
An ethical approval for the activities of the COST ENABLE Action, including this Delphi study, was granted by the Malaga Regional Research Ethics Committee (‘Comite de Etica de la Investigacion Provincial de Malaga’) on 29 April 2021 (online supplemental file 7). In addition, a data protection assessment was carried out by the Data Protection Officer at the University of Basel. According to this instance, the Delphi study protocol was determined as compliant regarding data protection and security (online supplemental file 8).
Supplemental material
Supplemental material
Future implications and challenges
The proposed scope and framework of attributes together with findings from this Delphi study will represent the first steps on the pathway to create an evidence-based, interoperable and user-friendly MATech repository. Following the Delphi consultation and integration of the repository module on the ENABLE website,57 providers of MATech (public or private) would be invited to upload information on their products via a MATech description form based on the final repository structure. The accuracy of the information would be verified by an independent review panel through a procedure yet to be established. Important challenges lay ahead, such as how to select MATech for inclusion in the repository given the broad scope of the definitions proposed, how to ensure accurate information about the technologies included, how to provide the information in other languages than English and in non-technical language accessible for all and how to maintain a representative and varied offer of technologies in the long term. Nevertheless, the ENABLE repository promises to bring together stakeholders from different backgrounds to build a common language which can have an important positive impact on medication adherence research and practice.
Dissemination
The repository will be publicly accessible for interested parties. The use of the repository will be promoted and supported by dissemination meetings, workshops and training schools. The findings of the study will be presented via publications (reports and manuscripts in open access peer-reviewed journals) and oral presentations to different stakeholders in conferences and meetings. The spirit of COST Actions is networking and dissemination of ideas; hence, the action is open for anybody who would wish to join or would like to be informed about its activities.
Ethics statements
Patient consent for publication
Ethics approval
An ethical approval for the activities of the COST ENABLE Action, including this Delphi study, was granted by the Malaga Regional Research Ethics Committee (“‘Comite de Etica de la Investigacion Provincial de Malaga”’) on 29th April 2021. In addition, a data protection assessment was carried out by the Data Protection Officer at the University of Basel. According to this instance, the Delphi study protocol was determined as compliant regarding data protection and security.
Acknowledgments
The authors would like to thank the ENABLE members for their feedback on the development of the ENABLE repository and early versions of this work, and the ENABLE Core Team for leadership and coordination of ENABLE activities, of which this work is part of.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Collaborators ENABLE Collaborators: Andrei Adrian Tica, Romania; Adriana Baban, Romania; Adriana E. Chis, Ireland; Alexandru Corlăteanu, Moldova; Anna Bryndis Blöndal, Iceland; Ane Erdal, Norwaay; Anne Gerd Granås, Norway; Anthony Karageorgos, Greece; Bernard Vrijens, Belgium; Bettina S. Husebø, Norway; Bjorn Wettermark, Sweden; Christos Petrou, Cyprus; Çiğdem Gamze Özkan, Turkey; Cristina Mihaela Ghiciuc, Romania; Daisy Volmer, Estonia; Dalma Erdősi, Hungary; Darinka Gjorgieva Ackova, North Macedonia; Dragana Drakul, Bosnia and Herzegovina; Dusanka Krajnovic, Serbia; Elena Kkolou, Cyprus; Elín Ingibjorg Jacobsen, Iceland; Emma Aarnio, Finland; Enkeleda Sinaj, Albania; Enrica Menditto, Italy; Elena Kkolou, Cyprus; Eric Van Ganse, France; Esra Uslu, Turkey; Fatjona Kamberi, Albania; Fedor Lehocki, Slovakia; Francisca Leiva-Fernandez, Spain; Freyja Jónsdóttir, Iceland; Fruzsina Mezei, Hungary; Gaye Hafez, Turkey; Gregor Bond, Austria; Guenka Petrova, Bulgaria; Hendrik Knoche, Denmark; Hilary Pinnock, UK; Horacio Gonzalez-Velez, Ireland; Indrė Trečiokienė, Lithuania; Ines Potočnjak, Croatia; Ingibjörg Gunnþórsdóttir, Iceland; Ioanna Chouvarda, Greece; Ioanna Tsiligianni, Greece; Isabel Leiva Gea, Spain; Isabelle Arnet, Switzerland; Ivett Jakab, Hungary; Jaime Correia de Sousa, Portugal; Jaime Espin Balbino, Spain; Janja Jazbar, Slovenia; Jesper Kjærgaard, Denmark; Jiří Vlček, Czech Republic; Joao Gregorio, Portugal; Job van Boven; The Netherlands; Jolanta Gulbinovic, Lithuania; Josip Culig, Croatia; Jovan Mihajlović, Serbia; Juris Barzdins, Latvia; Karin Svensberg, Sweden; Katarina Smilkov, North Macedonia; Katerina Mala-Ladova, Czech Republic; Katharina Blankart, Germany; Konstantin Doberer, Austria; Konstantin Tachkov, Bulgaria; Kristiina Sepp, Estonia; Laetitia Huiart, Luxembourg; Line Iden Berge, Norway; Liset van Dijk, The Netherlands; Maja Ortner Hadžiabdić, Croatia; Manon Belhassen, France; Marcia Vervloet, The Netherlands; Maria Cordina, Malta; Marie Ekenberg, Sweden; Marie Hidle Gedde, Norway; Marie McCarthy, Ireland; Marie Schneider, Switzerland; Marie Viprey, France; Marina Odalovic, Serbia; Martin Wawruch, Slovakia; Martina Bago, Croatia; Miriam Qvarnström, Sweden; Mitar Popovic, Montenegro; Mitja Kos, Slovenia; Natasa Duborija-Kovacevic, Montenegro; Noemi Bitterman, Israel; Omar S. Usmani, UK; Ott Laius, Estonia; Panagiotis Petrou, Cyprus; Paulo Félix Lamas, Spain; Paulo Moreira, Portugal; Petra Denig, The Netherlands; Przemyslaw Kardas, Poland; Quitterie Reynaud, France; Sabina De Geest, Switzerland; Seher Çakmak, Turkey; Stefan Bruno Velescu, Romania; Susanne Reventlow, Denmark; Tamás Ágh, Hungary; Valentina Marinkovic, Serbia; Valentina Orlando, Italy; Vered Shay, Israel; Vesna Vujic-Aleksic, Bosnia and Herzegovina; Vildan Mevsim, Turkey; Yasemin Cayir, Turkey; Yingqi Gu, Ireland; Zorana Kovacevic, Serbia.
Contributors All authors contributed to the work and formation of this manuscript. The first draft was prepared by UNM, CG, JR and AD. All other members of the steering committee (PB-F, FH, MTH, CJ, FMR, DS and IT) reviewed and upgraded the first version. All steering committee members (CG, JR, PB-F, FH, MTH, CJ, FMR, DS and IT) worked on development of the scope and framework of the attribute groups, UNM and AD coordinated the work. SPG was consulted as the expert in Delphi methodology, specifically the RAND/UCLA Appropriateness Method. The final version of the protocol was prepared by UNM and reviewed by all other authors (CG, JR, PB-F, SPG, FH, MTH, CJ, FMR, DS, IT and AD). All authors have read and approved the final version of the manuscript.
Funding This work was supported by COST Action ENABLE—European Network to Advance Best practices & technoLogy on medication adherencE’, number CA19132. CG was supported by a PhD grant financed by the Action LIONS Vaincre le Cancer. The work of IT was partially supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia (project no.451-03-68/2022-14/200161). AD was supported by an IDEXLYON grant (16-IDEX-0005; 2018-2021) during the preparation of this manuscript.
Disclaimer Funders had no role in the study design, content work and preparation or writing of the manuscript.
Competing interests SPG is a research team member for ExpertLens (an online platform and methodology for conducting modified-Delphi studies). SPG's spouse is a salaried employee of, and owns stock in, Eli Lilly and Company.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the 'Methods and analysis' section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.