You are here

Article Text

Article menu

Download PDFPDF

XML
Research methods
Protocol
ESPACOMP Medication Adherence Reporting Guidelines (EMERGE): a reactive-Delphi study protocol
  1. R Helmy1,
  2. L L Zullig2,3,
  3. J Dunbar-Jacob4,
  4. D A Hughes5,
  5. B Vrijens6,7,
  6. I B Wilson8,
  7. S De Geest1,9
  1. 1Institute of Nursing Science, Department of Public Health, Faculty of Medicine, University of Basel, Basel, Switzerland
  2. 2Center for Health Services Research in Primary Care, Durham Veterans Affairs Health Care System, Durham, North Carolina, USA
  3. 3Department of Medicine, Duke University, Durham, North Carolina, USA
  4. 4School of Nursing, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
  5. 5Centre for Health Economics and Medicines Evaluation, Bangor University, Bangor, UK
  6. 6Department of Public Health, University of Liège, Liège, Belgium
  7. 7WestRock Healthcare, Visé, Belgium
  8. 8Department of Health Services, Policy & Practice, Brown University School of Public Health, Providence, Rhode Island, USA
  9. 9Academic Centre for Nursing and Midwifery, KU Leuven, Leuven, Belgium
  1. Correspondence to Professor S De Geest; sabina.degeest{at}unibas.ch

Abstract

Introduction Medication adherence is fundamental to achieving optimal patient outcomes. Reporting research on medication adherence suffers from some issues—including conceptualisation, measurement and data analysis—that thwart its advancement. Using the ABC taxonomy for medication adherence as the conceptual basis, a steering committee of members of the European Society for Patient Adherence, COMpliance, and Persistence (ESPACOMP) launched an initiative to develop ESPACOMP Medication Adherence Reporting Guidelines (EMERGE). This paper is a protocol for a Delphi study that aims to build consensus among a group of topic experts regarding an item list that will support developing EMERGE.

Methods and analysis This study uses a reactive-Delphi design where a group of topic experts will be asked to rate the relevance and clarity of an initial list of items, in addition to suggesting further items and/or modifications of the initial items. The initial item list, generated by the EMERGE steering committee through a structured process, consists of 26 items distributed in 2 sections: 4 items representing the taxonomy-based minimum reporting criteria, and 22 items organised according to the common reporting sections. A purposive sample of experts will be selected from relevant disciplines and diverse geographical locations. Consensus will be achieved through predefined decision rules to keep, delete or modify the items. An iterative process of online survey rounds will be carried out until consensus is reached.

Ethics and dissemination An ethics approval was not required for the study according to the Swiss federal act on research involving human beings. The participating experts will be asked to give an informed consent. The results of this Delphi study will feed into EMERGE, which will be disseminated through peer-reviewed publications and presentations at conferences. Additionally, the steering committee will encourage their endorsement by registering the guidelines at the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network and other relevant organisations.

  • Medication adherence
  • Medication compliance
  • Health research reporting guidelines
  • Delphi method
  • Study protocol

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2016-013496

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Incorporates the input of many topic experts across various disciplines and geographical areas.

    • Preserves subject anonymity and reduces the effect of dominant individuals.

    • Uses statistical analysis techniques to reduce the potential of group pressure for conformity.

    • Requires large blocks of time to administer several survey rounds and consolidate their output.

    • Has potential to mould opinions based on the received aggregate feedback.

    Introduction

    Reporting research on medication adherence is suboptimal. Confusion prevails regarding the conceptual underpinning, adequate measurement and analysis of medication adherence data, hindering scientific progress in this field.1–3 Guidelines on the reporting of health research aim at enhancing publication quality and may focus on specific study designs, research areas or sections of a report. Examples of such guidelines can be found on the website of the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network (http://www.equator-network.org), and include STROBE4 (for observational studies in epidemiology), CONSORT5 (for trials) and TIDieR6 (for description and replication of interventions) guidelines. The introduction of such guidelines and their endorsement by professional societies and journals has proven to be helpful in enhancing the transparency and accuracy of health research reporting.7 ,8

    There are existing published guidelines and recommendations focusing on medication adherence research.1 ,9–12 However, these guidelines overlap considerably with those developed for general clinical research and are thus not particularly specific for medication adherence research.9 ,10 Additionally, they do not build on a clear conceptualisation of medication adherence,1 ,9 ,10 ,12 and focus on study design rather than reporting.1 ,11 ,12 Considering the shortcomings of the existing guidelines and the aforementioned deficits in the quality of medication adherence research, this field would benefit from specific reporting guidelines to boost the quality of reporting medication adherence research.

    A steering committee of members of the European Society for Patient Adherence, COMpliance, and Persistence (ESPACOMP, http://www.espacomp.eu) launched an initiative to develop ESPACOMP Medication Adherence Reporting Guidelines (EMERGE) that will tackle the aforementioned shortcomings in the following manner. Considering the conceptualisation of medication adherence, the anticipated guidelines will build on the ABC taxonomy for medication adherence.13 This taxonomy defines medication adherence as ‘the process by which patients take their medications as prescribed’. It divides this process into three inter-related phases: (1) initiation; (2) implementation and (3) persistence, each demanding a clear operational definition and appropriate measurement and data analysis. Hence, setting this conceptualisation as a standard for reporting medication adherence research is anticipated to remove the ambiguity surrounding medication adherence as a concept. Additionally, designing the guidelines attentively to avoid overlapping with the existing guidelines for general clinical research has a twofold benefit. First, it will steer clear of redundancy and, thus, any confusion that might arise accordingly. Second, it will allow the new guidelines to be applied to the different study designs and used in parallel with the general guidelines available for the corresponding study design. Finally, developing the guidelines as reporting guidelines will help both the authors in figuring out the important elements to be reported about their research and the reviewers in critically appraising the quality of the studies.

    Developing reporting guidelines, however, requires input by experts from various scientific backgrounds and needs to be relevant across geographical regions. This is particularly applicable in the context of medication adherence, which is a multidisciplinary science ranging from behavioural sciences through statistics and clinical medicine to economics. Delphi methodology, as a means for consensus building,14 enables integration of input from a wide variety of experts. It achieves consensus through an iterative process of survey rounds, providing an opportunity for its participants to revise their input in subsequent rounds based on collective group feedback from previous rounds. This method enables the neutral and efficient integration of feedback from different experts, is suggested as 1 of the 18 steps put forward by the EQUATOR network in their guidance on developing health research reporting guidelines15 and has been used in developing many similar guidelines.16 ,17

    This approach has many benefits.18–20 First, the iterative nature of the study and the feedback process allows the participants to reassess their initial judgements. Thus, consensus is reached through a gradual stepwise process based on rational thinking and input. Second, the participants do not interact directly and remain anonymous to each other until the survey rounds end. Hence, group domination by the views of certain individuals is avoided. Third, when the participants consider changing or sticking to their original opinion after checking group response, this decision is not affected by the desire to be seen concordant with senior or dominant individuals. Fourth, the experts can participate asynchronously and do not need to be present together while answering surveys as they receive collective feedback along with new surveys, which can be conducted through a web-based platform. Accordingly, it is feasible to use this approach to involve geographically distant participants. Moreover, the controlled feedback minimises the effect of noise, which can happen in face-to-face group discussions. Finally, the ability to use statistical analysis facilitates reaching a more objective consensus than that usually reached through face-to-face conversations. Consequently, the Delphi method is an appropriate method to include the knowledge of many experts to cover the numerous aspects of the medication adherence subject.

    Aim

    The aim of this Delphi study is to build consensus among a group of topic experts regarding an item list that will support developing guidelines for medication adherence research reporting.

    Methods and analysis

    Overview of the study design

    The study implements a reactive-Delphi design21 where a group of medication adherence experts will be provided, in the first survey round, with an initial list of 26 items that are specific to medication adherence research. They will be asked to rate the relevance and clarity of the items (on a scale from 1 to 4, where 1 is the lowest relevance/clarity and 4 is the highest relevance/clarity), with a possibility of suggesting modifications of the initial items, or new items to be added. After each round of survey, predefined decision rules will be applied to keep, delete or modify the items. This reactive-Delphi design allows for reduction of effort needed from the experts and faster arrival at consensus compared with the traditional Delphi design where experts are asked to come up with all items themselves.

    The steering committee

    The committee driving this initiative to develop guidelines for reporting research on medication adherence (also the authors of this study protocol) includes members and founding members of ESPACOMP. One of their major research interests is medication adherence. They represent a diversity of disciplines including biostatistics, health economics, health policy and management, health services research, medicine, nursing, pharmacy, pharmacology, and psychology. Some members of the committee were involved in development of the ABC taxonomy for medication adherence (BV, DAH and SDG).

    Role of the steering committee

    The steering committee developed the initial items and will select the sample of experts for this study. On the basis of the decision rules, the steering committee will also apply the corresponding actions after each survey round: (1) keeping or deleting an item based on relevance scores, and (2) modifying the wording of items based on clarity scores and the suggestions provided by the experts. For items that will not have achieved consensus on relevance after the survey rounds will be stopped, the steering committee will decide on each item individually based on internal consensus among its members, taking into consideration the aggregate input provided by the expert panel. All of the aforementioned tasks of the steering committee are further detailed later in this protocol.

    Sample and sampling procedures

    A purposive sample of medication adherence experts will be included in the study. The size of the group of experts needed for participation in any Delphi study does not depend on statistical power as representativeness in such studies is assessed based on the quality of the sample rather than its size.20 Accordingly, the steering committee identified medication adherence experts based on the representation of disciplines and geographical locations. In concordance with other similar projects aiming at developing guidelines for reporting of health research,16 ,17 the goal is to include a minimum of 20 experts in the final survey round. To compensate for possible initial refusal or attrition over rounds, oversampling was considered with a rate of 25% per round. With a literature-based average22 of three rounds until consensus is achieved, the starting sample will be a minimum of 40 experts.

    More specifically, the Delphi participants need to be established experts in the field of medication adherence and satisfy all the following main selection criteria:

    1. Having a minimum experience of 5 years in the field of medication adherence;

    2. Having an established international profile in this field, recognised by scientific publications, policy reports and/or extensive participation in specialised topic conferences, meetings, or interest groups; and

    3. Having good English proficiency to complete the surveys.

    To ensure representativeness of all relevant disciplines and geographical locations, the following variability selection criteria will be applied:

    1. The starting sample will include participants from each of the following disciplines:

      1. Health services research,

      2. Clinical research,

      3. Statistics,

      4. Medicine,

      5. Nursing,

      6. Pharmacy/pharmaceutical sciences,

      7. Pharmaceutical industry,

      8. Clinical pharmacology,

      9. Pharmacoepidemiology,

      10. Behavioural medicine/health psychology,

      11. Journal editing,

      12. Public health,

      13. Health policy;

    2. Geographical representation will be ensured by selecting experts from all continents.

    Each of the steering committee members will provide suggestions for experts to be included on the expert panel for the Delphi study based on the aforementioned criteria. Feedback and agreement on the proposed list by the whole steering committee will be sought before experts are invited to participate. The final choice of the experts to be included will be based on an optimal distribution and representation of experts in view of the sample selection criteria (main and variability criteria) and will be moderated by two members of the steering committee (SDG and RH). Delphi participants who will complete all rounds of the study will be listed in the final publication of the reporting guidelines in an acknowledgement section. However, the study will be fully anonymised and participants will not be known to each other during the survey rounds.

    Generation of the initial item list

    The initial item list was developed and fine-tuned by the steering committee. First, a literature review was performed to identify existing medication adherence research guidelines9–11 and recommendations1 ,12 by two members of the steering committee (RH and SDG). This information was summarised and discussed in an in-person meeting in Prague in November 2015 among all of the steering committee members. This discussion, guided by the ABC taxonomy for medication adherence as well as by a review of common sections of the existing reporting guidelines for health research reporting (eg, STROBE, CONSORT), led to the generation of a pool of items.

    Using a stepwise review process, the steering committee reviewed and further fine-tuned the items in view of relevance and clarity over four feedback rounds via email and conference calls. Items were gradually enhanced and revised to focus exclusively on aspects relevant to medication adherence. Redundancy between items of the existing reporting guidelines for health research (eg, STROBE, CONSORT) and items for reporting research on medication adherence was scrutinised and eliminated where needed.

    The resulting initial item list consists of 26 items distributed in two sections. The first section includes four items that reflect the conceptualisation of medication adherence as put forward by the ABC taxonomy for medication adherence and represent the taxonomy-based minimum reporting criteria. The second section includes 22 items specific to medication adherence research reporting and organised in a way congruent with common sections of reporting guidelines for major study types (eg, STROBE, CONSORT).

    Decision rules and definition of consensus on relevance and clarity of items

    Since the definition of consensus varies among Delphi studies according to the aims of each study, rating scales along with consensus rules for this study were inspired by the content validity index23 as it simplifies the decision-making process as explained below. Square (A) in figure 1 shows how the below decision rules will work together and their possible outcomes.

    Figure 1
    Figure 1

    An overview of the Delphi process.

    Item-level decision-making rules

    Scores for relevance will be used to decide on keeping or deleting an item. Consensus on relevance of an item is defined as 70% of the experts in the sample giving this item a score of 3 or more on a scale from 1 to 4 (1: not relevant; 2: somewhat relevant; 3: quite relevant; 4: highly relevant) during any survey round. Hence, this item will be kept on the final item list. Consensus on irrelevance of an item is defined as 70% of the respondents giving this item a score of 2 or less during any survey round. Consequently, this item will be deleted from the item list.

    Scores for clarity of wording will subsequently guide fine-tuning of the wording of the respective items to be included on the list. For simplifying the procedures, only one rule will be used for consensus on lack of clarity, defined as 70% of the respondents giving this item a score of 2 or less on a scale from 1 to 4 (1: not clear; 2: somewhat clear; 3: quite clear; 4: highly clear) during any survey round. Consensus on lack of clarity for any item will further lead to using comments provided by the experts to modify the wording of each corresponding item. This will be done by the steering committee after the rounds end.

    List-level decision-making rule

    One of the key methodological criteria of Delphi studies is having a stoppage rule on when the survey rounds will stop.22 For this purpose, a stoppage rule of having a consensus on relevance for 80% of all items on the list will be used. In other words, the Delphi rounds will be stopped once 80% of the items on the list at that point of time receive consensus on relevance (ie, 80% of the items received a score ≥3 by 70% of the experts). This rule will be applied from the first round if no new items will be suggested or from the second round after the experts will have provided scores for any suggested new items.

    Study procedures

    Once the initial item list and the participants’ sample are ready, the study will start with a preparatory period of 2 weeks to set up the online environment of the survey and test its functionality. Afterwards, the survey rounds will start and continue until the stoppage rule is fulfilled. An invitation for a feedback round will be sent to the expert group with a response deadline of 2 weeks after the invitation day. Reminders will be sent to those who will not respond, or will respond partially. Each round will be followed by an additional period for summarising and analysing the responses and integrating the results into the following version of the survey. An invitation for a next feedback round will be sent out as described before to the experts who will have completed all rounds from the beginning until then.

    The surveys will be conducted and the responses will be collected online via a survey platform (SurveyMonkey). Two members of the steering committee (SDG and RH) will be responsible for data collection and responding to possible inquiries from the experts. In case substantial issues are addressed by any member of the expert group, the other members of the committee will be consulted for advice and problem resolution.

    A more detailed explanation of the process is provided below:

    1. The first survey round will consist of:

      1. Providing a score for each of the items in view of relevance to the topic and clarity of wording;

      2. Justifying the scores chosen and/or suggesting modifications for each item—(optional—free-form text);

      3. Suggesting additional new items—(optional—free-form text).

    Additionally, the experts will provide information on their demographics, professional background and specific areas of expertise in adherence research for descriptive purposes as well as confirming their eligibility in the first survey round.

    Scores provided for the items will be summarised by their percentages and frequency distribution and inclusion or deletion of items in the following version of the survey will be guided by the aforementioned decision rules. Comments on potential adaptations of item wording will be summarised and integrated by the steering committee where deemed relevant.

    If no new items will be suggested and consensus on relevance is reached for 80% of the items on the initial item list, the survey rounds will be stopped. Otherwise, the scores and comments of the items that will not have reached consensus on relevance will be presented in the following round and/or suggestions of new items will be summarised and integrated into the following version of the survey. Further evaluation and decision-making will follow the methodology described before.

    1. The second round: an adapted item list including the scores and comments of the initial items as well as any new items from the first round will be sent to the experts. They will be invited to:

      1. Revise their opinion with new relevance and clarity scores, on the same scale as mentioned before, and comments for items that will not have achieved consensus on relevance in the first round;

      2. Score each of the new items in view of relevance to the topic and clarity of wording, on the same scale as mentioned before;

      3. Justify the scores chosen and/or suggest modifications for each item—(optional—free-form text).

    If list-level consensus is not reached by the second round, scores and comments will be summarised as described in the first round and integrated into the item list to be presented in further rounds until consensus is reached.

    1. Further rounds will consist of presenting consolidated feedback for all items that will not have achieved consensus on relevance in previous rounds with a chance for experts to revise their opinion accordingly with:

      1. New relevance and clarity scores for the remaining items;

      2. Justification for the scores chosen and/or modifications for each item—(optional—free-form text).

    Finally, the results of the final round will be consolidated and presented to the steering committee for proceeding with further steps. The flow chart in figure 1 delineates the survey rounds based on the input and output of each round.

    Ethical and legal considerations

    Since this study does not use health data of individuals, ethics approval is not required according to the Swiss federal act on research involving human beings. Participants in the Delphi study will be asked to provide informed consent to have their responses included in further analysis and dissemination of the results. Furthermore, they will be informed about confidentiality of the data and the corresponding legal obligations of not exposing such data to third parties. Additionally, they will be asked whether they would like to be acknowledged in the corresponding publications and dissemination of the guidelines. All data relevant to the study will be kept on password-encrypted computers which can be accessed by the steering committee only.

    Further planning of guidelines development

    The list of items reached through this study will be integrated by the steering committee into the planned EMERGE. The guidelines will be disseminated through peer-reviewed publications, presentations at conferences of ESPACOMP and other relevant organisations, and registration of the guidelines with the EQUATOR network. Endorsement of the guidelines by journals and relevant professional organisations will be encouraged. Two members of the steering committee (SDG and RH) will remain available to receive feedback and criticism after publication. Accordingly, further updates and revisions of the guidelines will be considered on an annual basis during ESPACOMP annual meetings, based on the EQUATOR guidance for developers of health research reporting guidelines.

    Outlook

    Medication non-adherence is a public health threat that causes poor patient outcomes and increased economic burden.24 ,25 The quality of medication adherence research as well as the quality of its reporting will determine the development and testing of effective and innovative solutions to enhance patients' adherence to medications and is therefore of paramount importance to many stakeholders. EMERGE aims at guiding researchers to report relevant aspects of medication adherence research in a standard manner. The use of the guidelines, in combination with other existing guidelines like STROBE or CONSORT, is expected to facilitate this task and, subsequently, help research in the medication adherence field advance towards achieving its ultimate goal of improved outcomes.

    Delphi study status

    The initial item list was developed and the online survey environment was set up and tested by the steering committee. The sample of experts was chosen and data collection (first round) started in June 2016.

    References

    1. ↵
      1. Bender B,
      2. Milgrom H,
      3. Apter A
      . Adherence intervention research: What have we learned and what do we do next? J Allergy Clin Immunol 2003;112:489–94. doi:10.1016/S0091-6749(03)01718-4
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Demonceau J,
      2. Ruppar T,
      3. Kristanto P, et al
      . Identification and assessment of adherence-enhancing interventions in studies assessing medication adherence through electronically compiled drug dosing histories: a systematic literature review and meta-analysis. Drugs 2013;73:545–62. doi:10.1007/s40265-013-0041-3
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Oberjé EJ,
      2. de Kinderen RJ,
      3. Evers SM, et al
      . Cost effectiveness of medication adherence-enhancing interventions: a systematic review of trial-based economic evaluations. Pharmacoeconomics 2013;31:1155–68. doi:10.1007/s40273-013-0108-8
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. von Elm E,
      2. Altman DG,
      3. Egger M, et al
      . Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 2007;335:806–8. doi:10.1136/bmj.39335.541782.AD
      OpenUrlFREE Full Text
    5. ↵
      1. Schulz KF,
      2. Altman DG,
      3. Moher D, et al
      . CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c332. doi:10.1136/bmj.c332
      OpenUrlFREE Full Text
    6. ↵
      1. Hoffmann TC,
      2. Glasziou PP,
      3. Boutron I, et al
      . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687. doi:10.1136/bmj.g1687
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Cobo E,
      2. Cortes J,
      3. Ribera JM, et al
      . Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ 2011;343:d6783. doi:10.1136/bmj.d6783
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Turner L,
      2. Shamseer L,
      3. Altman DG, et al
      . Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev 2012;1:60. doi:10.1186/2046-4053-1-60
      OpenUrlCrossRefPubMed
    9. ↵
      1. Peterson AM,
      2. Nau DP,
      3. Cramer JA, et al
      . A checklist for medication compliance and persistence studies using retrospective databases. Value Health 2007;10:3–12. doi:10.1111/j.1524-4733.2006.00139.x
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Gwadry-Sridhar FH,
      2. Manias E,
      3. Zhang Y, et al
      . A framework for planning and critiquing medication compliance and persistence research using prospective study designs. Clin Ther 2009;31:421–35. doi:10.1016/j.clinthera.2009.02.021
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Hutchins DS,
      2. Zeber JE,
      3. Roberts CS, et al
      . Initial medication adherence-review and recommendations for good practices in outcomes research: an ispor medication adherence and persistence special interest group report. Value Health 2015;18:690–9. doi:10.1016/j.jval.2015.02.015
      OpenUrl
    12. ↵
      1. Williams AB,
      2. Amico KR,
      3. Bova C, et al
      . A proposal for quality standards for measuring medication adherence in research. AIDS Behav 2013;17:284–97. doi:10.1007/s10461-012-0172-7
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Vrijens B,
      2. Geest S de, et al
      . A new taxonomy for describing and defining adherence to medications. Br J Clin Pharmacol 2012;73:691–705. doi:10.1111/j.1365-2125.2012.04167.x
      OpenUrlCrossRefPubMed
    14. ↵
      1. Brown BB
      . Delphi process: a methodology used for the elicitation of opinions of experts. Santa Monica, CA: RAND Corporation 1968. http://www.rand.org/pubs/papers/P3925.html
    15. ↵
      1. Moher D,
      2. Schulz KF,
      3. Simera I, et al
      . Guidance for developers of health research reporting guidelines. PLoS Med 2010;7:e1000217. doi:10.1371/journal.pmed.1000217
      OpenUrlCrossRefPubMed
    16. ↵
      1. Montgomery P,
      2. Grant S,
      3. Hopewell S, et al
      . Protocol for CONSORT-SPI: an extension for social and psychological interventions. Implement Sci 2013;8:99. doi:10.1186/1748-5908-8-99
      OpenUrl
    17. ↵
      1. Slade SC,
      2. Dionne CE,
      3. Underwood M, et al
      . Standardised method for reporting exercise programmes: protocol for a modified Delphi study. BMJ Open 2014;4:e006682. doi:10.1136/bmjopen-2014-006682
      OpenUrlAbstract/FREE Full Text
    18. ↵
      1. Sinha IP,
      2. Smyth RL,
      3. Williamson PR
      . Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011;8:e1000393. doi:10.1371/journal.pmed.1000393
      OpenUrlCrossRefPubMed
    19. ↵
      1. Dalkey NC
      . The Delphi method: an experimental study of group opinion. Santa Monica, CA: RAND Corporation 1969. http://www.rand.org/pubs/research_memoranda/RM5888.html
    20. ↵
      1. Hsu C,
      2. Sandford BA
      . The Delphi technique: making sense of consensus. Pract Assess Res Eval 2007;12. http://pareonline.net/getvn.asp?v=12&n=10 (accessed 29 Jun 2016).
    21. ↵
      1. Salkind NJ,
      2. Rasmussen K
      . Encyclopedia of measurement and statistics. Thousand Oaks, CA: SAGE Publications, 2007:243.
    22. ↵
      1. Diamond IR,
      2. Grant RC,
      3. Feldman BM, et al
      . Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014;67:401–9. doi:10.1016/j.jclinepi.2013.12.002
      OpenUrlCrossRefPubMed
    23. ↵
      1. Davis LL
      . Instrument review: Getting the most from a panel of experts. Appl Nurs Res 1992;5:194–7. doi:10.1016/S0897-1897(05)80008-4
      OpenUrlCrossRef
    24. ↵
      1. DiMatteo MR,
      2. Giordani PJ,
      3. Lepper HS, et al
      . Patient adherence and medical treatment outcomes: a meta-analysis. Med Care 2002;40:794–811. doi:10.1097/01.MLR.0000024612.61915.2D
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Iuga AO,
      2. McGuire MJ
      . Adherence and health care costs. Risk Manag Healthc Policy 2014;7:35–44. doi:10.2147/RMHP.S19801
      OpenUrlPubMed
    View Abstract

    Footnotes

    • Contributors All persons listed as authors contributed to preparing the manuscript and the International Committee of Medical Journal Editors (ICMJE) criteria for authorship were met. Specifically, the following contributions were made by the respective authors: RH contributed substantially to the conception and design of the study, and the acquisition, analysis, and interpretation of data for the work; drafted the work and revised it critically; approved the final version to be published; agrees to be accountable for all aspects of the work. JD-J, DAH, BV, IBW and LLZ contributed substantially to the acquisition, analysis and interpretation of data for the work; revised the work critically; approved the final version to be published; agree to be accountable for all aspects of the work. SDG is the project's principal investigator who contributed substantially to the conception and design of the study, and the acquisition, analysis, and interpretation of data for the work; drafted the work and revised it critically; approved the final version to be published; agrees to be accountable for all aspects of the work.

    • Funding This study is funded by the European Society for Patient Adherence, COMpliance, and Persistence (ESPACOMP).

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.