Objectives The aim of this Delphi survey was to establish an international consensus on the most useful outcome measures for research on the effectiveness of non-pharmacological interventions for migraine. This is important, since guidelines for pharmacological trials recommend measuring the frequency of headaches with 50% reduction considered a clinically meaningful effect. It is unclear whether the same recommendations apply to complementary (or adjunct) non-pharmacological approaches, whether the same cut-off levels need to be considered for effectiveness when used as an adjunct or stand-alone intervention, and what is meaningful to patients.
Setting University-initiated international survey.
Participants The expert panel was chosen based on publications on non-pharmacological interventions in migraine populations and from personal contacts. 35 eligible researchers were contacted, 12 agreed to participate and 10 completed all 3 rounds of the survey. To further explore how migraine patients viewed potential outcome measures, four migraine patients were interviewed and presented with the same measurement tools as the researchers.
Procedures The initial Delphi round was based on a systematic search of the literature for outcome measures used in non-pharmacological interventions for headache. Suggested outcome measures were rated by each expert, blinded towards the other members of the panel, for its usefulness on a 5-point Likert scale ranging from definitely not useful to extremely useful. Results were combined using median values and IQRs. Tools rated overall as definitely or probably not useful were excluded from subsequent rounds. Experts further suggested additional outcome measures that were presented to the panel in subsequent rounds. Additionally, experts were asked to rank the most useful tools and provide information on feasible cut-off levels for effectiveness for the three highest ranked tools.
Results Results suggest the use of the Migraine Disability Assessment (MIDAS), Headache Impact Test (HIT-6) and headache frequency as primary outcome measures. Patient experts suggested the inclusion of a measure of quality of life and evaluation of associated symptoms and fear of attacks.
Conclusions Recommendations are for the use of the MIDAS, the HIT-6 and headache frequency, in combination with an outcome measure for quality of life. Associated symptoms and fear of attacks should also be considered as secondary outcomes, if relevant for the individual target population. The cut-off level for effectiveness should be lower for non-pharmacological interventions, especially when used as an adjunct to medication.
Trial registration number German Register of Clinical Trials (DRKS00011777)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This manuscript provides an international expert consensus on outcome measures for trials evaluating the effectiveness of a non-pharmacological intervention for the reduction of migraine symptoms.
A Delphi survey with three rounds was conducted to reach a consensus.
Additional patient experts views contributed to a broader understanding on useful outcome measures including aspects of quality of life and attack-associated fear.
A limitation is that patients were not included in the Delphi process but interviewed on a separate occasion.
Migraine is a prevalent1 2 and disabling condition,3 with high socioeconomic costs mainly due to days off work.4–9 Migraine may become chronic when it occurs on ≥15 days per month for at least 3 months with features of migraine in ≥8 days.10
Guidelines for chronic and frequent episodic migraine recommend the use of non-pharmacological approaches as an adjunct or alternative to the first-line pharmacological management.11–16 Patients welcome this recommendation and usually report the use of complementary interventions to reduce the amount of acute pain medication.17 18 Especially, those patients who do not tolerate acute and/or prophylactic medicine because of side effects, have contraindications due to co-morbidities or may wish to avoid medication for other reasons, seek non-pharmacological alternatives.19 20 Recommended strategies include relaxation, cognitive-behavioural interventions and aerobic exercise; other interventions are mentioned but not recommended due to a lack of evidence for their effectiveness,11–16 primarily due to methodological shortcomings.21 22
Additional research is urgently needed to identify whether these other non-pharmacological treatment strategies are effective to complement pharmacological management as an adjunct or, in cases where patients do not tolerate acute and/or prophylactic medications because of side effects, have contraindications due to co-morbidities of other diseases or wish to avoid medication for other reasons, can manage their frequent episodic or chronic migraines prophylactically. However, before determining the effectiveness of either intervention, it is important to decide which outcome measures may be useful to indicate change following a non-pharmacological intervention in migraine patients. The International Headache Society (IHS) recommends the use of headache frequency as the primary outcome measure, usually assessed as headache days per month documented in a headache diary.23 However, this recommendation refers to studies investigating the effect of preventive medication. A guideline for behavioural trials confirms frequency as the primary outcome measure.24 No other clinical guidelines determining the main outcome for primary headache disorders exist for non-pharmacological trials. Additionally, no study has investigated whether the number of headache days is an improvement which is meaningful to patients. Therefore, it is unclear whether clinical trials investigating the effectiveness of treatments such as massage, passive mobilisation or manipulation, physical activity or any other non-pharmacological intervention as an adjunct or as a stand-alone intervention should also use headache frequency as a primary outcome measure, and at what cut-off level the intervention can be considered successful.
The cut-off level stated by the IHS and the guideline for behavioural treatment for the effectiveness of a treatment is a reduction of ≥50% of headache frequency.23 24 Both guidelines, however, acknowledge that these levels might be lowered for studies on chronic migraine populations or reported as ‘somewhat improved’ if a responder rate >25% is reached. Non-pharmacological interventions have a low risk of adverse events (AEs) and are less expensive. Intuitively, these approaches should therefore not be measured on the same scale as preventive medication since medication has a high risk of AEs. To reach a 50% level of improvement is particularly unrealistic if non-pharmacological interventions are evaluated as an add-on to the pharmacological management, especially since the net effect would then need to reach 75%.
The objective for this Delphi survey was to establish an international consensus on the most useful outcome measures for assessing the effectiveness of non-pharmacological interventions for frequent episodic or chronic migraine. A secondary aim of the Delphi study was to investigate whether the same level of improvement, that is, ≥50% in headache frequency, is required to conclude on its effectiveness as recommended for preventative migraine medication, when used as an adjunct or a stand-alone intervention.
The methods for this Delphi survey were based on published guidelines25 as well as on previous research.26 The data collection was conducted between October 2016 and August 2017. Selection of experts was based on personal contacts and a systematic search for primary authors in peer-reviewed journals on non-pharmacological intervention studies in headache populations. It must be noted that there is no guideline to define who is an expert for a Delphi survey or how many experts are required.27 Participants remained anonymous during all rounds to allow the expression of their opinions without the influence of dominant characters.28 After the final round, all experts were invited to co-author the report and subsequently received a first draft for revision and a final draft for approval prior to submission. The survey procedures were predesigned and included the following steps:
I. The initial survey was guided by a systematic evaluation of the literature on non-pharmacological interventions for migraine. PubMed was searched for articles published prior to 01.04.2017 using the following search strategy:
((((headache [Title/Abstract] OR migraine [Title/Abstract])) AND ((aerobic training or sports or exercise or acupuncture or chiropractic spinal manipulative therapy or progressive muscle relaxation or behavioural migraine management or cognitive behavioural therapy or biofeedback or physical activity or transcutaneous electrical nerve stimulation)) AND adult Filters: Humans.
II. In the first Delphi round, experts were asked to rate the usefulness of the outcome measures identified in step 1 for the use in non-pharmacological trials for frequent episodic or chronic migraine, on a 5-point Likert scale (1=‘extremely useful’, 2=’useful’, 3=‘do not know’, 4=‘probably not useful’, 5=‘definitely not useful’). Experts were further invited to provide and suggest additional outcome measurement tools that they also considered relevant. To allow for an informed decision on test properties and previous application in research, references were provided for each outcome measure. To promote a more reliable decision on outcome measures, experts were additionally asked to rank their three favourite tools in a hierarchical order and to provide the cut-off levels for effectiveness which they perceived feasible for non-pharmacological interventions for frequent episodic or chronic migraine. The questionnaire used for the rating procedure was sent as a word document via email after it had been tested among working group members for comprehensibility. If individual ratings were missing or questionnaires were not returned, reminder emails were sent.
III. For the second round, group responses were analysed and fed back to the experts (via email) including the new tests suggested. These were again rated for their usefulness as in round one. To provide more information on the specific tools, experts were invited to comment on the individual tools.
IV. A third round was conducted in the same manner as the second round to include all new information that emerged from round two and to reach a final agreement.
V. Definition of consensus: no further rounds are conducted if no new outcome measures are suggested. Median values across experts were used to identify the most useful tools. Consensus on the relevance of an outcome measure was assumed if 75% of the participants rated the tool as ‘useful’ or ‘extremely useful’. All outcome measures, rated as ‘definitely not useful’ or ‘probably not useful’ were excluded from subsequent Delphi rounds (based on the median value across experts). If multiple tools meet this definition of consensus, a hierarchical ranking was used to determine the tool considered most useful from the perspective of the experts in the panel.
All tools were analysed using the median and IQR across ratings and presented in box plots. Hierarchical ranking was analysed by allocating the highest-ranking tool 50 points, the second place 30 points and the third place 20 points. Sums across all ratings for each tool were calculated to identify the highest-ranking outcome measure. Comments on the outcome measures were summarised in a table with general quotes provided by the experts and for each tool.
Patient and public involvement
No patient involvement. Patient’s views were expressed and integrated into the results as stated in the section on ‘patients’ views’.
Patients repeatedly stated that there was so much more to their suffering than the number of headache days in their diaries. Rather than including them in the Delphi process, patients were invited to take part as experts for their symptoms in this study by being interviewed. It was anticipated that patients’ views were more multifaceted and diverse and that the ranking tasks requested during the Delphi rounds would do their opinions not sufficient justice. Therefore, in order to include the preferences of the target group, four women, known to the researchers from the university headache clinic, diagnosed with chronic migraine with >10 years history of migraine were recruited into the study. They were experienced in pharmacological and non-pharmacological trials and knowledgeable on a wide range of outcome measures throughout their experiences at a specialised university outpatient headache clinic. Patients were presented with the same outcome measures as the research experts and interviewed regarding their preference and opinion on the different tools. They were further asked whether they regarded a≥50% change in the number of headache days as an acceptable outcome level of improvement to justify a non-pharmacological treatment. Two patients were interviewed as a group to stimulate discussion and two were interviewed by telephone. Interviews were held at the outpatient headache clinic at the University Medical Center Hamburg-Eppendorf.
The interviews were recorded on a digital voice recorder. After the interviews, two researchers transcribed the interviews independently on personal computers to avoid potential risks of mishearing and misinterpretation.29 Transcripts were compared and discussed before they were coded and analysed.
A thematic analysis approach was used by categorising questions, themes and quotations.30 To focus on the research question, only those quotations were chosen in which the participants explained their views about the outcome measures.
A total of 35 international experts from seven countries were identified and invited via email to participate. The invitation included the researcher's details and meticulous information about the survey and the study design. Twelve experts agreed to participate and completed the first round, and 10 experts completed all three rounds of the survey.
Delphi round 1
The literature search identified seven different outcome measures used in studies evaluating the effectiveness of non-pharmacological interventions in migraine populations: Henry Ford-Hospital Headache Disability Inventory (HDI), Headache Impact Test (HIT-6), Migraine Disability Assessment (MIDAS), Numerical Rating Scale (NRS), Pain Disability Index (PDI), Short Form-36 (SF-36) and Short Form McGill Pain Questionnaire (SF-MPQ). These instruments were evaluated by a panel of 12 experts (eight physiotherapists, two chiropractors, two psychologists). All experts had an academic degree equivalent to PhD except one expert with an MSc. From these seven initial tests, PDI and SF-MPQ showed a median value of 3 (rated by three and four experts, respectively, as ‘do not know’) while the remaining tests showed a median rating of 2 (‘useful’) (figure 1). No outcome measure was discharged after this initial round. The ranking task placed the MIDAS first, followed by the HIT-6 and the NRS by the experts.
Ten additional outcome measures were suggested by the experts: Brief Pain Inventory, Craniofacial Pain and Disability Inventory (CF-PDI), Functional Disability Inventory (FDI), Fragebogen zu Kopfschmerzmanagement und Selbstwirksamkeit (FKMS), Headache Diary, Headache Frequency, Migraine Specific Quality of Life, Veterans RAND 12 Item Health Survey (VR-12), Rostocker Kopfschmerz-Fragenkomplex (RoKoKo) and DoloTest.
Delphi round 2
These outcome measures were rated by the experts participating in the second round (n=10). Two ratings (one on the RoKoKo and one on the FKMS) were missing. From the 10 newly evaluated outcome measures, headache diary and headache frequency received median ratings of ‘extremely useful’, while CF-PDI and FDI were excluded as ‘definitely not useful’ and ‘probably not useful’, respectively (figure 2). There is no English version available for the RoKoKo and the FKMS, making the feasibility of this tool not useful for English-speaking researchers. Comments provided by experts were summarised in a table with numbers behind quotes indicating the number of experts who expressed this opinion (table 1).
The revised ranking based on outcome measures from rounds one and two indicated that the MIDAS was the most useful tool, followed by the HIT-6 and the headache frequency. Headache diary, PDI and NPRS shared rank four. Suggested cut-off levels for each tool, indicating the acceptable effectiveness of a non-pharmacological intervention when used as an adjunct treatment to medication or as a stand-alone intervention, are listed in table 2.
Results from the patient interviews
All patients agreed that the SF-36 appeared to be the tool which best reflected their well-being. This was followed by the MIDAS, the HIT-6 and the HDI. The least useful tool from the patients’ point of view were NRS and McGill.
Patients commented that some aspects of their migraine were missing in the measuring tools available. One patient stated that sometimes her aura was the dominant problem but that this was never mentioned as an option in any of the tools. Another patient suggested adding a measure for fear of the next migraine attack. The fear itself was sometimes more disabling than any other symptom for this patient.
When asked about the minimum relevant change levels acceptable, patients agreed that any improvement of any of their symptoms would justify seeking a non-pharmacological treatment. A 50% reduction of headache days did not seem to be realistic for any patient for a non-pharmacological treatment. They explained how difficult it was to identify changes following a treatment because of the unpredictability of migraine. Quotations underlining the patients’ points of view are presented in table 3.
The present study reports on an international Delphi survey, aiming to reach consensus on the measurement tools to be used in non-pharmacological interventions for migraine. MIDAS, HIT-6 and headache frequency reached the definition of consensus and were ranked as the most relevant tools by the Delphi participants; conversely, patients valued measures of quality of life higher and suggested to measure additional aspects such as fear of the next attack.
While the outcome measures that were suggested by experts in this study largely reflect those used in pharmacological trials, MIDAS and HIT-6 were prioritised over headache frequency, potentially indicating that aspects of suffering and disability were valued higher than the simple counting of headache days.
Expert opinions varied widely with most outcome measures receiving ratings ranging from ‘extremely useful’ to ‘probably or definitely not useful’, indicating that none of the suggested outcome measures was perceived as ideal. This is partially explained by comments from experts and two of the four patients stating that the MIDAS is often misunderstood and not answered correctly and should be simplified. Furthermore, recalling a 3-month period might introduce bias and reduce precision of the measurement. The general criticism about the HIT-6 was that it does not accommodate the complexity of migraine and its impact on individuals. To prevent researchers from using different tools in future research and thereby not allowing for the comparability of results, these limitations should be addressed by, for example, providing a version of the MIDAS only reflecting on the past 4 weeks rather than on the past 3 months.
The highest rated outcome measures also reflected those best evaluated in the literature. MIDAS has been validated and translated in many languages31–36 and has shown good psychometric properties.31–39 Similar research is available on HIT-6.34 40–43 Some of the tools suggested were too specific for a particular pathology, such as the CF-PDI and FDI and are therefore only applicable in situations when painful temporomandibular dysfunctions play a dominant role. Others were too broad such as the SF-36 or VR-12 or PDI and therefore not considered as a specific outcome measure for studies on migraine. This might also be the reason, why no global rating tool, such as Patient Global Rating of Change was suggested or discussed by experts. Interestingly, patients viewed this topic differently by stating a clear preference for the SF-36. Patients also commented that the associated symptoms can be more disabling than the headache itself, such as the aura or fear of the next migraine attack. The migraine-specific quality of life questionnaire covers fear of the next attack and has been evaluated in various languages.44–50 It was initially unclear why the tool was not popular with the experts in this study. Few had used it because although free for researchers, it is only available from the company GlaxoSmithKline and the included 25 items require more time than the MIDAS or the HIT-6. Fear of migraine can also be investigated by a 4-item tool which has recently been developed.51 This tool has not been used in previous published trials on non-pharmacological treatments and was not suggested by the Delphi panel. Associated symptoms have largely been ignored in research. Wang et al recently reported on the development of a tool to capture these symptoms, but psychometric properties and testing procedures are not yet available.52
The commonly agreed cut-off value for effectiveness in pharmacological trials is a ≥50% reduction in headache days53 with an accepted reduction to a responder rate of 30% in chronic migraine populations.23 Previous research has identified the minimal clinically relevant change for HIT-6 to be −2.5 to −6 points in migraine.42 No recommendation was identified for the MIDAS. A ≥50% reduction of headache frequency is a target that is difficult to reach with non-pharmacological interventions, especially when provided as an adjunct to preventive medication and especially in chronic migraine. Lowering the target should be considered, especially since AEs are few, mild and transient,54 and most of the interventions are low cost. However, based on the expert panel of this survey, there was no clear agreement on a specific cut-off level for any of the tools, but it seems that more realistic targets are needed to show the true effect of a non-pharmacological intervention. A limitation of the survey design was that cut-off levels for effectiveness were not distinguished for studies using non-pharmacological interventions as an adjunct treatment to, for example, prophylactic medication and studies using non-pharmacological interventions as a stand-alone treatment. However, in both situations a lower target should be used, since non-pharamcological interventions do not (or to a much lower extent) have to consider the trade-off between effect and side-effects. Patients confirmed that any small improvements of any of their symptoms would justify the time and money spent on the intervention.
For future research on non-pharmacological interventions for frequent episodic or chronic migraine, it is highly important to use the same primary and/or secondary outcome measure to be able to compare and combine results to reach a higher level of evidence that can subsequently be included in national and international guidelines. At this stage, there does not seem to be an ideal outcome measure. Neither does it seem to be clear which aspects of migraine (such as its intensity, its impact on a person’s life, it’s uncertainty…) are the most important aspects to be measured. Within these limitations and with the literature currently available, we promote the use of the MIDAS, the HIT-6 and headache frequency, and an outcome measure for quality of life (eg, SF-36), which was preferred by patients and recently identified as a useful indicator for change,55 in addition to the newest clinical guideline recommendations by the IHS53 when conducting future non-pharmacological randomised clinical trials in patients with frequent episodic or chronic migraine. Associated symptoms and fear of attacks should be considered as secondary outcomes.
We recognise some potential limitations of the current study. First, there was a limited number of experts available to complete all three rounds of the Delphi process. However, they represented different professional backgrounds and reflect the variability in non-pharmacological migraine-research. Only four patients were interviewed; these were chosen based on their longstanding migraine history and previous participation in research trials, thus, acquiring a high level of expertise on outcome measures in relation to their own suffering levels. Outcome measures were presented to experts in a non-randomised order. While this might influence responses, it also helped to keep experts oriented by using a standardised order of outcome measures and response options.
There was some diversity between experts’ and patients’ opinions on the most important outcome measures for non-pharmacological trials. Recommendations are for the use of the MIDAS, the HIT-6 and headache frequency, in combination with an outcome measure for quality of life. Associated symptoms and fear of attacks should also be considered as secondary outcomes, if relevant for the individual target population. The cut-off level for effectiveness might be lower for non-pharmacological trials, especially when these are used as an adjunct to medication.
We would particularly like to thank the four migraine patients who shared their expertise during the focus group meetings and the telephone interviews.
Contributors All authors contributed substantially to the acquisition, analysis and interpretation of data. All authors revised it critically for important intellectual content. All authors approved of the final version submitted. KL and BW had the research idea and supervised the master students AB, SB, CS and MG who conducted the data acquisition. RC, AC, DF, CFdlP, TH, GJ, PK, BKM, BS, ES, PT and HvP were participants of the survey and after the data analysis invited to be co-authors of the paper to ensure that all statements were summarised correctly and to present the result as a recommendation supported by an international team of experts. As stated above, all authors substantially contributed to the manuscript by revising, writing and introducing ideas.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethical approval was granted by the local ethics authority (University of Hamburg, Germany, number 2017_89).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.