You are here

Article Text

Article menu

Download PDFPDF

XML
Health economics
Research
Are complementary therapies and integrative care cost-effective? A systematic review of economic evaluations
  1. Patricia M Herman1,
  2. Beth L Poindexter2,
  3. Claudia M Witt3,4,
  4. David M Eisenberg5,6,7
  1. 1Center for Health Outcomes and PharmacoEconomic Research, College of Pharmacy, University of Arizona, Tucson, Arizona, USA
  2. 2Zuckerman College of Public Health, University of Arizona, Tucson, Arizona, USA
  3. 3Institute for Social Medicine, Epidemiology and Health Economics, Charite’ University Medical Center, Berlin, Germany
  4. 4Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, USA
  5. 5Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
  6. 6Harvard School of Public Health, Boston, Massachusetts, USA
  7. 7Samueli Institute, Alexandria, Virginia, USA
  1. Correspondence to Dr Patricia M Herman; pherman{at}rand.org.

Abstract

Objective A comprehensive systematic review of economic evaluations of complementary and integrative medicine (CIM) to establish the value of these therapies to health reform efforts.

Data sources PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE were searched from inception through 2010. In addition, bibliographies of found articles and reviews were searched, and key researchers were contacted.

Eligibility criteria for selecting studies Studies of CIM were identified using criteria based on those of the Cochrane complementary and alternative medicine group. All studies of CIM reporting economic outcomes were included.

Study appraisal methods All recent (and likely most cost-relevant) full economic evaluations published 2001–2010 were subjected to several measures of quality. Detailed results of higher-quality studies are reported.

Results A total of 338 economic evaluations of CIM were identified, of which 204, covering a wide variety of CIM for different populations, were published 2001–2010. A total of 114 of these were full economic evaluations. And 90% of these articles covered studies of single CIM therapies and only one compared usual care to usual care plus access to multiple licensed CIM practitioners. Of the recent full evaluations, 31 (27%) met five study-quality criteria, and 22 of these also met the minimum criterion for study transferability (‘generalisability’). Of the 56 comparisons made in the higher-quality studies, 16 (29%) show a health improvement with cost savings for the CIM therapy versus usual care. Study quality of the cost-utility analyses (CUAs) of CIM was generally comparable to that seen in CUAs across all medicine according to several measures, and the quality of the cost-saving studies was slightly, but not significantly, lower than those showing cost increases (85% vs 88%, p=0.460).

Conclusions This comprehensive review identified many CIM economic evaluations missed by previous reviews and emerging evidence of cost-effectiveness and possible cost savings in at least a few clinical populations. Recommendations are made for future studies.

  • Complementary Medicine
  • Health Economics
  • Statistics & Research Methods

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

https://doi.org/10.1136/bmjopen-2012-001046

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Article summary

    Article focus

    • Given the limited nature of previous systematic reviews, what is the extent of evidence on the economic impacts of complementary and integrative medicine (CIM)?

    • What are the range of therapies and populations studied, and the quality of published economic evaluations of CIM?

    • What are the results of the higher-quality, more recent (and likely most cost-relevant) economic evaluations of CIM?

    Key messages

    • This study's comprehensive search strategy identified 338 economic evaluations of CIM, including 114 full evaluations published 2001–2010.

    • The cost-utility analyses found were of similar or better quality to those published across all medicine.

    • The higher-quality studies indicate potential cost-effectiveness, and even cost savings across a number of CIM therapies and populations.

    Strengths and limitations of this study

    • The strengths of this study are the comprehensive search strategy, the use of two reviewers, the use of multiple measures of study quality and the identification of higher-quality studies, for which results are reported in detail, via an objective short-list of quality criteria, which reduced the potential for bias.

    • The weaknesses of this study are similar to those of the other systematic reviews: reviewers were not blinded to journals and article authors, and some aspects of what makes a quality economic evaluation could not be judged from what was reported.

    • Publication bias was not assessed. However, it is not clear as to whether publication bias is relevant, given the purposes of this review.

    Introduction

    Between 1990 and 2007, four nationally representative surveys demonstrated that a third or more of US adults routinely used complementary and alternative medicine (CAM) therapies to treat their principal medical conditions.1–4 Total expenditures for CAM therapies were estimated at US$14 billion in 1990,1 US$27 billion in 19972 and US$34 billion in 2007.4 The 2007 US National Health Inventory Survey found that out-of-pocket expenditures for CAM therapies accounted for 11% of all out-of-pocket healthcare expenditures by Americans.4 Similar use numbers are seen in other countries.5–8 However, despite the popularity of and substantial expenditures on CAM therapies, their cost-effectiveness remains ill-defined and controversial.

    Economic evaluations allow costs to be included, alongside data on safety and effectiveness, in healthcare policy decisions. As healthcare costs rise, the availability of these economic evaluations becomes increasingly important to the formulation of disease management strategies which are both clinically effective and financially responsible. According to the National Center for Complementary and Alternative Medicine (NCCAM), CAM is ‘a group of diverse medical and healthcare systems, practices and products that are not generally considered part of conventional medicine’.9 In not being part of conventional medicine, individual complementary therapies and emerging models of integrative medicine (ie, coordinated access to both conventional and complementary care)—collectively termed as complementary and integrative medicine (CIM)—are often excluded in financial mechanisms commonly available for conventional medicine,2 and are rarely included in the range of options considered in the formation of healthcare policy. The availability of economic data could improve the consideration and appropriate inclusion of CIM in strategies to lower overall healthcare costs. In addition, economic outcomes are relevant to the licensure and scope of practice of practitioners, industry investment decisions (eg, the business case for integrative medicine), consumers and future research efforts (ie, through identifying decision-critical parameters for additional research10).

    A number of systematic reviews of economic evaluations of CIM have been published.11–23 Five of these prior reviews attempted to capture all economic evaluations of CIM therapies across all conditions.11 ,19–21 ,23 However, it is unclear as to whether all or even the majority of economic evaluations of CIM have been identified by these reviews. The searches are dated; the search strategy in the most recent review only captured articles published through 2007.23 The databases searched were limited—for example, only one used CINAHL,21 and only two others used EMBASE,19 ,23 in addition to Medline and AMED. Finally, these reviews generally used limited search terms to identify CIM studies. All but one only used variations on the terms ‘complementary’ or ‘alternative’ ‘medicine’ or ‘therapy’. Unfortunately, other reviewers have found that these search terms do not capture all CIM studies,24 ,25 which may be a reflection of the difficulty in defining what is and is not CIM.26 The search by Maxion-Bergemann et al11 also added individual therapies as search terms, but only included homeopathy, phytotherapy, traditional Chinese medicine, anthroposophic medicine and neural therapy. No search included ‘integrative medicine’.

    The goal of this paper is to identify, to the extent possible, all published economic evaluations of CIM, describe the types of CIM evaluated and the clinical conditions for which they have been evaluated, and identify the recent (and therefore, most cost-relevant) higher-quality studies and highlight their results for policy makers. We also make recommendations for future economic evaluations of CIM.

    Methods

    Six electronic databases were searched from their inception through December 2010: PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE. To be as comprehensive as possible, a combination of 11 medical subject headings (MeSH) and 39 other search terms were used (box 1). In addition, bibliographies of found articles and reviews were searched, and key researchers in various areas of CIM were contacted for their lists of known studies. Although non-English language articles were collected, they are not analysed in this review.

    Box 1

    Search terms used for the PubMed search: (Complementary Therapies (medical subject headings (MeSH)), Dietary Supplements (MeSH), Micronutrients (MeSH), Trace Elements (MeSH), Vitamins (MeSH), acupuncture, alternative medicine, ayurvedic medicine, chiropractic, biofeedback, collaborative medicine, complementary and alternative medicine, botanical medicine, complementary medicine, diet, energy medicine, herbal medicine, herbs, homeopathy, hypnosis, integrated medicine, integrative medicine, massage, meditation, mind-body medicine, minerals, naturopathic medicine, naturopathy, nutrients, nutritional supplements, relaxation, spa therapy, traditional Chinese medicine OR vitamins) AND (Cost-Benefit Analysis (MeSH), Cost Control (MeSH), Cost Savings (MeSH), Costs and Cost Analysis (MeSH), Economics (MeSH), economics (Subheading), Insurance (MeSH), cost benefit, cost effectiveness, cost identification, cost minimisation, cost utility, economic evaluation, insurance claims, managed care OR technology assessment). Searches in the other five databases used the same text words and (where available) analogous controlled vocabulary terms. All searches were restricted to human studies.

    Defining a comprehensive search strategy for CIM is challenging.24 ,27–29 There have been a number of efforts to develop a concise definition of CAM.26 ,30 This review used the one developed by the members of the Cochrane CAM Field31 and then added the terms ‘integrative’, ‘integrated’ and ‘collaborative’ medicine. The Cochrane CAM definition starts with the NCCAM definition9 and then refines it by specifically including all therapies ‘based upon the theories of a medical system outside the Western allopathic medical model’ (eg, traditional Chinese medicine and Reiki), and including others depending on the context and setting of their use. The context of use considers treatment/condition combinations and excludes those ‘currently considered to be standard treatment’, and the setting of use generally includes self-care and therapies delivered by CIM providers, but excludes therapies ‘delivered exclusively by conventionally credentialed medical personnel or exclusively within hospital settings’. Therefore, therapies such as chemotherapy regimens (eg, chronotherapy32), and therapies requiring surgical implantation (eg, neuroreflexotherapy33) or the placement of a feeding tube34 were not included even though these therapies appeared in our search. In cases where CIM therapies (eg, biofeedback or hypnosis) were included as part of a package of care (eg, with cognitive behavioural therapy), a judgement was made as to whether the CIM portion of the treatment made up half or more of the overall package of care under study. If so, the package of care was included as CIM. Because more than half of CIM users use multiple CIM therapies,35 studies of packages of therapies and coordinated care were identified as such.

    Articles were categorised as full economic evaluations if they compared the costs (inputs) and consequences (economic, clinical and/or humanistic outcomes36) of two or more therapeutic alternatives applied to the same patient population (ref.37, p. 11). Otherwise, they were considered partial evaluations, for example, cost-identification or cost-comparison studies.38 Studies that estimated resource utilisation were included as economic evaluations if the utilisation data were detailed enough to allow monetary valuation.

    Two reviewers (PMH and BLP) evaluated all articles for inclusion and extracted all data. Disagreements were resolved by discussion between the two review authors, or, if needed, by the other coauthors. Because the results of economic evaluations can rapidly lose relevance with time, mainly due to changes in practice patterns and cost structures, data were extracted only from the economic evaluations published 2001–2010. Extracted data were entered into an Excel template developed for a previous review.20 The type(s) of CIM evaluated and the target population were captured for all economic evaluations. Various indicators of study quality were captured for all full economic evaluations, and more detailed data and results were captured only for those full economic evaluations that met five quality criteria.

    The quality of an economic evaluation can be judged along two general dimensions: (1) whether the study was a quality measure of outcomes for its target population and location—that is, whether it was internally valid; and (2) whether enough information is provided for the study's results to be transferable (‘generalisable’).39 Health outcomes are to some extent considered generalisable across settings; however, because resource availability, practice patterns and relative prices can vary greatly, economic outcomes usually are not.40 Therefore, one goal in economic evaluation is to ensure the transferability of study results—that is, to provide enough study detail so that results can be adapted (usually via modelling) to apply to other settings.39 The 35-item BMJ checklist captures components of both dimensions of quality and was applied to all full economic evaluations.41 We also chose five quality criteria by which to identify a subset of full economic evaluations to highlight as being of most interest to policy makers. These quality criteria are based on recommendations made by the US Panel on Cost Effectiveness in Health and Medicine 42 and by well-known experts in the field,37 and focus on the quality of the underlying study (the first type of quality):

    • Because cost-effectiveness analysis (CEA) is comparative, to ensure that results are useful to decision makers, one of the alternatives to which the CIM intervention was compared must be some version of commonly available (routine, standard or usual) care.

    • The analysis must explicitly or implicitly use (and include all relevant costs from) at least one recognised perspective—for example, society, third-party payer, hospital or employer.

    • Since ‘an economic evaluation of a healthcare programme is only as good as the effectiveness data it is built upon’ (ref.43, p. 232), health outcomes must be from randomised controlled trials or non-randomised controlled trials using either statistical adjustment or matching to address baseline differences.

    • Since having patient-specific data on both costs and outcomes is an advantage for internal validity,44 resource use must be a measured outcome of the study. Modelling studies utilise the results of other published studies, therefore, are exempt from this criterion.

    • Because uncertainty in an economic evaluation comes not just from sample variation, but also from assumptions made,45 a sensitivity analysis is required.

    Because the prices used to value resources are highly location-specific and setting-specific,39 ,46 we also note, for the articles meeting the above criteria, the presence of a study reporting criterion essential for the transferability of study results (usually via modelling):39 ,40 separate reporting of unit costs from resource use for economic evaluations alongside trials, or from model parameters (eg, transition probabilities) for economic evaluations using models.

    Other data extracted for the economic evaluations which meet the five study-quality criteria are: treatment and study duration, primary clinical and economic outcome measures, the setting in which treatment took place, study design and sample size, the type (table 1) and perspective (ie, the point of view used to define costs) of the economic analysis, and incremental cost-effectiveness of the CIM alternative compared to usual care. Incremental cost-effectiveness is reported in 2011 US$ and is calculated from reported results by first converting the study currency to US$ using the Federal Reserve annual exchange rate47 for the study's currency year and then inflated to 2011 values using the medical care component of the Consumer Price Index.48

    View this table:
    Table 1

    Types of full economic evaluations

    Finally, up to three additional quality measures are included for each of these studies. The Tufts CEA Registry49 quality score is recorded when it was available (note it is only available for cost-utility analyses, CUAs). A Jadad score50 with minor modifications (the two possible points for blinding were replaced with one point for the use of a blinded assessor)51 was calculated for the economic evaluations that included a randomised trial. The percentage of the applicable items from the 35-item BMJ checklist that were met by each article is also reported.41

    Results

    As shown in figure 1, the database search identified 270 published economic evaluations. An additional 68 articles were added through the bibliography and expert-supplied list search for a total of 338 economic evaluations of CIM. Of these, 204 (60%) were published from 2001 through 2010 (114 full and 90 partial economic evaluations). Of the recent full economic evaluations almost all (103, 90%) examined the effect of one CIM therapy and most of the balance (10, 9%) examined the effect of two or more CIM therapies provided by the same practitioner. Only one looked at the effect of multiple CIM therapies provided by different CIM providers.52 CIM was generally evaluated as an adjunct to usual care.

    Figure 1
    Figure 1

    The flow of records and articles through the systematic review.

    As shown in table 2, the 204 economic evaluations published in the past 10 years are spread across a wide range of CIM therapies applied to a number of different study populations. The biggest concentration of full economic evaluations (19 in number) pertained to the use of NCCAM's definition of manipulative and body-based practices (eg, chiropractic, osteopathic manipulation, massage, etc) for the treatment of back pain.53–72 However, even this subgroup is fairly heterogeneous in terms of the therapy (or therapies) tested and/or the type of back pain treated. Eight of these comparisons involved chiropractic care for back pain; one for chronic,53 one for acute57and six for either type.59 ,60 ,63 ,64 ,67 ,68 Five evaluated spinal manipulation and manual therapy provided by physiotherapists for chronic back pain (one),65 acute back pain (two)58 ,69 or either (two).56 ,68 Four involved osteopathic manipulation; one for chronic71 and one for subacute back pain72 and two for musculoskeletal conditions including back pain.66 ,68 Three evaluated massage; two for chronic55 ,62 and one for acute back pain.57 The last two studies evaluated a musculoskeletal physician (treatment ‘with a combination of manual therapy, injections, acupuncture and other pain management techniques’) for orthopaedic referrals;54 and a Finnish folk medicine practice called ‘bone setting’ for the treatment of patients with chronic back pain.61

    Table 3 shows the results of the application of the 35-item BMJ checklist to the full economic evaluations published 2001–2010.41 On average, the number of applicable items met by each article stayed fairly constant during this period. However, the application of two key items (ie, the proper use of discounting and the inclusion of sensitivity analysis) and the disclosure of funding sources improved significantly, and reporting of the study time horizon worsened significantly. As expected, the average overall and individual-item percentages were higher for the higher-quality articles (those meeting the five study-quality criteria) and for CUAs of CIM. It is not surprising that CUA's quality is higher. They generally involve more effort than other CEAs and are required or recommended by various national guidelines.42 ,73–75 Nevertheless, it seems as though the quality of CUAs of CIM is generally comparable to, or slightly better than, that seen in CUAs across all medicine, at least in terms of the Tufts quality score, disclosure of funding sources and the five items where comparable data are available.76 ,77

    View this table:
    Table 2

    Types of individual complementary and integrative medicine (CIM) therapies studied for various conditions and in various populations: 2001–2010 (reported as the ratio of the total number of economic evaluations to the number of full economic evaluations)

    The number of full evaluations meeting each of the five study-quality criteria are: comparison to usual care 97 (85%), all costs from a recognised perspective 96 (84%), health outcomes from a randomised or matched-control trial 86 (75%), patient-specific data on costs and outcomes 105 (92%) and sensitivity analyses 37 (32%). Sixty-two (54%) of full evaluations met the first four of these and 31 (27%) met all five. A summary of the results of these 31 higher-quality articles (covering 28 different studies) is shown in table 4.54 ,60 ,62 ,68 ,71 ,78–103 Twenty-two of these articles (19 of the studies) reported resource use (trials) or model parameters (models) separate from unit prices—a minimum measure of study transferability.54 ,62 ,68 ,71 ,78 ,80–85 87–93 ,95 ,100 ,101 ,103 For those studies which included a randomised trial, the modified Jadad scores ranged from 2 to 4 on a scale from 0 to 4. The Tufts CEA Registry quality scores for the studies containing a CUA ranged from 4 to 6.5 on a scale from 1 to 7. The percentage of the applicable items on the BMJ checklist met by these studies ranged from 66% to 97%.

    View this table:
    Table 3

    Comparison of various quality measures between economic evaluations of complementary and integrative medicine (CIM) and conventional medicine

    View this table:
    Table 4

    Summary of results of complementary and integrative medicine (CIM) economic evaluations that met five study-quality criteria (31 articles representing 28 studies)

    Of the 56 comparisons made in these studies, 16 (29%) are cost saving—that is, the added CIM therapy had better health outcomes and lower costs than usual care alone. Cost savings were seen for acupuncture alone (instructional visits with an acupuncturist followed by home self-care by the partner for pregnant women with breech presentations at 33 weeks in terms of reductions in both breech presentation at birth and ceasareans in the Netherlands,91 and treatment by traditional Chinese medicine-trained licensed acupuncturists in private acupuncture clinics in the UK for low-back pain in terms of quality-adjusted life-years or QALYs from the societal perspective85) and in combination with other therapies (along with manual therapy, injections and other pain management for patients referred to an orthopaedic surgeon's office in Scotland who were unlikely to need surgery in terms of both improvements in health-related quality of life and QALYs54). Cost savings were also seen for manual therapy delivered by a physiotherapist, who is also a registered manual therapist, for neck pain in terms of perceived recovery, pain, neck disability and QALYs82; for preoperative oral supplementation with arginine and ω-3 fatty acids for patients with gastrointestinal cancer undergoing surgery102; for vitamin K1 supplementation for postmenopausal women with osteopenia and osteoporosis in terms of QALYs103; for supplementation with vitamins C and E and β-carotene for cataract prevention90; for fish oil supplementation in men with a history of heart attack87; for tai chi to prevent hip fractures in nursing home residents95 and for naturopathic care offered through a worksite clinic for chronic low-back pain in terms of both reductions in absenteeism and gains in QALYs.80

    Of the 28 cost-utility comparisons, one (massage for low-back pain62) was dominated— that is, had worse health outcomes and higher costs than usual care. Five (18%) are cost saving,54 ,80 ,82 ,85 ,103 5 (18%) have incremental cost-effectiveness ratios (ICERs) between US$0 and US$10 000 per quality-adjusted life-year (QALY),68 ,71 ,81 ,85 ,97 and 89% had ICERs less than US$50 000/QALY, a threshold often considered to represent the upper limit of society's value for a QALY.104 The cost-saving cost-utility studies were included in the paragraph above (ie, those that mention QALYs). The studies with cost-utility ICERs between US$0 and US$10 000 per QALY were: treatment by traditional Chinese medicine-trained licensed acupuncturists in private acupuncture clinics in the UK for low-back pain.85 hospital-based acupuncture by licensed oriental medical doctors in South Korea for 60-year-old women with first-time acute low-back pain,81 acupuncture from physicians with at least 140 h of training (A-diploma) in Germany for patients with dysmenorrhoea,97 osteopathic spinal manipulation by a general practitioner who is a registered osteopath in the UK for patients with subacute back pain,71 and an exercise programme plus spinal manipulation from a chiropractor, osteopath or physiotherapist at a private or National Health Service (NHS) site in the UK for low-back pain.68 The average percentage of applicable BMJ checklist items met by each study was slightly lower for those studies with at least one cost-saving comparison (85% vs 88%), but the difference was not statistically significant (t test=0.75, p value=0.460).

    Discussion

    This comprehensive systematic review identified 338 economic evaluations of CIM; 204 of which were published recently (2001–2010) covering a wide range of CIM therapies for a variety of populations. Although most patients who use CIM use more than one modality35 and despite the attention given to integrative medicine (coordinated access to conventional medicine and CIM),105 this systematic review found only one study that examined the effects of use of multiple CIM practitioners.52 In general, the quality of the recent full economic evaluations has held constant and is in line with what is seen in economic evaluations in conventional medicine. Details of the 31 recent higher-quality full economic evaluations indicate potential cost-effectiveness and cost savings across a variety of CIM therapies applied to different conditions. Owing to the non-generalisable nature of economic evaluations, the cost estimates shown are specific to their study settings.40 However, 22 articles provided at least the minimum information for study transferability. Therefore, their results could be adapted via modelling to determine the economic impact of these interventions in other settings.

    The strengths of this study are the comprehensive search strategy, which revealed a substantial number of published economic evaluations of CIM, the use of two reviewers and the use of multiple measures of study quality. Higher-quality studies were identified and highlighted for policy makers using a simple objective list of quality criteria, which reduced the potential for bias. The weaknesses of this study are similar to those of the other systematic reviews. The reviewers were not blinded to journals and article authors, which may have influenced results. Also, some aspects of what makes a quality economic evaluation could not be judged from what was reported. For example, ideally, pragmatic trials enrol patients typical of normal caseload in typical settings with typically trained and experienced practitioners following them under routine conditions (ref.37, p. 251). Judgements as to whether these criteria were met were not always possible from the reports, and were beyond the scope of this review. Finally, publication bias was not assessed. However, since the major goal of this study was to establish the extent of the published literature on this topic and to highlight the results of the higher-quality studies, it is not clear that publication bias is relevant here.

    The number of economic evaluations of CIM found and reviewed by this study far exceeds the numbers found in previous studies.11 ,19–21 ,23 This study found a total of 338 economic evaluations of CIM published between and including 1979 and 2010; 211 of these were full economic evaluations. White and Ernst19 identified 34 economic evaluations of CAM published 1987–1999; 11 of which were full economic evaluations. Between 1999 and October 2004, Herman et al20 identified 56 economic evaluations of CAM (39 full evaluations). Between 1994 and May 2004 Hulme and Long21 identified 19 full economic evaluations of CAM, and over a similar period (1995–2007) Doran et al23 found 43 economic evaluations (15 full evaluations). Maxion-Bergemann et al11 identified 5 (1 full) economic evaluations over an unspecified search period. The large number of economic evaluations found in this study reflects the facts that: (1) all evaluations from previous reviews were included; (2) a number of studies have been published since the last search dates of prior reviews and (3) a more extensive search strategy was used. It should be noted that 20% of the articles (68 of 338) in this review were identified through bibliography searches and from expert lists. Therefore, even the application of a long list of search terms to multiple databases does not guarantee that all CIM studies will be identified. However, there is some evidence that the indexing of these articles in medical databases is improving; studies from bibliographies and expert lists made up 32% of found articles published 2000 and before, but only 12% recent articles.

    There are several implications of this study for policy makers, clinicians and future researchers. First, there is a large and growing literature of quality economic evaluations in CIM. However, although indexing is improving in databases, finding these studies can require going beyond simple CIM-related search terms. Second, the results of the higher-quality studies indicate a number of highly cost-effective, and even cost saving, CIM therapies. Almost 30% of the 56 cost-effectiveness, cost-utility and cost-benefit comparisons shown in table 4 (18% of the CUA comparisons) were cost saving. Compare this to 9% of 1433 CUA comparisons found to be cost saving in a large review of economic evaluations across all medicine.106 Third, by meeting the five study-quality criteria, the studies shown in table 4 can each be considered a reasonable indicator of the health and economic impacts of the CIM therapy studied, at least in that population and setting. These studies, especially those showing cost savings, should be considered further for applicability in other settings. This requires the study to be transferable.39 Fortunately, the majority of the higher-quality studies met our measure of study transferability—resource use or model parameters, and unit costs were reported separately.

    Given the substantial number of economic evaluations of CIM found in this comprehensive review, even though it can always be said that more studies are needed, what is actually needed are better-quality studies—both in terms of better study quality (to increase the validity of the results for its targeted population and setting) and better transferability (to increase the usefulness of these results to other decision makers in other settings). Therefore, the following recommendations are made.

    1. That all studies measuring the effectiveness of CIM at least consider also measuring input costs and economic outcomes.

    2. That at least one arm of the study be some version of commonly available (usual) care, and that usual care and all interventions studied be described in sufficient detail that decision makers in other settings can determine what was done and whether the study's usual care comparator is applicable in their setting.

    3. That consideration be given to how CIM is typically used (eg, multiple CIM therapies) or can be used (eg, coordinated integrative care models) when designing studies.

    4. That changes in resource use be reported separately from unit costs in economic evaluations alongside clinical trials and that model parameters and unit costs be clearly reported in decision-analytic modelling studies.

    5. That all economic evaluations contain sensitivity analyses to increase the reliability of results.

    6. That more consideration be given to modelling as a method to estimate economic outcomes for existing effectiveness trial results, and to generalise existing quality economic evaluation results to other jurisdictions.

    Acknowledgments

    The authors wish to acknowledge and most gratefully thank Sandy Kramer of the University of Arizona Health Sciences Library for her assistance in the development and application of the search strategy and for eliminating duplicates from the search results. We would also like to thank Robert Scholten and P Scott Lapinski of the Harvard Medical School for their assistance with the EMBASE searches.

    References

    1. ↵
      1. Eisenberg DM,
      2. Kessler RC,
      3. Foster C,
      4. et al
      . Unconventional medicine in the United States: prevalence, costs, and patterns of use. N Engl J Med 1993;328:246–52.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Eisenberg DM,
      2. Davis RB,
      3. Ettner SL,
      4. et al
      . Trends in alternative medicine use in the United States, 1990–1997. JAMA 1998;280:1569–75.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Barnes PM,
      2. Powell-Griner E,
      3. McFann K,
      4. et al
      . Complementary and alternative medicine use among adults: United States, 2002. Advance data from Vital and Health Statistics. Hyattsville, MA: National Center for Health Statistics, 2004.
    4. ↵
      1. Nahin RL,
      2. Barnes PM,
      3. Stussman BJ,
      4. et al
      . Costs of complementary and alternative medicine (CAM) and frequency of visits to CAM practitioners: United States, 2007. National Health Statistics Reports. Hyattsville, MA: National Center for Health Statistics, 2009.
    5. ↵
      1. MacLennan AH,
      2. Wilson DH,
      3. Taylor AW
      . The escalating cost and prevalence of alternative medicine. Prev Med 2002;35:166–73.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Thomas KJ,
      2. Nicholl JP,
      3. Coleman P
      . Use and expenditure on complementary medicine in England: a population based survey. Complement Ther Med 2001;9:2–11.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Wolf U,
      2. Maxion-Bergemann S,
      3. Bornhoft G,
      4. et al
      . Use of complementary medicine in Switzerland. Forsch Komplementmed 2006;13:4–6.
      OpenUrlPubMed
    8. ↵
      1. Hartel U,
      2. Volger E
      . Use and acceptance of classical natural and alternative medicine in Germany—findings of a representative population-based survey. Forsch Komplementmed 2004;11:327–34.
      OpenUrl
    9. ↵
      National Center for Complementary and Alternative Medicine. What is complementary and alternative medicine (CAM)? National Center for Complementary and Alternative Medicine. Bethesda, Maryland: National Institutes of Health, 2011.
    10. ↵
      1. Claxton K,
      2. Posnett J
      . An economic approach to clinical trial design and research priority-setting. Med Econ 1996;5:513–24.
      OpenUrl
    11. ↵
      1. Maxion-Bergemann S,
      2. Wolf M,
      3. Bornhoft G,
      4. et al
      . Complementary and alternative medicine costs—a systematic literature review. Forsch Komplementmed 2006;13(Suppl 2):42–5.
      OpenUrlCrossRefPubMed
    12. ↵
      1. van der Roer N,
      2. Goossens MEJB,
      3. Evers SMAA,
      4. et al
      . What is the most cost-effective treatment for patients with low back pain? A systematic review. Best Pract Res Clin Rheumatol 2005;19:671–84.
      OpenUrlPubMed
    13. ↵
      1. Branson RA
      . Cost comparison of chiropractic and medical treatment of common musculoskeletal disorders: a review of the literature after 1980. Top Clin Chiropractic 1999;6:57–68.
      OpenUrl
    14. ↵
      1. Solomon DH,
      2. Bates DW,
      3. Panush RS,
      4. et al
      . Costs, outcomes, and patient satisfaction by provider type for patients with rheumatic and musculoskeletal conditions: a critical review of the literature and proposed methodological standards. Ann Intern Med 1997;127:52–60.
      OpenUrlPubMedWeb of Science
    15. ↵
      1. Kennedy DA,
      2. Hart J,
      3. Seely D
      . Cost-effectiveness of natural health products: a systematic review of randomized clinical trials. Evid-Based Complement Altern Med 2009;6:297–304.
      OpenUrlCrossRefPubMed
    16. ↵
      1. Gamber R,
      2. Holland S,
      3. Russo DP,
      4. et al
      . Cost-effective osteopathic manipulative medicine: a literature review of cost-effectiveness analyses for osteopathic manipulative treatment. J Am Osteopathic Assoc 2005;105:357–67.
      OpenUrlPubMed
    17. ↵
      1. Bornhoft G,
      2. Wolf U,
      3. Von Ammon K,
      4. et al
      . Effectiveness, safety and cost-effectiveness of homeopathy in general practice—summarized health technology assessment. Forsch Komplementmed 2006;13:19–29.
      OpenUrlCrossRefPubMed
    18. ↵
      1. Schneider CJ
      . Cost effectiveness of biofeedback and behavioral medicine treatments: a review of the literature. Biofeedback Self Regul 1987;12:71–92.
      OpenUrlPubMed
    19. ↵
      1. White AR,
      2. Ernst E
      . Economic analysis of complementary medicine: a systematic review. Complement Ther Med 2000;8:111–18.
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Herman PM,
      2. Craig BM,
      3. Caspi O
      . Is complementary and alternative medicine (CAM) cost-effective? A systematic review. BMC Complement Altern Med 2005;5:11.
      OpenUrlCrossRefPubMed
    21. ↵
      1. Hulme C,
      2. Long AF
      . Square pegs and round holes? A review of economic evaluation in complementary and alternative medicine. J Altern Complement Med 2005;11:179–88.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Canter PH,
      2. Coon JT,
      3. Ernst E
      . Cost-effectiveness of complementary therapies in the United Kingdom—a systematic review. Evid-Based Complement Altern Med 2006;3:425–32.
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Doran CM,
      2. Chang DH-T,
      3. Kiat H,
      4. et al
      . Review of economic methods used in complementary medicine. J Altern Complement Med 2010;16:591–5.
      OpenUrlPubMed
    24. ↵
      1. Pilkington K
      . Searching for CAM evidence: an evaluation of therapy-specific search strategies. J Altern Complement Med 2007;13:451–9.
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Shekelle PG,
      2. Morton SC,
      3. Suttorp MJ,
      4. et al
      . Challenges in systematic reviews of complementary and alternative medicine topics. Ann Intern Med 2005;142:1042–7.
      OpenUrlPubMedWeb of Science
    26. ↵
      1. Wootton JC
      . Classifying and defining complementary and alternative medicine. J Altern Complement Med 2005;11:777–8.
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Pilkington K,
      2. Richardson J
      . Exploring the evidence: the challenges of searching for research on acupuncture. J Altern Complement Med 2004;10:587–90.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Boddy K,
      2. Younger P
      . What a difference an interface makes: just how reliable are your search results? Focus Altern Complement Ther 2009;14:5–7.
      OpenUrl
    29. ↵
      1. Murphy LS,
      2. Reinsch S,
      3. Najm WI,
      4. et al
      . Spinal palpation: the challenges of information retrieval using available databases. J Manipulative Physiol Ther 2003;26:374–82.
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Furnham A
      . How the public classify complementary medicine: a factor analytic study. Complement Ther Med 2000;8:82–7.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Wieland LS,
      2. Manheimer E,
      3. Berman BM
      . Development and classification of an operational definition of complementary and alternative medicine for the Cochrane Collaboration. Altern Ther Health Med 2011;17:50–9.
      OpenUrlPubMed
    32. ↵
      1. Focan C
      . Pharmaco-economic comparative evaluation of combination chronotherapy vs. standard chemotherapy for colorectal cancer. Chronobiol Int 2002;19:289–97.
      OpenUrlPubMed
    33. ↵
      1. Kovacs FM,
      2. Llobera J,
      3. Abraira V,
      4. et al
      . Effectiveness and cost-effectiveness analysis of neuroreflexotherapy for subacute and chronic low back pain in routine general practice: a cluster randomized, controlled trial. Spine 2002;27:1149–59.
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Senkal M,
      2. Zumtobel V,
      3. Bauer KH,
      4. et al
      . Outcome and cost-effectiveness of perioperative enteral immunonutrition in patients undergoing elective upper gastrointestinal tract surgery: a prospective randomized study. Arch Surg 1999;134:1309–16.
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Wolsko PM,
      2. Eisenberg DM,
      3. Davis RB,
      4. et al
      . Insurance coverage, medical conditions, and visits to alternative medicine providers. Arch Intern Med 2002;162:281–7.
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Gunter MJ
      . The role of the ECHO model in outcomes research and clinical practice improvement. Am J Manag Care 1999;5:S217–24.
      OpenUrlPubMedWeb of Science
    37. ↵
      1. Drummond MF,
      2. Sculpher MJ,
      3. Torrance GW,
      4. et al
      . Methods for the economic evaluation of health care programmes. 3rd edn. Oxford: Oxford University Press, 2005.
    38. ↵
      1. Berger ML,
      2. Bingefors K,
      3. Hedblom E,
      4. et al
      . Health care cost, quality, and outcomes: ISPOR book of terms. Lawrenceville, NJ: International Society for Pharmacoeconomics and Outcomes Research, 2003.
    39. ↵
      1. Drummond M,
      2. Barbieri M,
      3. Cook J,
      4. et al
      . Transferability of economic evaluations across jurisdictions: ISPOR good research practices task force report. Value Health 2009;12:409–18.
      OpenUrlCrossRefPubMedWeb of Science
    40. ↵
      1. Drummond M,
      2. Manca A,
      3. Sculpher M
      . Increasing the generalizability of economic evaluations: recommendations for the design, analysis, and reporting of studies. Int J Technol Assess Health Care 2005;21:165–71.
      OpenUrlPubMedWeb of Science
    41. ↵
      1. Drummond MF,
      2. Jefferson TO
      , BMJ Economic Evaluation Working Party. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313:275–83.
      OpenUrlFREE Full Text
    42. ↵
      1. Gold MR,
      2. Siegel JE,
      3. Russell LB,
      4. et al
      . Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996.
    43. ↵
      1. Drummond MF,
      2. O'Brien B,
      3. Stoddart GL,
      4. et al
      . Methods for the economic evaluation of health care programmes. 2nd edn. Oxford: Oxford University Press, 1997.
    44. ↵
      1. Marshall DA,
      2. Hux M
      . Design and analysis issues for economic analysis alongside clinical trials. Med Care 2009;47:814–20.
      OpenUrl
    45. ↵
      1. Briggs A,
      2. Sculpher M,
      3. Buxton M
      . Uncertainty in the economic evaluation of health care technologies: the role of sensitivity analysis. Health Econ 1994;3:95–104.
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      1. Sculpher MJ,
      2. Pang FS,
      3. Manca A,
      4. et al
      . Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess 2004;8:1–213.
      OpenUrlPubMed
    47. ↵
      Board of Governors of the Federal Reserve System. Foreign Exchange Rates – G.5A, 1997–2012.
    48. ↵
      Bureau of Labor Statistics. Archived consumer price index detailed report information, 2000–2011.
    49. ↵
      Center for the Evaluation of Value and Risk in Health. Cost-effectiveness analysis registry: Institute for Clinical Research and Health Policy Studies. Tufts Medical Center is in Boston, MA, 2011.
    50. ↵
      1. Jadad AR,
      2. Moore RA,
      3. Carroll D,
      4. et al
      . Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:1–12.
      OpenUrlCrossRefPubMedWeb of Science
    51. ↵
      1. White AR,
      2. Ernst E
      . A systematic review of randomized controlled trials of acupuncture for neck pain. Rheumatology 1999;38:143–7.
      OpenUrlFREE Full Text
    52. ↵
      1. Robinson N,
      2. Donaldson J,
      3. Watt H
      . Auditing outcomes and costs of integrated complementary medicine provision–the importance of length of follow up. Complement Ther Clin Pract 2006;12:249–57.
      OpenUrlPubMed
    53. ↵
      1. Almog G,
      2. Lamond PJ,
      3. Gosselin G
      . Effects of chiropractic care on spinal symptomatology among professional drivers. Clin Chiropractic 2004;7:114–19.
      OpenUrl
    54. ↵
      1. Brown APL,
      2. Kennedy ADM,
      3. Torgerson DJ,
      4. et al
      . The OMENS trial: opportunistic evaluation of musculo-skeletal physician care among orthopaedic outpatients unlikely to require surgery. Health Bull 2001;59:199–210.
      OpenUrl
    55. ↵
      1. Cherkin DC,
      2. Eisenberg DM,
      3. Sherman KJ,
      4. et al
      . Randomized trial comparing traditional Chinese medical acupuncture, therapeutic massage, and self-care education for chronic low back pain. Arch Intern Med 2001;161:1081–8.
      OpenUrlCrossRefPubMedWeb of Science
    56. ↵
      1. Cook C,
      2. Cook A,
      3. Worrell T
      . Manual therapy provided by physical therapists in a hospital-based setting: a retrospective analysis. J Manipulative Physiol Ther 2008;35:338–43.
      OpenUrl
    57. ↵
      1. Eisenberg DM,
      2. Post DE,
      3. Davis RB,
      4. et al
      . Addition of choice of complementary therapies to usual care for acute low back pain: a randomized controlled trial. Spine 2007;32:151–8.
      OpenUrlCrossRefPubMedWeb of Science
    58. ↵
      1. Fritz JM,
      2. Brennan GP,
      3. Leaman H
      . Does the evidence for spinal manipulation translate into better outcomes in routine clinical care for patients with occupational low back pain? A case-control study. Spine J 2006;6:289–95.
      OpenUrlCrossRefPubMed
    59. ↵
      1. Grieves B,
      2. Menke JM,
      3. Pursel KJ
      . Cost minimization analysis of low back pain claims data for chiropractic vs medicine in a managed care organization. J Manipulative Physiol Ther 2009;32:734–9.
      OpenUrlPubMed
    60. ↵
      1. Haas M,
      2. Sharma R,
      3. Stano M
      . Cost-effectiveness of medical and chiropractic care for acute and chronic low back pain. J Manipulative Physiol Ther 2005;28:555–63.
      OpenUrlPubMed
    61. ↵
      1. Hemmila HM
      . Quality of life and cost of care of back pain patients in Finnish general practice. Spine 2002;27:647–53.
      OpenUrlPubMed
    62. ↵
      1. Hollinghurst S,
      2. Sharp D,
      3. Ballard K,
      4. et al
      . Randomised controlled trial of Alexander technique lessons, exercise, and massage (ATEAM) for chronic and recurrent back pain: economic evaluation. BMJ 2008;337:a2656.
      OpenUrlAbstract/FREE Full Text
    63. ↵
      1. Hurwitz EL,
      2. Morgenstern H,
      3. Harber P,
      4. et al
      . The effectiveness of physical modalities among patients with low back pain randomized to chiropractic care: findings from the UCLA low back pain study. J Manipulative Physiol Ther 2002;25:10–20.
      OpenUrlPubMed
    64. ↵
      1. Kominski GF,
      2. Heslin KC,
      3. Morgenstern H,
      4. et al
      . Economic evaluation of four treatments for low-back pain: results from a randomized controlled trial. Med Care 2005;43:428–35.
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      1. Lewis JS,
      2. Hewitt JS,
      3. Billington L,
      4. et al
      . A randomized clinical trial comparing two physiotherapy interventions for chronic low back pain. Spine 2005;30:711–21.
      OpenUrlCrossRefPubMedWeb of Science
    66. ↵
      1. Lipton JA,
      2. Meneses P,
      3. Martin JB,
      4. et al
      . Improved pain score outcomes achieved through the cooperative and cost-effective use of physical (osteopathic manipulative) medicine in the treatment of outpatient musculoskeletal complaints. Am Acad Osteopathy J 2002;12:26–32.
      OpenUrl
    67. ↵
      1. Stano M,
      2. Haas M,
      3. Goldberg B,
      4. et al
      . Chiropractic and medical care costs of low back care: results from a practice-based observational study. Am J Manag Care 2002;8:802–9.
      OpenUrlPubMed
    68. ↵
      UK Beam Trial Team. United Kingdom back pain exercise and manipulation (UK BEAM) randomised trial: cost effectiveness of physical treatments for back pain in primary care. BMJ 2004;329:1381.
      OpenUrlAbstract/FREE Full Text
    69. ↵
      1. Whitehurst DGT,
      2. Lewis M,
      3. Yao GL,
      4. et al
      . A brief pain management program compared with physical therapy for low back pain: results from an economic analysis alongside a randomized clinical trial. Arthritis Care Res 2007;57:466–73.
      OpenUrl
    70. ↵
      1. Wilkey A,
      2. Gregory M,
      3. Byfield D,
      4. et al
      . A comparison between chiropractic management and pain clinic management for chronic low-back pain in a National Health Service outpatient clinic. J Altern Complement Med 2008;14:465–73.
      OpenUrlCrossRefPubMed
    71. ↵
      1. Williams NH,
      2. Edwards RT,
      3. Linck P,
      4. et al
      . Cost-utility analysis of osteopathy in primary care: results from a pragmatic randomized controlled trial. Fam Pract 2004;21:643–50.
      OpenUrlAbstract/FREE Full Text
    72. ↵
      1. Williams NH,
      2. Wilkinson C,
      3. Russell I,
      4. et al
      . Randomized osteopathic manipulation study (ROMANS): pragmatic trial for spinal pain in primary care. Fam Pract 2003;20:662–9.
      OpenUrlAbstract/FREE Full Text
    73. ↵
      National Institute for Health and Clinical Excellence. Assessing cost-effectiveness. The guidelines manual. London: National Health Service, 2009:81–91.
    74. ↵
      Commonwealth Department of Health and Ageing. Guidelines for the pharmaceutical industry on preparaion of submissions to the pharmaceutical benefits advisory committee. Canberra: Commonwealth of Australia, 2002.
    75. ↵
      1. Glennie JL,
      2. Torrance GW,
      3. Baladi JF,
      4. et al
      . The revised Canadian guidelines for the economic evaluation of pharmaceuticals. Pharmacoeconomics 1999;15:459–68.
      OpenUrlCrossRefPubMedWeb of Science
    76. ↵
      1. Neumann PJ,
      2. Greenberg D,
      3. Olchanski NV,
      4. et al
      . Growth and quality of the cost-utility literature, 1976–2001. Value Health 2005;8:3–9.
      OpenUrlCrossRefPubMedWeb of Science
    77. ↵
      1. Neumann PJ
      . Costing and perspective in published cost-effectiveness analysis. Med Care 2009;47:S28–32.
      OpenUrlCrossRefPubMedWeb of Science
    78. ↵
      1. Black C,
      2. Clar C,
      3. Henderson R,
      4. et al
      . The clinical effectiveness of glucosamine and chondroitin supplements in slowing or arresting progression of osteoarthritis of the knee: a systematic review and economic evaluation. Health Technol Assess 2009;13:1–148.
      OpenUrlCrossRefPubMed
    79. ↵
      1. Franzosi MG,
      2. Brunetti M,
      3. Marchioli R,
      4. et al
      . Cost-effectiveness analysis of n-3 polyunsaturated fatty acides (PUFA) after myocardial infarction: results from Gruppo Italiano per lo Studio della Sopravvivenza nell'Infarto (GISSI)—Prevenzione Trial. Pharmacoeconomics 2001;19:411–20.
      OpenUrlCrossRefPubMedWeb of Science
    80. ↵
      1. Herman PM,
      2. Szczurko O,
      3. Cooley K,
      4. et al
      . Cost-effectiveness of naturopathic care for chronic low back pain. Altern Ther Health Med 2008;14:32–9.
      OpenUrlPubMed
    81. ↵
      1. Kim N,
      2. Yang B,
      3. Lee T,
      4. et al
      . An economic analysis of usual care and acupuncture collaborative treatment on chronic low back pain: a Markov model decision analysis. BMC Complement Altern Med 2010;10:74.
      OpenUrlPubMed
    82. ↵
      1. Korthals-de Bos IB,
      2. Hoving JL,
      3. van Tulder MW,
      4. et al
      . Cost effectiveness of physiotherapy, manual therapy, and general practitioner care for neck pain: economic evaluation alongside a randomised controlled trial. BMJ 2003;326:911.
      OpenUrlAbstract/FREE Full Text
    83. ↵
      1. Lamotte M,
      2. Annemans L,
      3. Kawalec P,
      4. et al
      . A multi-country health economic evaluation of highly concentrated n-3 polyunsaturated fatty acids in secondary prevention after myocardial infarction. Pharmacoeconomics 2006;24:783–95.
      OpenUrlCrossRefPubMedWeb of Science
    84. ↵
      1. Quilici S,
      2. Martin M,
      3. McGuire A,
      4. et al
      . A cost-effectiveness analysis of n−3 PUFA (Omacor®) treatment in post-MI patients. Int J Clin Pract 2006;60:922–32.
      OpenUrlCrossRefPubMedWeb of Science
    85. ↵
      1. Ratcliffe J,
      2. Thomas KJ,
      3. MacPherson H,
      4. et al
      . A randomised controlled trial of acupuncture care for persistent low back pain: cost effectiveness analysis. BMJ 2006;333:626.
      OpenUrlAbstract/FREE Full Text
    86. ↵
      1. Reinhold T,
      2. Witt CM,
      3. Jena S,
      4. et al
      . Quality of life and cost-effectiveness of acupuncture treatment in patients with osteoarthritis pain. Eur J Health Econ 2008;9:209–19.
      OpenUrlCrossRefPubMedWeb of Science
    87. ↵
      1. Schmier JK,
      2. Rachman NJ,
      3. Halpern MT
      . The cost-effectiveness of omega-3 supplements for prevention of secondary coronary events. Manag Care 2006;15:43–50.
      OpenUrlPubMed
    88. ↵
      1. Stevenson M,
      2. Lloyd-Jones M,
      3. Papaioannou D
      . Vitamin K to prevent fractures in older women: systematic review and economic evaluation. Health Technol Assess 2009;13:1–134.
      OpenUrlCrossRefPubMed
    89. ↵
      1. Thomas KJ,
      2. MacPherson H,
      3. Ratcliffe J,
      4. et al
      . Longer term clinical and economic benefits of offering acupuncture care to patients with chronic low back pain. Health Technol Assess 2005; 9:1–109.
      OpenUrlPubMed
    90. ↵
      1. Trevithick JR,
      2. Massel D,
      3. Robertson JM,
      4. et al
      . Modeling savings from prophylactic REACT antioxidant use among a cohort initially aged 50–55 years: a Canadian perspective. J Orthomolecular Med 2006;21:212–20.
      OpenUrl
    91. ↵
      1. van den Berg I,
      2. Kaandorp GC,
      3. Bosch JL,
      4. et al
      . Cost-effectiveness of breech version by acupuncture-type interventions on BL 67, including moxibustion, for women with a breech foetus at 33 weeks gestation: a modelling approach. Complement Ther Med 2010;18:67–77.
      OpenUrlPubMed
    92. ↵
      1. Van Tubergen A,
      2. Boonen A,
      3. Landewe R,
      4. et al
      . Cost effectiveness of combined spa-exercise therapy in ankylosing spondylitis: a randomized controlled trial. Arthritis Rheum 2002;47:459–67.
      OpenUrlCrossRefPubMedWeb of Science
    93. ↵
      1. Vickers AJ,
      2. Rees RW,
      3. Zollman CE,
      4. et al
      . Acupuncture of chronic headache disorders in primary care: randomised controlled trial and economic analysis. Health Technol Assess 2004;8:1–35.
      OpenUrlPubMed
    94. ↵
      1. Willich SN,
      2. Reinhold T,
      3. Selim D,
      4. et al
      . Cost-effectiveness of acupuncture treatment in patients with chronic neck pain. Pain 2006;125:107–13.
      OpenUrlCrossRefPubMedWeb of Science
    95. ↵
      1. Wilson CJ,
      2. Datta SK
      . Tai chi for the prevention of fractures in a nursing home population: an economic analysis. J Clin Outcomes Manag 2001;8:19–27.
      OpenUrl
    96. ↵
      1. Witt CM,
      2. Jena S,
      3. Selim D,
      4. et al
      . Pragmatic randomized trial evaluating the clinical and economic effectiveness of acupuncture for chronic low back pain. Am J Epidemiol 2006;165:487–96.
      OpenUrl
    97. ↵
      1. Witt CM,
      2. Reinhold T,
      3. Brinkhaus B,
      4. et al
      . Acupuncture in patients with dysmenorrhea: a randomized study on clinical effectiveness and cost-effectiveness in usual care. Am J Obstet Gynecol 2008;198:166.e1–8.
      OpenUrlPubMed
    98. ↵
      1. Witt CM,
      2. Reinhold T,
      3. Jena S,
      4. et al
      . Cost-effectiveness of acupuncture in women and men with allergic rhinitis: a randomized controlled study in usual care. Am J Epidemiol 2009;169:562–71.
      OpenUrlAbstract/FREE Full Text
    99. ↵
      1. Witt CM,
      2. Reinhold T,
      3. Jena S,
      4. et al
      . Cost-effectiveness of acupuncture treatment in patients with headache. Cephalalgia 2008;28:334–45.
      OpenUrlAbstract/FREE Full Text
    100. ↵
      1. Wonderling D,
      2. Vickers AJ,
      3. Grieve R,
      4. et al
      . Cost effectiveness analysis of a randomised trial of acupuncture for chronic headache in primary care. BMJ 2004;328:747.
      OpenUrlAbstract/FREE Full Text
    101. ↵
      1. Zijlstra TR,
      2. Braakman-Jansen LM,
      3. Taal E,
      4. et al
      . Cost-effectiveness of spa treatment for fibromyalgia: general health improvement is not for free. Rheumatol 2007;46:1454–9.
      OpenUrlAbstract/FREE Full Text
    102. ↵
      1. Braga M,
      2. Gianotti L,
      3. Vignali A,
      4. et al
      . Hospital resources consumed for surgical morbidity: effects of preoperative arginine and omega-3 fatty acid supplementation on costs. Nutrition 2005; 21:1078–86.
      OpenUrl
    103. ↵
      1. Stevenson MD,
      2. Jones ML
      . The cost effectiveness of a randomized controlled trial to establish the relative efficacy of vitamin K1 compared with alendronate. Med Decis Making 2010;31:43–52.
      OpenUrlPubMed
    104. ↵
      1. Grosse SD
      . Assessing cost-effectiveness in healthcare: history of the $50,000 per QALY threshold. Expert Rev Pharmacoecon Outcomes Res 2008;8:165–78.
      OpenUrlCrossRefPubMed
    105. ↵
      1. Schultz AM,
      2. Chao SM,
      3. McGinnis JM
      , eds. Integrative medicine and the health of the public: a summary of the February 2009 Summit. Washington DC: Institute of Medicine, 2009.
    106. ↵
      1. Bell CM,
      2. Urbach DR,
      3. Ray JG,
      4. et al
      . Bias in published cost effectiveness studies: systematic review. BMJ 2006;332:699–703.
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributions PMH conceived of the idea for the paper, designed the search strategy, reviewed the references found, extracted the data from each included article and is the guarantor for this study. In parallel, BLP also reviewed the references found, extracted data from included articles and worked with PMH to resolve any discrepancies between reviewers. CMW provided practical insight and an international perspective to the design of the paper and interpretation of results. DME participated in the early design of the study, including the data extraction plan, inclusion/exclusion criteria and the interpretation of results. All authors contributed to the drafting and editing of the manuscript.

    • Funding The Bernard Osher Foundation supports a portion of DME's time for research in integrative medicine. The Foundation had no control or influence over the design or execution of this study, nor no input into this manuscript.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The full list of found articles is available in a word document from the corresponding author.