User login
- Herbal medicines have been submitted to systematic reviews more frequently than any other complementary therapy, and it is here where the most positive evidence can be found.
- There is not much research into potential serious risks of complementary medicine. Possible risks range from the toxicity of herbs to vertebral artery dissection or nerve damage after chiropractic manipulation.
- Currently the Cochrane Library contains 34 systematic reviews of complementary medicine: 20 of herbal medicines, 7 of acupuncture, 3 of homeopathy, 2 of manual therapies, and 2 of other forms.
Complementary or alternative medicine has moved from the fringe of health care toward its center; recent figures show that in Germany, for instance, no less than three quarters of the general population use at least 1 complementary therapy.1 In the United States, the equivalent figures have increased from 33% in 1990 to 42% in 1997.2
Virtually all survey data agree that those most fascinated with complementary medicine are predominantly female, affluent, middleaged, and well-educated. Seventy-eight percent of all Medicaid programs provide coverage of at least 1 form of complementary medicine.3
At the same time, critics of complementary medicine often insist there is no good evidence to support these therapies,4,5 and that it is a waste of resources and a misuse of science to try establishing an evidence base for therapies that are essentially nonscientific, irrational, and implausible fads.6 But is this really true?
Research in complementary medicine
Over the last 30 years, the level of original research activity in complementary medicine has increased considerably.7 The best quality of evidence for or against the effectiveness of any therapy is usually provided by Cochrane reviews.8,9
Cochrane reviews
Currently the Cochrane Library contains 34 systematic reviews and 35 protocols of complementary medicine10 (depending on what one considers complementary/alternative and what mainstream, this figure might vary marginally). Twenty of the reviews are of herbal medicines, 7 of acupuncture, 3 of homeopathy, 2 of manual therapies, and 2 of other forms of comple-mentary medicine. Twelve reviews include a meta-analytic approach.
The 34 reviews comprise a total of 286 clinical, mostly randomized and often placebo-controlled, double-blind studies. In 1999, the Cochrane Library listed more than 4000 controlled trials of complementary medicine and a further 4000 awaited assessment.11
The single largest Cochrane review of complementary medicine is a meta-analysis of randomized clinical trials of St John’s wort for depression, based on 27 primary studies with a total of 2291 patients.12 Seven of the 34 Cochrane reviews are “negative”ie, do not suggest a positive clinical effect of the intervention under evaluation. Eleven are entirely inconclusive and 16 draw at least tentatively positive conclusions (Table). Given the dire funding situation for research in complementary medicine,13 this evidence base is remarkable.
TABLE
Cochrane reviews in complementary medicine with (tentatively) positive results
| First author (primary studies)* | Therapy | Indication | Reservations* |
|---|---|---|---|
| Furlan (8) | Massage | Low back pain | More studies required, some trials of poor quality |
| Green (4) | Acupuncture | Lateral elbow pain | More studies required, most trials of poor quality |
| Linde (7) | Acupuncture | Asthma | Evidence only positive for peak expiratory flow rate, effect size small |
| Linde (27) | St John’s wort (Hypericum perforatum)† | Depression | Some trials of poor quality, few equivalence studies |
| Little (11) | Various herbal medicines | Rheumatoid arthritis | More studies required, some trials of poor quality |
| Little (5) | Various herbal medicines | Osteoarthritis | More studies required, some trials of poor quality |
| Melchart (26) | Acupuncture | Headache | Evidence positive only for migraine headaches, effect size small |
| Pittler (7) | Kava (Piper methysticum)† | Anxiety | Concern over safety |
| Pittler (13) | Horse chestnut (Aesculus hippocastanum)† | Chronic venous insufficiency | Scarcity of long-term studies |
| Pittler (4) | Feverfew (Tanacetum parthenium)† | Migraine prevention | More studies required, effect size small |
| Pittler (2) | Globe artichoke (Cynara scolymus)† | Hypercholesterolemia | More studies required effect size small |
| Vickers (7) | Oscillococcinum | Influenza | More studies required, effect size small |
| Wilt (21) | Saw palmetto (Serenoa repens)† | Benign prostate hypertrophy | Effect size moderate, scarcity of long-term studies |
| Wilt (18) | African prune (Pygeum africanum)† | Benign prostate hypertrophy | Scarcity of long-term studies |
| Wilt (4) | Cernilton† | Benign prostate hypertrophy | More studies required, scarcity of long-term trials |
| Wilt (4) | Beta-sitosterols† | Benign prostate hypertrophy | More studies required, scarcity of long-term trials |
| All data extracted from The Cochrane Library, 2003. “More studies required” means that volume of data was small; “trials of poor quality” means that the average quality of the evidence was lowered by flawed studies; “effect size” describes the difference in clinical response to active and control treatment. | |||
| *As expressed by authors of respective review. | |||
| †Plant-based treatments. | |||
Other reviews
The Cochrane database may be the best but certainly is not the only source of systematic reviews of complementary medicine. My unit has published about 100 systematic reviews (a full list is available from the author), and most were not in the Cochrane format.
Linde and Willich have analyzed selected systematic reviews of acupuncture, homeopathy, and herbal medicine, and have shown that their methodological approach differed considerably.14
Methods of reviewing complementary medicine
The conclusions of these methodologically diverse articles were still surprisingly consistent.15 Some maintain that complementary medicine cannot be evidence-based in the conventional sense of the word16 ; that “softer” types of evidence need to be taken into consideration as well17 ; that placebo effects must not be dismissed as nonbeneficial18 ; that the healing encounter includes significant factors that may never be quantifiable19 ; that “the scientific method cannot measure hope, divine intervention, or the power of belief.”20 And, obviously, research in complementary medicine “must consider social, cultural, political, and economic contexts.”21
The debate about what constitutes the best research methods for complementary medicine has been going on for decades. There are no simple answers except that, like in any type of scientific inquiry, there are no intrinsically good or bad methods, only good and bad matches between the research question posed and the methodology employed.22
Herbal medicines have been submitted to systematic reviews more frequently than any other complementary therapy, and it is in this area where most of the positive evidence can be found (also outside Cochrane reviews).23 The medical conditions treated with complementary medicine are often chronic benign diseases for which existing conventional treatments fail to offer a cure or a risk-free reduction of symptoms (Table). Given the popularity of complementary medicine and the economic importance of these conditions, it seems ill-conceived to argue against further research in this area. 6
Firm conclusions of the Cochrane (or other) reviews of complementary medicine are often hampered by the paucity of primary studies; 10 of the 16 reviews in Table are based on fewer than 10 primary studies. The average methodological quality of the primary data is in some but by no means all disappointing.24-26
Problems in testing complementary medicine
There is little doubt that rigorous trials of complementary medicine can pose formidable problems.22 What, for instance, is an adequate placebo for a study of massage therapy, and how should one blind patients in such a trial? The biggest obstacle to good research is perhaps the notorious lack of research funding in this area, which is all the more acute because costs can be particularly high for trials of time-intensive forms of complementary medicine.13
The average size of the overall therapeutic effect associated with complementary medicine is usually modest and the numbers needed to treat are often high. In other words, the difference between benefit from complementary medicine and no therapy or placebo may be statistically significant but critics might argue that it is of debatable clinical relevance.5 Even minor adverse effects would therefore critically disturb the delicate balance of risk and benefit.23
Assessing the risks
It follows that the potential risks of complementary medicine require careful attention and more systematic study. Our fragmentary knowledge indicates that the issues are complex.27 They range from toxicity of herbs to vertebral artery dissection or nerve damage after chiropractic manipulation.
They also include more subtle indirect hazards. Some practitioners of complementary medicine, for instance, tend to advise their clients against employing important medical interventions.28 At present there are no reliable incident figures regarding serious adverse effects of complementary medicine,5,23,27 rendering this area perhaps the most urgent topic for further research.
Moving forward
At a time when healthcare systems universally are strapped for money, the decision whether to integrate complementary medicine into routine medicine will undoubtedly be influenced by economic considerations. Virtually all research on this issue is inconclusive or flawed or both.29 We therefore cannot be sure whether such an integration would save public funds or cost extra money.
In the US, about 41 million people have no health insurance and are thus not covered for even the most basic health care. About the same number of Americans are underinsured. In this situation, it seems difficult to argue without convincing data that the integration of complementary medicine would present a solution to the economic problems in healthcare.
In conclusion, during recent years the evidence in support of complementary medicine has been considerably strengthened, primarily through numerous Cochrane reviews10 and other documents.7,23,30 A large range of promising interventions could be at our fingertips. At a time when whole populations are voting with their feet in favor of complementary medicine,1,2 it would be in everyone’s interest to invest in rigorous research of this area.
Correspondence
Complementary Medicine, Peninsula Medical School, Universities of Exeter and Plymouth, 25 Victoria Park Road, Exeter EX2 4NT, United Kingdom. E-mail: [email protected].
1. Institut fÜr Demoskopie Allensbach. Naturheilmittel: Wichtigste Erkenntnisse aus Allensbacher Trendstudien. [Natural cures: The most important conclusions from Allensbach trend studies.] Available at: www.demoskopie.de.Accessed on May 27, 2003.
2. Eisenberg DM, David RB, Ettner SL, et al. Trends in alternative medicine use in the United States, 1990–1997: results of a follow-up national survey. JAMA 1998;280:1569-1575.
3. Steyer TE, Freed GL, Lantz PM. Medicaid reimbursement for alternative therapies. Alt Ther Health Med 2002;8:84-88.
4. Marcus DM. Alternative medicine and the Arthritis Foundation. Arthritis Rheum 2002;47:5-7.
5. Fitzcharles M. Is it time for rheumatologists to rethink the use of manual therapies? J Rheumatol 2002;29:1117-1120.
6. Charlton BG. Randomized trials in alternative/complementary medicine. Q J Med 2002;95:643-645.
7. Barnes J, Abbot NC, Harkness E, Ernst E. Articles on complementary medicine in the mainstream medical literature: an investigation of MEDLINE, 1966 through 1996. Arch Intern Med 1999;159:1721-1725.
8. Olsen O, Middleton P, Ezzo J, et al. Quality of Cochrane reviews: assessment of sample from 1998. BMJ 2001;323:829-832.
9. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ 2000;320:537-540.
10. The Cochrane Library [database online] Oxford, UK: Update Software; 2002.Available at: www.cochranelibrary.com. Accessed on May 23, 2003.
11. Vickers A. Evidence-based medicine and complementary medicine. ACP J Club 1999;130:A13-14.
12. Linde K. St John’s wort for depression. The Cochrane Library 2000;4:1-17.
13. Ernst E. Funding research into complementary medicine: the situation in Britain. Complement Ther Med 1999;7:250-253.
14. Linde K, Willich SN. How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med 2003;96:17-22.
15. Ernst E. How objective are systematic reviews? J R Soc Med 2003;96:56-57.
16. Tonelli MR, Callahan TC. Why alternative medicine cannot be evidence-based. Acad Med 2001;76:1213-1220.
17. Jonas WB. The evidence house: how to build an inclusive base for complementary medicine. West J Med 2001;175:79-80.
18. Walach H. The efficacy paradox in randomized controlled trials of CAM and elsewhere: beware of the placebo trap. J Altern Complement Med 2001;7:213-218.
19. Redwood D. Methodological changes in the evaluation of complementary and alternative medicine: issued raised by Sherman et al and Hawk et al. J Altern Complement Med 2002;8:5-6.
20. Puchalski CM. Reconnecting the science and art of medicine. Acad Med 2001;76:1224-1225.
21. Bodeker G, Kronenberg F. A public health agenda for traditional, complementary, and alternative medicine. Am J Public Health 2002;92:1582-1591.
22. Vickers A, Cassileth B, Ernst E, et al. How should we research unconventional therapies? A panel report from the conference on Complementary and Alternative Medicine Research Methodology, National Institute of Health. Int J Technol Assess Health Care 1997;13:111-121.
23. Ernst E, Pittler MH, Stevinson C, White AR. The desktop guide to complementary and alternative medicine. London: Mosby; 2001.
24. Bloom BS, Retbi A, Dahan S, Jonsson E. Evaluation of randomized controlled trials on complementary and alternative medicine. Int J Technol Assess Health Care 2000;16:13-21.
25. Linde K, Jonas WB, Melchart D, Willich S. The methodological quality of randomised controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol 2001;30:526-531.
26. Moher D, Sampson M, Campbell K, et al. Assessing the quality of reports of randomised trials in pediatric complementary and alternative medicine. BMC Pediatr 2002;2:1-6.
27. Ernst E. Risks associated with complementary therapies. In MNG Dukes and JR Aronson, eds, Meyler’s Side Effects of Drugs. 14th ed. Oxford and New York: Elsevier Science; 2000:1649-1681.
28. Schmidt K, Ernst E. Aspects of MMR. Survey shows that some homeopaths and chiropractors advise against MMR. BMJ 2002;325-597.
29. White AR, Ernst E. Economic analysis of complementary medicine a systematic review. Complement Ther Med 2000;8:111-118.
30. Yuan C-S, Bieber EJ, eds. Textbook of complementary and alternative medicine. Boca Raton, Fla: Parthenon; 2003.
- Herbal medicines have been submitted to systematic reviews more frequently than any other complementary therapy, and it is here where the most positive evidence can be found.
- There is not much research into potential serious risks of complementary medicine. Possible risks range from the toxicity of herbs to vertebral artery dissection or nerve damage after chiropractic manipulation.
- Currently the Cochrane Library contains 34 systematic reviews of complementary medicine: 20 of herbal medicines, 7 of acupuncture, 3 of homeopathy, 2 of manual therapies, and 2 of other forms.
Complementary or alternative medicine has moved from the fringe of health care toward its center; recent figures show that in Germany, for instance, no less than three quarters of the general population use at least 1 complementary therapy.1 In the United States, the equivalent figures have increased from 33% in 1990 to 42% in 1997.2
Virtually all survey data agree that those most fascinated with complementary medicine are predominantly female, affluent, middleaged, and well-educated. Seventy-eight percent of all Medicaid programs provide coverage of at least 1 form of complementary medicine.3
At the same time, critics of complementary medicine often insist there is no good evidence to support these therapies,4,5 and that it is a waste of resources and a misuse of science to try establishing an evidence base for therapies that are essentially nonscientific, irrational, and implausible fads.6 But is this really true?
Research in complementary medicine
Over the last 30 years, the level of original research activity in complementary medicine has increased considerably.7 The best quality of evidence for or against the effectiveness of any therapy is usually provided by Cochrane reviews.8,9
Cochrane reviews
Currently the Cochrane Library contains 34 systematic reviews and 35 protocols of complementary medicine10 (depending on what one considers complementary/alternative and what mainstream, this figure might vary marginally). Twenty of the reviews are of herbal medicines, 7 of acupuncture, 3 of homeopathy, 2 of manual therapies, and 2 of other forms of comple-mentary medicine. Twelve reviews include a meta-analytic approach.
The 34 reviews comprise a total of 286 clinical, mostly randomized and often placebo-controlled, double-blind studies. In 1999, the Cochrane Library listed more than 4000 controlled trials of complementary medicine and a further 4000 awaited assessment.11
The single largest Cochrane review of complementary medicine is a meta-analysis of randomized clinical trials of St John’s wort for depression, based on 27 primary studies with a total of 2291 patients.12 Seven of the 34 Cochrane reviews are “negative”ie, do not suggest a positive clinical effect of the intervention under evaluation. Eleven are entirely inconclusive and 16 draw at least tentatively positive conclusions (Table). Given the dire funding situation for research in complementary medicine,13 this evidence base is remarkable.
TABLE
Cochrane reviews in complementary medicine with (tentatively) positive results
| First author (primary studies)* | Therapy | Indication | Reservations* |
|---|---|---|---|
| Furlan (8) | Massage | Low back pain | More studies required, some trials of poor quality |
| Green (4) | Acupuncture | Lateral elbow pain | More studies required, most trials of poor quality |
| Linde (7) | Acupuncture | Asthma | Evidence only positive for peak expiratory flow rate, effect size small |
| Linde (27) | St John’s wort (Hypericum perforatum)† | Depression | Some trials of poor quality, few equivalence studies |
| Little (11) | Various herbal medicines | Rheumatoid arthritis | More studies required, some trials of poor quality |
| Little (5) | Various herbal medicines | Osteoarthritis | More studies required, some trials of poor quality |
| Melchart (26) | Acupuncture | Headache | Evidence positive only for migraine headaches, effect size small |
| Pittler (7) | Kava (Piper methysticum)† | Anxiety | Concern over safety |
| Pittler (13) | Horse chestnut (Aesculus hippocastanum)† | Chronic venous insufficiency | Scarcity of long-term studies |
| Pittler (4) | Feverfew (Tanacetum parthenium)† | Migraine prevention | More studies required, effect size small |
| Pittler (2) | Globe artichoke (Cynara scolymus)† | Hypercholesterolemia | More studies required effect size small |
| Vickers (7) | Oscillococcinum | Influenza | More studies required, effect size small |
| Wilt (21) | Saw palmetto (Serenoa repens)† | Benign prostate hypertrophy | Effect size moderate, scarcity of long-term studies |
| Wilt (18) | African prune (Pygeum africanum)† | Benign prostate hypertrophy | Scarcity of long-term studies |
| Wilt (4) | Cernilton† | Benign prostate hypertrophy | More studies required, scarcity of long-term trials |
| Wilt (4) | Beta-sitosterols† | Benign prostate hypertrophy | More studies required, scarcity of long-term trials |
| All data extracted from The Cochrane Library, 2003. “More studies required” means that volume of data was small; “trials of poor quality” means that the average quality of the evidence was lowered by flawed studies; “effect size” describes the difference in clinical response to active and control treatment. | |||
| *As expressed by authors of respective review. | |||
| †Plant-based treatments. | |||
Other reviews
The Cochrane database may be the best but certainly is not the only source of systematic reviews of complementary medicine. My unit has published about 100 systematic reviews (a full list is available from the author), and most were not in the Cochrane format.
Linde and Willich have analyzed selected systematic reviews of acupuncture, homeopathy, and herbal medicine, and have shown that their methodological approach differed considerably.14
Methods of reviewing complementary medicine
The conclusions of these methodologically diverse articles were still surprisingly consistent.15 Some maintain that complementary medicine cannot be evidence-based in the conventional sense of the word16 ; that “softer” types of evidence need to be taken into consideration as well17 ; that placebo effects must not be dismissed as nonbeneficial18 ; that the healing encounter includes significant factors that may never be quantifiable19 ; that “the scientific method cannot measure hope, divine intervention, or the power of belief.”20 And, obviously, research in complementary medicine “must consider social, cultural, political, and economic contexts.”21
The debate about what constitutes the best research methods for complementary medicine has been going on for decades. There are no simple answers except that, like in any type of scientific inquiry, there are no intrinsically good or bad methods, only good and bad matches between the research question posed and the methodology employed.22
Herbal medicines have been submitted to systematic reviews more frequently than any other complementary therapy, and it is in this area where most of the positive evidence can be found (also outside Cochrane reviews).23 The medical conditions treated with complementary medicine are often chronic benign diseases for which existing conventional treatments fail to offer a cure or a risk-free reduction of symptoms (Table). Given the popularity of complementary medicine and the economic importance of these conditions, it seems ill-conceived to argue against further research in this area. 6
Firm conclusions of the Cochrane (or other) reviews of complementary medicine are often hampered by the paucity of primary studies; 10 of the 16 reviews in Table are based on fewer than 10 primary studies. The average methodological quality of the primary data is in some but by no means all disappointing.24-26
Problems in testing complementary medicine
There is little doubt that rigorous trials of complementary medicine can pose formidable problems.22 What, for instance, is an adequate placebo for a study of massage therapy, and how should one blind patients in such a trial? The biggest obstacle to good research is perhaps the notorious lack of research funding in this area, which is all the more acute because costs can be particularly high for trials of time-intensive forms of complementary medicine.13
The average size of the overall therapeutic effect associated with complementary medicine is usually modest and the numbers needed to treat are often high. In other words, the difference between benefit from complementary medicine and no therapy or placebo may be statistically significant but critics might argue that it is of debatable clinical relevance.5 Even minor adverse effects would therefore critically disturb the delicate balance of risk and benefit.23
Assessing the risks
It follows that the potential risks of complementary medicine require careful attention and more systematic study. Our fragmentary knowledge indicates that the issues are complex.27 They range from toxicity of herbs to vertebral artery dissection or nerve damage after chiropractic manipulation.
They also include more subtle indirect hazards. Some practitioners of complementary medicine, for instance, tend to advise their clients against employing important medical interventions.28 At present there are no reliable incident figures regarding serious adverse effects of complementary medicine,5,23,27 rendering this area perhaps the most urgent topic for further research.
Moving forward
At a time when healthcare systems universally are strapped for money, the decision whether to integrate complementary medicine into routine medicine will undoubtedly be influenced by economic considerations. Virtually all research on this issue is inconclusive or flawed or both.29 We therefore cannot be sure whether such an integration would save public funds or cost extra money.
In the US, about 41 million people have no health insurance and are thus not covered for even the most basic health care. About the same number of Americans are underinsured. In this situation, it seems difficult to argue without convincing data that the integration of complementary medicine would present a solution to the economic problems in healthcare.
In conclusion, during recent years the evidence in support of complementary medicine has been considerably strengthened, primarily through numerous Cochrane reviews10 and other documents.7,23,30 A large range of promising interventions could be at our fingertips. At a time when whole populations are voting with their feet in favor of complementary medicine,1,2 it would be in everyone’s interest to invest in rigorous research of this area.
Correspondence
Complementary Medicine, Peninsula Medical School, Universities of Exeter and Plymouth, 25 Victoria Park Road, Exeter EX2 4NT, United Kingdom. E-mail: [email protected].
- Herbal medicines have been submitted to systematic reviews more frequently than any other complementary therapy, and it is here where the most positive evidence can be found.
- There is not much research into potential serious risks of complementary medicine. Possible risks range from the toxicity of herbs to vertebral artery dissection or nerve damage after chiropractic manipulation.
- Currently the Cochrane Library contains 34 systematic reviews of complementary medicine: 20 of herbal medicines, 7 of acupuncture, 3 of homeopathy, 2 of manual therapies, and 2 of other forms.
Complementary or alternative medicine has moved from the fringe of health care toward its center; recent figures show that in Germany, for instance, no less than three quarters of the general population use at least 1 complementary therapy.1 In the United States, the equivalent figures have increased from 33% in 1990 to 42% in 1997.2
Virtually all survey data agree that those most fascinated with complementary medicine are predominantly female, affluent, middleaged, and well-educated. Seventy-eight percent of all Medicaid programs provide coverage of at least 1 form of complementary medicine.3
At the same time, critics of complementary medicine often insist there is no good evidence to support these therapies,4,5 and that it is a waste of resources and a misuse of science to try establishing an evidence base for therapies that are essentially nonscientific, irrational, and implausible fads.6 But is this really true?
Research in complementary medicine
Over the last 30 years, the level of original research activity in complementary medicine has increased considerably.7 The best quality of evidence for or against the effectiveness of any therapy is usually provided by Cochrane reviews.8,9
Cochrane reviews
Currently the Cochrane Library contains 34 systematic reviews and 35 protocols of complementary medicine10 (depending on what one considers complementary/alternative and what mainstream, this figure might vary marginally). Twenty of the reviews are of herbal medicines, 7 of acupuncture, 3 of homeopathy, 2 of manual therapies, and 2 of other forms of comple-mentary medicine. Twelve reviews include a meta-analytic approach.
The 34 reviews comprise a total of 286 clinical, mostly randomized and often placebo-controlled, double-blind studies. In 1999, the Cochrane Library listed more than 4000 controlled trials of complementary medicine and a further 4000 awaited assessment.11
The single largest Cochrane review of complementary medicine is a meta-analysis of randomized clinical trials of St John’s wort for depression, based on 27 primary studies with a total of 2291 patients.12 Seven of the 34 Cochrane reviews are “negative”ie, do not suggest a positive clinical effect of the intervention under evaluation. Eleven are entirely inconclusive and 16 draw at least tentatively positive conclusions (Table). Given the dire funding situation for research in complementary medicine,13 this evidence base is remarkable.
TABLE
Cochrane reviews in complementary medicine with (tentatively) positive results
| First author (primary studies)* | Therapy | Indication | Reservations* |
|---|---|---|---|
| Furlan (8) | Massage | Low back pain | More studies required, some trials of poor quality |
| Green (4) | Acupuncture | Lateral elbow pain | More studies required, most trials of poor quality |
| Linde (7) | Acupuncture | Asthma | Evidence only positive for peak expiratory flow rate, effect size small |
| Linde (27) | St John’s wort (Hypericum perforatum)† | Depression | Some trials of poor quality, few equivalence studies |
| Little (11) | Various herbal medicines | Rheumatoid arthritis | More studies required, some trials of poor quality |
| Little (5) | Various herbal medicines | Osteoarthritis | More studies required, some trials of poor quality |
| Melchart (26) | Acupuncture | Headache | Evidence positive only for migraine headaches, effect size small |
| Pittler (7) | Kava (Piper methysticum)† | Anxiety | Concern over safety |
| Pittler (13) | Horse chestnut (Aesculus hippocastanum)† | Chronic venous insufficiency | Scarcity of long-term studies |
| Pittler (4) | Feverfew (Tanacetum parthenium)† | Migraine prevention | More studies required, effect size small |
| Pittler (2) | Globe artichoke (Cynara scolymus)† | Hypercholesterolemia | More studies required effect size small |
| Vickers (7) | Oscillococcinum | Influenza | More studies required, effect size small |
| Wilt (21) | Saw palmetto (Serenoa repens)† | Benign prostate hypertrophy | Effect size moderate, scarcity of long-term studies |
| Wilt (18) | African prune (Pygeum africanum)† | Benign prostate hypertrophy | Scarcity of long-term studies |
| Wilt (4) | Cernilton† | Benign prostate hypertrophy | More studies required, scarcity of long-term trials |
| Wilt (4) | Beta-sitosterols† | Benign prostate hypertrophy | More studies required, scarcity of long-term trials |
| All data extracted from The Cochrane Library, 2003. “More studies required” means that volume of data was small; “trials of poor quality” means that the average quality of the evidence was lowered by flawed studies; “effect size” describes the difference in clinical response to active and control treatment. | |||
| *As expressed by authors of respective review. | |||
| †Plant-based treatments. | |||
Other reviews
The Cochrane database may be the best but certainly is not the only source of systematic reviews of complementary medicine. My unit has published about 100 systematic reviews (a full list is available from the author), and most were not in the Cochrane format.
Linde and Willich have analyzed selected systematic reviews of acupuncture, homeopathy, and herbal medicine, and have shown that their methodological approach differed considerably.14
Methods of reviewing complementary medicine
The conclusions of these methodologically diverse articles were still surprisingly consistent.15 Some maintain that complementary medicine cannot be evidence-based in the conventional sense of the word16 ; that “softer” types of evidence need to be taken into consideration as well17 ; that placebo effects must not be dismissed as nonbeneficial18 ; that the healing encounter includes significant factors that may never be quantifiable19 ; that “the scientific method cannot measure hope, divine intervention, or the power of belief.”20 And, obviously, research in complementary medicine “must consider social, cultural, political, and economic contexts.”21
The debate about what constitutes the best research methods for complementary medicine has been going on for decades. There are no simple answers except that, like in any type of scientific inquiry, there are no intrinsically good or bad methods, only good and bad matches between the research question posed and the methodology employed.22
Herbal medicines have been submitted to systematic reviews more frequently than any other complementary therapy, and it is in this area where most of the positive evidence can be found (also outside Cochrane reviews).23 The medical conditions treated with complementary medicine are often chronic benign diseases for which existing conventional treatments fail to offer a cure or a risk-free reduction of symptoms (Table). Given the popularity of complementary medicine and the economic importance of these conditions, it seems ill-conceived to argue against further research in this area. 6
Firm conclusions of the Cochrane (or other) reviews of complementary medicine are often hampered by the paucity of primary studies; 10 of the 16 reviews in Table are based on fewer than 10 primary studies. The average methodological quality of the primary data is in some but by no means all disappointing.24-26
Problems in testing complementary medicine
There is little doubt that rigorous trials of complementary medicine can pose formidable problems.22 What, for instance, is an adequate placebo for a study of massage therapy, and how should one blind patients in such a trial? The biggest obstacle to good research is perhaps the notorious lack of research funding in this area, which is all the more acute because costs can be particularly high for trials of time-intensive forms of complementary medicine.13
The average size of the overall therapeutic effect associated with complementary medicine is usually modest and the numbers needed to treat are often high. In other words, the difference between benefit from complementary medicine and no therapy or placebo may be statistically significant but critics might argue that it is of debatable clinical relevance.5 Even minor adverse effects would therefore critically disturb the delicate balance of risk and benefit.23
Assessing the risks
It follows that the potential risks of complementary medicine require careful attention and more systematic study. Our fragmentary knowledge indicates that the issues are complex.27 They range from toxicity of herbs to vertebral artery dissection or nerve damage after chiropractic manipulation.
They also include more subtle indirect hazards. Some practitioners of complementary medicine, for instance, tend to advise their clients against employing important medical interventions.28 At present there are no reliable incident figures regarding serious adverse effects of complementary medicine,5,23,27 rendering this area perhaps the most urgent topic for further research.
Moving forward
At a time when healthcare systems universally are strapped for money, the decision whether to integrate complementary medicine into routine medicine will undoubtedly be influenced by economic considerations. Virtually all research on this issue is inconclusive or flawed or both.29 We therefore cannot be sure whether such an integration would save public funds or cost extra money.
In the US, about 41 million people have no health insurance and are thus not covered for even the most basic health care. About the same number of Americans are underinsured. In this situation, it seems difficult to argue without convincing data that the integration of complementary medicine would present a solution to the economic problems in healthcare.
In conclusion, during recent years the evidence in support of complementary medicine has been considerably strengthened, primarily through numerous Cochrane reviews10 and other documents.7,23,30 A large range of promising interventions could be at our fingertips. At a time when whole populations are voting with their feet in favor of complementary medicine,1,2 it would be in everyone’s interest to invest in rigorous research of this area.
Correspondence
Complementary Medicine, Peninsula Medical School, Universities of Exeter and Plymouth, 25 Victoria Park Road, Exeter EX2 4NT, United Kingdom. E-mail: [email protected].
1. Institut fÜr Demoskopie Allensbach. Naturheilmittel: Wichtigste Erkenntnisse aus Allensbacher Trendstudien. [Natural cures: The most important conclusions from Allensbach trend studies.] Available at: www.demoskopie.de.Accessed on May 27, 2003.
2. Eisenberg DM, David RB, Ettner SL, et al. Trends in alternative medicine use in the United States, 1990–1997: results of a follow-up national survey. JAMA 1998;280:1569-1575.
3. Steyer TE, Freed GL, Lantz PM. Medicaid reimbursement for alternative therapies. Alt Ther Health Med 2002;8:84-88.
4. Marcus DM. Alternative medicine and the Arthritis Foundation. Arthritis Rheum 2002;47:5-7.
5. Fitzcharles M. Is it time for rheumatologists to rethink the use of manual therapies? J Rheumatol 2002;29:1117-1120.
6. Charlton BG. Randomized trials in alternative/complementary medicine. Q J Med 2002;95:643-645.
7. Barnes J, Abbot NC, Harkness E, Ernst E. Articles on complementary medicine in the mainstream medical literature: an investigation of MEDLINE, 1966 through 1996. Arch Intern Med 1999;159:1721-1725.
8. Olsen O, Middleton P, Ezzo J, et al. Quality of Cochrane reviews: assessment of sample from 1998. BMJ 2001;323:829-832.
9. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ 2000;320:537-540.
10. The Cochrane Library [database online] Oxford, UK: Update Software; 2002.Available at: www.cochranelibrary.com. Accessed on May 23, 2003.
11. Vickers A. Evidence-based medicine and complementary medicine. ACP J Club 1999;130:A13-14.
12. Linde K. St John’s wort for depression. The Cochrane Library 2000;4:1-17.
13. Ernst E. Funding research into complementary medicine: the situation in Britain. Complement Ther Med 1999;7:250-253.
14. Linde K, Willich SN. How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med 2003;96:17-22.
15. Ernst E. How objective are systematic reviews? J R Soc Med 2003;96:56-57.
16. Tonelli MR, Callahan TC. Why alternative medicine cannot be evidence-based. Acad Med 2001;76:1213-1220.
17. Jonas WB. The evidence house: how to build an inclusive base for complementary medicine. West J Med 2001;175:79-80.
18. Walach H. The efficacy paradox in randomized controlled trials of CAM and elsewhere: beware of the placebo trap. J Altern Complement Med 2001;7:213-218.
19. Redwood D. Methodological changes in the evaluation of complementary and alternative medicine: issued raised by Sherman et al and Hawk et al. J Altern Complement Med 2002;8:5-6.
20. Puchalski CM. Reconnecting the science and art of medicine. Acad Med 2001;76:1224-1225.
21. Bodeker G, Kronenberg F. A public health agenda for traditional, complementary, and alternative medicine. Am J Public Health 2002;92:1582-1591.
22. Vickers A, Cassileth B, Ernst E, et al. How should we research unconventional therapies? A panel report from the conference on Complementary and Alternative Medicine Research Methodology, National Institute of Health. Int J Technol Assess Health Care 1997;13:111-121.
23. Ernst E, Pittler MH, Stevinson C, White AR. The desktop guide to complementary and alternative medicine. London: Mosby; 2001.
24. Bloom BS, Retbi A, Dahan S, Jonsson E. Evaluation of randomized controlled trials on complementary and alternative medicine. Int J Technol Assess Health Care 2000;16:13-21.
25. Linde K, Jonas WB, Melchart D, Willich S. The methodological quality of randomised controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol 2001;30:526-531.
26. Moher D, Sampson M, Campbell K, et al. Assessing the quality of reports of randomised trials in pediatric complementary and alternative medicine. BMC Pediatr 2002;2:1-6.
27. Ernst E. Risks associated with complementary therapies. In MNG Dukes and JR Aronson, eds, Meyler’s Side Effects of Drugs. 14th ed. Oxford and New York: Elsevier Science; 2000:1649-1681.
28. Schmidt K, Ernst E. Aspects of MMR. Survey shows that some homeopaths and chiropractors advise against MMR. BMJ 2002;325-597.
29. White AR, Ernst E. Economic analysis of complementary medicine a systematic review. Complement Ther Med 2000;8:111-118.
30. Yuan C-S, Bieber EJ, eds. Textbook of complementary and alternative medicine. Boca Raton, Fla: Parthenon; 2003.
1. Institut fÜr Demoskopie Allensbach. Naturheilmittel: Wichtigste Erkenntnisse aus Allensbacher Trendstudien. [Natural cures: The most important conclusions from Allensbach trend studies.] Available at: www.demoskopie.de.Accessed on May 27, 2003.
2. Eisenberg DM, David RB, Ettner SL, et al. Trends in alternative medicine use in the United States, 1990–1997: results of a follow-up national survey. JAMA 1998;280:1569-1575.
3. Steyer TE, Freed GL, Lantz PM. Medicaid reimbursement for alternative therapies. Alt Ther Health Med 2002;8:84-88.
4. Marcus DM. Alternative medicine and the Arthritis Foundation. Arthritis Rheum 2002;47:5-7.
5. Fitzcharles M. Is it time for rheumatologists to rethink the use of manual therapies? J Rheumatol 2002;29:1117-1120.
6. Charlton BG. Randomized trials in alternative/complementary medicine. Q J Med 2002;95:643-645.
7. Barnes J, Abbot NC, Harkness E, Ernst E. Articles on complementary medicine in the mainstream medical literature: an investigation of MEDLINE, 1966 through 1996. Arch Intern Med 1999;159:1721-1725.
8. Olsen O, Middleton P, Ezzo J, et al. Quality of Cochrane reviews: assessment of sample from 1998. BMJ 2001;323:829-832.
9. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ 2000;320:537-540.
10. The Cochrane Library [database online] Oxford, UK: Update Software; 2002.Available at: www.cochranelibrary.com. Accessed on May 23, 2003.
11. Vickers A. Evidence-based medicine and complementary medicine. ACP J Club 1999;130:A13-14.
12. Linde K. St John’s wort for depression. The Cochrane Library 2000;4:1-17.
13. Ernst E. Funding research into complementary medicine: the situation in Britain. Complement Ther Med 1999;7:250-253.
14. Linde K, Willich SN. How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med 2003;96:17-22.
15. Ernst E. How objective are systematic reviews? J R Soc Med 2003;96:56-57.
16. Tonelli MR, Callahan TC. Why alternative medicine cannot be evidence-based. Acad Med 2001;76:1213-1220.
17. Jonas WB. The evidence house: how to build an inclusive base for complementary medicine. West J Med 2001;175:79-80.
18. Walach H. The efficacy paradox in randomized controlled trials of CAM and elsewhere: beware of the placebo trap. J Altern Complement Med 2001;7:213-218.
19. Redwood D. Methodological changes in the evaluation of complementary and alternative medicine: issued raised by Sherman et al and Hawk et al. J Altern Complement Med 2002;8:5-6.
20. Puchalski CM. Reconnecting the science and art of medicine. Acad Med 2001;76:1224-1225.
21. Bodeker G, Kronenberg F. A public health agenda for traditional, complementary, and alternative medicine. Am J Public Health 2002;92:1582-1591.
22. Vickers A, Cassileth B, Ernst E, et al. How should we research unconventional therapies? A panel report from the conference on Complementary and Alternative Medicine Research Methodology, National Institute of Health. Int J Technol Assess Health Care 1997;13:111-121.
23. Ernst E, Pittler MH, Stevinson C, White AR. The desktop guide to complementary and alternative medicine. London: Mosby; 2001.
24. Bloom BS, Retbi A, Dahan S, Jonsson E. Evaluation of randomized controlled trials on complementary and alternative medicine. Int J Technol Assess Health Care 2000;16:13-21.
25. Linde K, Jonas WB, Melchart D, Willich S. The methodological quality of randomised controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol 2001;30:526-531.
26. Moher D, Sampson M, Campbell K, et al. Assessing the quality of reports of randomised trials in pediatric complementary and alternative medicine. BMC Pediatr 2002;2:1-6.
27. Ernst E. Risks associated with complementary therapies. In MNG Dukes and JR Aronson, eds, Meyler’s Side Effects of Drugs. 14th ed. Oxford and New York: Elsevier Science; 2000:1649-1681.
28. Schmidt K, Ernst E. Aspects of MMR. Survey shows that some homeopaths and chiropractors advise against MMR. BMJ 2002;325-597.
29. White AR, Ernst E. Economic analysis of complementary medicine a systematic review. Complement Ther Med 2000;8:111-118.
30. Yuan C-S, Bieber EJ, eds. Textbook of complementary and alternative medicine. Boca Raton, Fla: Parthenon; 2003.