That’s the question many physicians ask when patients mention complementary and alternative medicine (CAM). It is the very question PDSA’s medical advisors asked when we set out to update the complementary medicine portion of our web site. Since it is such a frequently asked question, it deserves some discussion.

First, we need to define evidence, in the medical sense. Medical evidence requires a published study. If something seems to help a group or even a single patient, it is only considered evidence if someone writes an article and a journal publishes it. Researchers then rank the published evidence according to quality. In general, double-blind, placebo-controlled studies are considered the highest quality evidence.

Another measure of quality is the reputation and reach of the journal in which the research is published. One measure of journal quality is the Impact Factor, the number of times the articles in the journal have been referenced by other journal articles in the past two years. Some of the journals on the high-impact list are familiar to many, such as the New England Journal of Medicine. Despite controversy over the validity of the journal impact criteria, it is still a widely used, sometimes subjective, measure.²

The same publishing and grading systems are used for complementary treatments, although there are considerable barriers at each step in the evidence process for those types of treatments.The first barrier is the cost. “The cost per patient of running Phase 3 clinical studies of new pharmaceuticals [larger-enrollment studies required for establishing the cost/benefit and prescribing regimen] exceeds $26,000, on average…Survey data reveal that Phase 3 studies are the most costly as measured on a per-patient basis. Phase 2 trials are comparatively cheaper, with the average per-patient cost falling just over $19,300 per patient. Phase 1 trials, which test drugs’ safety on a fraction of the number of patients tested in Phase III trials, are even less expensive at nearly $15,700 per patient.”⁴

Unless the treatment has a large profit potential, a placebo-controlled, phase III trial or even a smaller trial is prohibitive for many wanting to study complementary treatments. Some clinical trials in the US are funded by the National Institutes of Health. However the budget for the Center for Complementary and Alternative Medicine at the National Institutes of Health is less than one half of one percent of the total NIH budget and that is to cover all the possible complementary treatments for all diseases. Other Institutes and non-profits may fund some CAM trials, but very few so this as a large part of their budget or their mission.

Another barrier is publication.There are many journals that welcome conventional research results, but in the top journals the bar is higher for complementary medical results, especially if they are positive. “More positive than negative trials of complementary therapies are published, except in high-impact factor MM [mainstream medicine]-journals. In non-impact factor CAM-journals positive studies were of poorer methodological quality than the corresponding negative studies.”³ It is important to note that while both positive and negative results are published for CAM in high-impact journals, 80% of industry-sponsored trials for conventional medicine are positive and 50% of trials are never published, potentially because the results are negative.⁵

A third barrier is expertise. Many of the journal articles reporting results on clinical trials for pharmaceutical and biotech treatments are drafted by professional medical writers paid by corporations. The clinical research and publication drafts in large research institutions, excluding drug and biotech clinical trials, are usually done by lower-level research staff. In the complementary medical community, the expertise and rewards center on treating patients, not designing, writing, or publishing research studies and the depth of the staff is often shallow.

A fourth barrier is language. CAM research is global, with much of the research done and published outside of the US. That means the results and full journal articles, even if published are often inaccessible in English.  Some English abstracts are in PubMed, but not all.

All the barriers taken together, it is not surprising that there are fewer CAM studies, sometimes considered of a lower quality, than their conventional counterparts. In recent years there has been a greater emphasis on CAM research and more visibility of the bias in some main stream medical research¹. Hopefully, there will be more equality in the future.

*****************

1. Freedman D. “Lies, Damned Lies, and Medical Science.” Atlantic Magazine. Nov. 2010.http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/

2. Impact Factor: http://en.wikipedia.org/wiki/Impact_factor

3. Pittler MH et al. “Location bias in controlled clinical trials of complementary/alternative therapies.” Journal of Clinical Epidemiology. May 2000, Volume 53, Issue 5, Pages 485-489. http://www.ncbi.nlm.nih.gov/pubmed/10812320

4. “Phase 3 Clinical Trial Costs Exceed $26,000 per Patient.” PRNewswire. October 13, 2006. http://www.lifesciencesworld.com/news/view/11080

5. Sharav VH. "Black Hole" of medical research--Negative Results Don't get Published - JAMA, WSJ” Alliance for Human Research Protection. June 5, 2004. http://www.ahrp.org/infomail/04/06/05.php