A Clarification of Critical Appraisal, Evidence, and Education

Joel J. Gagnier, BA(Hons), ND, MSc(Cand Clin Epi)


April 29, 2004

Editor's Notes:
This is a revision of an earlier draft letter which was published erroneously. The author was provided with this revised letter and chose to not respond further.

I would like to thank the editors of Medscape General Medicine for allowing me the opportunity to comment on the narrative review entitled "Naturopathy: A Critical Appraisal."

As a first comment, I would like to offer some clarification. The term "Critical Appraisal" can be defined in numerous ways. When this term is used in reference to an area of medicine, critical appraisal is a process by which one uses predetermined criteria to rationally evaluate a published article, some intervention (therapeutic, diagnostic, prognostic, etc), or possibly an entire field of medical care, as Atwood attempts. Typically, a set of criteria are identified that are appropriate for the specific focus; these are then applied to the literature surrounding the topic, and logical conclusions are then derived.[1,2] This process has been the focus of a growing trend in medical practice and education: evidence-based medicine.[1,2] Atwood's article does not employ critical appraisal in this sense. I propose that Atwood's article is a criticism in the form of a narrative review. The definition of a criticism is "The act of criticizing usually unfavourably: fault finding, disapproval and objection."[3] Thus, the title would more appropriately read "Naturopathy: A Criticism." Also, it is well known that narrative reviews, such as this, are biased and lag far behind the current data.[4,6] But of course, because objectivity is implied in the words "critical appraisal," one can see why the author has chosen the published title. Kudos!

In the introduction of this article, Atwood quotes a dated (1968) Department of Health, Education, and Welfare (HEW) report and a section of a recent book to convince readers that practices of naturopathic medicine are not supported by research. The latter quote, from investigators at Simon Fraser University, Burnaby, British Columbia, Canada, is confusing. The quote states that they could not find any properly controlled trials to support the claims of the profession. However, search strategies employed, keywords used, and databases searched are not reported. Let me clarify the amount of literature available that appraises the interventions used by naturopathic doctors (NDs). There are well over 6500 controlled clinical trials[7] and over 1000 systematic reviews on the various complementary and alternative medicines (CAM) that NDs are taught and regularly use. Empirical evidence indicates that the methodologic quality of controlled CAM trials is similar to conventional medicine trials.[8,10] In fact, there is empirical evidence that many of the interventions taught in naturopathic medical schools are efficacious. It is not feasible to explore each individual intervention in this short letter.

But do not be fooled! Medicine is not a purely evidence-based adventure. Nor is naturopathic medicine. Current research indicates that about 50% of the interventions used in general internal medicine are supported by randomized controlled studies.[11] Of course, these figures do not include the literature on standard medical preventive, diagnostic, and prognostic practices, of which NDs are trained. The amount of evidence supporting interventions used by NDs has not been adequately explored.

Atwood fails at supporting his hypothesis that NDs are inadequately trained to be primary care practitioners. If medical doctor (MD) training is considered to be the "gold standard," let us compare naturopathic educational programs with MD programs. A comparison between MD and ND training reveals that the basic science instruction (anatomy, physiology, biochemistry, pharmacology, pathology, microbiology, and immunology) is virtually identical in terms of quantity, content, and intensity.[12] Additionally, both primarily have doctorates delivering the curriculum. Clinical education (physical diagnosis, clinical diagnosis, radiologic diagnosis, minor surgery, etc) is also very similar between the 2 schools. The main difference between these 2 educations is clinical instruction, interning, and residencies. Naturopathic medical schools rely almost exclusively on outpatient clinics, whereas MD clinical instruction uses outpatient clinics, emergency rooms, intensive care units, and other hospital environments.[12] ND training includes a 1-year internship and optional 2-year residency (residency positions are limited), whereas MD training includes a 2- to 6-year residency, depending on specialty. Both professions sit rigorous licensing exams that include basic science and clinical science components. Both licensing exams are national, standardized, and designed to assess competence relative to each profession.[12]

Atwood states that the validity of what NDs do "can be determined only by reference to the facts of nature and by rigorous testing of biologically plausible claims." The issue of validity of interventions used by NDs is addressed above. In short, it is not possible in this space to critically appraise all evidence for all the CAM therapies used by NDs. I can address individual areas if Dr. Atwood would like to contact me. The author seems to claim that biological plausibility is a prerequisite for doing any controlled testing and that rigorous controlled testing is required to establish validity. Although I do agree with the latter, biological plausibility is not required for controlled testing. Many therapies used in medicine today were discovered through serendipitous findings in unrelated research. That is, a known mechanism or a theoretically probable mechanism is not required for controlled testing. Therefore, efficacy of an intervention may be shown despite a known biological mechanism. Nonetheless, to establish causation requires a review of all principles of causation, not just biological plausibility.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: