CHEST. 2003;123(3) 

In This Article

Clinician or Statistician?

Evidence-based medicine is not a new concept. It is rather the attention to study quality and to the classification of the level of evidence that has changed recently. Simply publishing evidence in a peer-reviewed journal does not constitute a reason to change individual clinical practices. The evidence must be scrutinized closely. The validity of the statistical analysis is an integral element of such scrutiny. The challenge then becomes to offer adequate information about the design and methods of a study without overwhelming the reader (usually a clinician) with complex statistical terms and calculations.

In the absence of universal criteria for the reporting of statistical methodologies, it is up to the authors to include the pertinent parts of such methodologies. With clinicians receiving little formal training in biostatistics, and with only the occasional participation of statisticians in the authorship list, it comes as no surprise that "the stats" of a study are often reported wrongly or inadequately. In this issue of CHEST (see page 923), Moss and colleagues attempt to address this issue as it relates to the pulmonary and critical care literature. They evaluate the literature from five prominent pulmonary and critical care journals over a 6-month period and focus on the use of multivariable logistic models. Using criteria established by Concato et al in 1993[1] < they examine 81 published manuscripts for 12 potential limitations that are related to the reporting of multivariable logistic regression analysis. Deficiencies were found in all 12 categories. The most consistent deficiency was in the reporting of collinearity, which was mentioned only in 1 of these 81 articles. Collinearity occurs when independent variables are so highly correlated that it becomes difficult or impossible to distinguish their individual influences on the response variable. Clearly, not testing for collinearity may lead to significant flaws in the results and conclusions. It is difficult to understand whether the absence of reference to this important value indicates the failure to produce it or the failure to report it. Given the intimate involvement of statistical consultants in such studies, the latter is more likely. The lack of information on other important details speaks about a casual attitude toward reporting statistical methodologies. Although every author would mention the type and manufacturer of the ultrasonographic device that was used to perform complex ultrasonography procedures, 31% of the studies examined by Moss et al did not report on the statistical software used for the complex statistical analysis.

If the problem lies merely in inadequate reporting, it should be easy to fix. As suggested by Campillo[2] and Lang and Secic,[3] strict guidelines must be in place for publications using complex statistical analysis such as multivariable logistic regression. This would force authors to clearly state their statistical methods, and any flaws could be attributed to the methods used rather than to the lack of reporting. On the other hand, if the problem lies in the misuse or misunderstanding of logistic regression and other statistical methods, the problem is harder to remedy. It would call for a higher level of education in biostatistics for clinicians performing high-quality research. This should not be interpreted as a call for physicians to analyze their own results. Assistance by qualified statisticians is imperative and irreplaceable. Clinicians can hardly become statisticians. However, a more-than-superficial understanding of statistics can ensure smooth collaboration and valid results. The old dictum "garbage in, garbage out" cannot be better represented than by the case of a clinician with no statistical knowledge working with a statistician with no clinical perspective.

A second layer of improvement can occur during the peer review process. As noted by Goodman et al,[4] 82% of journals with circulations of > 25,000 copies per year maintain a statistical consultant, compared to only 31% of those journals with circulations of < 4,100 copies per year. It would be desirable to include routinely a statistician among the reviewers of articles that use complex statistical analysis. Comments on the statistical methods can only improve the quality of a study, the same way the clinical reviewers' comments improve the content and display of the results. A balanced view should be maintained between reporting essential information and filling up space with unnecessary details.

Obviously, these are not problems limited to the pulmonary and critical care literature. In 1993, Concato et al[1] revealed similar problems in another two leading medical journals, The Lancet and The New England Journal of Medicine. If such problems occur in journals of undisputed scientific magnitude, they are likely to occur throughout the medical literature. It is incumbent on those who write the articles to use statistical analysis appropriately, honestly, and competently, and to report the methods used without ambiguity. Also, the peer reviewers must demand that a precise but succinct statistical report be included in the manuscripts that are submitted. Finally, the consumer (clinician) must look at all that is written with a jaundiced eye and be sure that conclusions stand on a solid statistical foundation before being applied to patients. Clinicians cannot be statisticians, but they can become solid interpreters of essential statistical methods.

Carlos V.R. Brown, MD, George C. Velmahos, MD, Los Angeles, CA

Drs. Brown and Velmahos are associated with the Los Angeles County-University of Southern California Medical Center. Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (e-mail: permissions@chestnet.org).

Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (e-mail: permissions@chestnet.org).

Correspondence to: George C. Velmahos, MD, LAC + USC Medical Center, 1200 North State St, Room 9900, Los Angeles, CA 90033.

  1. Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariable models. Ann Intern Med 1993; 118:201-210

  2. Campillo C. Standardizing criteria for logistic regression models. Ann Intern Med 1993; 119:540-541

  3. Lang TA, Secic M. How to report statistics in medicine. Philadelphia, PA: American College of Physicians, 1997

  4. Goodman SN, Altman DG, George SL. Statistical reviewing policies of medical journals: caveat lector? J Gen Intern Med 1998; 13:753-756