ABIM to Test Open-Book Exam for MOC

April 07, 2016

Physicians have less and less need to rely on memorized know-how when online clinical reference material is at their fingertips in the exam room.

The American Board of Internal Medicine (ABIM) is heeding that trend as it begins to study switching to an open-book exam for its controversial maintenance of certification (MOC) program. Richard Baron, MD, the president and chief executive officer of the ABIM, said that testing such an exam is imperative, given the science of psychometrics. Dr Baron wants to know, for example, whether test-takers will look up so much information online that they won't have time to answer all the questions.

A legion of irate ABIM diplomates have damned the group's MOC requirements as too time-consuming, too expensive, and irrelevant to their clinical experience. Part of their experience is looking up drug information, practice guidelines, and journal articles online at the point of care. So why should MOC exams be any different?

Last month the ABIM announced that it had begun a study on making at least part of its MOC assessments open book. The study dovetails with the ABIM's plan to replace the recertification test, now conducted every 10 years, with shorter, more frequent tests.

The ABIM has invited a select group of internal medicine diplomates to complete a 5-hour pilot exam at Pearson VUE testing centers across the country between April 17 and September 1. It is not an actual MOC exam that will affect their certification status, and it covers only about 75% of the real thing anyway, Dr Baron told Medscape Medical News. However, because the pilot exam represents a bona fide learning experience, participants will receive 20 MOC points in addition to a $250 honorarium.

One group of randomly assigned physicians will take the pilot exam with access to UpToDate, an online information resource for clinicians. A second group will not have access to any online material. The ABIM will compare the test scores of the two groups, survey them about their experience, and solicit recommendations to improve the test. Dr Baron said his group hopes to publish the results by year's end.

If and when open-book testing becomes a reality, the ABIM probably would consider a variety of online information resources, and not just UpToDate, he said.

Time Management a Key Issue

The study of open-book testing for MOC will center largely on physicians' time management, said Dr Baron. It's an issue physicians habitually face in the exam room, he said.

"Doctors who are seeing scheduled patients don't have time to look up something on every patient that they see," he said.

The question then is how much test-takers will use UpToDate.

"When we give the exam now, most people get through all the questions," said Dr Baron. "If you try to look up every question [online], you'll probably run out of time."

The open-book test will also shed light on the quality of individual questions, which must not be so easy that everyone gets the right answer, or so hard that everyone whiffs. "Let's say that 60% of physicians get a certain question right," said Dr Baron. "If you give them access to UpToDate, what will the percentage be then?"

One critic of the ABIM's MOC program has called the pilot for an open-book exam a waste of time. Writing in his blog titled Dr Wes, cardiologist Westby Fisher, MD, recently cautioned his colleagues to "carefully consider...participating in this latest ABIM MOC research project that promises to potentiate this expensive and corrupt re-credentialing program."

Dr Fisher said the $250 honorarium amounted to less than $50 per hour and noted that physicians won't receive extra money for travel. And the 20 MOC points, he said, "seems like bribery," given that membership on a hospital staff and participation in an insurer-provider network hinge on board certification.

In response, Dr Baron said the physicians in the pilot exam are volunteering to help improve the testing process. "We're not pretending to value their time in a market-based way," he said. Awarding the 20 MOC points is reasonable, he added, because some physicians will treat the pilot exam as a warm-up for the actual exam and prepare beforehand. And one cohort will be researching questions during the exam using UpToDate.

"We think this is a high-value educational experience," said Dr Baron.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.