Sorting Through the Diagnosis of Physician Dyscompetence

William Norcross, MD


November 23, 2009


The UCSD PACE Program is the largest program for assessment of physician competence and performance in North America. In addition to its own unique set of procedures and tools, PACE is a site for the Assessment Center Program of the Post-Licensure Assessment System (PLAS), a collaborative program of the National Board of Medical Examiners (NBME) and the Federation of State Medical Boards (FSMB). In the following sections, Dr. W's case is used to illustrate the Phase I and II Assessments of the UCSD PACE Program.

Phase I Assessment

Complete history and physical examination. This examination revealed no significant physical or mental health problems that would interfere with Dr. W's ability to practice medicine.

PACE health professional intake data form. The intake form revealed no personal, professional, academic, or lifestyle issues likely to contribute to substandard practice.

PACE assessment of ongoing medical education and professional development. This assessment revealed no noteworthy problems, with skills and practices used to maintain professional growth. He attended about 1 continuing medical education (CME) meeting annually, usually the Scientific Assembly of the American Academy of Family Physicians. He enjoys reading, both professional journals and science fiction. He subscribes to American Family Physician, Annals of Family Medicine, The Journal of Family Practice, and The New England Journal of Medicine. He said that he tried to read the abstracts of all the articles in a journal, and used that information to read the full articles that seemed to be of practical value to his practice.

Oral clinical examination in family medicine (performed by a member of the PACE faculty in family medicine). He received 9.5 of a possible 10 -- an excellent score.

NBME tests. Dr. W was given several examinations developed by the NBME with the following results:

  • NBME Clinical Subject Exam in Family Medicine: a standardized paper-and-pencil multiple-choice examination that is specific for the specialty of family medicine, seventh decile;

  • NBME Mechanisms of Disease: a standardized paper-and-pencil multiple-choice examination testing basic medical school knowledge, eighth decile; and

  • NBME Ethics and Communication: a standardized paper-and-pencil multiple-choice examination assessing knowledge and judgment in ethical decision making and communication, ninth decile.

A fourth examination, the NBME Primum and Transaction Stimulated Recall (The Primum), was of particular interest. This is an interactive computerized program, in which the physician is given initial basic information about 8 patients and then is asked to request information from the history, physical examination, and lab and imaging studies for those patients. The physician is then asked to make a diagnosis and manage each one. The analysis provided by the NBME showed a generally good performance on the Primum when Dr. W was compared with the comparison group: 6 cases at the quartile above the median, 1 case at the quartile below the median, and 1 case at the bottom quartile.

Immediately after completing the Primum, the NBME provides PACE with a printout of all of the "transactions" that the physician requested during his/her evaluation of each patient scenario. The transaction-stimulated recall is analogous to the chart-stimulated recall process, but it uses the transactions from the Primum as the basis of exploring the index physician's clinical decision-making processes and clinical judgment in the diagnosis and management of each scenario.

The transaction-stimulated recall interview, which was performed by a member of the PACE faculty, disclosed some important findings. Dr. W missed important diagnoses on 2 cases; the interviewer found that he became too focused too quickly on the primary diagnosis, which caused him to miss the other diagnoses. The faculty member wrote: "Dr. W is intelligent and well-trained, and has good basic skills. He has a high volume practice and tends to focus on the diagnosis and evaluate patients quickly. Most of the time he is right. However, he has the tendency, and danger, of missing critical clinical information...."

Examining a mock patient. Dr. W performed a complete history and physical examination on a mock patient. Evaluation and feedback were provided by an observing PACE faculty member and by the mock patient. Dr. W performed a thorough, competent history and physical examination. He was nervous, but very professional, respectful, and caring.

PACE chart audit. A member of the PACE faculty of the same specialty as Dr. W performed a longitudinal audit of 10 of Dr. W's charts, which were randomly selected, using a standardized instrument. Deficiencies were noted in all charts reviewed and included:

  • Illegible handwriting at times;

  • Gross disorganization, with an inability to follow the management of chronic health problems or preventive care longitudinally;

  • No use of problem lists, medication records, or flow sheets for chronic problems (eg, diabetes) or preventive care;

  • Failure to follow-up or treat problems, eg, total cholesterol of 388 mg/dL and triglycerides of 635 mg/dL not addressed in follow-up visits; and

  • Failure to treat chronic problems to levels of commonly accepted clinical guidelines, eg, glycohemoglobin values above 9% tolerated without increasing intensity of therapy and low-density lipoprotein cholesterol values of well over 100 mg/dL tolerated in patients with adult-onset diabetes without increasing intensity of therapy).

All of the charts showed illegible handwriting, gross disorganization, marked deviation from accepted standards and clinical guidelines, and significant problems that were lost to follow-up or inadequately treated.

Phase II Assessment

The PACE Phase II Assessment is a 5-day on-site clinical observership. In the case of Dr. W, it took place at 2 UCSD Medical Group family medicine clinics under the mentorship of 7 full-time faculty family physicians. The PACE participating physician, in this case Dr. W, accompanies the faculty member on his/her clinical activities, meets patients, and engages in discussion and questioning. The participating physician has no responsibility for patient care, performs no procedures, and makes no entries into the medical record. PACE participants are also asked to keep a patient log and develop a portfolio during Phase II.

Dr. W was judged by all Phase II faculty assessors to be polite, courteous, friendly, punctual, and professional in appearance and behavior. Typical faculty comments included the following:

  • "Very nice man. Good fund of knowledge. Tends to jump to conclusions, however...."

  • "Polite, seems to have a solid fund of knowledge, but does not employ it readily. Seems in a hurry. Was able to come up with more possibilities in the differential diagnosis of a patient with chest pain when I coaxed him, but his innate tendency was to rush to the diagnosis of esophageal spasm...."

  • "Pleasant to have in clinic. Was very impressed with our electronic health record and our frequent point-of-contact use of the Internet for seeking current clinical information."

As part of the Phase II experience, a standardized charting exercise was administered. Dr. W was asked to view a videotaped history and physical examination that a faculty member had performed on a standardized patient with chest pain. He was encouraged to take notes and allowed to go over the 15-minute tape as often as he wanted in order to get all of the information accurately. His instructions were to construct an accurate and complete medical record of the interaction "as if he were the PACE faculty member writing up the encounter." He was asked to create the very best record that he could. His results were excellent: accurate, complete, legible, and well organized.

In order to address practice-based learning and improvement, as well as to ensure that the physician has basic computer competency, the participating physician is also required to complete 2 tests using knowledge of the Internet, called the Evidence-Based Medicine Project and the World Wide Web Clinical Information Treasure Hunt. Dr. W's performance on both was satisfactory.