USMLE Clinical Skills Testing: End It or Change It Entirely

The lowest-value, highest-cost portion of the United States Medical Licensing Examination (USMLE) series is finally gone. At least for now.

Five years ago, as a medical student, I co-organized #EndStep2CS, a nationwide movement demanding an end to the USMLE Step 2 Clinical Skills (CS) exam. Despite widespread support — including nearly 20,000 signatures from students, faculty, and physicians — the USMLE refused to budge. Instead, it raised the passing score, arguably as a punitive response to our protest.

All it took was a global pandemic to change things.

Because Step 2 CS involves face-to-face interactions between candidates and standardized patients, it was appropriately deemed high risk during the COVID-19 outbreak. In March, the USMLE suspended the exam, which will remain defunct for at least a year. A test that was previously defended as essential to patient safety and critical to physician licensing is now on hold, without any subsequent significant interruption to the production of new doctors. In the meantime, USMLE says it is working on a plan to reimagine the test, a mission it calls "Revitalize CS."

Since its inception in 2004, the Step 2 CS has been controversial and unpopular. Some have highlighted its significant cost, including the $1300 testing fee and associated travel expenses (as it is administered at only five sites). Others point to its low or unproven value and its contribution to burnout. These and other concerns, including the lack of time for student remediation, were raised before the exam was even required.

Given these flaws, if Step 2 CS cannot be truly "revitalized," then it should be outright eliminated.

What Is It Good For?

We can all agree that doctors should have essential clinical skills. The question is whether the Step 2 CS is a meaningful way of assessing those skills or whether it causes more problems than it's worth.

The test itself is fairly new; it was introduced in 2004. It purports to assess physical examination, communication, and documentation skills before a doctor can be licensed. Because an incompetent trainee may pose a risk to patients, this screening makes sense on the surface. Unfortunately, there is ample reason to doubt that Step 2 CS identifies "unskilled" future doctors before they can be licensed.

Step 2 CS is internally validated by the USMLE. Notably, that organization has a monopoly on medical licensing in the United States. Although it is a nonprofit, it makes millions of dollars in revenue per year, despite having no significant external checks or balances. The test's sensitivity and specificity in identifying unprepared trainees have never been meaningfully presented. We still do not know the expected prevalence of "unqualified trainees" in the tested population. Thus, the test's actual value remains unknown.

Clinical tests that screen for disease in our patients must demonstrate objective benefit that outweighs harm before they are routinely used in practice. Step 2 CS should be held to the same standard. In the more than 15 years that Step 2 CS has been implemented, we have been presented no significant evidence that it has actually improved patient safety, reduced morbidity or mortality, improved resident physician communication, or in any way improved clinical skills. Furthermore, no data suggest that Step 2 CS is better able to identify a problematic medical student than the traditional screening method: review by the faculty and deans at licensed and accredited medical schools.

Yet any opposition to Step 2 CS is dismissed as an opposition to protecting patient safety and the "greater good."

Can It Be Saved?

If the test is to be saved, the USMLE must truly "revitalize" Step 2 CS during the current suspension. As a community, we must advocate for several meaningful improvements and alterations.

An easy first step would be to increase the exam's availability. Quite simply, the USMLE must open more testing sites. Students must currently travel significant distances to take the exam. Think of the burden this places on rural students or those in states like Hawaii or Alaska. This testing must be done in a cost-neutral way for students.

If Step 2 CS comes back, the test could be administered at medical schools throughout the country. The National Board of Medical Examiners (NBME), which comprises the USMLE along with the Federation of State Medical Boards (FSMB), probably has sufficient funds to help upgrade clinical simulation sites around the country to meet its standards. It should also be able to have teams routinely audit the decentralized testing process. To further help with cost concerns, financial assistance should be available to students who need it.

Behind the scenes, the USMLE should include more medical students on its board. They should be from diverse backgrounds, including those who have limited finances as well as ESL students, and should be involved at various levels. They should also be compensated for their time. This would help ensure that financially vulnerable students are incentivized to commit their time toward improving the test.

Moreover, because all medical students are affected, all of them should be surveyed about the test moving forward. Sure, some — or even most — of the feedback may be negative, but this may provide excellent ideas that would not otherwise emerge. After completing licensing, many doctors stop thinking about these tests. Students are the greatest untapped pool for revitalizing the process.

Or Should It Be Abolished?

If those changes seem conservative, why not start fresh?

Instead of a universally applied standardized test like Step 2 CS, clinical skills assessments could be returned to individual medical schools (or school collaborations). The USMLE could create a rubric with clear guidelines and train medical school faculty how to observe third-year students in real patient interactions. Students could then be given a chance to improve, and schools could provide meaningful remediation between attempts. If the NBME wants to ensure that this is sufficient for licensing, it should work with accrediting bodies to evaluate school-based clinical skills testing.

Any proposal to end the test will be met with the same responses we have heard time and again. To those who claim that Step 2 CS protects patient safety, I would again stress that there has never been any evidence that Step 2 CS improves patient safety, clinical skills, or communication. In fact, a 2016 meta-analysis of large-scale licensing examinations in highly developed countries found "a significant knowledge gap [around] the impact of licensure examinations on subsequent patient care and on the profession." The authors concluded that the debate is "characterized by strong opinions but is weak in terms of validity evidence."

Others claim that getting rid of Step 2 CS as a requirement for licensing would look bad to the general public. This claim ignores the fact that medical schools routinely assess students' clinical skills. Keeping a useless, costly test in order to keep up appearances to those outside of medicine is an insufficient reason for its preservation.

The need to assess international medical graduates (IMGs) is also used as a reason to preserve the exam. As per NBME representatives in 2004, "allowing a person to bypass a licensing requirement solely on the basis of the location of his or her medical school shows a lack of concern about patient safety and the quality of health care."

For many reasons, we need strong pathways to facilitate IMGs joining clinical practice in the United States. However, international medical schools are not beholden to the same accreditation as US schools. This is why we have an Educational Commission for Foreign Medical Graduates (ECFMG). That organization offered a clinical skills assessment before Step 2 CS came into existence. It would seem reasonable to return to that system.

Acknowledging Current Reality

Despite arguments made in favor of Step 2 CS, the value of the exam must be assessed in the context of current reality. If the test can be halted by the COVID-19 pandemic without anyone losing sleep over an intolerable risk to patient safety or the impossibility of assessing the quality of applicants to residency programs, then its benefits have clearly been overstated.

Step 2 CS does not meet the bare minimum expectations of any screening test. We have reached this point only because we have allowed a monopoly on licensure by a virtually unchecked organization. It does not need to be this way.

The #EndStep2CS movement failed because medical students are a transient population, with little power or incentives to go against hierarchy. The NBME saw $24 million in added revenue the year after Step 2 CS was introduced. That is a lot of money for them to defend. It is imperative that doctors use the current suspension to speak out, lobby, and demand that the USMLE be subject to meaningful checks and balances or that Step 2 CS be ended outright.

No matter how we get there, we need to step away from Step 2 CS as it currently exists — not because clinical skills are unimportant, but because they are too important to be subjected to such a flawed process.

Lydia A. Flier, MD, is a primary care physician and an instructor at Harvard Medical School. In addition to #EndStep2CS, her work has included development of a curriculum to teach medical students how to teach and professionalism training for medical students entering wards. As a medical student, she co-wrote and co-directed "The Gunner Song" and "What Does the Spleen Do?"

Follow Medscape on Facebook, Twitter, Instagram, and YouTube

Comments

Commenting is limited to medical professionals. To comment please Log-in.

Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.