New Admission Test Flawed, Won't Produce Better Doctors

The medical school admissions process in the United States demands applicants earn high grades in relevant undergraduate courses and an impressive score on the Medical College Admission Test (MCAT). However, the MCAT cannot gauge many factors that make a good physician. These factors include qualities such as empathy, emotional intelligence, communication skills, and even common sense. To fill this void, more than 35 American medical schools now require or are considering imposing so-called situational judgment tests (SJTs). These tests are designed to assess reactions to numerous scenarios applicants are likely to encounter while practicing medicine in attempt to measure qualities the MCAT cannot.

SJTs have a four-decade history of successful use in nonmedical settings. Versions that measure interpersonal skills appear to have some value in predicting both internship and job performance in medical students. Data suggest that incorporating SJTs into the admissions process may increase the proportion of matriculating female students and those from lower socioeconomic backgrounds. Medical educators champion these tests as an equalizing force in the admissions process.

SJTs may help increase diversity in certain ways, including helping to identify candidates with important "softer skills." However, they may actually decrease other types of diversity and inflict additional psychological and financial burdens on students. The real danger is that the adoption of these tests may give the appearance of fully addressing concerns about diversity and prevent the search for a better, less cumbersome tool.

Why SJTs Are Problematic

Certain demographic groups—historically, disadvantaged minorities and individuals from lower socioeconomic backgrounds—have long been significantly underrepresented among medical students. SJTs have been praised as being marginally better than the MCAT, which has been associated with aspects of structural racism. However, the use of SJTs alone cannot produce medical school classes that remotely mirror the overall United States population. Applicants with disadvantaged backgrounds still score lower on SJTs than their privileged peers, even if the difference is less than with the MCAT. Celebrating the minimal improvement associated with SJTs may perpetuate the false belief that the ongoing demographic problem is currently being "fixed."

Additionally, students can prepare for an SJT as easily as they can for any other standardized examination. SJT tutors are already popping up alongside MCAT tutors. In addition, many of the skills tested on SJTs, such as how to navigate a professional setting, are those that affluent students often learn through socialization at an early age. These skills range from having the confidence to question a more senior clinician to being able to ask for feedback in a direct and constructive manner. Absorbing cultural norms of the medical setting is far easier when a person has parents from backgrounds similar to most medical professionals. Although other students from different backgrounds may acquire these skills, they may only do so after exposure to these norms during medical training.

For example, affluent students who have witnessed their own parents negotiate with medical providers on more equal footing may be more willing to question the conduct of others in a hospital setting. Students who have inherently had less bargaining power, due to various socioeconomic factors, may be reluctant to engage in such negotiation, even when a scenario in an SJT calls for doing so. Perhaps more significantly, SJTs are generally scored by independent raters who are usually physicians already socialized to the medical atmosphere.

Many SJTs are also plagued by a lack of transparency in scoring, which is necessary to maintain the integrity of the tests. For example, the website for CASPer explains, "We do not provide applicants with their CASPer score nor do we provide feedback on their answers... [Applicants] will not receive their CASPer score in the same way that they would not expect to get 'results' from an interview, reference letter, or personal statement." Scores, both a numeric value and a percentile, are provided directly to medical schools at the request of applicants, and the schools are permitted to use the scores as they see fit.

However, in preparing their candidacies, applicants often receive feedback on their interviewing skills and statements. Even applicants taking the MCAT are provided their scores directly, allowing them to gauge their performances and prospects for admission. Rather than allowing test takers to understand and overcome their shortcomings, SJTs obscure the process.

Comments

Commenting is limited to medical professionals. To comment please Log-in.

Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.