Surgery and IM Residents Both Overestimate Surgical Risk

Veronica Hackethal, MD

October 13, 2017

Internal medicine (IM) and general surgery trainees almost universally overestimate the risk for postoperative complications and death in surgically complex patients, according to a study published online October 11 in JAMA Surgery.

The study is the first to evaluate how accurately medical and surgical residents assess operative risk in surgically complex patients compared with a standard risk calculator.

"The study underscores the importance of availability and refinement of risk-adjusted models, such as the [American College of Surgeons' National Surgical Quality Improvement Program] NSQIP risk calculator, to provide a consistent, individualized, and evidence-based assessment of surgical risk to patients," James Healy, MD, from Yale University School of Medicine, New Haven, Connecticut, and colleagues write.

Many physicians develop a sense of surgical risk during the course of their careers. However, such assessments can be subject to bias, including a tendency to use anecdotal evidence. More objective risk calculators have been developed, but these do not always consider multiple medical comorbidities.

Surgeons and IM practitioners often work together in making treatment decisions about complex patients, but whether one group performs better than the other is unclear.

To investigate the issue, researchers conducted a study with 76 general surgery residents (34.2% women) and 76 IM residents (52.6% women). Participants used an anonymous, online tool to estimate the likelihood of complications and death on a scale of 0% (not probable) to 100% (highly probable). They assessed seven scenarios in surgically complex patients: colectomy, duodenal ulcer repair, inguinal hernia repair, perforated viscus exploration, small bowel resection, cholecystectomy, and mastectomy.

Then researchers compared residents' responses with risk-adjusted outcomes from the NSQIP online calculator, a nationally representative tool that allows users to input 21 patient characteristics to estimate 30-day risk for 12 different types of complications.

The researchers also asked about residents' confidence on a variety of factors related to surgical risk assessment. Respondents used a 5-point Likert scale, with 1 indicating strongly disagree and 5 indicating strongly agree. The study had an overall response rate of 64%.

General surgery residents had significantly more confidence in predicting postoperative surgical complications compared with IM residents (mean response, 3.6 vs 2.8, respectively; P < .001).

General surgery residents were significantly more confident than IM residents about not offering operations (4.3 vs 3.7; P < .001), were less likely to discuss code status (3.2 vs 3.8; P < .001), and were less likely to consult risk-adjusted models such as the NSQIP (2.9 vs 3.7; P < .001).

Both general surgery and IM residents similarly overestimated the risk for postsurgical complications. In 91% of clinical estimates, both groups overestimated every type of risk by 26% to 33% compared with NSQIP estimates.

Results also showed a wide range of estimates, suggesting lack of consensus and the use of anecdotal evidence when assessing surgical risk.

The authors mention several limitations of the study. The accuracy of the NSQIP calculator is under debate and may have the potential for underestimating risk. However, the authors explain that the NSQIP remains the most comprehensive, nationally representative tool for assessing surgical outcomes available.

"The estimates that it provides can offer an important starting point for discussion when surgically complex patients and their families are faced with the decision to undergo an operation. Additional studies are needed to evaluate general risk estimation practices across the United States and to validate the accuracy of currently available risk-adjusted models," they conclude.

In an invited commentary, Rajesh Aggarwal, MBBS, PhD, FRCS, from Thomas Jefferson University, Philadelphia, Pennsylvania, also emphasizes that surgical risk assessment plays an important role in shared decision-making discussions with patients.

To improve surgical risk assessment, he stresses physician education about the use of risk-adjusted models. He also emphasizes improving the quality of care for surgically complex patients by documenting risk and care pathways before surgery, along with discussion of outcomes after surgery. Finally, he recommends health system strategies, such as timely support for clinicians, with multidisciplinary input from anesthesia, critical care, palliative care, social care, and others.

"The best management should take into account big data from risk calculators but should not be shackled by it. It is with this comprehensive and collaborative approach that we can be better equipped to serve patients and their families in their times of greatest need," he concludes.

The authors have disclosed no relevant financial relationships. Dr Aggarwal reports consulting for Applied Medical.

JAMA Surg. Published online October 11, 2017. Article full text, Commentary extract

For more news, join us on Facebook and Twitter

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.

processing....