'Extended Reality' Comes to the Cath Lab

Debra L Beck

July 03, 2018

Rapid innovation in mobile computing is maturing extended-reality technologies that can be adapted to cardiovascular medicine, promising real advances in medical education and clinical care.

In a paper recently published in JACC: Basic to Translational Science, pediatric electrophysiologist Jennifer N.A. Silva, MD, Washington University, St. Louis, Missouri, and colleagues review a number of applications for extended reality in cardiac care that may ultimately help physicians learn more quickly, interpret images more accurately, and accomplish interventions in less time.

"Pokémon Go is a really good example of augmented reality…but what was really impressive to me was how it captured the imagination. The way in which it allowed people the ability to see something new and different was almost magical," Silva said in an interview with theheart.org | Medscape Cardiology.

"The corollary is when we first started looking at electroanatomical maps of the heart, it was very clear to me that I was seeing something I'd never seen before. It was a new way to see and interact with the heart, and it helped me realize how little we understand these electroanatomical relationships that we're trying to impact," she said. "So beyond improving our accuracy and patient outcomes, the scientist in me is incredibly optimistic that we might be able to use this tool to determine some basic mechanisms of arrhythmias."

Reality Is Relative

The extended-reality spectrum ranges from fully immersive experiences, such as virtual reality (VR), to augmented reality (AR), which allows an unobstructed experience with virtual objects within a natural background.

"Virtual reality has limited use in medicine because the technology is fully immersive in that it transports you out of your location and into a fully virtual environment. This is great for video games but less great for physicians who are trying to interact with their patients," said Silva.

"The technologic barrier that was crossed with augmented reality is that you can remain in your natural environment — still in a room talking to your patient, your anesthesiologist, or your nurses — but you can import and visualize digital data at the same time."

A first real-time use of AR came in a 2016 first-in-human case report, in which a cardiologist used Google Glass to project three-dimensional (3D) CT  angiography (CTA) reconstructions of the right coronary artery directly in the physician's field of vision, allowing him to visualize the artery while at the same time verifying the direction of the guidewire as he advanced it toward the occluded vessel segment. Recanalization was successful and two stents were implanted.

"For those of us who are tech junkies, watching this from afar, this was the most exciting thing ever," said Silva.

Maksymilian Opolski, MD, PhD, from the Cardinal Wyszynski National Institute of Cardiology in Warsaw, Poland, was the operator behind that Google Glass, and his research has advanced. In collaboration with cardiologist Paul Knaapen, MD, from the VU University Medical Center in Amsterdam, the Netherlands, Opolski is recruiting for the AR-PCI trial, a noninferiority study comparing outcomes after augmented-reality CTA-guided percutaneous coronary intervention (PCI) vs standard angiography-guided PCI.

The trial is using Epson Moverio BT-350 Glass, which offers two fully transparent lenses and higher resolution compared with  Google Glass.

"Thanks to the simple and intuitive nature of the app, cardiologists can navigate through the images of the patient's heart using voice commands, not just making the software easy to use, but also ideal for maintaining the sterile conditions of the operating theater," Opolski said in an email exchange with theheart.org | Medscape Cardiology.

Explained Silva, "When mixed reality came on the scene, it allowed us to not just to stay in our natural environment and import digital images, but it lets the person wearing the headset control those images."

At the recent Heart Rhythm Society (HRS) 2018 Scientific Sessions, Silva and colleagues presented observational data on 10 patients (mean age, 13 years) undergoing ablation of supraventricular tachycardias using their mixed-reality system. ELVIS, or the Enhanced Electrophysiology and Interaction System, displays intraprocedural 3D holograms of patient-specific cardiac geometry and electroanatomic maps with real-time catheter locations, all while allowing direct control of the display without breaking sterility.

The study confirmed feasibility and the system's ability to meet minimum performance criteria, as judged by a second electrophysiology  team that observed the procedure from the control room but offered no feedback or guidance to the performing physician. Latency within the system was less than 131 milliseconds.

Silva and her husband, biomedical engineer Jonathan Silva, PhD, are cofounders of SentiAR, the company spun out of Washington University's School of Medicine and School of Engineering in 2017 to commercialize their 3D-visualization technology. The company was awarded a $2.2 million National Institutes of Health grant earlier this year to advance augmented-reality technology in cardiac surgery and interventional procedures and is hoping to get US Food and Drug Administration approval for their system by late 2018.

"We're not going to know the real power of this until we've put it in the hands of users from all different backgrounds representing all different types of electrophysiology," said Silva. "For some it might increase their efficiency, for others their accuracy, but either way, we'll have to be able to improve patient outcome and patient experiences. But I think we'll find multiple ways to that end goal."

Other groups are working on similar technology, including an Israeli start-up company, RealView Imaging, which has teamed with the pediatric cardiology group at Schneider's Children's Medical Center (Petach Tikva, Israel) to develop a computer-generated holography system. The RealView system displays holographic images of the heart using intraprocedural data from 3D rotational angiography and live 3D transesophageal echocardiography. The holograms are interactive and can be rotated, marked, measured, and sliced.

The technology has demonstrated feasibility and real-time anatomical accuracy, but the company has yet to release data showing improvements in procedure-related parameters, such as fluoroscopy time, procedure duration, and outcomes.

Defining Reality in Medical Education

"The tools that we train with are the tools we want when we go into practice, so I think if we start teaching medical students and residents with these technologies, they're going to become attendings who are used to these technologies and the answers they provide," said Silva.

While mixed reality will likely find its "match" intraprocedurally, VR is a natural fit for medical education, she thinks. Extended reality offers a wide range of possibilities in the educational realm, including applications that leverage the immersion that VR enables in order to simulate an operating environment.

More simply, the Stanford Virtual Heart Project uses an immersive VR headset to allow medical students to visualize normal and abnormal anatomies and understand how congenital anomalies affect physiology. Trainees can inspect and manipulate models, enhancing and speeding their learning of the complex abnormal physiologies and their hemodynamic sequelae.

In another example of how these technologies promise to revolutionize medical education, researchers from Case Western Reserve University in Cleveland, Ohio, are developing a holographic dissection application. Wearers of the HoloLens (Microsoft) will see a representation of a human body in 3D and can navigate through layers of skin, muscle, blood vessels, and organs to the skeleton below. They can see the heart pumping blood around the body and learn how and where the veins and arteries feed into circulation. Working alone or in teams, HoloAnatomy trainees can dissect and reconstruct the 3D human body models over and over again.

"The old way, once you do a dissection, you're done," said Silva. "But this allows you to take it apart and put it back together again — again and again. And there's something about building it as well as taking it apart that will really help students understand how things fit and work together. It's what engineers do!"

The Road Ahead

Silva points out that many challenges and limitations remain.

"These technologies are still constrained due to cost, size, weight and power to achieve the highest visual quality, mobility, processing speed and interactivity," she said. "Every design decision to mitigate these challenges affects applicability for use in each procedural environment."

Said Opolski, "I have no doubt that AR will shake up clinical practice in the near future."

In interventional cardiology and radiology, Opolski thinks the biggest challenge relates to obtaining wider cooperation between individual users of AR devices and the vendors of angiography systems, so that the technology can be readily implemented in the cath lab.

"This will require the readiness on the side of angiography providers to customize their products with open-source operating systems so that one could easily connect and share imaging data with AR devices," said Opolski.

Jennifer Silva and Jonathan Silva are cofounders of, consultants to, and serve on the board of directors for SentiAR. The second author, Michael Southworth, MS, is a SentiAR shareholder . Opolski has disclosed no relevant financial relationships.

JACC Basic Transl Sci. 2018;3:420-430. Full text

For more from theheart.org | Medscape Cardiology, follow us on Twitter and Facebook


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.