Earlier this year, Dr. Marc Tewfik, MUHC director of Rhinology and assistant professor in the department of Otolaryngology at the McGill University Health Centre, broke ground by becoming the first surgeon in North America to use augmented reality (AR) for sinus operations. Dr. Tewfik, who has completed several AR-aided surgeries to date, previously relied on a pointer, static scans and educated guesses to conduct procedures. Thanks to AR technology, he can now navigate 3D maps of his patients’ nasal systems, all in real time while operating.

Augmented reality for surgery has been discussed in academic and industrial research since the 1990s, however it is only recently that physicians and surgeons have started adopting the technology, identifying potential applications for it and immersing themselves into high-resolution, 3D representations of their patients’ anatomies.

In this article we share some of the different ways that AR is transforming surgery and enabling surgeons worldwide to plan, teleguide, simulate, educate and perform complex operations.


Touch Surgery
As surgical trainees, Jean Nehme and Andrew Chow experienced many of the challenges that are facing the global healthcare industry today: lack of access to training, a shrinking global surgical workforce and slow adoption of innovative techniques. With the goal of using technology to improve the standards of surgical training worldwide, the duo founded Touch Surgery in 2013, an interactive mobile surgical simulator application that guides users – medically trained or otherwise – through every step of an operation.

With over 200 digitized surgical procedures and more than 1.5 million users to date, Touch Surgery has recently delved into AR and VR with hopes of bringing the technologies into the operating room. Earlier this year, at CES 2017 the company announced its support for a new type of deeply immersive surgery training that will make use of HoloLens and DAQRI wearable devices.


Proximie is an AR platform that enables surgeons to virtually transport themselves into ORs to guide, train, teach, and support other surgeons and medical experts, all through the screen of a tablet. Founded by Nadine Haram and Talal Ali Ahmad in 2016, the platform is based around the concept of having two mobile devices or tablets, one in each location, with a camera for viewing and a screen/microphone setup for communication. Using AR, the remote ‘assisting’ surgeon can guide the operation and operating surgeon with commentary or screen markings.

Last year, Dr. Ghassan Abu Sitta, Head of the Plastic Surgery Division at AUBMC, used the software to lead operations in the Gaza strip from Beirut and demonstrate in real time the actions needed to be taken. In the US, Dr. Raj Vyas, a surgeon at The University of California Riverside (UCR), also worked in partnership with Proximie to train two senior plastic surgeons based in Peru in treating cleft lips and palates. In addition, the application is being used to give medical students hands-on education and surgical experience.


Royal Philips
Traditionally an open surgery procedure with large incisions, in recent years, spine surgery has been complimented by the use of minimally-invasive techniques. The techniques, which involve manipulating surgical tools through small incisions in the patient’s skin, minimize blood loss and soft tissue damage; however also reduce the visibility of the spine. Therefore, to aid surgeons, Royal Philips, a leader in integrated image-guided therapy solutions, is developing a new AR navigation system for spinal surgeries.

Using high-resolution optical cameras mounted on a x-ray detector to image the surface of the patient, the technology constructs a 3D AR image of the patient’s external and internal anatomy. The resulting 3D view aims to improve procedure planning, operation times, and surgical tool navigation and even has a higher overall accuracy than non-AR assisted surgical navigation according to a study in medical journal Spine.


During surgeries, doctors must look away from their patients in order to check vital signs and other medical data, resulting in longer surgery times and interrupted focus. To tackle this issue, European scientists are using photonics technology to develop a medical visor known as VOSTARS (Video Optical See-Through Augmented Reality Surgical system), a head-mounted display that projects a patient’s anesthetic data, heart rate, body temperature, blood pressure and breathing rates into the surgeon’s field of vision. 

The VOSTARS system works by capturing what the surgeon sees from a head-mounted camera and merging it with the patient’s medical images from CT, MRI or 3DUS scans. The central processer then presents a real-time hybrid image on the visor’s dashboard to the surgeon. The aim behind the visor is to reduce surgery times, the time spent under anesthetic, and the cost involved in any operation.


Scopis Holographic Navigation
Scopis, a surgical navigation and medical AR and mixed reality technology company, announced earlier this year the launch of their newest development: the Holographic Navigation Platform. The platform, which is designed to offer surgeons greater precision, speed and enhanced visibility using Microsoft’s HoloLens headset,  allows for better outcomes for patients undergoing minimally invasive spinal procedures.

Wearing a pair of the HoloLens glasses, surgeons can see a 3D overlay of the pre-planned positioning of screws, allowing them to interactively align their instruments with the holographic visualization and find the precise locations more quickly. Moreover, surgeons can keep their eye on the operative field and use gestures to place virtual monitors in their visual field. Beyond surgeons, Scopis’ technology also benefits patients and medical professionals by reducing the radiation exposure from fluoroscopy devices that are used to determine the optimal position for screw placement during surgery.








Catch up on what you've missed