At IMAIOS, we are particularly passionate about innovation and experimentation. So, when we heard about the launch of the Apple Vision Pro, we knew we had to test it and explore its possibilities in the medical field.
What is Apple Vision Pro?
It is a mixed reality headset that Apple describes as the first spatial computer. Its most revolutionary feature is undoubtedly its eye-tracking control, which is truly game-changing. Beyond that, the image quality is simply stunning, and the user experience is unmatched compared to previous-generation headsets, thanks to the micro-OLED display. However, despite weighing only 600 grams, prolonged use can make the headset feel heavy on the face.
- The micro-OLED screen eliminates the screen-door effect, a genuine breakthrough.
- The weight distribution could be improved as it eventually becomes burdensome.
A Revolution in Medicine?
The Vision Pro medical device could be the most advanced on the market. Experts in the field will find it a valuable tool for daily practice, training, and interventional radiology.
In March 2024, a historic milestone was reached in the UK when, for the first time, a team of surgeons used Apple’s Vision Pro headset during a surgical procedure. This marked a significant advancement in integrating mixed reality into the medical field. The surgeon was assisted by a theatre nurse at London’s private Cromwell Hospital, who used the Vision Pro to prepare for the operation, monitor the procedure, and select the appropriate tools. This technology allowed the surgical team to navigate the operation in augmented reality, enhancing visibility and precision.
Enhancing Medical Education
The highly immersive quality of virtual reality, augmented reality and extended reality technologies means that students can actively participate in realistic simulations, life-like medical scenarios and interactive anatomical models.
The implementation of Apple Vision Pro has the potential to transform medical education and training by offering a more practical and immersive learning experience.
Virtual reality can provide cost-effective, standardised, and reproducible clinical training, particularly through:
- Interactive simulations: Hands-on training is often difficult to provide, but VR can overcome this challenge through simulation-based learning. Reproducing real-world scenarios and offering performance feedback has been shown to be more effective than traditional clinical training, delivering long-lasting results1.
- Real-time access to information: Trainees could access relevant clinical data, practice guidelines, and case studies while performing procedures or interacting with patients, enabling informed, up-to-date decision-making.
- Remote collaboration: The headset could facilitate remote collaboration between students, lecturers, and healthcare professionals during training sessions. This would be particularly beneficial for distance learning and consulting on complex cases across different locations.
What Does This Mean for Radiology?
These technologies provide radiologists with cutting-edge tools to analyse complex three-dimensional (3D) medical images, improving their understanding of human anatomy and facilitating radiological diagnosis.
More precise diagnostics leading to better patient care
Radiological visualisation is being revolutionised through the use of advanced computer vision techniques for imaging and 3D reconstruction. Future developments include real-time 3D reconstruction algorithms, volumetric rendering techniques, and virtual reality visualisation tools, offering an immersive and interactive exploration of medical imaging data. Coupled with enhanced image resolution provided by modern equipment, diagnostics will become more accurate and efficient.
The visualisation of three-dimensional medical imaging in an immersive space creates exciting opportunities to improve patient care.
Radiologists can view medical images with unprecedented precision and clarity, enabling more accurate diagnoses and better-informed treatment plans. By leveraging state-of-the-art visualisation tools, radiologists can enhance patient outcomes and ensure higher-quality care.
A World First: CT-Guided Percutaneous Vertebroplasty with Apple Vision Pro
We tested the AVP in real-world conditions. The percutaneous vertebroplasty was performed by Dr Denis Hoa under the supervision of Dr Antoine Micheau.
Key advantages we observed include:
- Gesture recognition and eye tracking facilitating the visualisation of pre-procedural imaging exams, aiding intervention preparation.
- The ability to position the display in the most convenient location for the operator, such as overlaying images directly on the patient’s body.
- Enhanced visualisation through real-time image overlay near the operator.
- Reduced cognitive load by minimising the need to shift attention between different areas.
- Lower risk of contamination, as zooming, moving, and rotating screens can be done without physical contact.
In short, the Apple Vision Pro has the potential to change the way healthcare professionals are trained by providing advanced educational resources, practical simulations and real-time collaboration tools. The result could be more effective medical training and professionals who are better prepared to face the challenges of clinical practice.
The Future: IMAIOS App Development
We are currently in the research phase, exploring potential tools while assessing feasibility. One initial integration idea is native 3D reconstruction within the IMAIOS DICOM Viewer for iOS.
We also strongly believe that remote training could benefit from this technology. Live training sessions detailing surgical or interventional radiology procedures could become a reality, making this another key area for development.
1 Waisberg E, Ong J, Masalkhi M, Zaman N, Sarker P, Lee AG, Tavakkoli A. Apple Vision Pro and the advancement of medical education with extended reality. Can Med Educ J. 2024 Feb 29;15(1):89-90. doi: 10.36834/cmej.77634.
