Adaptive Anatomy is a new approach to image display in the operating room using mixed reality

Project website: http://adaptiveimaging.org/adaptive-anatomy

With the improved capability of sensor and display technologies, medical screens have become mobile, touch-sensitive, flexible and, most recently, transparent. The introduction of transparent displays has the potential to fundamentally change the concepts of intra-operative imaging and interaction in the operating room. While standard computer monitors generally assign viewers a static and passive position, adaptive imaging combines imaging technology and design principles from image fusion and augmented reality to display patient-related information directly into the surgeon’s field of view. Seen through a transparent head-mounted display, medical data overlays and annotates the surgeon’s vision with anatomical information by coinciding with the scale and position of the patient in real time. As it eliminates the offset between image and patient, surgeons do not need to continuously switch back and forth between screen and surgical site in order to view information which results in improved hand-eye coordination and spatial awareness.

While a broad range of studies have proven the technical feasibility and limitations of mixed reality applications and presented clinical usecases, the appropriate workflows and their impact on the decision-making processes in the operating room have yet to be validated in order to fulfil the constantly increasing market expectations. To do so Adaptive Anatomy determines a) the intra-operative situations and workflows that benefit from mixed reality visualisations, it identifies b) the design strategies necessary to efficiently integrate 4D images into the surgical workflow regarding both patient-specific as well as task specific data and it defines c) the expertise that is required for surgeons to cope with this new architecture of image display in the operating room. By looking beyond the technical means of 4D visualisations (tracking, registration, image quality etc.) adaptive anatomy provides an approach for surgical staff in order to better understand the problems that mixed reality is actually able to solve.

The project is conducted as part of the Cluster of Excellence Image Knowledge Gestaltung, and funded by the German Research Foundation as well as by the Future Lab at the Department of Experimental Surgery at Charité University Hospital Berlin.