Precision surgery with AI-supported 3D planning for gastroenterology

Motivation

Different interventional procedures are used for early gastrointestinal neoplasia depending on the type of tumor. Minimally invasive methods such as endoscopic mucosal resection (EMR) or endoscopic submucosal dissection (ESD) are particularly gentle, while radical surgery is necessary if deep tumor invasion is suspected. The choice of procedure is guideline-based, based on the type of growth and gross morphology (Paris classification) as well as additional features (e.g. JNET or pit-pattern classification). However, the subjective assessment of the three-dimensional (3D) tumor structure based on two-dimensional (2D) images from flexible endoscopy often leads to varying diagnoses. Elasticity measurements, for example by endosonographic elastography, help to identify deep invasions (tumor hotspots), but require a subjective combination with the white light endoscopy images by the physician, which can lead to errors. EndoPlan-3D therefore creates an AI-supported 3D reconstruction with integrated tissue elasticity, which is provided via interactive technologies and thus increases the precision of surgical planning.

Goals and Procedure

EndoPlan-3D aims to create a Al-supported, interactive 3D environment that optimizes surgical interventions for early gastrointestinal neoplasia. The central aim of the project is to enable more precise and reliable planning of endoscopic interventions by processing multimodal data from white light endoscopy and endosonography using AI algorithms to create dynamic, individual 3D models. Doctors will be able to reliably identify tumor boundaries, critical hotspots and optimal resection methods using a VR-based interaction environment. The planned technology includes the creation of realistic 3D visualizations from endoscopic video data using innovative AI processes. These models are supplemented by acoustic and tactile feedback for better identification of critical tumor sites and for determining optimal surgical instrument settings. The results are exploited both economically and scientifically: EndoPlan-3D not only aims to reduce incorrect decisions and lower the risk of recurrence, but also to improve surgical quality and patient care in the long term.

Innovations and Perspectives

The developed system uses video data to create a three-dimensional model of the esophagus that shows the exact boundaries of the tumor, thereby facilitating surgical planning. The technology can also be applied to other surgical procedures.

Cooperation Partner

• Erbe Elektromedizin GmbH
• University Hospital Augsburg
• University of Augsburg
• OTH Regensburg (Coordination)

Funding

BMFTR project "EndoPlan-3D: Precision surgery with AI-supported 3D planning for gastroenterology",
Funding reference: 16SV9556

BMFTR project page

Period and Volume