Cookie Settings

We use cookies to optimize our website. These include cookies that are necessary for the operation of the site, as well as those that are only used for anonymous statistic. You can decide for yourself which categories you want to allow. Further information can be found in our data privacy protection .


These cookies are necessary to run the core functionalities of this website and cannot be disabled.

Name Webedition CMS
Purpose This cookie is required by the CMS (Content Management System) Webedition for the system to function correctly. Typically, this cookie is deleted when the browser is closed.
Name econda
Purpose Session cookie emos_jcsid for the web analysis software econda. This runs in the “anonymized measurement” mode. There is no personal reference. As soon as the user leaves the site, tracking is ended and all data in the browser are automatically deleted.

These cookies help us understand how visitors interact with our website by collecting and analyzing information anonymously. Depending on the tool, one or more cookies are set by the provider.

Name econda
Purpose Statistics
External media

Content from external media platforms is blocked by default. If cookies from external media are accepted, access to this content no longer requires manual consent.

Name YouTube
Purpose Show YouTube content
Name Twitter
Purpose activate Twitter Feeds

Research at the Division of Intelligent Medical Systems

The mission of our division is to improve the quality of interventional healthcare in a data-driven manner using Artificial Intelligence (AI) as a central concept. Our research can be categorized in the three main pillars: Intelligent Medical Imaging Systems, Methods for Intelligent Systems, and Validation of Intelligent Systems with a strong focus on Applications in Interventional Healthcare and Open Science.

→ Team philosophy

Intelligent Medical Imaging Systems

A core component of all interventional data science lies in the real-time perception of the environment. Current clinical decision-making is largely based on human perception relying mainly on visual and tactile feedback (palpation). This holds particularly true for the field of surgery. Most currently available medical imaging modalities commonly rely on the use of ionizing radiation, suffer from poor resolution or contrast and/or lack the capacity to operate in real time, thus not fulfilling the needs of an interventional environment. We challenge the current state of the art by proposing novel interventional imaging concepts based on biophotonics techniques. In this context we pioneered machine learning as an approach to solving the inverse problem of reconstructing clinically relevant tissue properties from optical or photoacoustic spectral measurements obtained with multispectral optical or photoacoustic measurements.

Our novel interventional imaging concepts serve as the basis for a wide range of tasks including (a) tumor detection and localization, (b) perfusion monitoring, and (c) full semantic scene segmentation for context-aware assistance.

Representative publications - Intelligent Medical Imaging Systems

Ayala, L., Adler, T. et al. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. Science Advances 2023.

Gröhl, J. et al. Learned spectral decoloring enables photoacoustic oximetry. Scientific Reports 2021.

Seidlitz, S., Sellner, J. et al. Robust deep learning-based semantic organ segmentation in hyperspectral images. Medical Image Analysis 2022.

Methods for Intelligent Systems

To provide assistance to caregivers, the imaging data must be interpreted in the context of the relevant, available knowledge and according to key clinical questions. In this regard, the surgical domain faces several (often) unique challenges pertaining to data science methodology. Specifically, surgical data science often suffers from extremely high data variability and heterogeneity and a lack of representative, annotated data. To overcome this hurdle, we have pioneered different concepts to address data sparsity in the surgical domain, with a specific focus on synthetic data. As building trust in AI is key for bringing AI-based solutions into clinical practice, a particular second focus of our research has further been dedicated to explainable, uncertainty-aware machine learning, with a novel class of invertible neural networks serving as the core underlying technique.

Reliable machine learning in cancer care. Uncertainty-aware machine learning is achieved with a novel class of invertible neural networks serving as core underlying technique, here for representing ambiguity in registration problems.

Representative publications - Methods for Intelligent Systems

Eisenmann, M. et al. Why Is the Winner the Best? Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023. 

Godau, P., Kalinowski, al. Deployment of Image Analysis Algorithms Under Prevalence Shifts. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. 

Adler, T. J. et al. Out of Distribution Detection for Intra-operative Functional Imaging. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures (MICCAI UNSURE) 2019.

Ardizzone, L. et al. Analyzing Inverse Problems with Invertible Neural Networks. International Conference on Learning Representations (ICLR) 2019.

Maier-Hein, L. et al. Surgical data science for next-generation interventions. Nature Biomedical Engineering 2017.

Validation of Intelligent Systems

Increasing evidence points to poor validation being one of the major reasons for the failure of AI-based solutions in clinical practice. Our division is working on new methodology and corresponding open source tools for the reliable and robust validation of algorithms.

Problem-aware metric recommendations. Popular performance metrics often fail to capture clinical interest and adversely impact validation. To overcome this issue, we formed the international ‘Metrics Reloaded’ consortium comprising more than 70 experts worldwide under DKFZ and Helmholtz Imaging lead, with the goal of addressing these pitfalls by enabling the selection of metrics that match the domain interest in a problem-aware approach (Maier-Hein et al. 2024).

Representative publications - Validation of Intelligent Systems

Maier-Hein, L. et al. Metrics reloaded: Recommendations for image analysis validation. Nature Methods 2024. 

Reinke, A. et al. Understanding metric-related pitfalls in image analysis validation. Nature Methods 2024. 

Wiesenfarth, M. et al. Methods and open-source toolkit for analyzing and visualizing challenge results. Scientific Reports 2021.

Reinke, A. How to Exploit Weaknesses in Biomedical Challenge Design and Organization. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2018.

Maier-Hein, L. et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications 2018.

Applications in Interventional Healthcare

The ultimate and most challenging goal regarding surgical data is their usage to provide real-time, context-aware assistance to the physician in and beyond the operating room. Clinical applications include:

  • Surgical action recognition (1st prize MICCAI CholecTriplet Challenge 2022)
  • Instrument detection (1st prize MICCAI PitVis Workflow Recognition in Endoscopic Pituitary Surgery Challenge2023)
  • Polyp detection (1st prize IEEE EndoCV Challenge 2022)
  • Tissue classification (1st prize MICCAI GIANA Challenge 2021)
  • Perfusion monitoring in laparoscopic kidney surgery (Ayala et al. 2023)
  • AI-assisted surgical training (Müller et al. 2022)
  • Ultrasound-navigated percutaneous needle insertions (Franz et al. 2017)

Examples for applications in interventional healthcare. (a): First real-time in-human perfusion monitoring in laparoscopic cancer therapy with spectral imaging. Novel machine learning methodology enables the real-time detection of tissue ischemia based on spectral imaging data (Ayala et al. 2023). (b): AI-assisted surgical training. Robust tracking of gestures enables intuitive teaching of surgical trainees. In collaboration with University Hospital Heidelberg (Müller et al. 2022).

Representative publications - Applications in Interventional Healthcare

Ayala, L. et al. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. Science Advances 2023.

Franz, A. M. et al. First clinical use of the EchoTrack guidance approach for radiofrequency ablation of thyroid gland nodules. International Journal of Computer Assisted Radiology and Surgery 2017.

Maier-Hein, L. et al. Surgical data science – from concepts toward clinical translation. Medical Image Analysis 2022.

Müller, L.-R. et al. Robust hand tracking for surgical telestration. International Journal of Computer Assisted Radiology and Surgery 2022.

Open Science

Our division is strongly committed to the principle of open science.

to top
powered by webEdition CMS