Cookie Settings

We use cookies to optimize our website. These include cookies that are necessary for the operation of the site, as well as those that are only used for anonymous statistic. You can decide for yourself which categories you want to allow. Further information can be found in our data privacy protection .


These cookies are necessary to run the core functionalities of this website and cannot be disabled.

Name Webedition CMS
Purpose This cookie is required by the CMS (Content Management System) Webedition for the system to function correctly. Typically, this cookie is deleted when the browser is closed.
Name econda
Purpose Session cookie emos_jcsid for the web analysis software econda. This runs in the “anonymized measurement” mode. There is no personal reference. As soon as the user leaves the site, tracking is ended and all data in the browser are automatically deleted.

These cookies help us understand how visitors interact with our website by collecting and analyzing information anonymously. Depending on the tool, one or more cookies are set by the provider.

Name econda
Purpose Statistics
External media

Content from external media platforms is blocked by default. If cookies from external media are accepted, access to this content no longer requires manual consent.

Name YouTube
Purpose Show YouTube content
Name Twitter
Purpose activate Twitter Feeds

Research at the Division of Intelligent Medical Systems

Medical interventions are a central pillar of modern oncology. At the same time, death after medical intervention is one of the leading causes of mortality and morbidity; in fact, death within 30 days after surgery was recently identified as the third-biggest contributor to death worldwide [Nepogodiev et al, 2019]. Oncological surgeries are among those with particularly high complication and mortality rates [Friese & Aiken, 2008]. An increasing body of research further suggests that a large proportion of surgery-related these deaths adverse outcomes can be avoided. The driving hypothesis of our research is that:

Data science has the potential to revolutionize interventional cancer care by systematically elevating its safety, efficiency and quality.

The mission of our division is thus to improve the quality of interventional healthcare in a data-driven manner using Artificial Intelligence (AI) as a central concept. Our research can be categorized in the three main pillars: Intelligent Medical Imaging Systems, Methods for Intelligent Systems, and Validation of Intelligent Systems with a strong focus on Applications in Interventional Healthcare and Open Science.

→ Team philosophy

Intelligent Medical Imaging Systems

A core component of all interventional data science lies in the real-time perception of the environment (Fig. 1 Task 1). Current clinical decision-making is largely based on human perception relying mainly on visual and tactile feedback with a focus on visual and tactile perception (palpation). This holds particularly true for the field of surgery. Most currently available medical imaging modalities commonly rely on the use of ionizing radiation, suffer from poor resolution or contrast and/or lack the capacity to operate in real time, thus not fulfilling the needs of an interventional environment. We challenge the current state of the art by proposing novel interventional imaging concepts based on biophotonics techniques. In this context we pioneered machine learning as an approach to solving the inverse problem of reconstructing clinically relevant tissue properties from optical or photoacoustic spectral measurements obtained with multispectral optical or photoacoustic measurements.

Our novel interventional imaging concepts serve as the basis for a wide range of tasks including (a) tumor detection and localization, (b) perfusion monitoring, and (c) full semantic scene segmentation for context-aware assistance.

Representative publications

Ayala, L. et al. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. Science Advances 2023.

Gröhl, J. et al. Learned spectral decoloring enables photoacoustic oximetry. Scientific Reports 2021.

Seidlitz, S. et al. Robust deep learning-based semantic organ segmentation in hyperspectral images. Medical Image Analysis 2022.

Methods for Intelligent Systems

To provide context-aware assistance to caregivers, the perceived data must reliably be interpreted in the context of the relevant, available knowledge and according to key clinical questions (see Fig. 1 Task 2) . In this regard, the surgical domain faces several (often) unique challenges pertaining to data science methodology. Specifically, surgical data science often suffers from extremely high data variability and heterogeneity and a lack of representative, annotated data. To overcome this hurdle, we have pioneered different concepts to address data sparsity in the surgical domain. As building trust in AI is key for bringing AI-based solutions into clinical practice, a particular focus of our research has further been dedicated to explainable, uncertainty-aware machine learning, with a novel class of invertible neural networks serving as the core underlying technique.

Reliable machine learning in cancer care. Uncertainty-aware machine learning is achieved with a novel class of invertible neural networks serving as core underlying technique, here for representing ambiguity in registration problems.

Representative publications

Adler, T. J. et al. Out of Distribution Detection for Intra-operative Functional Imaging. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures (MICCAI UNSURE) 2019.

Ardizzone, L. et al. Analyzing Inverse Problems with Invertible Neural Networks. International Conference on Learning Representations (ICLR) 2019.

Godau, P. et al. Task Fingerprinting for Meta Learning in Biomedical Image Analysis. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021.

Maier-Hein, L. et al. Surgical data science for next-generation interventions. Nature Biomedical Engineering 2017.

Validation of Intelligent Systems

Increasing evidence points to poor validation being one of the major reasons for the failure of AI-based solutions in clinical practice. Our division is working on new methodology and corresponding open source tools for the reliable and robust validation of algorithms.

Problem-aware metric recommendations. Popular performance metrics often fail to capture clinical interest and adversely impact validation. To overcome this issue, we formed the international ‘Metrics Reloaded’ consortium comprising more than 70 experts worldwide under DKFZ and Helmholtz Imaging lead, with the goal of addressing these pitfalls by enabling the selection of metrics that match the domain interest in a problem-aware approach (Maier-Hein et al. 2022).

Representative publications

Maier-Hein, L. et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications 2018.

Maier-Hein, L. et al. Metrics reloaded: Recommendations for image analysis validation. preprint arXiv 2022.

Reinke, A. How to Exploit Weaknesses in Biomedical Challenge Design and Organization. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2018.

Reinke, A., et al. Common Limitations of Image Processing Metrics: A Picture Story, preprint arXiv 2022.

Wiesenfarth, M. et al. Methods and open-source toolkit for analyzing and visualizing challenge results. Scientific Reports 2021.

Applications in Interventional Healthcare

The ultimate and most challenging goal regarding surgical data is their usage to provide real-time, context-aware assistance to the physician in and beyond the operating room. Clinical applications include:

  • Context-aware assistance by surgical action detection (winner of international competition CholecTriplet surgical action classification)
  • Colon cancer detection (winner of international competition EndoCV 2022)
  • Colon cancer classification (winner of international competition GIANA 2021)
  • Perfusion monitoring in laparoscopic kidney surgery
  • AI-assisted surgical training
  • Ultrasound-navigated percutaneous needle insertions

Examples for applications in interventional healthcare. (a): First real-time in-human perfusion monitoring in laparoscopic cancer therapy with spectral imaging. Novel machine learning methodology enables the real-time detection of tissue ischemia based on spectral imaging data (Ayala et al. 2023). (b): AI-assisted surgical training. Robust tracking of gestures enables intuitive teaching of surgical trainees. In collaboration with University Hospital Heidelberg (Müller et al. 2022).

Representative publications

Ayala, L. et al. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. Science Advances 2023.

Franz, A. M. et al. First clinical use of the EchoTrack guidance approach for radiofrequency ablation of thyroid gland nodules. International Journal of Computer Assisted Radiology and Surgery 2017.

Maier-Hein, L. et al. Surgical data science – from concepts toward clinical translation. Medical Image Analysis 2022.

Müller, L.-R. et al. Robust hand tracking for surgical telestration. International Journal of Computer Assisted Radiology and Surgery 2022.

Open Science

Our division is strongly committed to the principle of open science.

to top
powered by webEdition CMS