Research at the Division of Intelligent Medical Systems
Medical interventions are a central pillar of modern oncology. At the same time, death after medical intervention is one of the leading causes of mortality and morbidity; in fact, death within 30 days after surgery was recently identified as the third-biggest contributor to death worldwide [Nepogodiev et al, 2019]. Oncological surgeries are among those with particularly high complication and mortality rates [Friese & Aiken, 2008]. An increasing body of research further suggests that a large proportion of surgery-related these deaths adverse outcomes can be avoided. The driving hypothesis of our research is that:
Data science has the potential to revolutionize interventional cancer care by systematically elevating its safety, efficiency and quality.
The mission of our division is thus to improve the quality of interventional healthcare in a data-driven manner using Artificial Intelligence (AI) as a central concept. Our research can be categorized in the three main pillars: Intelligent Medical Imaging Systems, Methods for Intelligent Systems, and Validation of Intelligent Systems with a strong focus on Applications in Interventional Healthcare and Open Science.
Intelligent Medical Imaging Systems
A core component of all interventional data science lies in the real-time perception of the environment (Fig. 1 Task 1). Current clinical decision-making is largely based on human perception relying mainly on visual and tactile feedback with a focus on visual and tactile perception (palpation). This holds particularly true for the field of surgery. Most currently available medical imaging modalities commonly rely on the use of ionizing radiation, suffer from poor resolution or contrast and/or lack the capacity to operate in real time, thus not fulfilling the needs of an interventional environment. We challenge the current state of the art by proposing novel interventional imaging concepts based on biophotonics techniques. In this context we pioneered machine learning as an approach to solving the inverse problem of reconstructing clinically relevant tissue properties from optical or photoacoustic spectral measurements obtained with multispectral optical or photoacoustic measurements.
Our novel interventional imaging concepts serve as the basis for a wide range of tasks including (a) tumor detection and localization, (b) perfusion monitoring, and (c) full semantic scene segmentation for context-aware assistance.
© dkfz.de
Representative publications
Ayala, L. et al. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. Science Advances 2023. https://doi.org/10.1126/sciadv.add6778
Gröhl, J. et al. Learned spectral decoloring enables photoacoustic oximetry. Scientific Reports 2021. https://doi.org/10.1038/s41598-021-83405-8
Seidlitz, S. et al. Robust deep learning-based semantic organ segmentation in hyperspectral images. Medical Image Analysis 2022. https://doi.org/10.1016/j.media.2022.102488
Methods for Intelligent Systems
To provide context-aware assistance to caregivers, the perceived data must reliably be interpreted in the context of the relevant, available knowledge and according to key clinical questions (see Fig. 1 Task 2) . In this regard, the surgical domain faces several (often) unique challenges pertaining to data science methodology. Specifically, surgical data science often suffers from extremely high data variability and heterogeneity and a lack of representative, annotated data. To overcome this hurdle, we have pioneered different concepts to address data sparsity in the surgical domain. As building trust in AI is key for bringing AI-based solutions into clinical practice, a particular focus of our research has further been dedicated to explainable, uncertainty-aware machine learning, with a novel class of invertible neural networks serving as the core underlying technique.
Reliable machine learning in cancer care. Uncertainty-aware machine learning is achieved with a novel class of invertible neural networks serving as core underlying technique, here for representing ambiguity in registration problems.
© dkfz.de
Further information
Representative publications
Adler, T. J. et al. Out of Distribution Detection for Intra-operative Functional Imaging. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures (MICCAI UNSURE) 2019. https://doi.org/10.1007/978-3-030-32689-0_8
Ardizzone, L. et al. Analyzing Inverse Problems with Invertible Neural Networks. International Conference on Learning Representations (ICLR) 2019. http://arxiv.org/abs/1808.04730
Godau, P. et al. Task Fingerprinting for Meta Learning in Biomedical Image Analysis. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021. https://doi.org/10.1007/978-3-030-87202-1_42
Maier-Hein, L. et al. Surgical data science for next-generation interventions. Nature Biomedical Engineering 2017. https://doi.org/10.1038/s41551-017-0132-7
Validation of Intelligent Systems
Increasing evidence points to poor validation being one of the major reasons for the failure of AI-based solutions in clinical practice. Our division is working on new methodology and corresponding open source tools for the reliable and robust validation of algorithms.
Problem-aware metric recommendations. Popular performance metrics often fail to capture clinical interest and adversely impact validation. To overcome this issue, we formed the international ‘Metrics Reloaded’ consortium comprising more than 70 experts worldwide under DKFZ and Helmholtz Imaging lead, with the goal of addressing these pitfalls by enabling the selection of metrics that match the domain interest in a problem-aware approach (Maier-Hein et al. 2022).
Representative publications
Maier-Hein, L. et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications 2018. https://doi.org/10.1038/s41467-018-07619-7
Maier-Hein, L. et al. Metrics reloaded: Recommendations for image analysis validation. preprint arXiv 2022. http://arxiv.org/abs/2206.01653
Reinke, A. How to Exploit Weaknesses in Biomedical Challenge Design and Organization. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2018. https://doi.org/10.1007/978-3-030-00937-3_45
Reinke, A., et al. Common Limitations of Image Processing Metrics: A Picture Story, preprint arXiv 2022. http://arxiv.org/abs/2104.05642
Wiesenfarth, M. et al. Methods and open-source toolkit for analyzing and visualizing challenge results. Scientific Reports 2021. https://doi.org/10.1038/s41598-021-82017-6
Applications in Interventional Healthcare
The ultimate and most challenging goal regarding surgical data is their usage to provide real-time, context-aware assistance to the physician in and beyond the operating room. Clinical applications include:
- Context-aware assistance by surgical action detection (winner of international competition CholecTriplet surgical action classification)
- Colon cancer detection (winner of international competition EndoCV 2022)
- Colon cancer classification (winner of international competition GIANA 2021)
- Perfusion monitoring in laparoscopic kidney surgery
- AI-assisted surgical training
- Ultrasound-navigated percutaneous needle insertions
Examples for applications in interventional healthcare. (a): First real-time in-human perfusion monitoring in laparoscopic cancer therapy with spectral imaging. Novel machine learning methodology enables the real-time detection of tissue ischemia based on spectral imaging data (Ayala et al. 2023). (b): AI-assisted surgical training. Robust tracking of gestures enables intuitive teaching of surgical trainees. In collaboration with University Hospital Heidelberg (Müller et al. 2022).
Representative publications
Ayala, L. et al. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. Science Advances 2023. https://doi.org/10.1126/sciadv.add6778
Franz, A. M. et al. First clinical use of the EchoTrack guidance approach for radiofrequency ablation of thyroid gland nodules. International Journal of Computer Assisted Radiology and Surgery 2017. https://doi.org/10.1007/s11548-017-1560-2
Maier-Hein, L. et al. Surgical data science – from concepts toward clinical translation. Medical Image Analysis 2022. https://doi.org/10.1016/j.media.2021.102306
Müller, L.-R. et al. Robust hand tracking for surgical telestration. International Journal of Computer Assisted Radiology and Surgery 2022. https://doi.org/10.1007/s11548-022-02637-9
Open Science
Our division is strongly committed to the principle of open science.