Computational Photoacoustics
Figure 1: Visualization of the photoacoustic effect: a. Tissue is irradiated by laser light and the laser light diffusely propagates through the tissue; b. where absorbed, the deposited energy leads to the expansion of the tissue which in turn c. leads to the emergence of sound waves that can be measured with an ultrasound transducer.
© dkfz.de
Photoacoustic tomography (PAT) is a modality that can provide tomographic images at a depth range of up to several centimeters by exploiting the photoacoustic effect (Fig. 1). In the last two decades, a lot of progress has been made towards the translation of PAT into clinical routine and photoacoustics has proven to be an invaluable tool for basic research due to its ability to image functional information, such as local blood oxygenation. Our research contributions can be subdivided into methodological research as well as medical applications.
Challenge: PAI comprises ill-posed inverse problems
Acoustic inverse problem: Measurements of the photoacoustic times series data need to be reconstructed into a spatial distribution of the initial pressure distribution. The main issues that arise are that usually, the reconstruction algorithm is tasked to cope with incomplete data and with a limited detection bandwidth of the measurement device.
Optical inverse problem: A measurement of the initial pressure distribution, even though proportional to the absorption of chromophores, is also dependent on the light fluence which is in turn highly dependent on the distribution of chromophores and scatterers within the tissue. As a consequence, quantification of chromophore concentrations from the underlying initial pressure distribution is an ill-posed inverse problem.
Goal: Quantitative Image Reconstruction
We presented the first machine learning based method towards the optical inverse problem (Kirchner, Gröhl et al., 2018). Here, a machine learning algorithm uses calculated "context images" to estimate the fluence in each voxel, which – in turn – can be used to correct the optical absorption measured. The patented concept (Maier-Hein et al., 2019) has yielded extremely promising results in silico and thus has high potential for solving one of the primary issues related to PAT.
Complementary to contributions to classical methods to solve the acoustic inverse problem (Kirchner, Sattler et al, 2018), we have also introduced deep learning concepts for PAT image reconstruction (Waibel, Gröhl, Isensee, Kirchner et al. 2018) and have made contributions that take into account uncertainty estimation in both inverse problems (Gröhl, Kirchner, Adler et al., 2018; Gröhl, Kirchner, Maier-Hein, 2018). We also developed techniques for 3D photoacoustic vessel angiography with optical tracking (Kirchner et al., 2016).
Figure 2: General principle how to extract chromophore concentrations and subsequently calculate functional tissue properties from multispectral photoacoustic images.
© dkfz.de
Application: Measuring hemodynamic changes in the brain
Our translational contributions in the context of PAT are related to calculation of haemodynamics in the human carotid artery (Kirchner, Gröhl et al., 2018) and open brain surgery (Kirchner, Gröhl, Holzwarth et al., 2019), also using deep learning techniques (Gröhl et al., 2019).
We used the ability of PAT to recover tissue oxygenation to examine hemodynamic mechanisms while suffering from a stroke. Using PAT to measure such local haemodynamic changes, we were able to visualize waves of spreading depolarization up to a centimeter deep in the brain cortex (Kirchner, Gröhl, Herrera et al., 2019).
Figure 3: Multispectral photoacoustic (PA) imaging of blood oxygenation (sO2). After a 15min baseline scan, spreading depolarization (SD) was induced by potassium chloride (KCl) stimulation in the left brain hemisphere. The sagittal plane was continuously measured for 51min. (a) PA sO2 estimation before stimulation with marked regions of interest (ROI). (b) Time evolution of estimated sO2 in the two ROI. (c) The change in blood oxygenation (ΔsO2) is shown for three example time steps 30 seconds apart (T1, T2 and T3), corresponding to the dashed lines in (b). Figure and caption reprinted from the CC-BY licensed publication [Kir19a].
© dkfz.de
Team
- Dr. Alexander Seitel (Project Lead)
- Melanie Schellenberg (Doctoral Student)
- Kris Dreher (Doctoral Student)
- Niklas Holzwarth (Doctoral Student)
- Jan-Hinrich Nölke (Doctoral Student)
- Vahid Abdi (Master's Student)
- Patricia Vieten (Master's Student)
- Tom Rix (Intern)
- Prof. Dr. Lena Maier-Hein (Principal Investigator)
Alumni
- Dr. Janek Gröhl (Doctoral Student)
- Dominik Waibel (Master's Student)
- Angelika Laha (Master's Student)
- Franz Sattler (Student Assistant)
- Dr. Thomas Kirchner (Doctoral Student)
Key Collaborators
- M.D. Dr. med. Edgar Santos, Universität Heidelberg, University Clinic for Neurosurgery
- PD Dr. med. Sebastian Adeberg, Heidelberg University Hospital at the Department of Radiation Oncology
- Thomas Held, Heidelberg University Hospital at the Department of Radiation Oncology
Publications
Bohndiek, S., Brunker, J., Gröhl, J., Hacker, L., Joseph, J., Vogt, W. C., Armanetti, P., Assi, H., Bamber, J. C., Beard, P. C., & others. (2019). International Photoacoustic Standardisation Consortium (IPASC): Overview (Conference Presentation). Photons Plus Ultrasound: Imaging and Sensing 2019, 10878, 108781N.
Gröhl, J., Schellenberg, M., Dreher, K. K., Holzwarth, N., Tizabi, M. D., Seitel, A., & Maier-Hein, L. (2021). Semantic segmentation of multispectral photoacoustic images using deep learning. Photons Plus Ultrasound: Imaging and Sensing 2021 Conference Abstract.
Gröhl, Janek. (2020). Data-driven quantitative photoacoustic imaging [PhD Thesis]. Universität Heidelberg.
Gröhl, Janek. (2019). International Photoacoustic Standardisation Consortium (IPASC): Recommendations for standardized data exchange in photoacoustic imaging (Conference Presentation). Photons Plus Ultrasound: Imaging and Sensing 2019, 10878, 108781S.
Gröhl, Janek, & Hacker, L. (2020). International Photoacoustic Standardisation Consortium (IPASC): Progress in the data acquisition and management theme (Conference Presentation). Photons Plus Ultrasound: Imaging and Sensing 2020, 11240, 112401F
Gröhl, Janek, Kirchner, T., Adler, T. J., Hacker, L., Holzwarth, N., Hernández-Aguilera, A., Herrera, M. A., Santos, E., Bohndiek, S. E., & Maier-Hein, L. (2021). Learned spectral decoloring enables photoacoustic oximetry. Scientific Reports, 11(1), 6565. https://doi.org/10.1038/s41598-021-83405-8
Gröhl, Janek, Kirchner, T., Adler, T., & Maier-Hein, L. (2018). Confidence Estimation for Machine Learning-Based Quantitative Photoacoustics. Journal of Imaging, 4(12), 147. https://doi.org/10.3390/jimaging4120147
Gröhl, Janek, Kirchner, T., Adler, T., & Maier-Hein, L. (2019). Estimation of blood oxygenation with learned spectral decoloring for quantitative photoacoustic imaging (LSD-qPAI). ArXiv Preprint ArXiv:1902.05839.
Gröhl, Janek, Kirchner, T., Adler, T., & Maier-Hein, L. (2020). Deep learning-based oxygenation estimation for multispectral photoacoustic imaging (Conference Presentation). Photons Plus Ultrasound: Imaging and Sensing 2020, 11240, 112402P.
Gröhl, Janek, Kirchner, T., & Maier-Hein, L. (2017). Abstract: Quantitative Photoakustische Tomografie durch lokale Kontextkodierung. In Bildverarbeitung für die Medizin 2017 (pp. 153–153).
Gröhl, Janek, Kirchner, T., & Maier-Hein, L. (2018). Confidence estimation for quantitative photoacoustic imaging. Photons Plus Ultrasound: Imaging and Sensing 2018, 10494, 104941C.
Gröhl, Janek, Schellenberg, M., Dreher, K., & Maier-Hein, L. (2021). Deep learning for biomedical photoacoustic imaging: A review. Photoacoustics, 22, 100241. https://doi.org/10.1016/j.pacs.2021.100241
Holzwarth, N., Schellenberg, M., Gröhl, J., Dreher, K. K., Seitel, A., Tizabi, M., Müller-Stich, B. P., & Maier-Hein, L. (n.d.). Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern. Retrieved March 1, 2021, from https://arxiv.org/abs/2011.04997
Kirchner, T. (2019). Real-time blood oxygenation tomography with multispectral photoacoustics [PhD Thesis]. Universität Heidelberg.
Kirchner, T., Gröhl, J., Herrera, M. A., Adler, T., Hernández-Aguilera, A., Santos, E., & Maier-Hein, L. (2019). Photoacoustics can image spreading depolarization deep in gyrencephalic brain. Scientific Reports, 9(1), 1–9.
Kirchner, T., Gröhl, J., Holzwarth, N., Herrera, M. A., Hernández-Aguilera, A., Santos, E., & Maier-Hein, L. (2019). Photoacoustic monitoring of blood oxygenation during neurosurgical interventions. Photons Plus Ultrasound: Imaging and Sensing 2019, 10878, 108780C.
Kirchner, T., Gröhl, J., & Maier-Hein, L. (2018). Context encoding enables machine learning-based quantitative photoacoustics. Journal of Biomedical Optics, 23(5), 056008.
Kirchner, T., Gröhl, J., Sattler, F., Bischoff, M. S., Laha, A., Nolden, M., & Maier-Hein, L. (2019). An open-source software platform for translational photoacoustic research and its application to motion-corrected blood oxygenation estimation. ArXiv Preprint ArXiv:1901.09781.
Kirchner, T., Gröhl, J., Sattler, F., M.d, M. S. B., Laha, A., Nolden, M., & Maier-Hein, L. (2018). Real-time in vivo blood oxygenation measurements with an open-source software platform for translational photoacoustic research (Conference Presentation). Photons Plus Ultrasound: Imaging and Sensing 2018, 10494, 1049407. https://doi.org/10.1117/12.2288363
Kirchner, T., Sattler, F., Gröhl, J., & Maier-Hein, L. (2018). Signed real-time delay multiply and sum beamforming for multispectral photoacoustic imaging. Journal of Imaging, 4(10), 121.
Kirchner, T., Wild, E., Maier-Hein, K. H., & Maier-Hein, L. (2016). Freehand photoacoustic tomography for 3D angiography using local gradient information. Photons Plus Ultrasound: Imaging and Sensing 2016, 9708, 97083G. https://doi.org/10.1117/12.2209368
Nölke, J.-H., Adler, T., Gröhl, J., Kirchner, T., Ardizzone, L., Rother, C., Köthe, U., & Maier-Hein, L. (2021). Invertible Neural Networks for Uncertainty Quantification in Photoacoustic Imaging. In: Bildverarbeitung für die Medizin 2021 (pp. 330–335). https://doi.org/10.1007/978-3-658-33198-6_80
Sattler, F., Kirchner, T., Gröhl, J., & Maier-Hein, L. (2018). Real-time delay multiply and sum beamforming for multispectral photoacoustics (Conference Presentation). Photons Plus Ultrasound: Imaging and Sensing 2018, 10494, 104942Q. https://doi.org/10.1117/12.2285862
Waibel, D., Gröhl, J., Isensee, F., Kirchner, T., Maier-Hein, K., & Maier-Hein, L. (2018). Reconstruction of initial pressure from limited view photoacoustic images using deep learning. Photons Plus Ultrasound: Imaging and Sensing 2018, 10494, 104942S. https://doi.org/10.1117/12.2288353
Waibel, D., Gröhl, J., Isensee, F., Maier-Hein, K. H., & Maier-Hein, L. (2018). Abstract: Rekonstruktion der initialen Druckverteilung photoakustischer Bilder mit limitiertem Blickwinkel durch maschinelle Lernverfahren. In: Bildverarbeitung für die Medizin 2018 (pp. 201–201).
Patents
Maier-Hein, L., Kirchner, T., Groehl, J., MACHINE LEARNING-BASED QUANTITATIVE PHOTOACOUSTIC TOMOGRAPHY (PAT) European patent pending under no. EP16177204.1
Holzwarth N., Schellenberg M., Gröhl J., Maier-Hein L. Method and system for context-aware photoacoustic imaging. European patent pending under no. EP20193102.9
Awards
First Prize Science Slam SMIT (2019)
Kris Dreher and Niklas Holzwarth for their contribution about photoacoustic imaging
DKFZ PhD Retreat Best Poster Award (2018)
Janek Gröhl for is poster at the Annual PhD Retreat of the German Cancer Research Center
1st prize conhIT Nachwuchspreis (2017)
Janek Gröhl for his Master's thesis "Machine learning based quantitative photoacoustic tomography"
Thomas-Gessmann-Special-Award (2017)
Janek Gröhl for his Master's thesis "Machine learning based quantitative photoacoustic tomography"