Cookie Settings

We use cookies to optimize our website. These include cookies that are necessary for the operation of the site, as well as those that are only used for anonymous statistic. You can decide for yourself which categories you want to allow. Further information can be found in our data privacy protection .


These cookies are necessary to run the core functionalities of this website and cannot be disabled.

Name Webedition CMS
Purpose This cookie is required by the CMS (Content Management System) Webedition for the system to function correctly. Typically, this cookie is deleted when the browser is closed.
Name econda
Purpose Session cookie emos_jcsid for the web analysis software econda. This runs in the “anonymized measurement” mode. There is no personal reference. As soon as the user leaves the site, tracking is ended and all data in the browser are automatically deleted.

These cookies help us understand how visitors interact with our website by collecting and analyzing information anonymously. Depending on the tool, one or more cookies are set by the provider.

Name econda
Purpose Statistics
External media

Content from external media platforms is blocked by default. If cookies from external media are accepted, access to this content no longer requires manual consent.

Name YouTube
Purpose Show YouTube content
Name Twitter
Purpose activate Twitter Feeds

AI and multispectral imaging facilitate keyhole surgery

No. 12 | 13/03/2023 | by FB

How can camera images be used during minimally invasive surgeries to assess whether the operated organ is sufficiently perfused with blood? Scientists from the German Cancer Research Center (DKFZ) and the Städtisches Klinikum Karlsruhe have now succeeded in automatic monitoring of ischemia during a kidney surgery using only the optical properties of the tissue and not relying on injection of contrast agent. To do this, they combined a special imaging technique with artificial intelligence methods. The new method has now been used for the first time in patients with kidney tumors.

While conventional devices (right) generate an RGB video image, the newly developed laparoscope (left) uses a multispectral camera. This also makes it possible to visualize functional properties of the tissue.
© Leonardo Ayala / DKFZ

Minimally invasive surgical techniques, also known as "keyhole surgery", are increasingly replacing conventional open operations - also in cancer care. For example, certain tumors of the internal organs can be removed in such laparoscopic procedures. This involves the use of an instrument known as a laparoscope, which is inserted into the abdomen through a small incision in the skin and then allows the physician to view the affected organs.

The video camera connected to the laparoscope delivers an image signal consisting of the three color channels red, green and blue (RGB). This color space is also used, for example, in televisions or smartphones. However, it has been shown that human tissue has optical properties that cannot be represented with this image information alone. In the scope of the European Research Council (ERC) project NEURAL spicing and in collaboration with the Städtisches Klinikum Karlsruhe, Lena Maier-Hein (DKFZ) and her team therefore investigate the use of multispectral imaging in this context. These multispectral camera systems not only capture the three wavelength ranges of the optical spectrum mentioned above, but provide a multispectral measurement covering over 16 wavelength ranges. This makes it possible to visualize certain functional properties of the tissue that are invisible to conventional camera systems, such as the blood perfusion status of an organ.

This is relevant, for example, in patients with kidney cancer when the tumor has to be removed surgically. During this intervention, the blood flow to the affected areas must be interrupted by clamping the arteries. Only relying on the conventional laparoscopic RGB video makes it virtually impossible to assess the state of tissue perfusion. Only by using a fluorescent contrast agent, which is injected into the patient and subsequently accumulates in the perfused tissues, the physician is able to see whether the correct segments of the organ have been cut off from the blood supply. If this was not successful, the contrast medium has to be washed out for about 30 minutes so that the procedure can then be repeated.

"We were able to show that blood perfusion in the kidney can be visualized without contrast agents and in real time using a combination of multispectral imaging and artificial intelligence," says Lena Meier-Hein. The new technology has now been successfully used in a study in ten patients who had to undergo partial nephrectomy. "Because our approach allows the blood perfusion status of the tissue to be monitored live, the clamping of the blood vessels can be corrected immediately if necessary. This reduces the duration of the procedure, making it safer for patients," says Leonardo Ayala, lead author of the study. The new technology also eliminates the risk of an allergic reaction to the contrast agent.

Two aspects played a decisive role in the acquisition and processing of the image data. The use of multispectral imaging systems in minimally invasive surgery has failed so far, among other things, due to the time required by the systems to acquire and process the data. If several seconds are needed to calculate a new image, the information is practically unusable during the procedure. By using novel multispectral camera technology, the researchers have now been able to solve this problem. Their newly developed system delivers 25 frames per second and thus a fluid video image. On the other hand, the perfusion estimation for the first time could be accomplished using AI methods that do not rely on large amounts of training data. The special feature here is that the AI learns to distinguish perfused from non-perfused tissue for each patient anew. "This is necessary because the tissue is not the same in every patient, so the optical properties can also be very different," explains Leonardo Ayala. Therefore, at the beginning of a surgery, a short video sequence of the kidney is recorded for each patient with the help of a laparoscope. "The algorithm then calculates, based on the patient's individual image data, how the state of blood flow affects the optical properties of the tissue, so that the degree of blood flow can be displayed during surgery," Tim Adler, second lead author complements.

Comparable methods could also be used for other surgical issues in the future. Dogu Teber, clinical lead of the project, sees great potential for this new technological approach: "Spectral imaging in combination with new AI-based analysis tools could become an important tool for fast, reliable and safe functional imaging in minimally invasive surgery."

Leonardo Ayala, Tim J. Adler, Silvia Seidlitz, Sebastian Wirkert, Christina Engels, Alexander Seitel, Jan Sellner, Alexey Aksenov, Matthias Bodenbach, Pia Bader, Sebastian Baron, Anant Vemuri, Manuel Wiesenfarth, Nicholas Schreck, Diana Mindroc, Minu Tizabi, Sebastian Pirmann, Brittaney Everitt, Annette Kopp-Schneider, Dogu Teber, Lena Maier-Hein:
Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery, Science Advances 2023, DOI: 10.5281/zenodo.7529803

A picture is available for download: 

Caption: While conventional devices (right) generate an RGB video image, the newly developed laparoscope (left) uses a multispectral camera. This also makes it possible to visualize functional properties of the tissue.

Note on the use of images for press releases
Use is free of charge. The German Cancer Research Center (DKFZ) permits one-time use in connection with reporting on the subject of the press release or on DKFZ in general. Please indicate as photo credit: "Source: Leonardo Ayala / DKFZ".
Distribution of the photographic material to third parties is only permitted after prior consultation with the DKFZ Press Office (Tel. 06221 42 2854, e-mail: Use for commercial purposes is prohibited.

The German Cancer Research Center (Deutsches Krebsforschungszentrum, DKFZ) with its more than 3,000 employees is the largest biomedical research institution in Germany. More than 1,300 scientists at the DKFZ investigate how cancer develops, identify cancer risk factors and search for new strategies to prevent people from developing cancer. They are developing new methods to diagnose tumors more precisely and treat cancer patients more successfully. The DKFZ's Cancer Information Service (KID) provides patients, interested citizens and experts with individual answers to all questions on cancer.

Jointly with partners from the university hospitals, the DKFZ operates the National Center for Tumor Diseases (NCT) in Heidelberg and Dresden, and the Hopp Children's Cancer Center KiTZ in Heidelberg. In the German Consortium for Translational Cancer Research (DKTK), one of the six German Centers for Health Research, the DKFZ maintains translational centers at seven university partner locations. NCT and DKTK sites combine excellent university medicine with the high-profile research of the DKFZ. They contribute to the endeavor of transferring promising approaches from cancer research to the clinic and thus improving the chances of cancer patients.

The DKFZ is 90 percent financed by the Federal Ministry of Education and Research and 10 percent by the state of Baden-Württemberg. The DKFZ is a member of the Helmholtz Association of German Research Centers.


Subscribe to our RSS-Feed.

to top
powered by webEdition CMS