Cookie Settings

We use cookies to optimize our website. These include cookies that are necessary for the operation of the site, as well as those that are only used for anonymous statistic. You can decide for yourself which categories you want to allow. Further information can be found in our data privacy protection .

Essential

These cookies are necessary to run the core functionalities of this website and cannot be disabled.

Name Webedition CMS
Purpose This cookie is required by the CMS (Content Management System) Webedition for the system to function correctly. Typically, this cookie is deleted when the browser is closed.
Name econda
Purpose Session cookie emos_jcsid for the web analysis software econda. This runs in the “anonymized measurement” mode. There is no personal reference. As soon as the user leaves the site, tracking is ended and all data in the browser are automatically deleted.
Statistics

These cookies help us understand how visitors interact with our website by collecting and analyzing information anonymously. Depending on the tool, one or more cookies are set by the provider.

Name econda
Purpose Statistics
External media

Content from external media platforms is blocked by default. If cookies from external media are accepted, access to this content no longer requires manual consent.

Name YouTube
Purpose Show YouTube content
Name Twitter
Purpose activate Twitter Feeds

Efficient data annotation

With the breakthrough success of deep learning-based algorithms for automatic image annotation, the availability of reference annotations for their training is one of the major bottlenecks in many fields. In computer assisted minimally-invasive surgery, for example, the amount of publically available annotated data is steadily increasing. However, it is extremely challenging to capture all the variations (e.g. all medical devices applied in an intervention) that may occur in clinical practice. Furthermore, algorithms trained on data of one specific domain (e.g. the specific hardware in a specific hospital for a specific intervention) typically do not generalize well to new domains. This is an important limitation as curation of (sufficient) training data is extremely labor-intensive and is currently hindering progress in the field. Several different approaches have been introduced to address this issue:

Crowdsourcing:

In crowdsourcing-based approaches, annotation tasks are outsourced to masses of anonymous workers in an online community. We have explored this concept in the context of 3D diagnostic image annotation (Heim et al., 2015; Heim et al., 2018) and endoscopic video annotation (Maier-Hein et al., 2016) . In order to deal with the high variation in annotation quality, we proposed a novel approach to estimate the skills of a user based on the interaction with the annotation software (Heim et al., 2018). Furthermore, we investigated first concepts to crowd-algorithm collaboration for large-scale image annotation (Maier-Hein et al., 2016).

Figure 1: Automated quality control in crowdsourcing. A machine learning algorithm analyzes the interaction behaviour of the users and detects spammers accordingly (Heim et al., 2018).
© dkfz.de

Self-supervised learning

Another approach to achieve efficient data annotation is to create methods that exploit natural structure in data to "self-supervise'' their learning. This concept holds great potential for the field of Surgical Data Science as the bottleneck related to training data acquisition is often due to the small amount of annotated medical image data rather than the amount of raw medical data that. One of our contributions (Ross et al., 2018), for example, was based on the hypothesis that unlabeled video data can be used to learn a representation of the target domain (here: the specific hardware setup and surgical procedure that an instrument segmentation algorithm is applied in) to boost the performance of state-of-the-art machine learning algorithms. The core component of this approach is a so-called auxiliary task (here: recolorization) that is used to pre-train a convolutional neural network (CNN) for the target task.

Domain adaptation

The concept of domain adaptation has successfully been applied in various fields to compensate for the fact that algorithms trained on a specific training domain typically do not generalize well on the target domain. In (Wirkert et al., 2017), for example, we used domain adaptation in the context of oxygenation estimation from multispectral imaging data in order to compensate that no reference labels for oxygenation could be obtained. The approach involved using unlabeled real measurements of the target domain to give different weights to training samples that were generated with a Monte-Carlo approach, in order to train the algorithm.

Alumni

  • Dr. Eric Heim (Doctoral Student)
  • Martina Zündel (Student Assistant)

Key Collaborators

  • Dr. Daniel Kondermann, QUALITY MATCH, http://www.quality-match.com
  • Dr. Hannes Kenngott, Division of Visceral Surgery of the Department of General Surgery, University of Heidelberg
  • Prof. Dr. Stefanie Speidel, Department for Translational Surgical Oncology, National Center for Tumor Diseases Dresden

Publications

Bittel, S., Roethlingshoefer, V., Kenngott, H., Wagner, M., Bodenstedt, S., Ross, T., Speidel, S., & Meier-Hein, L. (2017). How to create the largest in-vivo endoscopic dataset. Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis.

Cardoso, M. J., Arbel, T., Lee, S., Cheplygina, V., Balocco, S., Mateus, D., Zahnd, G., Maier-Hein, L., Demirci, S., Granger, E., & others. (2017). Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis. CVII-STENT and Second International Workshop, LABELS.

Ganz, M., Kondermann, D., Andrulis, J., Knudsen, G. M., & Maier-Hein, L. (2017). Crowdsourcing for error detection in cortical surface delineations. International Journal of Computer Assisted Radiology and Surgery, 12(1), 161–166.

Heim, E., Roß, T., Seitel, A., März, K., Stieltjes, B., Eisenmann, M., Lebert, J., Metzger, J., Sommer, G., Sauter, A. W., & others. (2018). Large-scale medical image annotation with crowd-powered algorithms. Journal of Medical Imaging, 5(3), 034002.

Heim, E., Seitel, A., Andrulis, J., Isensee, F., Stock, C., Ross, T., & Maier-Hein, L. (2017). Clickstream analysis for crowd-based object segmentation with confidence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12), 2814–2826.

Heim, E., Seitel, A., Stock, C., Ross, T., & Maier-Hein, L. (2017). Clickstreamanalyse zur Qualitätssicherung in der crowdbasierten Bildsegmentierung. In Bildverarbeitung für die Medizin 2017 (pp. 17–17). Springer.

Maier-Hein, L., Kondermann, D., Roß, T., Mersmann, S., Heim, E., Bodenstedt, S., Kenngott, H. G., Sanchez, A., Wagner, M., Preukschas, A., & others. (2015). Crowdtruth validation: A new paradigm for validating algorithms that rely on image correspondences. International Journal of Computer Assisted Radiology and Surgery, 10(8), 1201–1212.

Maier-Hein, L., Mersmann, S., Kondermann, D., Bodenstedt, S., Sanchez, A., Stock, C., Kenngott, H. G., Eisenmann, M., & Speidel, S. (2014). Can masses of non-experts train highly accurate image classifiers? International Conference on Medical Image Computing and Computer-Assisted Intervention, 438–445.

Maier-Hein, L., Ross, T., Gröhl, J., Glocker, B., Bodenstedt, S., Stock, C., Heim, E., Götz, M., Wirkert, S., Kenngott, H., & others. (2016). Crowd-algorithm collaboration for large-scale endoscopic image annotation with confidence. International Conference on Medical Image Computing and Computer-Assisted Intervention, 616–623.

Maier-Hein, L., Mersmann, S., Kondermann, D., Stock, C., Kenngott, H. G., Sanchez, A., Wagner, M., Preukschas, A., Wekerle, A.-L., Helfert, S., & others. (2014). Crowdsourcing for reference correspondence generation in endoscopic images. International Conference on Medical Image Computing and Computer-Assisted Intervention, 349–356.

Maier-Hein, L., Wagner, M., Ross, T., Reinke, A., Bodenstedt, S., Full, P. M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., & others. (2020). Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room. ArXiv Preprint ArXiv:2005.03501.

Ross, T., Reinke, A., Full, P. M., Wagner, M., Kenngott, H., Apitz, M., Hempe, H., Filimon, D. M., Scholz, P., Tran, T. N., Bruno, P., Arbeláez, P., Bian, G.-B., Bodenstedt, S., Bolmgren, J. L., Bravo-Sánchez, L., Chen, H.-B., González, C., Guo, D., ... Maier-Hein, L. (2020). Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge. Medical Image Analysis, 101920. https://doi.org/10.1016/j.media.2020.101920

Ross, T., Zimmerer, D., Vemuri, A., Isensee, F., Wiesenfarth, M., Bodenstedt, S., Both, F., Kessler, P., Wagner, M., Müller, B., & others. (2018). Exploiting the potential of unlabeled endoscopic video data with self-supervised learning. International Journal of Computer Assisted Radiology and Surgery, 13(6), 925–933.

Awards

IPCAI Best Presentation Award: Runner Up (2018)
Tobias Roß for the presentation "Exploiting the potential of unlabeled endoscopic video data with self-supervised learning"


KUKA IJCARS Best Paper Award, 3rd Prize (2016)
M. Ganz, D. Kondermann, J. Andrula, G. M. Knubben and L. Maier-Hein for the paper "Crowdsourcing for error detection in cortical surface delineations."


CURAC Best Paper Award (2015)
E. Heim, T. Ross, T. Norajitra, M. Nolden, K. März, D. Kondermann, S. Speidel, K. Maier-Hein and L. Maier-Hein for the paper "Crowdgestützte Organsegmentierung: Möglichkeiten und Grenzen."

to top
powered by webEdition CMS