Computer-assisted forensic medicine
Mobile markerless augmented reality and its application to forensic medicine
During autopsy, forensic pathologists today mostly rely on visible indication, tactile perception and experience to determine the cause of death. Although computed tomography (CT) data is often available for the bodies under examination, these data are rarely used due to the lack of radiological workstations in the pathological suite. The data may prevent the forensic pathologist from damaging evidence by allowing him to associate, for example, external wounds to internal injuries.
To address these issues, we propose a mobile and markerless concept based on the recently emerging range cameras. For the first time, our concept enables both (1) intuitive augmented reality visualization and (2) surface documentation of forensic data. In contrast to other state-of-the-art systems, the concept is non-invasive, cost-efficient and easy to integrate into the clinical workflow. Our concept involves a range camera, mounted on a tablet computer, to acquire depth and color information in real time. The idea is to visualize CT data in color images directly over the body part of interest. Because it is portable, the device can be freely placed almost anywhere close to the body and is easy to handle. By fusing color images and CT data, external wounds and internal injuries are shown in an intuitive and non-invasive manner (e.g., shot wound and organ lesions or bruises and fractures), thus avoiding destruction of potential evidence. Our concept facilitates online and offline visualization of multimodal forensic data. The medical examiner can access this data during the forensic examination to obtain an overview of the case. Images/videos and the 3D textured CT surfaces can further be used for documentation and for intuitive presentation in trial. More information can be found in [1].
With kind permission from Springer Science+Business Media: International Journal of Computer Assisted Radiology and Surgery, Mobile markerless augmented reality and its application in forensic medicine, volume 10, 2015, page 573-586, Kilgus, T., Heim, E., Haase, S., Prüfer, S., Müller, M., Seitel, A., Fangerau, M., Wiebe, T., Iszatt, J., Schlemmer, H.-P., Hornegger, J., Yen, K., Maier-Hein, L., figure number 12.
Concept of mobile markerless augmented reality: A mobile device shows internal structures from the viewpoint of the device. A continuous update of the virtual camera pose enables a vivid and intuitive presentation of multi-modal (video and tomographic) forensic data.
© dkfz.de
Figure 2: Visualization of multi-modal forensic data (computed tomography data and photographic images). Reprinted from [1] with permission of Springer.
© dkfz.de
Key collaborators
Prof. Yen, Institute for Forensic Medicine and Traffic Medicine, Heidelberg
Prof. Hornegger, Sven Haase, Department of Computer Science, Pattern Recognition Lab, Friedrich-Alexander University Erlangen-Nuremberg
PhD Michael Müller, Markus Fangerau, mbits GmbH, Heidelberg
Selected publications
[1] Kilgus, T., Franz, A.M., Seitel, A., März, K., Bartha, L., Fangerau, M., Mersmann, S., Groch, A., Meinzer, H.-P., Maier-Hein, L., 2012. Registration of partially overlapping surfaces for range image based augmented reality on mobile devices, in: SPIE Medical Imaging. International Society for Optics and Photonics, p. 83160T–83160T.
[2] Kilgus, T., Heim, E., Haase, S., Prüfer, S., Müller, M., Seitel, A., Fangerau, M., Wiebe, T., Iszatt, J., Schlemmer, H.-P., Hornegger, J., Yen, K., Maier-Hein, L., 2014. Mobile markerless augmented reality and its application in forensic medicine. International Journal of Computer Assisted Radiology and Surgery 10, 573–586.
[3] Maier-Hein, L., Franz, A.M., Fangerau, M., Schmidt, M., Seitel, A., Mersmann, S., Kilgus, T., Groch, A., Yung, K., Santos, T.R. dos, Meinzer, H.-P., 2011. Towards Mobile Augmented Reality for On-Patient Visualization of Medical Images, in: Handels, H., Ehrhardt, J., Deserno, T.M., Meinzer, H.-P., Tolxdorff, T. (Eds.), Bildverarbeitung Für Die Medizin, Informatik Aktuell. Springer, pp. 389–393.
[4] Müller, M., Rassweiler, M.-C., Klein, J., Seitel, A., Gondan, M., Baumhauer, M., Teber, D., Rassweiler, J.J., Meinzer, H.-P., Maier-Hein, L., 2013. Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. International Journal of Computer Assisted Radiology and Surgery 8, 663–675.