Research topics

Explainable, intuitive and reliable decisions

Two researchers with certificates at the interview during the award ceremony.
VRVis researchers Dimitrios Lenis and David Major receive the eAward 2021 in the category "Artificial Intelligence" for their research work on XAI.

How an artificial intelligence-based classification algorithm assigns a category, such as ‘diseased’ or ‘healthy’, to a radiological image, is typically not intelligible for the user. The therefore predominantly used neuronal networks behave like a black box – the learned image features guiding the networks decision remain elusive for the user.

For years, VRVis has been dedicated to researching AI solutions that are not only explainable and intuitive, but also reliable, producing comprehensible and trustworthy decisions. To achieve this goal, we are developing methods that make the neural network's decision-making process transparent to the user. These methods not only give medical professionals additional feedback on the reliability of the decision but also help developers in verifying the quality of the trained network.

Analysis of mammography data

Breast cancer is the most common type of cancer in women worldwide. Thus, the exact analysis of mammography images is an important task in radiology, since the early detection of risk factors and possible suspicious regions in the breast is crucial for successful therapy.

In cooperation with our long-term partner Agfa HealthCare, we developed various deep learning approaches for the analysis of mammography data. We combine several deep learning models that are focused to specific medical questions, such as where pathological lesions and microcalcifications are located in an image or whether a patient has denser breast tissue. This allows us to improve patient-level predictions based on the specific results of the individual models. The modular design of our solution and the possibility to access task specific decisions individually is increasing the explainability of the model and trust in its overall decision. To further increase trust in individual black box deep learning models and their decisions, we developed several novel approaches for explaining AI decisions in the context of image-based decision making.

    Segmentation of soft tissue tumors in multimodal data

    A female researcher stands in front of a presentation wall and presents the results of her deep learning based research work.
    VRVis researcher Theresa Neubauer presents the MICCAI paper "Soft Tissue Sarcoma Co-Segmentation in Combined MRI and PET/CT Data".

    The exact segmentation of soft tissue tumors in multimodal data is of high importance in various medical domains, e.g. for surgery and biopsy planning or in radiotherapy. For this purpose multimodal data like MRI and PET/CT data is acquired, whereby the tumor segmentation may look different, depending on the modality and clinical task

    In cooperation with the Medical University of Vienna, we developed and compared different methods to segment soft tissue tumors. We focused specifically on the modality- and task-specific soft tissue tumor segmentation, and how we can fuse multimodal data efficiently to improve the quality of the segmentation.

      Tuberculosis detection by AI-supported screening

      Three X-ray images of the chest area next to each other, on which areas affected by tuberculosis are marked/classified in different ways.
      The ground truth of an X-ray of a tuberculosis disease is shown on the left. The middle image shows the results of a standard Gradcam method; this method only partially detects the diseased areas. The third image shows the much more precise XAI method from VRVis, which was used in the next step/further project: The colored areas show the disease, the color gradient shows the severity of the disease.

      In order to support medical professionals in patient screening to detect tuberculosis infections, VRVis has developed a deep learning-based application in collaboration with Agfa Healthcare. With large numbers of patients, it assists in identifying diseased individuals more rapidly, optimising further diagnosis and treatment courses. Find more information in our press release.

      Medical image registration

      [Translate to English:] Zu sehen sind zwei Spalten an medizinischen Röntgenbildern des Brustbereichs, daneben sind zwei Spalten von medizinischer Bildregistrierung derselben Aufnahmen.
      Chest X-Ray images of the same patient taken at two different medical exams. They show different stages of various diseases. The previous state-of-the-art method uses lung penalized registration, obfuscating significant changes in the X-ray images. The VRVis method is capable of highlighting large and subtle differences at the same time. Precise localisation of disease markers is important to medical professionals to monitor disease progression.

      Medical image registration is an integral part of medical analysis workflows. It allows different images of one or more patients to be analyzed in a directly comparable way by transforming the images into the same coordinate system. Registration is necessary to compare images between studies acquired at different time points or between different modalities.

      VRVis has developed methods for classical and deep learning based image registration in collaboration with its medical partners, and in particular investigated how visualization can support registration workflows and highlight problems.

        Segmentation and annotation of blood vessels

        Two images of the human skeleton with segmented and color-highlighted arteries.
        The automatic segmentation of arteries in different parts of the body: on the left in the chest area, on the right in the shoulder, neck and head area.

        According to the World Health Organization, arteriosclerosis ist the no.1 cause of death globally. A detailed analysis of blood vessels in 3D angiograms is generally very time-consuming. In collaboration with Agfa Healthcare, VRVis has developed several methods for efficient semi-automated and fully automated segmentation of peripheral and coronary blood vessels, as well as their semantic annotation; some of these methods have won awards, been published in high-impact journals, patented and integrated into our customers’ products.

        General approaches to feature detection and segmentation of organs

        You can see two rows of 3 CT scans showing images of the spine and liver as well as several colored markers segmenting the corresponding organs.
        Segmentation of organs on CT images, the top row shows the spine and the bottom row the liver.

        VRVis has developed and patented a number of basic methods for medical image analysis. These include widely cited AI methods for simultaneous segmentation of several objects on radiology images, sequential segmentation methods for CT image data, efficient feature detection methods, as well as several basic methods for statistical shape and texture models which, for example, are still used in combination with AI methods for difficult tasks in organ segmentation.

        Fully automated semantic annotation of the spinal column

        A VRVis researcher shows two computer screens on which spinal scans can be seen.
        The VRVis solution recognizes human vertebrae on various MR and CT scans fully automatically and annotates the spinal cords anatomically correctly.

        The spine is an important reference system in the human body for describing the site of pathologies during a radiological examination. For this reason, the anatomical name of each visible vertebra in the image must be known and, in many countries, has to be labelled on the image by radiologists manually. This time-consuming work has to be performed by radiologists in addition to the actual diagnosis within a short space of time. VRVis has developed a group of algorithms which can fully automatically recognise and correctly annotate the anatomy of the human vertebrae from different MR and CT scans independent of the image section. This important automated preparatory work simplifies the radiologist’s work and provides more time for the actual diagnosis.

        Cardiac analysis

        Today, cardiovascular diseases are the most common cause of death in Austria. A detailed analysis of (time-dependent) 3D image data of the beating heart and coronary arteries in a patient is extremely time-consuming. Fully automated methods which simplify the quantification of heart pathologies can therefore make a considerable contribution to accelerating the diagnosis by the radiologist. VRVis has already developed several solutions for segmenting the heart ventricles on CT and MR images as well as an award-winning solution for fully automated segmentation of coronary arteries.

        Several patents on AI

        Three female researchers in front of a screen. The standing researcher points to a detail on the monitor.
        The Biomedical Image Informatics research team led by Katja Bühler has researched and developed more than 20 biomedical patents in recent years.

        In association with our application-oriented research work for our business partner Agfa Healthcare, many patents for AI-based solutions have been produced in our in-house basic research, for example on the fully automated segmentation and annotation of blood vessels or human vertebrae. You can find out more about our patents.