Interactive 3D visualization for space exploration

Die Marsoberfläche mit geologischen Markierungen versehen.
Die Shaler Region des Mars mit geologischen Annotationen in PRo3D.

The exploration of space and far-away planets such as Mars demands a wide range of different technological instruments through which information encoded in image and sensor data can be sent to Earth. In the context of the Mars rover missions Perseverance of NASA as well as ExoMars of the ESA, VRVis, jointly with JOANNEUM Research, who plays a leading role in the development of the rover's main cameras, has developed the innovative 3D viewer PRo3D, which makes it possible to reconstruct orbiter and Mars rover camera images down to the smallest detail and to visualize them in 3D. Immediately after the landing of NASA's Perseverance rover on Mars, the versatile 3D visualization tool was used to generate the first 3D overflight videos to share the NASA rover's first steps on the distant planet with the general public. PRo3D's high-resolution digital surface models, which can be rendered in just a few steps, also provide a fundamental basis for planetary geology to bring distant worlds, such as asteroid surfaces in the context of impact research, to Earth.

Analysis tools for planetary geology

You can see a cross-section of rock layers on which geological annotations were made in colored lines.
Creation of a geologic log by selecting points on the stratigraphic boundaries of an outcrop.

Planetary scientists can also create geologic logs directly from annotations in the 3D view. They select points on the boundaries of stratigraphic layers to define a cross section of an outcrop. This results in a 2D plot of the layering. Finally, they combine several of these logs from different locations by aligning their layer boundaries. This results in so called correlation panels, which is a geologic interpretation of the stratigraphy of a larger area. We presented this extension of PRo3D at the IEEE VIS 2020 conference (see paper video or the publication "InCorr: Interactive Data-Driven Correlation Panels for Digital Outcrop Analysis").

Planetary Defence

Screenshot of PRo3D showing a model of Dimorphos
First rough model of Dimorphos, the target of NASA's Dart impactor. It is loaded into PRo3D to test annotation tools on an asteroid surface reconstruction. This manual reconstruction is based on first images from Dart. © NASA/APL/ESA/ JOANNEUM RESEARCH/VRVis

VRVis is also involved in ESA’s HERA mission, which is part of the joint ESA-NASA AIDA mission (Asteroid Impact & Deflection Assessment) and focuses on planetary defence. Its goal is to test if asteroids approaching Earth can be deflected by an impactor. PRo3D will be extended to analyse data form HERA instruments, which is essential to assess the effect of the impact. More about the HERA project.

Simulation and generation of training data for deep learning

Visualisierung in PRo3D mit Positionierung eines Strahlenkegels
Maske für die Positionierung eines Strahlenkegels in der Shaler-Region des Mars.

For Mars rover missions, identifying promising scientific targets is of utmost importance. Since mission planning from Earth is challenging, the ideal solution would be to develop Mars rovers further so that they can "autonomously" identify relevant targets for measurements and on-site sampling. For the training of a deep learning system developed by project partner SLR, VRVis also created a method to automatically render a wide variety of images from existing reconstructions of the Martian surface. The DL system was trained to detect shatter cones caused by asteroid impacts.

Publikation: Bechtold, Paar, Garolla, Nowak, Fritz, Traxler, Sidla, Köberl (2023): Planetary Scientific Target Detection via Deep Learning: A Case Study for Finding Shatter Cones in Mars Rover Images

Extended reality for space weather exploration

Black user interface with many images and graphs on the topic of space weather
User interface of the CosmoWeather Expert protoype for virtual visualization of space weather events showing the most important parameters. (c) DLR

Space weather and the effects that weather phenomena such as solar storms can have on Earth are receiving more and more attention as a research question. However, due to the high complexity, it is crucial to present the topic in a way understandable for scientists and the interested general public. Using innovative VR and AR solutions, extended reality offers an excellent opportunity to visualize complex weather data in a low-threshold way: for human-centered analysis, simulation, and communication.

Super Resolution Reconstruction for the optimization of satellite imagery

Two satellite images side by side, the original image on the left is pixelated, the image post-processed by VRVis (on the right) is much clearer.
VRVis' AI-assisted solution enhances the resolution and level of detail of Sentinel-2 satellite imagery (left) using specially applied Super Resolution Reconstruction technology (right).

Satellite images contain valuable information about the Earth's surface and the surrounding atmosphere, which makes them major data sources in the fight against climate change. However, many satellite images, especially those freely available, lack the level of detail necessary for effective work and research. To address this, VRVis is developing AI-based solutions that refine the resolution of freely available satellite imagery using adapted Super Resolution Reconstruction methods. The current goal of the research work is to create AI-based monitoring of small-scale agricultural areas that supports sustainable agriculture in the sense of the European Green Deal.


By playing this video, YouTube may save personal data such as your IP address.
By playing this video, YouTube may save personal data such as your IP address.
By playing this video, YouTube may save personal data such as your IP address.
By playing this video, YouTube may save personal data such as your IP address.

Media reports