The exploration of space and far-away planets such as Mars demands a wide range of different technological instruments through which information encoded in image and sensor data can be sent to Earth. In the context of the Mars rover missions Perseverance of NASA as well as ExoMars of the ESA, VRVis, jointly with JOANNEUM Research, who plays a leading role in the development of the rover's main cameras, has developed the innovative 3D viewer PRo3D, which makes it possible to reconstruct orbiter and Mars rover camera images down to the smallest detail and to visualize them in 3D. Immediately after the landing of NASA's Perseverance rover on Mars, the versatile 3D visualization tool was used to generate the first 3D overflight videos to share the NASA rover's first steps on the distant planet with the general public. PRo3D's high-resolution digital surface models, which can be rendered in just a few steps, also provide a fundamental basis for planetary geology to bring distant worlds, such as asteroid surfaces in the context of impact research, to Earth.
Planetary scientists can also create geologic logs directly from annotations in the 3D view. They select points on the boundaries of stratigraphic layers to define a cross section of an outcrop. This results in a 2D plot of the layering. Finally, they combine several of these logs from different locations by aligning their layer boundaries. This results in so called correlation panels, which is a geologic interpretation of the stratigraphy of a larger area. We presented this extension of PRo3D at the IEEE VIS 2020 conference (see paper video or the publication "InCorr: Interactive Data-Driven Correlation Panels for Digital Outcrop Analysis").
VRVis is also involved in ESA’s HERA mission, which is part of the joint ESA-NASA AIDA mission (Asteroid Impact & Deflection Assessment) and focuses on planetary defence. Its goal is to test if asteroids approaching Earth can be deflected by an impactor. PRo3D will be extended to analyse data form HERA instruments, which is essential to assess the effect of the impact. More about the HERA project.
For Mars rover missions, identifying promising scientific targets is of utmost importance. Since mission planning from Earth is challenging, the ideal solution would be to develop Mars rovers further so that they can "autonomously" identify relevant targets for measurements and on-site sampling. For the training of a deep learning system developed by project partner SLR, VRVis also created a method to automatically render a wide variety of images from existing reconstructions of the Martian surface. The DL system was trained to detect shatter cones caused by asteroid impacts.
Publikation: Bechtold, Paar, Garolla, Nowak, Fritz, Traxler, Sidla, Köberl (2023): Planetary Scientific Target Detection via Deep Learning: A Case Study for Finding Shatter Cones in Mars Rover Images
Space weather and the effects that weather phenomena such as solar storms can have on Earth are receiving more and more attention as a research question. However, due to the high complexity, it is crucial to present the topic in a way understandable for scientists and the interested general public. Using innovative VR and AR solutions, extended reality offers an excellent opportunity to visualize complex weather data in a low-threshold way: for human-centered analysis, simulation, and communication.
Satellite images contain valuable information about the Earth's surface and the surrounding atmosphere, which makes them major data sources in the fight against climate change. However, many satellite images, especially those freely available, lack the level of detail necessary for effective work and research. To address this, VRVis is developing AI-based solutions that refine the resolution of freely available satellite imagery using adapted Super Resolution Reconstruction methods. The current goal of the research work is to create AI-based monitoring of small-scale agricultural areas that supports sustainable agriculture in the sense of the European Green Deal.
Within the framework of “AI for Green”-funded research project develops AI-based solutions aimed at optimizing free satellite data for monitoring agricultural areas of all sizes.
PanCam-3D focuses on the further development of interactive 3D visualizations for the ExoMars 2022 mission.
WIBSTAC addresses the usage of wide baseline stereo 3D reconstruction for medium- and long-range mapping of the Martian surface, based on imagery from panoramic rover camera instruments.
The CosmoWeather research project aims to develop VR and AR solutions for experts and the general public to make the research field of space weather more accessible and understandable.
The "Mars-DL" project is investigating how a deep learning system can support the research work of planetary scientists through object and pattern recognition. For this project VRVis has extended the functionality of PRo3D to automatically render shatter cone training images.
Support for planetary research: Visual analysis of reconstructions of the Mars surface and view planning for rover camera instruments.
Visual Analysis of Asteroid Deflection.
Virtual exploration and geological analysis of reconstructed Mars surfaces and rock outcrops.
MINERVA is an integrated framework for planetary scientists allowing members of different instrument teams to cooperate synergistically in virtual workspaces by sharing observations, analyses and annotations of heterogonous mission data.
Planetary Robotics Vision Data Exploitation.
Strategic Research in Scalable, Semantic Rendering.