Deep Pictures

The project has the mission to create expressive vector graphics in a time and cost efficient process and
to enable blind and visually impaired people to experience visual arts.

Due to its analytic nature, vector graphics allow many convenient and more semantic image editing tasks compared to common pixel-based graphics. However, pixel graphics are still much more common today, with few exceptions in representing text, technical CAD drawings or geographic maps. We attribute this fact to two major problems that are described in turn. We present numerous ideas for solving these problems, which we will investigate in our project.

The first problem is that no efficient tools are available for creating vector graphics from natural, pixel-based images. Existing fully automatic approaches perform very poorly. As the general image vectorization problem is to some extent of artistic nature, we propose semi-automatic approaches that combine user input with powerful automatic image processing methods. As a particularly interesting sub-problem, we will merge raster images with widely varying zoom levels to form a single vector graphics representation. Another interesting sub problem is the efficient conversion of texture images with repetitive content. A second problem is the limited expressiveness associated with traditional vector graphics. Based on our recent results, we will visually improve vector representations so that natural images appear more realistic. Here we especially consider image details, as present in textured objects, without adding excessive amounts of geometry beyond what can be handled by a user.

Aside from purely visual results, our research will be driven by a second, particularly challenging application of vector graphics: the creation of relief interpretations from paintings, used as learning tools (touch reliefs) for blind and visually impaired people. Exploiting the analytic nature of vector graphics as geometric support structure has the potential to significantly simplify and to speed up this tedious process. By using the result of our image vectorization as outlined above, we aim to develop techniques for shape creation of individual scene elements, including free form modeling tools and making use of available 3D objects, where a particular challenge is the modeling of undercuts in high reliefs. We will also investigate how the modeled objects can be efficiently composed to a scene. This includes tools to optimally utilize a given depth budget, and 2D scene layout so that important objects can be enlarged, as is needed for touch reliefs. The resulting system will enable artistically trained people to quickly create a visually and haptically convincing, high-quality relief interpretation. A selection of the modeled touch reliefs will be physically produced, and we will evaluate their quality and expressiveness with blind and visually impaired people, experts of fine arts and art historians. This way, our results will directly be beneficial for all parties involved, enabling more institutions to open their collection to all people without barrier.