Building Bridges With Interactive Visual Technologies

Adeline Joffres (adeline.joffres@huma-num.fr), Huma-Num, France and Rocio Ruiz Rodarte (rocioruizrodarte@gmail.com), Tecnológico de Monterrey, Mexico and Roberto Scopigno (roberto.scopigno@isti.cnr.it), CNR-ISTI, Italy and George Bruseker (bruseker@ics.forth.gr), ICS-FORTH, Greece and Anaïs Guillem (anais.guillem@gmail.com), UC Merced, USA and Marie Puren (marie.puren@inria.fr), INRIA, France and Charles Riondet (charles.riondet@inria.fr), INRIA, France and Pierre Alliez (pierre.alliez@inria.fr), INRIA, France and Franco Niccolucci (franco.niccolucci@gmail.com), PIN, Italy

1. General presentation of the panel :

Digital technologies offer modern instruments to create strong and persistent bridges between different cultures, countries, disciplines or communities. Among those, interactive visual technologies (3D graphics, or the several incarnations of 2D images) offer unprecedented capabilities to link different contexts:
- visual technologies bridge the scholar or the professional to the reality, by providing high fidelity digital clones of the works under study;
- they offer excellent methods to bridge different scientific domains, by providing tools for integrating different data and show their interplay, thus strongly supporting multidisciplinary investigation;
- they help bridging different cultures and communities, since the visual presentation of different heritage is the first step towards improved mutual knowledge and discovery of commonalities;
- they help bridging academic research with the public, since those visual and interactive media are extremely powerful instruments to disseminate to the public in museums or on the web;
- and, finally, they help bridging the past with the future, since visual representations together with linked semantic data are the key resource to preserve our current knowledge for the future (documenting not just the conservation status but also the reasoning process that leaded us to some insight) and to support future work on the same subjects.

Such benefits are secured by research infrastructures allowing experts to foster and share their best practices and allowing less experienced scholars to acquire knowledge on 3D data management. Another ambition of these infrastructures is to promote interoperability in research practices by giving access to guidance and services. Furthermore, these infrastructures also provide tools through iterative and collaborative processes. In that perspective, as far as 3D standards and practices are concerned, the H2020 project PARTHENOS (http://www.parthenos-project.eu/) is building a service called "Standardization Survival Kit (SSK)" which is an example of such a collaborative approach.

The panel will review the status of a few visual technologies and related CH/DH workflows, mostly focusing on the technologies supporting:
- the digitization of CH assets;
- the visual analysis and comprehension;
- the semantic enrichment and preservation of knowledge.
It will also demonstrate how these technologies' best practices can be secured by the PARTHENOS standardization policy materialized by the SSK.

The panelists will present the status of the different sub-domains with very short talks (10 minutes each), highlighting opportunities, consolidated approaches and open issues. This state of the art review will build a common space for the further Q&A discussion with the audience on the perceived strengths and weaknesses of current digital instruments used in the DH domain.

2. Panelists

- Rocio RUIZ RODARTE (Tecnológico de Monterrey, Mexico, https://tec.mx/es)
- Pierre ALLIEZ (Inria, France, https://team.inria.fr/titane/pierre-alliez/ )
- Roberto SCOPIGNO (CNR-ISTI, Italy, PARTHENOS member, http://vcg.isti.cnr.it/~scopigno/ )
- George BRUSEKER (ICS-FORTH, Greece, http://www.ics.forth.gr/isl/index_main.php?l=e&c=452 ) & Anais GUILLEM (UC Merced, USA)
- Marie PUREN or Charles RIONDET, Inria, France, PARTHENOS members

3. Chairs

- Franco NICCOLUCCI, PIN, PARTHENOS’s project coordinator.
- Adeline JOFFRES, CNRS Huma-Num, PARTHENOS member.

4. Panelists - Talks abstracts

4.1. Talk 1 by Rocio RUIZ RODARTE, Tecnológico de Monterrey (Mexico)
Reconstruction 3D, from Archaeological Reports to Digital Museographies.

3D visualization has served as a bridge between the world of archaeologists, academics and general public in a country like Mexico where geography and budget do not always help this union to arise in a simple way.

Seventeen years ago we combined the knowledge of the archeologist of Calakmul, Ramon Carrasco, with those of students and professors of Architecture and Robotics to develop digital reproductions of the findings of this ancient site hidden within the Mayan jungle.

We used simple methods such as AutoCAD, 3DMax, Unreal Engine and ARToolKit to reconstruct buildings and burials with the purpose of making virtual reality tours, augmented reality installations and VRML models for web pages. All these three-dimensional archives served the archaeologists and restorers of Calakmul to make presentations at conferences and present work reports for several years. Some, the least, were placed in a museum near Calakmul.

The project also served to promote the interest among Mexican researchers in cultural heritage themes. Some of the initiatives that are presented today in national forums as DH2018, arose from projects developed back in 2000 in institutions such as CNA, UAM, UNAM and Tecnológico de Monterrey.

For this event, we have made an upgrade of our projects with the same intention of making them accessible to be viewed by general public. The simple methods we used for those projects still serve us today for virtual tours with the 4th version of Unreal Engine and more commercial applications of augmented reality such as Aurasma. Nowadays we also have the possibility to embed those 3D models inside eBooks for tablets.

There is always people who have the knowledge, the content, as well as developers with extensive technological skills and advanced tools. These forums allow us to get to know each other and establish the bridges to communicate and work together.

4.2. Talk 2 by Pierre ALLIEZ, INRIA, France
3D Digitization and Reconstruction

The technological advances of geometric measurement devices have revolutionized our ability to digitize the world in 3D. This revolution has made possible the development of many new technologies to digitize cultural heritage artefacts and scenes. In addition to sensor technologies, a key issue at the heart of this revolution is that of surface reconstruction, which consists in converting the raw measurements into a computerized surface representation. Beyond faithful topology and geometry, cultural heritage artefacts requires capturing additional properties such as complex interactions between light and materials. Key issues to generate meaningful cultural digital resources include the documentation and quality assessment of the acquisition process, the capturing of conservation status and the recording of provenance information. Open issues are related to the continuous update and consolidation of cultural digital resources. Instead of centralized and static acquisition by a single expert, a dire need is to shift to collaborative and dynamic approaches where communities, active sensor networks and active data resources cooperate to continuously create meaningful cultural resources and generate new knowledge with high relevance to cultural heritage practitioners.

4.3. Talk 3 by Roberto SCOPIGNO, CNR-ISTI, Italy
Visualizing Complex and Information-rich Visual Media

Digital technologies are now mature for producing high quality digital replicas of Cultural Heritage (CH) artefacts. Those digital clones are becoming important assets in many DH activities.

Many applications require to share the models produced, to support the cooperative work of professionals or scholars. This is an emerging need in the DH/CH community: results of digitisation should not remain of restricted use of the scholar or museum who commissioned the digitisation, but should be open to the large community of experts and practitioners. Therefore, the web is the ideal distribution and sharing context. But publishing on the web a complex 3D model was not easy until recently.

The focus of the talk will be to show and discuss practical solutions for the easy publication on the web and visualisation of high-fidelity 3D models. The talk will show some practical examples where high-quality 3D models have been transformed in web-compliant format to be used in CH research, restoration and conservation. The examples will include tools developed to enrich the visual data or to produce insight from the interactive visual analysis.

4.4. Talk 4 by George Bruseker (ICS-FORTH, Greece), Anais Guillem (UC Merced, USA)
3D Models, Humanities & Ontologies

The versatility that 3D modelling techniques offer scholars to pursue new and old lines of research, has led to an intense interest in its application in fields across the humanities. The enthusiastic uptake of these techniques raises important meta-methodological questions in terms of the correct scientific documentation of 3D models and the long term ability to objectively test their validity. Facing these questions demands interdisciplinary cooperation between computer science, information management, and the implicated humanities themselves foremost. A key question to answer is how to elaborate a methodological and technical means to trace the provenance of digital objects, from the point of their original creation through digitization or hypothesis, through their various iterations and adaptations. We argue that only a formal ontology solution, such as CRMdig, that generalizes over a series of popular metadata formats, together with technically aided documentation of process by scholars during their 3D research can achieve these goals.

Investment in developing and applying ontologies to the 3D modelling problem is justified by its increasing importance in research. The range of the application scenario for the use of 3D modelling has long moved past simple visualization for presentations to become an analytic research tool in its own right that can aid in primary research. 3D models can be used to carry out research on physical structures and spaces at a micro or macro level. Accurately capturing physical dimensions in detail can help in the study of design, style, function, provenance and decay of objects. Such models provide an empirical base for pursuing broader research questions such as understanding intention, continuity and change. Virtual reconstruction work, which extends measurement of extant partial objects with evidence based hypothetical reconstructions of past states, allows for complex arguments to be pursued in relation to scholarly sound digital models of partially lost past works.

Given the extant investment of time and thought in such models, it is imperative that this knowledge be preserved. The challenge of doing so arises due to the plethora of tools, institutions and methods which are brought to bear in applying 3D techniques. There is a dizzying area of products and tools already in the ‘market’ and these increase every day. Meanwhile, owing to a lack of standards and a general meta-methodology for the treatment of such models, work done in the past that involved important capital and human investment, is now unusable. The situation calls for a remedy.

We argue that there can be no top-down solution to this problem, imposing certain metadata standards or certain techniques for all. Because of the on-going and developing nature of this research approach, such approaches have in the past and will continue to fail. Rather, what is needed is, on the one hand, the creation of an awareness of the provenance tracking requirement, to create adequate metadata in the first place, and then the application of sufficiently broad ontological models, in order to allow technical solutions for the integration of heterogeneous data from multiple repositories into long term viable storage.

4.5. Talk 5 by Charles RIONDET & Marie PUREN, INRIA, France
PARTHENOS, a European Project Building and Disseminating Collaborative Tools : the Example of 3D Standards within the Standardization Survival Kit

The lifecycle of 3D objects, from their production to their reuse, including processing, description and visualization, involves many highly technical steps and the use of a wide range of technologies, methods and tools, evolving quickly. For each step, the diversity of practices and protocols is a hindrance to agree on a unique standardized solution that could fit all users' needs and solve all the possible problems. Consequently, a crucial task consists in producing guidelines and documenting research practices, in particular at both ends of the 3D objects' lifecycle:
- the digitization/modelling phase, in the course of which these objects are created and thus have to be documented properly,
- and the data reuse phase, in the course of which the availability of proper metadata and accessible archives is a pre-condition for further reuse of digital assets.

PARTHENOS, in particular with the development of the Standardization Survival Kit, aims at being the place where these good practices can be recorded and presented, by means of specific research scenarios where the handling of 3D objects is the core activity, and that would be presented together with documentation, literature and technical resources.

The Standardization Survival Kit can be seen as the host for the state of art documentation of standards related to 3D (amongst others). It is currently under development and will be soft launched in the beginning of 2018, so that a demonstration could be provided during the conference in June 2018. The 3D scenarios will be created by domain experts - partners of the PARTHENOS project or external scholars -, and based on the white paper "Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation", result of a 2016 PARTHENOS Workshop (Alliez, P. et al., 2017).


Appendix A

Bibliography
  1. Alliez, P. and Bergerot, L. and Bernard, J. F. and Boust, C. and Bruseker G., et al. (2017) Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation. White paper: A result of the PARTHENOS Workshop held in Bordeaux at the Maison des Sciences de l’Homme d’Aquitaine and at Archeovision Lab. (France), November 30th - December 2nd, 2016. PARTHENOS. Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation, Nov 2016, Bordeaux, France. pp.71, 2017. https://hal.inria.fr/hal-01526713v2