Spectral and Multi-light Imaging for Cultural Heritage Material Analysis and Appearance Reconstruction

The visual characteristics of an object are defined by complex light-matter interaction phenomena. The field of visual computing studies these phenomena in order to better understand optical properties of real world scenes so that ultimately, the reality can be approximated and reconstructed by life-like models. These models can be rooted in theoretical frameworks, or they can be constructed through the analysis of data, collected to enclose different aspects of the reality. Nowadays, there is a plethora of sensing techniques, able to capture various dimensions of an object, such as its spatial distribution, depth, color and reflectance, and even its aging behaviour. This thesis pivots around the capture and analysis of spectral and multi-light images towards the recovery of material properties and appearance attributes, with a focus on cultural heritage applications. As part of the spectral module, the following techniques are used to measure the electromagnetic fingerprint of materials: reflectance image spectroscopy and microfading. Multi-light imaging records the interplay between the light and an object, when the former moves at different angles in the hemisphere above the latter. The spectral signals and multi-light image stacks are processed to create prototype representations of a given object, whereby its color, material and shape are reconstructed. For cultural heritage, the term reconstruction carries a double meaning. On the one hand, it refers to the estimation of appearance properties from the analysis of data and images. On the other hand, it alludes to the restoration of damaged objects. Artworks have different ages that can date far back in history, as well as an inherent sensitivity to environmental factors. For this reason, many have incurred losses or degradation that altered their original appearance when they were first created. This thesis proposes novel solutions to both meanings of appearance reconstruction for a diverse range of cultural heritage objects: bas-reliefs, paintings, drawings, murals. A share of the research outcomes is concerned with the assessment of shape reconstruction for fluorescent and non-fluorescent objects, relying on polynomial models to explain the multi-light image collections. In addition, characterization of material properties is performed using spectral signatures. Then, another area of research is dedicated to the restoration of artworks. In particular, a color-consistent method for loss infilling is proposed for the retouching of wall paintings. Furthermore, this thesis considers the temporal dimension related to light-induced aging, and integrates it to digitally simulate the past and future appearance of an artwork, with a method anchored in multivariate algebra. This innovative spatio-temporal simulation approach bridges the data collected with two spectral capture techniques, namely microfading and hypespectral imaging. Finally, this thesis reviews the ethical implications of imaging workflows for cultural heritage applications, from acquisition to storage in digital repositories, processing, manipulation and reproduction.