Image Registration for the Alignment of Digitized Historical Documents

Image Registration for the Alignment of Digitized Historical Documents

AmirAbbas Davari1*, Tobias Lindenberger1*, Armin Häberle2, Vincent Christlein1, Andreas Maier1, Christian Riess1

1. Pattern Recognition Lab, Computer Science Department, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany [amir.davari, tobias.lintob.lindenberger, vincent.christlein, andreas.maier, christian.riess] @fau.de
2. Bibliotheca Hertziana - Max-Planck-Institut für Kunstgeschichte, Rome, Italy Haeberle@biblhertz.it

• Both authors contributed equally

1 Introduction Novel imaging and image processing techniques provide technical tools to art historians for a better understanding of the creation of an artwork. The approach of a “work process analysis” of an artwork (e.g. a drawing or painting) by art historians aims at segmenting and differentiating the unique steps of production and thus, following the artist’s path from the starting point to his final image. About 70-75% of all old master drawings consist of multiple materials, such as chalks of distinct colors, graphite and/or ink, which mostly have been applied in a step by step manner. This fact opens the opportunity for a chronological reconstruction of the genesis of the work. To this end, material decomposition of drawn layers is oftentimes the most accurate way to follow the artistic workflow. One way to perform layer separation is by spectroscopy. However, this approach is oftentimes destructive to the examined material. To allow for an examination of old master drawings while preserving the drawing to the best extend possible, it is also possible to acquire a multi- or hyperspectral image of the drawing, and separate the layers within the range of visible wavelengths. To obtain a reliable and consistent separation of artwork layers as a basis for art historical interpretation, a number of technical challenges have yet to be solved [Dav17]. First of all, there is the need for a pixel-wise “ground truth” map to objectively compare competing approaches. One possibility to get such a ground truth is to mimic the creation of a step-by-step layered artwork, and to image it after completing each work step. A map for a layer can then be obtained by subtracting two subsequent layers. However, one substantial issue lies in the fact that the acquisitions of two subsequent layers do not exactly match onto each other, and therefore have to be aligned. There are many possible reasons for the mismatch, among which are distortions, mechanical motion, and spherical and chromatic aberration of the optical devices. An example mismatch is illustrated in Fig. 1.1. Similarly, a mismatch must be compensated when the output of an algorithm for layer separation should be mapped 2

to the computed ground truth. This compensation must be a pixel-wise alignment. This is done by a process that is, in the field of image processing, referred to as “image registration”. In this work, we investigate different classical image registration methods for the purpose of creating an accurate ground truth map for hyperspectral historical document processing. We first narrow down the number of possibilities for solving this task by considering problem-specific constraints. Then, we quantitatively and qualitatively compare the two most promising approaches on a phantom document. This paper is organized as follows: in the main document, we state the key findings of this study. A more technical presentation and justification of the intermediate choices is provided in the appendix.

(a) (b) (c) Figure 1.1: Importance of image registration for layer separation in old Master Drawings using image processing is depicted here. (a) image of a phantom data that is acquired by a board scanner, (b) sample channel of hyperspectral image from the same phantom data, (c) false color overlapping image of (a) and (b). As it can be observed, the two images that are acquired by the board scanner and the hyperspectral camera are not pixel-wise aligned. Therefore, the output of layer separation algorithm on the hyperspectral image cannot be numerically evaluated. Image registration would solve this problem.

2 Methods

2.1 Hyperspectral Image Acquisition Hyperspectral imaging combines normal spatial imaging with spectroscopy. The spatial and spectral information of a target is stored in a stack of grayscale images. Each individual image in the stack represents the target recorded at a different 3

wavelength. The stack can be used for further image analysis in the spatial and spectral domain [Mid16c]. For this project, we used a hyperspectral push-broom camera. This camera type features a two dimensional detector array that is combined with a spectrograph. The target is illuminated along one of the spatial axes of the detector. This line contains the full spectrum of the camera’s spectral axis. In push-broom scanning

…(Full text truncated)…