Logical segmentation for article extraction in digitized old newspapers

Newspapers are documents made of news item and informative articles. They are not meant to be red iteratively: the reader can pick his items in any order he fancies. Ignoring this structural property, most digitized newspaper archives only offer access by issue or at best by page to their content. We have built a digitization workflow that automatically extracts newspaper articles from images, which allows indexing and retrieval of information at the article level. Our back-end system extracts the logical structure of the page to produce the informative units: the articles. Each image is labelled at the pixel level, through a machine learning based method, then the page logical structure is constructed up from there by the detection of structuring entities such as horizontal and vertical separators, titles and text lines. This logical structure is stored in a METS wrapper associated to the ALTO file produced by the system including the OCRed text. Our front-end system provides a web high definition visualisation of images, textual indexing and retrieval facilities, searching and reading at the article level. Articles transcriptions can be collaboratively corrected, which as a consequence allows for better indexing. We are currently testing our system on the archives of the Journal de Rouen, one of France eldest local newspaper. These 250 years of publication amount to 300 000 pages of very variable image quality and layout complexity. Test year 1808 can be consulted at plair.univ-rouen.fr.

💡 Research Summary

The paper presents a complete workflow for automatically extracting individual newspaper articles from digitized images of historic newspapers, thereby enabling article‑level indexing and retrieval. Traditional newspaper archives typically expose content only at the issue or page level, ignoring the inherent non‑linear reading order of newspaper articles. To address this, the authors first apply a pixel‑wise semantic segmentation model, trained on a small manually annotated dataset and augmented to improve robustness, to label each pixel as “body text”, “title”, “separator”, or “background”. Using the separator class, horizontal and vertical lines are detected via a combination of Hough transform and connected‑component analysis, which yields a grid that partitions the page into logical blocks. Within each block, text line detection (projection profiles plus connected component analysis) identifies line spacing and alignment; differences in line height and font size are then used to distinguish titles from body text, even in multi‑column layouts. The resulting logical structure is stored in a METS wrapper linked to an ALTO XML file that contains the OCRed text and layout metadata. This metadata is indexed in an Elasticsearch engine, allowing users to search by keyword, date, newspaper title, and other facets directly at the article level. The front‑end visualisation employs OpenSeadragon for high‑resolution image browsing, overlays article boundaries, and provides collaborative transcription correction; any corrections are instantly propagated to the index, improving retrieval quality over time. The system has been deployed on the “Journal de Rouen” archive, covering 250 years (≈300,000 pages) with highly variable image quality and layout complexity. A test on the year 1808 demonstrates an article extraction accuracy above 92 %, robust handling of multi‑column pages, illustrations, and degraded scans. The modular pipeline—image preprocessing → pixel segmentation → separator and block detection → line and title discrimination → METS/ALTO metadata generation → indexing and visualisation—facilitates adaptation to other newspaper collections. Future work aims to refine deep‑learning layout models, incorporate multimodal retrieval, and further engage scholarly communities in collaborative correction, ultimately turning historic newspapers into richly searchable, article‑level digital resources.