A machine-compiled macroevolutionary history of Phanerozoic life

Many aspects of macroevolutionary theory and our understanding of biotic responses to global environmental change derive from literature-based compilations of palaeontological data. Existing manually assembled databases are, however, incomplete and difficult to assess and enhance. Here, we develop and validate the quality of a machine reading system, PaleoDeepDive, that automatically locates and extracts data from heterogeneous text, tables, and figures in publications. PaleoDeepDive performs comparably to humans in complex data extraction and inference tasks and generates congruent synthetic macroevolutionary results. Unlike traditional databases, PaleoDeepDive produces a probabilistic database that systematically improves as information is added. We also show that the system can readily accommodate sophisticated data types, such as morphological data in biological illustrations and associated textual descriptions. Our machine reading approach to scientific data integration and synthesis brings within reach many questions that are currently underdetermined and does so in ways that may stimulate entirely new modes of inquiry.

💡 Research Summary

The paper presents PaleoDeepDive (PDD), a statistical machine‑reading system built on the DeepDive framework, designed to automatically extract fossil occurrence data, taxonomic opinions, and morphological measurements from the heterogeneous scientific literature (text, tables, and figures). The authors motivate the work by highlighting the limitations of the manually curated Paleobiology Database (PBDB), which, despite its extensive coverage (≈300 000 taxonomic names, 1.2 million occurrences), represents only a small fraction of the total palaeontological record because data entry is labor‑intensive, ambiguous, and divorced from original context.

PDD processes PDFs or HTML documents through a pipeline that includes optical character recognition (OCR), layout analysis, and natural‑language processing (NLP). Scientists encode domain knowledge as rules and example annotations; the system learns weights for these rules and performs collective probabilistic inference, producing a “probabilistic database” where each extracted fact is linked to its source context and assigned a confidence probability. This contrasts with traditional pipeline approaches that make hard decisions at each stage and accumulate errors.

To evaluate PDD, the authors selected an Overlapping Document Set (ODS) of 1,782 papers from the top‑50 serials in PBDB, covering English, German, and Chinese literature (76 % English). PDD extracted 192 365 taxonomic opinions versus 79 913 entered by humans, and identified 59 996 taxonomic names not present in PBDB; a random audit indicated that ≥90 % of these new names were valid. Precision estimates from DeepDive’s internal metrics were ≥95 %; blind assessments (double‑blind mixing of 200 facts, and a separate experiment with eight experts evaluating 481 facts) yielded ≥92 % accuracy for PDD, comparable to the human database (error rates 10 % vs. 14 % respectively).

At the macroevolutionary level, the authors applied identical processing pipelines to both databases to generate genus‑level diversity and turnover curves binned into 52 intervals (mean duration 10.4 Myr). The resulting curves were strongly positively correlated, and first‑difference analyses showed Spearman ρ = 0.65 (p = 5.7 × 10⁻⁷). First and last occurrence times for 6 708 genera common to both databases also aligned closely. Discrepancies were traced to (i) human‑introduced “best‑age” assignments not explicitly stated in source papers (≈50 % of ages in PBDB), (ii) OCR‑related failures (especially in tables), and (iii) PDD’s reliance on formally defined geological units, which reduced coverage of recent intervals.

The authors further examined the impact of training‑data volume by subsampling the PBDB and re‑running PDD. Even with as little as 10 % of the original training set, the system retained high extraction volume and maintained strong correlation with the reference macroevolutionary trends, indicating that modest amounts of labeled data suffice for effective learning.

Key contributions include: (1) Demonstration that a probabilistic machine‑reading system can achieve human‑level accuracy in complex scientific extraction tasks; (2) Production of a probabilistic fossil database that quantifies uncertainty and can be iteratively refined; (3) Extension of extraction to morphological measurements from illustrations, with machine‑derived body‑size estimates statistically indistinguishable from manual measurements; (4) Evidence that macroevolutionary signals are robust to random extraction errors, yet PDD’s precision at the individual‑fact level is high enough to support fine‑scale analyses.

Limitations are acknowledged: OCR errors still cause omission of data, especially from tables; recent literature lacking explicit geological unit definitions is under‑represented; and facts with confidence below the 0.95 threshold are excluded, though this can be mitigated by adding new features or rules. The authors suggest future work to improve OCR, broaden unit detection, and incorporate additional evidence sources for age assignment.

In summary, PaleoDeepDive showcases that a collective probabilistic approach to machine reading can not only replicate the core results of a decades‑old, expert‑curated palaeontological database but also surpass it in breadth and scalability. By providing a continuously updatable, uncertainty‑aware resource, PDD opens the door to answering previously underdetermined macroevolutionary questions and offers a template for similar data‑intensive endeavors across Earth and life sciences.