From Data to the p-Adic or Ultrametric Model

We model anomaly and change in data by embedding the data in an ultrametric space. Taking our initial data as cross-tabulation counts (or other input data formats), Correspondence Analysis allows us to endow the information space with a Euclidean metric. We then model anomaly or change by an induced ultrametric. The induced ultrametric that we are particularly interested in takes a sequential - e.g. temporal - ordering of the data into account. We apply this work to the flow of narrative expressed in the film script of the Casablanca movie; and to the evolution between 1988 and 2004 of the Colombian social conflict and violence.

💡 Research Summary

The paper proposes a novel framework for detecting anomalies and changes in complex data sets by embedding the data first in a Euclidean space using Correspondence Analysis (CA) and then inducing an ultrametric structure that respects a sequential (typically temporal) ordering. The authors begin with raw data presented as cross‑tabulation counts or similar contingency tables. Such tables are high‑dimensional and often sparse, making direct distance calculations noisy and uninformative. CA addresses this by performing a χ²‑based singular value decomposition, which projects rows (e.g., events, script lines) and columns (e.g., time periods, characters) onto a low‑dimensional Euclidean space while preserving the dominant sources of variation. Each observation is thus represented by a small set of coordinates that capture the essential structure of the original contingency matrix.

Having obtained a Euclidean embedding, the next step is to convert the ordinary metric into an ultrametric. An ultrametric satisfies a strengthened triangle inequality: d(x, z) ≤ max{d(x, y), d(y, z)}. This property is characteristic of hierarchical clustering trees, where the distance between two points is determined by the height of their lowest common ancestor. The authors introduce a “temporal ultrametric” that explicitly incorporates the ordering of observations. They compute Euclidean distances between successive time points, treat these distances as edge weights in a fully connected graph, extract a Minimum Spanning Tree (MST), and then transform the MST into a binary tree. The distance assigned to each internal node is the maximum of its two child edges, ensuring the ultrametric condition. Consequently, the resulting tree encodes the hierarchical proximity of observations: large jumps in the tree correspond to abrupt changes in the underlying data, while smooth climbs reflect gradual evolution.

A key theoretical insight is the connection between the induced ultrametric and p‑adic number theory. In a p‑adic metric, the distance between two numbers depends on the highest power of a prime p that divides their difference; this yields a natural hierarchy of “closeness.” The ultrametric tree constructed from the data is isomorphic to a p‑adic space, allowing the authors to interpret high‑p‑adic‑order nodes as points of significant deviation (anomalies) and low‑order nodes as stable regions. This mathematical bridge provides a rigorous foundation for anomaly detection that goes beyond heuristic clustering.

The methodology is illustrated with two distinct case studies. The first applies the pipeline to the screenplay of the classic film Casablanca. The script is tokenized into scenes, lines, and characters, and a contingency table is built linking lines to scenes. CA reduces this to a two‑dimensional representation, after which the temporal ultrametric is imposed across the ordered sequence of scenes. The resulting ultrametric tree highlights narrative turning points: for example, the famous line “Here’s looking at you, kid” appears as a pronounced peak, indicating a sharp shift in emotional tone and plot direction. By quantifying these peaks, the authors demonstrate how the model can objectively locate and rank narrative climaxes, offering a tool for film scholars and automated story‑analysis systems.

The second case study examines the evolution of the Colombian social conflict from 1988 to 2004. Annual counts of various event types—violent incidents, peace negotiations, human‑rights violations, arms trafficking, etc.—are assembled into a yearly contingency matrix. After CA projection, the temporal ultrametric captures the hierarchical dynamics of the conflict. The tree reveals two major inflection points: a steep rise in violence during the early 1990s and a subsequent flattening beginning around 2002, coinciding with intensified peace negotiations. These inflection points correspond to high‑p‑adic‑order nodes, flagging periods of heightened instability. The model thus provides a compact, mathematically grounded summary of a complex, multi‑dimensional socio‑political process.

Overall, the paper makes several contributions. First, it integrates correspondence analysis with ultrametric induction, delivering a unified pipeline that preserves essential structure while enabling hierarchical anomaly detection. Second, it leverages the equivalence between ultrametrics and p‑adic distances to give a rigorous interpretation of hierarchical changes. Third, it validates the approach on both narrative and socio‑political data, showing its versatility across domains that involve ordered, high‑dimensional observations. Finally, the ultrametric tree is inherently visual and interpretable, facilitating collaboration between data scientists and domain experts. The authors argue that this combination of statistical embedding, hierarchical geometry, and number‑theoretic insight offers a powerful new lens for understanding change and detecting anomalies in a wide range of complex data sets.