The development of an information criterion for Change-Point Analysis

Change-point analysis is a flexible and computationally tractable tool for the analysis of times series data from systems that transition between discrete states and whose observables are corrupted by noise. The change-point algorithm is used to identify the time indices (change points) at which the system transitions between these discrete states. We present a unified information-based approach to testing for the existence of change points. This new approach reconciles two previously disparate approaches to Change-Point Analysis (frequentist and information-based) for testing transitions between states. The resulting method is statistically principled, parameter and prior free and widely applicable to a wide range of change-point problems.

💡 Research Summary

The paper presents a novel information‑theoretic framework for change‑point analysis (CPA) that overcomes the limitations of traditional model‑selection criteria such as AIC and BIC when applied to singular models. CPA seeks to locate the time indices at which a noisy time series switches between discrete states. The authors first formalize a CPA model as consisting of two distinct sets of parameters: (i) continuous “state” parameters θ that describe the statistical properties within each segment, and (ii) discrete change‑point indices i that delimit the segments. While the state parameters are regular (non‑zero Fisher information), the change‑point indices are essentially non‑identifiable: their Fisher information is zero because moving a change point within a homogeneous segment does not affect the likelihood. This singularity causes AIC, whose penalty equals the number of free parameters, to dramatically underestimate model complexity, leading to over‑segmentation.

To address this, the authors introduce the Frequentist Information Criterion (FIC). FIC replaces the unknown true distribution p with the model’s own distribution q(·|M) in the expectation that defines the complexity term K. The resulting information criterion is
 IC = h(data | MLE) + K_FIC,
where h is the negative log‑likelihood evaluated at the maximum‑likelihood estimate (MLE). The key innovation is the decomposition of the complexity increment between successive nested models into a “nesting complexity” k(n) = K_FIC(n) − K_FIC(n‑1). Two regimes are distinguished:

1. Identifiable parameters – when the added parameters are regular, the nesting penalty reduces to the familiar AIC term k⁺ = d, where d is the number of added continuous parameters.
2. Unidentifiable (change‑point) parameters – when a new change point is introduced, the penalty is derived from a new statistic U(N,d). By mapping the information change onto a d‑dimensional discrete Brownian bridge, the authors show that the expected penalty equals k⁻ = 2 E