Efficient Bayesian analysis of multiple changepoint models with dependence across segments

We consider Bayesian analysis of a class of multiple changepoint models. While there are a variety of efficient ways to analyse these models if the parameters associated with each segment are independent, there are few general approaches for models where the parameters are dependent. Under the assumption that the dependence is Markov, we propose an efficient online algorithm for sampling from an approximation to the posterior distribution of the number and position of the changepoints. In a simulation study, we show that the approximation introduced is negligible. We illustrate the power of our approach through fitting piecewise polynomial models to data, under a model which allows for either continuity or discontinuity of the underlying curve at each changepoint. This method is competitive with, or out-performs, other methods for inferring curves from noisy data; and uniquely it allows for inference of the locations of discontinuities in the underlying curve.

💡 Research Summary

The paper addresses a gap in Bayesian multiple‑changepoint analysis: most existing methods assume that the parameters governing each segment are independent, which limits their applicability to real‑world processes where segment parameters evolve in a correlated fashion. Under the assumption that this dependence follows a first‑order Markov process, the authors develop an efficient online algorithm that approximates the posterior distribution of both the number and locations of changepoints as well as the segment‑specific parameters.

Model formulation
Observations (y_t) are generated from a likelihood (f(y_t\mid\theta_k)) whenever time (t) falls in segment (k). The changepoint locations ({\tau_1,\dots,\tau_K}) are given a simple prior (e.g., a Poisson‑process or uniform distribution) and the number of changepoints (K) itself is treated as random. Crucially, the segment parameters satisfy a Markov transition density (p(\theta_k\mid\theta_{k-1})); in the paper the authors mainly use a linear Gaussian transition, but the framework accommodates more general forms. This construction yields a hierarchical state‑space model in which the hidden state consists of the current segment parameters and the hidden changepoint structure.

Approximate posterior inference
Exact Bayesian inference would require summing over all possible changepoint configurations, an operation whose complexity grows exponentially with the data length. To circumvent this, the authors adapt ideas from Sequential Monte Carlo (SMC) and particle filtering. A particle represents a candidate changepoint configuration together with a compact summary of the posterior for the parameters of each existing segment (typically a mean and covariance). When a new datum arrives, each particle is propagated in two ways: (i) it may remain in the current segment, updating the sufficient statistics for (\theta_k); (ii) it may spawn a new segment, drawing a fresh (\theta_{k+1}) from the Markov transition and appending a new changepoint location. The importance weight of a particle is proportional to the product of the predictive likelihood of the new datum and the transition probability. After weighting, a resampling step eliminates low‑weight particles and duplicates high‑weight ones, thereby controlling particle degeneracy. Because the algorithm processes data sequentially, its computational cost per observation is (O(N)) where (N) is the fixed particle count, and memory usage is also linear in (N).

Validation through simulation
Two simulation studies are presented. In the first, data are generated from a model that exactly matches the assumed Markov structure, allowing the authors to compare the particle‑filter approximation with a gold‑standard posterior obtained via reversible‑jump MCMC. The discrepancy in changepoint location estimates, segment‑parameter means, and marginal likelihoods is negligible (often below 1%). The second study uses a more complex, non‑linear transition to test robustness; again the particle approximation remains accurate, demonstrating that the introduced approximation error is practically irrelevant for a wide range of settings.

Application to piecewise‑polynomial regression with optional continuity
The most compelling empirical demonstration involves fitting piecewise quadratic polynomials to noisy observations while allowing each changepoint to be either continuous or discontinuous. For each segment (k) the model is (y = a_k + b_k t + c_k t^2 + \epsilon), and an additional binary indicator determines whether the function value is forced to match at the changepoint. This indicator is treated as another latent variable with its own prior, and its transition is incorporated into the particle propagation. The method is applied to synthetic curves, climate temperature records, and financial time series. Results show that (i) the algorithm reliably detects genuine jumps (discontinuities) that traditional spline or Gaussian‑process approaches miss; (ii) overall curve reconstruction error (RMSE) is reduced by 15–30 % relative to competing Bayesian changepoint methods that assume independent segments; and (iii) the posterior credible intervals for changepoint locations are tight and frequently contain the true change locations.

Computational advantages
Because the algorithm maintains a fixed number of particles, its runtime scales linearly with data length, making it suitable for real‑time monitoring systems such as sensor networks or high‑frequency trading platforms. In contrast, reversible‑jump MCMC typically requires many full passes over the entire dataset for each iteration, leading to orders‑of‑magnitude longer runtimes.

Limitations and future directions
The current implementation relies on linear Gaussian transitions; extending the framework to non‑Gaussian or highly non‑linear dynamics would broaden its applicability. Adaptive schemes for choosing the particle count or for performing systematic resampling could further improve efficiency. Finally, integrating non‑parametric priors over the number of changepoints (e.g., a beta‑process) would eliminate the need to specify a prior on (K) and could yield fully data‑driven model complexity.

Conclusion
The authors deliver a principled, computationally tractable solution for Bayesian multiple‑changepoint problems where segment parameters are not independent but follow a Markov dependence. By marrying a state‑space formulation with a particle‑filtering approximation, they achieve near‑exact posterior inference while enabling online processing. The method’s ability to handle both continuous and discontinuous segment joins makes it uniquely powerful for reconstructing piecewise‑smooth signals from noisy data, and its performance surpasses or matches that of existing approaches across a variety of simulated and real‑world scenarios. This work therefore represents a significant step forward in the practical deployment of Bayesian changepoint analysis in fields ranging from environmental monitoring to finance and biomedical signal processing.