Residual analysis methods for space--time point processes with applications to earthquake forecast models in California

Modern, powerful techniques for the residual analysis of spatial-temporal point process models are reviewed and compared. These methods are applied to California earthquake forecast models used in the Collaboratory for the Study of Earthquake Predictability (CSEP). Assessments of these earthquake forecasting models have previously been performed using simple, low-power means such as the L-test and N-test. We instead propose residual methods based on rescaling, thinning, superposition, weighted K-functions and deviance residuals. Rescaled residuals can be useful for assessing the overall fit of a model, but as with thinning and superposition, rescaling is generally impractical when the conditional intensity $\lambda$ is volatile. While residual thinning and superposition may be useful for identifying spatial locations where a model fits poorly, these methods have limited power when the modeled conditional intensity assumes extremely low or high values somewhere in the observation region, and this is commonly the case for earthquake forecasting models. A recently proposed hybrid method of thinning and superposition, called super-thinning, is a more powerful alternative.

💡 Research Summary

This paper reviews and compares modern residual analysis techniques for space‑time point‑process models, focusing on their application to earthquake forecasting models used by the Collaboratory for the Study of Earthquake Predictability (CSEP) in California. Traditional evaluation tools such as the L‑test (likelihood‑based) and N‑test (count‑based) are low‑power and often fail to reveal detailed spatial‑temporal mis‑fits. The authors therefore explore a suite of more sophisticated methods: rescaling, thinning, superposition, weighted K‑functions, deviance residuals, and a recently introduced hybrid called super‑thinning.

Rescaling transforms observed event times and locations using the conditional intensity λ(x,t) so that, under a correctly specified model, the transformed points follow a homogeneous Poisson process. While useful for assessing overall fit, the method becomes numerically unstable when λ is highly volatile or takes extreme values, which is common in seismic forecasts. Thinning removes points in regions where λ is high, whereas superposition adds simulated points where λ is low, both aiming to produce a uniform intensity. These approaches can highlight local areas of poor fit, but their power collapses when λ approaches zero or infinity, because too few or too many points remain after the operation.

Super‑thinning combines thinning and superposition by specifying a target intensity μ. For each location‑time pair, if λ>μ a point is randomly deleted with probability 1‑μ/λ; if λ<μ a synthetic point is added with probability μ/λ‑1. This yields a residual process with constant intensity μ, preserving information about where the original model over‑ or under‑predicts. The authors demonstrate that super‑thinning retains high power even when λ varies dramatically, making it a more reliable diagnostic tool.

Weighted K‑functions incorporate λ as a weight, allowing the comparison of empirical and theoretical K‑statistics while accounting for spatial heterogeneity. Deviance residuals compute the log‑likelihood difference for each spatio‑temporal cell, providing a direct measure of relative model performance across the study region.

The methods are applied to several CSEP forecast models (e.g., ETAS, Poisson, smoothed seismicity). While L‑ and N‑tests deem most models acceptable, residual analyses reveal systematic mis‑fits: over‑prediction along the coastal fault zones, under‑prediction in the Sierra Nevada, and scale‑dependent discrepancies captured by weighted K‑functions. Super‑thinning, in particular, isolates the problematic cells with clear visual maps, and when combined with deviance residuals it suggests concrete model refinements (e.g., adjusting background rates or aftershock productivity).

The paper concludes with practical recommendations: accurate estimation of λ is essential; boundary effects must be corrected; computational cost of super‑thinning is modest compared with simulation‑based tests; and the suite of residual tools should become standard in seismic forecast validation. Future work may extend these diagnostics to real‑time monitoring, multi‑model ensembles, and other natural‑hazard point processes such as volcanic eruptions or flood events.