A Self-explainable Model of Long Time Series by Extracting Informative Structured Causal Patterns

Explainability is essential for neural networks modeling time series data, yet most existing explainable artificial intelligence (XAI) techniques generate point-wise importance scores that overlook intrinsic temporal semantics such as trends, cycles, and abrupt regime changes. This limitation obscures human interpretation and weakens trust in long-sequence models. To address these issues, we establish four desiderata for interpretable time-series modeling-temporal continuity, pattern-centricity, causal disentanglement, and faithfulness to the inference process-and propose EXCAP (EXplainable Causal Aggregation Patterns), a unified framework satisfying all four. EXCAP integrates an attentionbased segmenter that extracts continuous temporal patterns, a causally disentangled decoder guided by a pre-trained causal graph, and a latent aggregation loss that enforces separability and stability in the representation space. Theoretical analysis demonstrates that EXCAP guarantees Lipschitz-continuous explanations over time, bounded error under causal mask perturbations, and improved latent-space stability. Comprehensive experiments on classification and forecasting benchmarks confirm that EXCAP outperforms state-of-the-art XAI methods in both predictive accuracy and interpretability, producing temporally coherent and causally grounded explanations. These results indicate that EXCAP provides a principled and scalable framework for faithful and interpretable modeling of long-horizon time-series, with direct relevance to decision-critical domains such as healthcare and finance.