Meta-learning to Address Data Shift in Time Series Classification

Across engineering and scientific domains, traditional deep learning (TDL) models perform well when training and test data share the same distribution. However, the dynamic nature of real-world data, broadly termed \textit{data shift}, renders TDL models prone to rapid performance degradation, requiring costly relabeling and inefficient retraining. Meta-learning, which enables models to adapt quickly to new data with few examples, offers a promising alternative for mitigating these challenges. Here, we systematically compare TDL with fine-tuning and optimization-based meta-learning algorithms to assess their ability to address data shift in time-series classification. We introduce a controlled, task-oriented seismic benchmark (SeisTask) and show that meta-learning typically achieves faster and more stable adaptation with reduced overfitting in data-scarce regimes and smaller model architectures. As data availability and model capacity increase, its advantages diminish, with TDL with fine-tuning performing comparably. Finally, we examine how task diversity influences meta-learning and find that alignment between training and test distributions, rather than diversity alone, drives performance gains. Overall, this work provides a systematic evaluation of when and why meta-learning outperforms TDL under data shift and contributes SeisTask as a benchmark for advancing adaptive learning research in time-series domains.

💡 Research Summary

This paper investigates whether meta‑learning can mitigate the performance degradation caused by data shift in time‑series classification, using seismic event detection as a representative scientific problem. Traditional deep learning (TDL) models assume that training and test data share the same distribution; when this assumption is violated—due to sensor drift, environmental changes, or other sources of distributional shift—model accuracy drops sharply, demanding costly relabeling and retraining. Meta‑learning, specifically optimization‑based first‑order methods (FOMAML and Reptile), offers a “learning‑to‑learn” paradigm: a meta‑model is trained across many related tasks so that it can quickly adapt to a new task with only a few labeled examples.

The authors introduce a novel, semi‑synthetic benchmark called SeisTask. SeisTask consists of 243 tasks generated by a full factorial design over five seismic simulation parameters (circles, layers, velocity, frequency, source type), each task containing 420 two‑channel waveforms (half signal, half noise) with a fixed low signal‑to‑noise ratio (SNR = 1/5). An auxiliary out‑of‑distribution set, OOD‑STEAD, is built from real STEAD recordings, organized into 35 tasks distinguished by SNR bins, to test generalisation on truly unseen data.

To confirm that the tasks exhibit meaningful data shift, the authors train a task‑specific model for each of the 243 tasks and evaluate cross‑task performance. They compute a proportion metric (p_u) that measures how often other task‑specific models outperform the model trained on the same task, and they use linear Center Kernel Alignment (CKA) to quantify similarity between task representations. Both analyses reveal that tasks are neither identical nor completely unrelated, satisfying the prerequisite for meta‑learning.

Experiments compare three learning regimes across multiple model sizes (small, medium, large) and data regimes (5–20 shots per task up to hundreds of shots):

1. TDL + fine‑tuning (train on all available data, then fine‑tune on the target task).
2. Meta‑learning (FOMAML, Reptile) (meta‑train on many tasks, then adapt with a few shots).
3. Training from scratch with the same limited shots.

Key findings:

• In data‑scarce regimes (≤ 20 labeled examples per task), meta‑learning consistently outperforms TDL, achieving 5–15 % higher accuracy and converging in fewer epochs. The advantage is most pronounced for small networks (2–3 layers, 64–128 units), where meta‑learning also reduces over‑fitting, as evidenced by a smaller train‑validation gap.
• When abundant data are available (≥ 200 examples per task) and larger architectures are used (≥ 5 layers, > 500 k parameters), the performance gap narrows; TDL with fine‑tuning matches or slightly exceeds meta‑learning. In this regime, the overhead of meta‑training offers little benefit.
• Task diversity matters, but not in a naïve way. Adding diverse tasks improves meta‑learning only when the added tasks are similar to the target distribution. Emphasizing similarity between training and test tasks yields the largest gains; indiscriminate diversity can introduce noise and degrade performance.
• On the OOD‑STEAD set, meta‑learned models generalise well when the test tasks share characteristics with the meta‑training tasks, confirming that the learned adaptation strategy transfers to real seismic data. When the test distribution is markedly different, performance aligns with that of TDL fine‑tuning, indicating the limits of transfer.

The authors discuss practical implications: meta‑learning is especially valuable in scientific domains where labeling is expensive and data are inherently scarce, and where rapid adaptation to new conditions (e.g., after a sensor upgrade) is required. However, when large labeled datasets are already available, the simpler TDL pipeline may be preferable due to lower computational overhead. The study also highlights the importance of carefully curating the meta‑training task pool to reflect the target domain, rather than merely maximizing task count.

Limitations include the focus on first‑order meta‑learning methods; future work could explore second‑order MAML, domain‑adaptation techniques, test‑time adaptation, or continual‑learning frameworks. Moreover, while SeisTask provides controlled variability, real‑world deployments will encounter additional complexities (non‑stationary noise, missing modalities) that merit further investigation.

In summary, the paper provides a thorough empirical evaluation of meta‑learning versus traditional deep learning under data shift for time‑series classification. It demonstrates that meta‑learning delivers faster, more stable adaptation and reduced over‑fitting in low‑data, small‑model settings, while its benefits diminish as data and model capacity grow. The introduction of the SeisTask benchmark and the analysis of task similarity effects constitute valuable contributions for the broader community interested in adaptive learning for physical‑science time‑series data.