Dual-Criterion Curriculum Learning: Application to Temporal Data

Curriculum Learning (CL) is a meta-learning paradigm that trains a model by feeding the data instances incrementally according to a schedule, which is based on difficulty progression. Defining meaningful difficulty assessment measures is crucial and most usually the main bottleneck for effective learning, while also in many cases the employed heuristics are only application-specific. In this work, we propose the Dual-Criterion Curriculum Learning (DCCL) framework that combines two views of assessing instance-wise difficulty: a loss-based criterion is complemented by a density-based criterion learned in the data representation space. Essentially, DCCL calibrates training-based evidence (loss) under the consideration that data sparseness amplifies the learning difficulty. As a testbed, we choose the time-series forecasting task. We evaluate our framework on multivariate time-series benchmarks under standard One-Pass and Baby-Steps training schedules. Empirical results show the interest of density-based and hybrid dual-criterion curricula over loss-only baselines and standard non-CL training in this setting.

💡 Research Summary

The paper introduces Dual‑Criterion Curriculum Learning (DCCL), a novel curriculum learning framework that simultaneously leverages two complementary measures of instance difficulty: a loss‑based criterion and a density‑based criterion computed in a learned representation space. Traditional curriculum learning relies on a single difficulty estimator—often a hand‑crafted heuristic or the model’s own loss—and this single view can be insufficient, especially when the data exhibit structural heterogeneity or noise. DCCL addresses this limitation by first learning an embedding function ϕθ (which can be any model capable of producing vector representations, such as a pretrained Transformer, an LSTM, or a simple feature extractor). In this embedding space, instance density is estimated either via k‑nearest‑neighbors (k‑NN) counting or Kernel Density Estimation (KDE). The density score reflects how “typical” an instance is: high density indicates a prototypical, easy example, while low density signals a rare, potentially harder example.

The loss‑based difficulty δloss is obtained by evaluating the current model’s prediction error on each instance (e.g., mean‑squared error). Low loss corresponds to easy samples, high loss to hard ones. Both scores are normalized to