Tight Lower Bounds and Improved Convergence in Performative Prediction

Performative prediction is a framework accounting for the shift in the data distribution induced by the prediction of a model deployed in the real world. Ensuring rapid convergence to a stable solution where the data distribution remains the same after the model deployment is crucial, especially in evolving environments. This paper extends the Repeated Risk Minimization (RRM) framework by utilizing historical datasets from previous retraining snapshots, yielding a class of algorithms that we call Affine Risk Minimizers and enabling convergence to a performatively stable point for a broader class of problems. We introduce a new upper bound for methods that use only the final iteration of the dataset and prove for the first time the tightness of both this new bound and the previous existing bounds within the same regime. We also prove that utilizing historical datasets can surpass the lower bound for last iterate RRM, and empirically observe faster convergence to the stable point on various performative prediction benchmarks. We offer at the same time the first lower bound analysis for RRM within the class of Affine Risk Minimizers, quantifying the potential improvements in convergence speed that could be achieved with other variants in our framework.

💡 Research Summary

The paper tackles a central challenge in performative prediction: the data distribution changes as a deployed model influences the environment, and rapid convergence to a performatively stable point is essential. Existing work on Repeated Risk Minimization (RRM) – notably Perdomo et al. (2020) and Mofakhami et al. (2023) – provides convergence guarantees under different sensitivity assumptions (Wasserstein vs. χ²) and strong convexity conditions. However, these analyses only give upper bounds on the convergence rate, and it remained unclear whether those bounds are tight or improvable.

The authors introduce a new algorithmic family called Affine Risk Minimizers (ARM). Instead of training on the distribution induced by the current model alone, ARM forms an affine combination of the distributions from all previous training snapshots. Formally, at iteration t the algorithm minimizes the risk under a mixture distribution Dₜ = Σ_{i=0}^{t‑1} αᵗ_i D(fθ_i), where the coefficients αᵗ_i are non‑negative and sum to one. A particularly simple instantiation uses the average of the last two snapshots (α = ½ for each). The authors prove that the set of performatively stable points for ARM coincides with that of standard RRM, so stability is not sacrificed.

The theoretical contributions are threefold. First, they sharpen the existing upper bound for RRM under the χ²‑sensitivity framework by removing an extraneous constant C, yielding a linear convergence rate of (√(ε M γ))^t when √(ε M γ) < 1. Second, they establish matching lower bounds for both the χ²‑sensitivity (Theorem 2) and the Wasserstein‑sensitivity (Theorem 3) settings, showing that the previously known upper bounds are indeed tight: there exist problem instances for which any RRM algorithm that only uses the current distribution converges no faster than Ω((√(ε M γ))^t) or Ω((β ε γ)^t) respectively. Third, they prove that ARM can break these lower bounds. Lemma 1 shows that with a ½–½ mixture of the last two snapshots, the contraction factor becomes √(3/2 · ε M γ), strictly smaller than √(ε M γ). Consequently, ARM achieves a provably faster linear rate while converging to the same stable point.

Empirically, the authors evaluate ARM on several performative prediction benchmarks, most notably strategic classification tasks where agents manipulate features in response to the classifier. Across four datasets, ARM reaches the stable point with 30–40 % fewer iterations than standard RRM, confirming the theoretical speedup. They also observe that overly emphasizing very old snapshots can degrade performance, highlighting the need for careful weight design.

Beyond the immediate algorithmic advance, the paper’s lower‑bound analysis delineates a fundamental barrier for any method that relies solely on the current induced distribution. By demonstrating that incorporating historical data can surpass this barrier, the work opens a new research direction: exploring richer combinations of past distributions (non‑affine, adaptive weighting) and extending the analysis to other divergence measures such as KL or total variation. The results have practical implications for any decision‑dependent learning system—regulation, healthcare, education—where rapid stabilization after deployment is critical.

In summary, the paper provides (i) a tighter convergence upper bound for RRM, (ii) the first matching lower bounds proving the optimality of those rates, (iii) a novel algorithmic class (ARM) that leverages past data to beat the lower bound, and (iv) extensive experiments validating the theoretical gains. This constitutes a significant step forward in understanding and improving the dynamics of performative prediction.