Sequential Membership Inference Attacks

Modern AI models are not static. They go through multiple updates in their lifecycles. Thus, exploiting the model dynamics to create stronger Membership Inference (MI) attacks and tighter privacy audits are timely questions. Though the literature empirically shows that using a sequence of model updates can increase the power of MI attacks, rigorous analysis of the optimal' MI attacks is limited to static models with infinite samples. Hence, we develop an optimal’ MI attack, SeMI*, that uses the sequence of model updates to identify the presence of a target inserted at a certain update step. For the empirical mean computation, we derive the optimal power of SeMI*, while accessing a finite number of samples with or without privacy. Our results retrieve the existing asymptotic analysis. We observe that having access to the model sequence avoids the dilution of MI signals unlike the existing attacks on the final model, where the MI signal vanishes as training data accumulates. Furthermore, an adversary can use SeMI* to tune both the insertion time and the canary to yield tighter privacy audits. Finally, we conduct experiments across data distributions and models trained or fine-tuned with DP-SGD demonstrating that practical variants of SeMI* lead to tighter privacy audits than the baselines.

💡 Research Summary

The paper tackles a fundamental yet under‑explored aspect of modern machine learning systems: they are rarely static, but evolve through a series of updates as new data arrives, models are fine‑tuned, or federated learning rounds are completed. Existing membership inference (MI) attacks and privacy audits typically focus on a single snapshot of the model, ignoring the wealth of information that intermediate checkpoints can provide. The authors ask whether access to the whole sequence of model outputs can be leveraged to design optimal MI attacks and consequently obtain tighter lower bounds on differential privacy (DP) parameters.

To formalize the problem, they define a sequential mechanism M consisting of T intermediate mechanisms M₁,…,M_T. At each step t a fresh batch D_t of size n is drawn i.i.d. from a possibly time‑varying distribution D_t, and the accumulated dataset S_t = ⋃_{i=1}^t D_i is fed to M_t, producing an output o_t. The global mechanism returns the full sequence o = (o₁,…,o_T). A “crafter” may optionally insert a chosen target point (the canary) z* into a randomly selected batch D_τ at a randomly chosen position j; the insertion indicator B∈{0,1} and the insertion time τ are drawn from user‑specified distributions ν_B and ν_τ.

The core of the work is the design of a sequential MI test (SeMI) that decides, based on the observed sequence, whether B=0 (no canary) or B=1 (canary present at time τ). When τ is known, the problem reduces to a simple hypothesis test between H₀ (no insertion) and H_τ¹ (insertion at τ). By invoking the Neyman‑Pearson lemma, the authors derive the likelihood‑ratio (LR) test that is provably optimal in the sense of maximizing power for a fixed Type‑I error.

For analytical tractability they focus first on the empirical mean mechanism under Gaussian data. Each batch mean (\bar X_t) is aggregated recursively into (\hat\mu_t = (1-1/t)\hat\mu_{t-1} + (1/t)\bar X_t). They show that the joint likelihood ratio factorizes such that all terms cancel except the transition from (\hat\mu_{\tau-1}) to (\hat\mu_{\tau}). Consequently the LR depends only on the batch mean of the insertion step, a property they call the Isolation Property. In the multivariate case the log‑LR is

\