Modern AI models are not static. They go through multiple updates in their lifecycles. Thus, exploiting the model dynamics to create stronger Membership Inference (MI) attacks and tighter privacy audits are timely questions. Though the literature empirically shows that using a sequence of model updates can increase the power of MI attacks, rigorous analysis of the `optimal' MI attacks is limited to static models with infinite samples. Hence, we develop an `optimal' MI attack, SeMI*, that uses the sequence of model updates to identify the presence of a target inserted at a certain update step. For the empirical mean computation, we derive the optimal power of SeMI*, while accessing a finite number of samples with or without privacy. Our results retrieve the existing asymptotic analysis. We observe that having access to the model sequence avoids the dilution of MI signals unlike the existing attacks on the final model, where the MI signal vanishes as training data accumulates. Furthermore, an adversary can use SeMI* to tune both the insertion time and the canary to yield tighter privacy audits. Finally, we conduct experiments across data distributions and models trained or fine-tuned with DP-SGD demonstrating that practical variants of SeMI* lead to tighter privacy audits than the baselines.
Machine Learning (ML) models memorize training data, creating privacy risks for individuals whose data were used to develop them (Shokri et al., 2017;Carlini et al., 2022a). Privacy auditing quantifies these risks by testing whether specific records can be detected in the training set. Membership Inference (MI) attacks serve as the primary tool for auditing: an attacker who distinguishes members from non-members with high confidence establishes an empirical Specifically, standard auditing practice targets a specific snapshot of a model. The auditor selects a target point (a.k.a. canary), queries the model, and applies a membership test. Target selection matters: points 'far' from the data distribution leak more information than typical points (Azize & Basu, 2025;Carlini et al., 2022b). Modern ML models are dynamic: they rarely exist as isolated snapshots.
Continuous training updates the deployed models as new data arrives. Fine-tuning adapts the base models to specific tasks. Federated learning aggregates updates from different users across rounds. In each case, multiple model versions become accessible, whether through public release, API versioning, or checkpoint access during training. Jagielski et al. (2023) demonstrate that combining the MI scores across model updates improves attack accuracy. They apply standard MI attacks separately to each model snapshot, and then, aggregate the scores via difference or ratio operations. Related work on model updates includes reconstruction attacks (Salem et al., 2020) that recover data from online learning updates, machine unlearning attacks (Chen et al., 2020), and leakage analysis of snapshots in NLP (Zanella-Béguelin et al., 2020). Our approach differs by deriving the optimal test via likelihood ratio analysis and identifying the insertion time of a canary as a new actionable lever for tighter audits.
We study Membership Inference (MI) when the auditor observes a sequence of T model snapshots rather than a single static one. The sequential setting offers two levers for designing tight audits: selecting targets that leak information, and choosing when to insert them. Specifically, we ask:
- Are there advantages of the MI tests accessing a sequence of models in comparison to a static test on the final model? 2. Does the knowledge of the insertion time affect the power of sequential MI tests? 3. Can sequential MI tests lead to theoretically and practically tighter privacy audits?
Our contributions address these questions affirmatively.
(1) Optimal MI test with sequential access. Motivated by the optimal MI attack for the static setting (Azize & Basu, 2025) derived from the Neyman-Pearson lemma (Neyman & Pearson, 1933), we derive the optimal MI attack in the sequential setting, which is a sequential likelihood ratio test crafted with an insertion time. We refer to it as SeMI * . Specifically, for the empirical mean computation from Gaussian data, we derive a closed-form analysis of the sequential MI attack.
(2) Isolation property of sequential MI test. We further observe a unique phenomenon for the sequential MI test on the empirical mean, namely isolation property. This means that consecutive outputs reveal the batch mean at insertion time τ exactly. The likelihood ratio depends only on this recovered batch statistic. Rather than detecting the target among all cumulative samples, the sequential test operates on an isolated batch of size n. For the empirical mean under stationary conditions, test power depends only on batch size and target distance from the mean, not on τ or T . By contrast, a single-observation test applied to the final output must detect the target among nT samples; its influence diminishes as training data accumulates. Sequential observation eliminates this dilution.
(3) Adapting MI tests with the insertion time. As insertion time emerges as a unique lever for sequential MI tests, we study the impact of the knowledge of the insertion time. We study three likelihood ratio based tests, SeMI * , SeMI unif and SeMI max , that respectively assume τ is known, uniformly distributed, or can be anything. We derive their powers for the empirical mean mechanism. Our numerical studies show that SeMI * achieves the best power while the other two behave similarly.
(4) Extending sequential MI tests to Gradient Descent (GD). Since GD-type algorithms (Kiefer & Wolfowitz, 1952) and their variants are the standard to train modern ML models, we further extend the design of sequential MI tests to Stochastic GD (SGD) and DP-SGD (Abadi et al., 2016) for batch training, namely SeMI SGD . We clarify the practical relaxations needed to extend the sequential MI tests to the practical ML models. Our numerical studies for SGD and DP-SGD show similar improvements achieved by the sequential MI test and impact of the insertion time as in the empirical mean setting.
(5) From tighter sequential MI tests to sequential privacy audits. First, we formalize sequential privac
This content is AI-processed based on open access ArXiv data.