Efficient, Differentially Private Point Estimators

Differential privacy is a recent notion of privacy for statistical databases that provides rigorous, meaningful confidentiality guarantees, even in the presence of an attacker with access to arbitrary side information. We show that for a large class of parametric probability models, one can construct a differentially private estimator whose distribution converges to that of the maximum likelihood estimator. In particular, it is efficient and asymptotically unbiased. This result provides (further) compelling evidence that rigorous notions of privacy in statistical databases can be consistent with statistically valid inference.

💡 Research Summary

The paper “Efficient, Differentially Private Point Estimators” addresses a fundamental tension in statistical data analysis: how to protect individual privacy while preserving the statistical efficiency of estimators. The authors focus on parametric families of probability distributions that satisfy standard regularity conditions (smoothness, bounded parameter space, existence of Fisher information). Under these assumptions, the maximum‑likelihood estimator (MLE) is asymptotically normal with variance equal to the inverse Fisher information, and it is asymptotically efficient (i.e., it attains the Cramér‑Rao lower bound). However, the MLE is not differentially private because a single data point can influence the estimate arbitrarily.

To reconcile privacy with efficiency, the authors propose a simple yet powerful construction based on the “sample‑and‑aggregate” paradigm. The data set of size n is partitioned into k disjoint blocks, each containing t = n/k observations. On each block the authors compute a bias‑corrected MLE, denoted (\hat\theta_{bc}). Bias correction removes the leading O(1/n) bias term of the ordinary MLE, reducing the bias to O(n^{-3/2}), which is negligible compared to the variance for the asymptotic analysis. The block‑wise estimates (z_j) are then averaged to obtain (\bar z = \frac{1}{k}\sum_{j=1}^k z_j).

Differential privacy is achieved by adding Laplace noise (R\sim\text{Lap}(\lambda)) with scale (\lambda = \Lambda/(k\varepsilon)), where (\Lambda) is the diameter of the bounded parameter space (\Theta) and (\varepsilon) is the privacy budget. The final estimator is \