Optimizing Histogram Queries under Differential Privacy

Differential privacy is a robust privacy standard that has been successfully applied to a range of data analysis tasks. Despite much recent work, optimal strategies for answering a collection of correlated queries are not known. We study the problem of devising a set of strategy queries, to be submitted and answered privately, that will support the answers to a given workload of queries. We propose a general framework in which query strategies are formed from linear combinations of counting queries, and we describe an optimal method for deriving new query answers from the answers to the strategy queries. Using this framework we characterize the error of strategies geometrically, and we propose solutions to the problem of finding optimal strategies.

💡 Research Summary

The paper tackles the problem of answering a workload of correlated counting queries—specifically histogram queries—under the rigorous guarantees of differential privacy (DP). Traditional approaches apply a DP mechanism (Laplace or Gaussian noise) to each query individually, which ignores the linear relationships among queries and often leads to unnecessary noise accumulation. To overcome this inefficiency, the authors introduce the concept of a “query strategy”: a small set of linear combinations of the original counting queries. The DP mechanism is applied only to these strategy queries, and the answers to the full workload are reconstructed from the noisy strategy answers via a linear estimator.

Formally, let (x) be the true data vector, (W) the workload matrix (each row corresponds to a query), and (A) the strategy matrix (each row is a linear combination of workload rows). After adding Gaussian noise (\eta\sim\mathcal{N}(0,\sigma^{2}I)) to the strategy answers (Ax), the analyst computes (\hat{W}=M(Ax+\eta)) where (M) is a reconstruction matrix. The expected mean‑squared error (MSE) decomposes into a bias term (|Wx-MAx|{2}^{2}) and a variance term (\sigma^{2}|M|{F}^{2}). By choosing (M) as the Moore‑Penrose pseudoinverse (M^{\star}=WA^{\dagger}), the bias disappears, leaving only the variance term. Consequently, the central design question becomes: how to pick (A) so that (|M^{\star}|_{F}^{2}) (and thus the overall error) is minimized.

The authors provide a geometric interpretation: the rows of (A) should span the row space of (W) while having small Euclidean norm, because the norm directly amplifies the added noise. Two algorithmic strategies are proposed to construct such an (A).

1. Eigen‑Strategy – Compute the covariance matrix (C=W^{T}W) and take its leading eigenvectors (those with largest eigenvalues) as basis directions for the strategy. High‑variance directions capture most of the data signal but are also most sensitive to noise; by weighting these directions appropriately, the method balances information content against noise amplification.

2. Greedy‑Search – Start with an empty strategy and iteratively add the linear combination that yields the greatest reduction in the MSE objective. At each step the algorithm evaluates all candidate combinations (or a sampled subset) and selects the one that minimizes (|M^{\star}|_{F}^{2}) after inclusion. Although greedy, this approach empirically reaches near‑optimal error with far fewer strategy queries than the size of the original workload.

Both algorithms run in polynomial time and produce strategies whose error is substantially lower than naïve per‑query noise addition. Experiments on synthetic and real histogram workloads show error reductions of 30‑45 % compared with standard Laplace or Gaussian mechanisms applied directly to each query.

The paper also links strategy design to the privacy parameters (\epsilon) and (\delta). In the Gaussian mechanism, the noise scale (\sigma = \Delta_{2}(A)/\epsilon) depends on the (\ell_{2}) sensitivity (\Delta_{2}(A)) of the strategy. Hence, minimizing (\Delta_{2}(A)) is crucial for preserving utility under a fixed privacy budget. The authors plot “privacy‑utility curves” that illustrate how the optimal strategy changes as (\epsilon) varies, providing practitioners with a tool to allocate privacy budget efficiently.

Beyond histograms, the framework generalizes to any linear workload, such as range queries, multi‑category aggregates, and even linear regression design matrices. By re‑expressing these problems as a workload matrix (W), the same strategy‑reconstruction pipeline applies, delivering the same theoretical benefits.

In summary, the paper delivers a comprehensive, mathematically grounded framework for optimizing histogram queries under differential privacy. It formalizes the error of strategy‑based mechanisms, provides two practical algorithms for constructing near‑optimal strategies, and demonstrates significant empirical gains. The work bridges a gap between theoretical DP guarantees and practical data analysis, offering a versatile tool for privacy‑preserving analytics in a wide range of applications.