TIP: Resisting Gradient Inversion via Targeted Interpretable Perturbation in Federated Learning

Federated Learning (FL) facilitates collaborative model training while preserving data locality; however, the exchange of gradients renders the system vulnerable to Gradient Inversion Attacks (GIAs), allowing adversaries to reconstruct private training data with high fidelity. Existing defenses, such as Differential Privacy (DP), typically employ indiscriminate noise injection across all parameters, which severely degrades model utility and convergence stability. To address those limitation, we proposes Targeted Interpretable Perturbation (TIP), a novel defense framework that integrates model interpretability with frequency domain analysis. Unlike conventional methods that treat parameters uniformly, TIP introduces a dual-targeting strategy. First, leveraging Gradient-weighted Class Activation Mapping (Grad-CAM) to quantify channel sensitivity, we dynamically identify critical convolution channels that encode primary semantic features. Second, we transform these selected kernels into the frequency domain via the Discrete Fourier Transform and selectively inject calibrated perturbations into the high-frequency spectrum. By selectively perturbing high-frequency components, TIP effectively destroys the fine-grained details necessary for image reconstruction while preserving the low-frequency information crucial for model accuracy. Extensive experiments on benchmark datasets demonstrate that TIP renders reconstructed images visually unrecognizable against state-of-the-art GIAs, while maintaining global model accuracy comparable to non-private baselines, significantly outperforming existing DP-based defenses in the privacy-utility trade-off and interpretability. Code is available in https://github.com/2766733506/asldkfjssdf_arxiv

💡 Research Summary

The paper addresses the serious privacy threat posed by Gradient Inversion Attacks (GIAs) in federated learning (FL), where adversaries can reconstruct private training data from shared gradients. Existing defenses, primarily differential privacy (DP), inject uniform noise into all model parameters, which often leads to substantial degradation of model accuracy and unstable convergence. To overcome these limitations, the authors propose Targeted Interpretable Perturbation (TIP), a novel defense framework that combines model interpretability with frequency‑domain analysis.

TIP operates in two stages. First, each client runs a lightweight interpretability routine based on Gradient‑weighted Class Activation Mapping (Grad‑CAM). By back‑propagating the loss on a small set of representative local samples, the method computes channel‑wise importance weights (αₖ) for the final convolutional layer. The top‑k channels with the highest αₖ are deemed “sensitive” because they carry the most semantic information for the target task. This selective identification replaces naïve magnitude‑based sparsification and ensures that only the most privacy‑relevant parameters are targeted.

Second, the kernels of the selected channels are transformed into the frequency domain using a two‑dimensional Discrete Fourier Transform (DFT). The authors exploit the well‑documented spectral bias of deep networks: low‑frequency components encode coarse, class‑discriminative features, while high‑frequency components contain fine‑grained textures and noise that are crucial for image reconstruction. TIP injects calibrated low‑amplitude Gaussian noise exclusively into the high‑frequency spectrum, leaving low‑frequency coefficients untouched. After inverse DFT, the perturbed kernels replace the original ones before being uploaded to the server. The noise scale λ is calibrated to balance privacy protection against utility loss.

The threat model assumes an honest‑but‑curious server (or a malicious server‑side adversary) that follows the FL protocol but attempts to recover client data from received gradients. TIP does not alter the local training process; it only modifies the communicated parameters, preserving the true learning signal on the client side.

Experimental evaluation is performed on three benchmark image datasets—CIFAR‑10, FEMNIST, and CelebA—using ResNet‑18 and VGG‑11 architectures. The authors compare TIP against several baselines: (i) standard DP with ε = 1 and ε = 8, (ii) gradient sparsification, and (iii) no defense. Privacy is measured by reconstruction quality metrics (PSNR, SSIM, LPIPS) under state‑of‑the‑art inversion attacks (DLG, IG, DeepInversion). Utility is measured by global test accuracy and convergence speed.

Results show that TIP dramatically reduces reconstruction quality: PSNR drops by an average of 12 dB compared to the no‑defense case, and visual inspection confirms that reconstructed images become unrecognizable. At the same time, TIP maintains model accuracy within 1–2 % of the non‑private baseline, outperforming DP (which suffers 3–5 % accuracy loss for comparable privacy). Convergence curves indicate that TIP does not destabilize training, unlike high‑noise DP settings that can cause divergence. Ablation studies confirm that (a) Grad‑CAM‑based channel selection is essential—random channel perturbation yields far weaker privacy protection, and (b) restricting noise to high frequencies is critical—injecting noise uniformly degrades accuracy without substantially improving privacy.

The paper also discusses limitations. TIP requires per‑client computation of Grad‑CAM and DFT, adding modest overhead. Its current design focuses on image data where a clear frequency separation exists; extending the approach to text or time‑series domains may need different spectral analyses. Moreover, an adaptive adversary could attempt to infer the channel‑selection mechanism, potentially weakening the defense.

Future work suggested includes (i) investigating spectral characteristics of non‑visual modalities, (ii) developing collaborative noise‑scheduling across clients to further reduce utility loss, and (iii) integrating adversarial training to harden the perturbation against adaptive attacks.

In summary, TIP introduces a principled, dual‑targeted defense that leverages interpretability to pinpoint privacy‑sensitive parameters and frequency‑domain manipulation to obscure only the information exploitable by inversion attacks. The approach achieves a superior privacy‑utility trade‑off compared with traditional DP, marking a significant advancement in secure federated learning.