PDF: PUF-based DNN Fingerprinting for Knowledge Distillation Traceability

Knowledge distillation transfers large teacher models to compact student models, enabling deployment on resource-limited platforms while suffering minimal performance degradation. However, this paradigm could lead to various security risks, especially model theft. Existing defenses against model theft, such as watermarking and secure enclaves, focus primarily on identity authentication and incur significant resource costs. Aiming to provide post-theft accountability and traceability, we propose a novel fingerprinting framework that superimposes device-specific Physical Unclonable Function (PUF) signatures onto teacher logits during distillation. Compared with watermarking or secure enclaves, our approach is lightweight, requires no architectural changes, and enables traceability of any leaked or cloned model. Since the signatures are based on PUFs, this framework is robust against reverse engineering and tampering attacks. In this framework, the signature recovery process consists of two stages: first a neural network-based decoder and then a Hamming distance decoder. Furthermore, we also propose a bit compression scheme to support a large number of devices. Experiment results demonstrate that our framework achieves high key recovery rate and negligible accuracy loss while allowing a tunable trade-off between these two key metrics. These results show that the proposed framework is a practical and robust solution for protecting distilled models.

💡 Research Summary

The paper addresses the emerging security threat of model theft in knowledge distillation pipelines by introducing a lightweight fingerprinting framework that leverages Physical Unclonable Functions (PUFs). Instead of embedding watermarks in model parameters or relying on secure enclaves, the authors superimpose device‑specific PUF signatures directly onto the teacher model’s logits. Each PUF key is a binary vector; its bits are mapped to a signed perturbation of magnitude ε and added to the clean logits (z_PUF = z_t + δ). The teacher is quantized (e.g., 8‑bit) and fine‑tuned with these perturbed logits, after which a student model is trained using a simple mean‑squared error loss against the fingerprinted outputs.

To emulate real hardware, intra‑device noise is simulated by XOR‑ing a Bernoulli‑distributed mask with a fixed flip probability (1‑5 %) onto the key, producing a noisy key matrix that generates realistic perturbations. This ensures that the fingerprint survives typical variations in temperature, voltage, and aging.

Key recovery proceeds in two stages. First, a neural‑network decoder is trained on a synthetic dataset that models the distribution of logit differences Δz = z_s − z_t under varying ε and Gaussian noise σ. The decoder outputs per‑bit probabilities, which are thresholded to obtain a binary estimate of the PUF key. Second, a Hamming‑distance decoder compares this estimate against a pre‑registered database of all device keys, selecting the entry with minimal Hamming distance. This second stage corrects occasional bit errors from the neural decoder, yielding high recovery accuracy.

The authors also introduce a bit‑compression scheme that packs multiple PUF bits into a compact logit vector, dramatically increasing the number of distinguishable devices without additional overhead. Experiments on CIFAR‑10/100 and ImageNet show that with ε in the range 0.01–0.05 the student model’s accuracy drops by less than 0.2 % while the key recovery rate exceeds 95 %. Compression does not significantly degrade recovery performance, demonstrating scalability to thousands of devices.

Overall, the proposed system offers (1) minimal computational and storage overhead, (2) robustness against reverse engineering and tampering thanks to hardware‑rooted PUF signatures, and (3) post‑theft accountability by enabling traceability of any leaked or re‑distilled model back to the originating device. This makes it a practical solution for protecting distilled AI models on edge platforms.