ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models

As large language models (LLMs) are used in sensitive fields, accurately verifying their computational provenance without disclosing their training datasets poses a significant challenge, particularly in regulated sectors such as healthcare, which have strict requirements for dataset use. Traditional approaches either incur substantial computational cost to fully verify the entire training process or leak unauthorized information to the verifier. Therefore, we introduce ZKPROV, a novel cryptographic framework allowing users to verify that the LLM’s responses to their prompts are trained on datasets certified by the authorities that own them. Additionally, it ensures that the dataset’s content is relevant to the users’ queries without revealing sensitive information about the datasets or the model parameters. ZKPROV offers a unique balance between privacy and efficiency by binding training datasets, model parameters, and responses, while also attaching zero-knowledge proofs to the responses generated by the LLM to validate these claims. Our experimental results demonstrate sublinear scaling for generating and verifying these proofs, with end-to-end overhead under 3.3 seconds for models up to 8B parameters, presenting a practical solution for real-world applications. We also provide formal security guarantees, proving that our approach preserves dataset confidentiality while ensuring trustworthy dataset provenance.

💡 Research Summary

ZKPROV addresses a critical gap in the verification of large language models (LLMs) used in regulated domains such as healthcare and finance. While prior work has focused on proving inference correctness or the integrity of the entire training process, none have provided a cryptographic guarantee that a model was trained on specific, authorized datasets without exposing the data itself. ZKPROV introduces a zero‑knowledge framework that binds a model’s response to the exact datasets used during training, while keeping both the dataset contents and model parameters confidential.

The system relies on four main cryptographic building blocks. First, a “binding value” hashes together the user query, the model’s answer, the dataset commitment, and the model‑parameter commitment, ensuring any change invalidates the proof. Second, HyperNova‑based recursive zk‑SNARKs are used to express each transformer layer’s computation as a Rank‑1 Constraint System (R1CS) and then fold the layer‑wise proofs into a single succinct proof. HyperNova’s logarithmic verification complexity yields near‑constant verification time regardless of model depth. Third, Kate‑Zaverucha‑Goldberg (KZG) polynomial commitments are employed to commit to dataset hashes and model weights; KZG enables short opening proofs that can be verified via pairings without revealing the underlying polynomials. Fourth, Boneh‑Lynn‑Shacham (BLS) signatures, with their aggregation property, allow multiple data‑source authorities to sign their datasets and for a verifier to check a single aggregated signature.

The protocol proceeds in four stages. (1) Data providers compute a KZG commitment of their dataset hash and obtain a BLS signature from the appropriate authority. (2) Model developers embed the dataset commitment and a KZG commitment of the initial model parameters into the training pipeline, binding any subsequent fine‑tuning updates to the same commitment structure. (3) Upon receiving a user query, the model generates an answer together with the associated binding value and a HyperNova zk‑SNARK proof that the answer was derived from the committed model and dataset. (4) The verifier checks the BLS signatures, validates the KZG commitments, and runs the HyperNova verification, thereby confirming—under zero‑knowledge—that the response originates from a model trained on the authorized data.

Security analysis formally defines data confidentiality, parameter confidentiality, and provenance integrity, and proves zero‑knowledge, binding strength, and computational soundness. Because the binding value links the query to the specific dataset commitment, an adversary cannot substitute unauthorized data or replay old commitments without being detected. The use of recursive zk‑SNARKs ensures that proof generation and verification scale sublinearly with model size, making the approach practical for real‑world LLMs.

Empirical evaluation uses Llama models ranging from 1 B to 8 B parameters. Proof generation averages 1.8 seconds, verification 1.5 seconds, and total end‑to‑end latency (including model inference) stays below 3.3 seconds across all model sizes. The overhead remains essentially constant as model depth grows, confirming the claimed sublinear scaling. Experiments also demonstrate the efficiency of aggregated BLS signatures when multiple datasets from different authorities are involved, a scenario common in healthcare where patient records, treatment guidelines, and regulatory documents must all be certified.

Limitations include the current focus on pre‑training or fine‑tuning phases; extending the framework to fully online learning would require additional protocol design. Moreover, proof generation still demands considerable GPU memory and compute, which may be prohibitive for low‑resource deployments. Future work aims to design lighter arithmetic circuits, explore hardware acceleration, and integrate retrieval‑augmented generation (RAG) so that external knowledge bases can be cryptographically linked to the same provenance guarantees.

In summary, ZKPROV delivers a practical, zero‑knowledge solution that lets regulators, auditors, and end‑users verify that an LLM’s outputs are grounded in authorized, certified datasets without leaking any sensitive information. This bridges the compliance‑privacy divide that has long hindered the adoption of powerful AI systems in high‑stakes domains.