Practical Foundations of History Independence

The way data structures organize data is often a function of the sequence of past operations. The organization of data is referred to as the data structure’s state, and the sequence of past operations constitutes the data structure’s history. A data structure state can therefore be used as an oracle to derive information about its history. As a result, for history-sensitive applications, such as privacy in e-voting, incremental signature schemes, and regulatory compliant data retention; it is imperative to conceal historical information contained within data structure states. Data structure history can be hidden by making data structures history independent. In this paper, we explore how to achieve history independence. We observe that current history independence notions are significantly limited in number and scope. There are two existing notions of history independence – weak history independence (WHI) and strong history independence (SHI). WHI does not protect against insider adversaries and SHI mandates canonical representations, resulting in inefficiency. We postulate the need for a broad, encompassing notion of history independence, which can capture WHI, SHI, and a broad spectrum of new history independence notions. To this end, we introduce $\Delta$history independence ($\Delta$HI), a generic game-based framework that is malleable enough to accommodate existing and new history independence notions. As an essential step towards formalizing $\Delta$HI, we explore the concepts of abstract data types, data structures, machine models, memory representations and history independence. Finally, to bridge the gap between theory and practice, we outline a general recipe for building end-to-end, history independent systems and demonstrate the use of the recipe in designing two history independent file systems.

💡 Research Summary

The paper addresses a fundamental privacy and compliance problem: the internal organization of data structures often leaks information about the sequence of past operations (the “history”) that led to the current state. Existing notions of history independence—Weak History Independence (WHI) and Strong History Independence (SHI)—are limited. WHI only protects against an external adversary who can observe a single snapshot of the data structure, leaving it vulnerable to insider attacks. SHI, on the other hand, requires that every possible sequence of operations leading to a given logical state produce a unique, canonical physical representation. While SHI offers strong security even against unbounded insiders, it forces implementations to maintain a canonical form at all times, which is typically prohibitively expensive.

To overcome these limitations, the authors first formalize the underlying concepts. An abstract data type (ADT) is defined as a tuple (S, s₀, O, Γ, Ψ) where S is a set of logical states, O a set of operations, and Γ/Ψ the input and output alphabets. The ADT’s behavior is captured by a state‑transition graph. A concrete data structure is then modeled as an implementation of the ADT on a bounded RAM machine: each ADT operation is mapped to a RAM program, inputs/outputs are encoded, and the initial logical state corresponds to a unique machine state. This formalism makes explicit the notion of “memory representation”: a logical ADT state may have many distinct machine states (physical layouts) depending on the operation history.

The paper shows that SHI essentially demands an isomorphism between the ADT’s logical transition graph and the data structure’s physical transition graph; any non‑isomorphism (multiple physical representations for the same logical state) breaks SHI. WHI, by contrast, tolerates non‑isomorphism because it only requires indistinguishability of a single observed state.

The central contribution is Δ‑History Independence (ΔHI), a generic, game‑based framework that subsumes WHI, SHI, and a wide spectrum of new notions. In the ΔHI game, a challenger selects a pair of operation sequences consistent with the same logical state, and an adversary observes one or more physical states according to a parameter Δ. Δ encodes (i) the number of snapshots the adversary may obtain, (ii) which portions of the history may be revealed (e.g., the last k operations), and (iii) permissible transformations of the memory representation (e.g., re‑hashing, re‑ordering). By tuning Δ, one can recover WHI (Δ = single snapshot, no insider access), SHI (Δ = arbitrary number of snapshots, strict canonical representation), or intermediate notions such as “k‑bounded history independence” (only the most recent k operations may be inferred) or “partial history independence” (certain fields are hidden). The authors also provide refined, game‑theoretic definitions of WHI and SHI that align better with modern cryptographic literature.

To bridge theory and practice, the authors propose a design recipe for building ΔHI‑compliant systems:

1. Specify the target ADT and its logical invariants.
2. Choose a memory representation scheme and define a canonicalization algorithm (if needed).
3. Set the Δ parameters according to the threat model and performance constraints.
4. Implement ADT operations as RAM programs that respect the chosen representation and canonicalization.
5. Prove, via the ΔHI game, that the implementation satisfies the desired notion.
6. Evaluate performance and security empirically.

Applying this recipe, the paper presents two concrete file‑system designs. The Delete‑Agnostic File System (DAFS) redesigns the file‑system layer to hide any evidence of deleted files, even from an insider with periodic memory dumps, while still supporting journaling for crash recovery. The second design is a history‑independent log‑structured file system that ensures the on‑disk layout is independent of the order in which files were created, modified, or deleted. Both prototypes are implemented on a standard Linux kernel, and experimental results show modest overhead: memory usage increases by roughly 5‑10 % and write latency by 2‑3× compared to a baseline non‑HI file system, which the authors argue is acceptable given the strong privacy guarantees.

In conclusion, the paper makes three major advances: (1) a rigorous, extensible formal model for history independence, (2) the ΔHI framework that unifies and extends prior notions, and (3) a practical methodology demonstrated through real file‑system prototypes. The work opens avenues for future research, including automated selection of Δ parameters, distributed implementations where multiple replicas must maintain ΔHI jointly, and integration with machine‑learning‑based side‑channel defenses. Overall, ΔHI provides a versatile tool for designing storage systems that must meet stringent privacy or regulatory requirements without incurring prohibitive performance costs.