Symbolic Backwards-Reachability Analysis for Higher-Order Pushdown Systems

Higher-order pushdown systems (PDSs) generalise pushdown systems through the use of higher-order stacks, that is, a nested “stack of stacks” structure. These systems may be used to model higher-order programs and are closely related to the Caucal hierarchy of infinite graphs and safe higher-order recursion schemes. We consider the backwards-reachability problem over higher-order Alternating PDSs (APDSs), a generalisation of higher-order PDSs. This builds on and extends previous work on pushdown systems and context-free higher-order processes in a non-trivial manner. In particular, we show that the set of configurations from which a regular set of higher-order APDS configurations is reachable is regular and computable in n-EXPTIME. In fact, the problem is n-EXPTIME-complete. We show that this work has several applications in the verification of higher-order PDSs, such as linear-time model-checking, alternation-free mu-calculus model-checking and the computation of winning regions of reachability games.

💡 Research Summary

The paper addresses the backwards‑reachability problem for higher‑order alternating pushdown systems (APDS), a powerful extension of higher‑order pushdown systems (PDS) that incorporates alternation. A higher‑order PDS uses an n‑store, i.e., a nested “stack of stacks” structure, where each level supports push, pop, and duplication operations. While the forward‑reachability set of a higher‑order PDS is generally non‑regular, previous work showed that for order‑one PDS the set of configurations that can reach a given regular set is itself regular and can be computed by a saturation algorithm on finite automata. For higher‑order systems, Bouajjani and Meyer introduced nested‑store automata to handle order‑two systems with a single control state, but extending this to multiple control states and to the alternating setting proved non‑trivial.

The authors overcome these obstacles by employing alternating automata as the symbolic representation of configuration sets. In an alternating automaton, an existential player (Eloise) selects a transition set, and a universal player (Abelard) then chooses a concrete successor state, mirroring the alternation in APDS. The main technical contribution is a “cascading fixed‑point” construction: starting from the highest order n, the algorithm repeatedly adds all transitions that are enabled by the current transition relation, updates the labels of existing transitions, and freezes the state set at that order once no new transitions appear. The process then proceeds to order n‑1, re‑applying the same steps. Because each order’s transition addition is bounded (the number of new transitions is finite) and the alphabet at order‑1 is finite, the whole procedure terminates after a finite number of cascades. The resulting alternating automaton precisely accepts the set Pre* (C_init), i.e., all configurations that can reach the regular initial set C_init.

Complexity analysis shows that the algorithm runs in n‑EXPTIME time and is n‑EXPTIME‑complete, improving on the earlier doubly‑exponential bound for non‑alternating higher‑order processes. The use of alternating automata also resolves the difficulty of naïvely taking a product of a multi‑automaton (representing control states) with a nested‑store automaton, which would otherwise generate infinitely many new states and break termination arguments.

The paper then demonstrates three major applications of this result:

1. Linear‑time (LTL) model checking – By translating an LTL formula into a Büchi automaton and intersecting it with the APDS transition system, the backwards‑reachability algorithm computes the set of configurations that violate the specification, yielding a decision procedure for LTL model checking of higher‑order PDS.

2. Alternation‑free μ‑calculus model checking – The μ‑calculus formula is unfolded into a finite set of sub‑formulas; each sub‑formula’s satisfaction set is obtained via a backwards‑reachability computation, and the results are combined according to the μ‑calculus semantics, providing a complete model‑checking algorithm for this fragment.

3. Reachability games – By encoding a two‑player game on a higher‑order PDS as an APDS where Eloise’s choices correspond to existential transitions and Abelard’s to universal ones, the winning region for Eloise is exactly the backwards‑reachability set of the target configurations. Hence the algorithm computes winning regions for reachability (and, via known reductions, parity) games on higher‑order pushdown graphs.

The authors also discuss the relationship between their notion of regularity (BM‑regularity, based on nested‑store automata) and the alternative C‑regularity introduced by Carayol, noting that their results apply to BM‑regular sets while C‑regular sets are MSO‑definable and may be strictly larger.

In conclusion, the paper provides a robust symbolic technique for computing backwards‑reachability sets of higher‑order alternating pushdown systems, establishes optimal n‑EXPTIME complexity, and leverages this technique to solve important verification problems such as LTL and μ‑calculus model checking and the analysis of higher‑order pushdown games. The work bridges the gap between higher‑order recursion schemes, Caucal’s hierarchy of infinite graphs, and practical verification tools for higher‑order programs.