Automated Formalization of Probabilistic Requirements from Structured Natural Language

Integrating autonomous and adaptive behavior into software-intensive systems presents significant challenges for software development, as uncertainties in the environment or decision-making processes must be explicitly captured. These challenges are amplified in safety- and mission-critical systems, which must undergo rigorous scrutiny during design and development. Key among these challenges is the difficulty of specifying requirements that use probabilistic constructs to capture the uncertainty affecting these systems. To enable formal analysis, such requirements must be expressed in precise mathematical notations such as probabilistic logics. However, expecting developers to write requirements directly in complex formalisms is unrealistic and highly error-prone. We extend the structured natural language used by NASA’s Formal Requirement Elicitation Tool (FRET) with support for the specification of unambiguous and correct probabilistic requirements, and develop an automated approach for translating these requirements into logical formulas. We propose and develop a formal, compositional, and automated approach for translating structured natural-language requirements into formulas in probabilistic temporal logic. To increase trust in our formalizations, we provide assurance that the generated formulas are well-formed and conform to the intended semantics through an automated validation framework and a formal proof. The extended FRET tool enables developers to specify probabilistic requirements in structured natural language, and to automatically translate them into probabilistic temporal logic, making the formal analysis of autonomous and adaptive systems more practical and less error-prone.

💡 Research Summary

The paper addresses a critical gap in the engineering of autonomous and adaptive systems: the specification and formal analysis of requirements that involve probabilistic constructs. While NASA’s Formal Requirement Elicitation Tool (FRET) already enables engineers to write unambiguous requirements in a structured natural‑language format called FRETish, the original version supports only deterministic properties and cannot express uncertainty. To bridge this gap, the authors extend the FRETish language with two new elements: a probability field that allows the explicit declaration of probability bounds (e.g., “WITH PROBABILITY ≥ 0.95”) and a new condition keyword (“UPON”) that enriches the triggering logic. These extensions increase the number of possible template keys from 160 to 560, dramatically expanding the expressive power of the language.

The core contribution is a compositional, automated translation pipeline that maps each extended FRETish template to a formula in Probabilistic Computation Tree Logic Star (PCTL*). The pipeline consists of a pre‑computation phase, where a “Probabilistic Formalizer” generates a JSON cache of all template‑to‑formula mappings, and a user interaction phase, where a “Probabilistic Instantiator” extracts variables from the user’s input, retrieves the appropriate PCTL* skeleton, and instantiates it. The translation leverages the Structured Assertion Language for Temporal Logic (SALT) to simplify embedded Linear Temporal Logic (LTL) cores before emitting the final property in the PRISM property language, ensuring compatibility with widely used model‑checking tools such as PRISM, Storm, and ISCASMC.

To increase confidence in the generated specifications, the authors introduce an automated validation framework. This framework comprises three components: (1) a Probabilistic Model Generator that builds a Markov‑chain model reflecting the semantics of the requirement, (2) a Probabilistic Oracle that mathematically interprets the template and produces the expected truth value for a given model, and (3) a Probabilistic Semantics Evaluator that runs the generated PCTL* formula on the model using PRISM and checks for agreement with the oracle. The framework thus provides an oracle‑based proof that the translation is both syntactically well‑formed and semantically faithful.

The approach is evaluated on a corpus of 334 requirements drawn from research literature, public repositories, and an industry case study with RTX (autonomous taxiing, take‑off, and landing). The evaluation addresses three research questions: (RQ1) the expressiveness of the extended language, (RQ2) the success rate of fully automated translation and validation, and (RQ3) practical usefulness in a real‑world setting. Results show that over 96 % of the requirements are translated without manual intervention, and the validation framework confirms semantic correctness for the same proportion. The RTX case study demonstrates a ~70 % reduction in time required to produce formally verified properties compared with manual encoding, and no verification errors were observed.

The paper also discusses related work, highlighting that while several pattern‑based or restricted‑natural‑language tools (e.g., ProProST, PSPWizard, QUARTET) support probabilistic specifications, none combine a free‑form structured language with automated compositional translation and systematic validation. Threats to validity are identified, including the reliance on a finite set of templates (which may need extension for more exotic constructs) and the assumption that the underlying probabilistic model can be represented as a discrete Markov chain. The authors note challenges in integrating large language models (LLMs) for bridging unrestricted natural language and FRETish, suggesting future research directions.

In conclusion, the extended FRET tool provides a practical, trustworthy bridge from human‑readable probabilistic requirements to machine‑checkable formal specifications. By automating both translation and validation, it reduces the expertise barrier for engineers, mitigates human error, and facilitates rigorous analysis of safety‑critical autonomous systems. Future work will focus on (1) automated generation and evolution of new templates, (2) support for continuous probability distributions, and (3) tighter integration with LLM‑based natural‑language processing to further streamline the requirements engineering workflow.