Requirements Engineering Challenges in Building AI-Based Complex Systems

This paper identifies and tackles the challenges of the requirements engineering discipline when applied to development of AI-based complex systems. Due to their complex behaviour, there is an immanent need for a tailored development process for such systems. However, there is still no widely used and specifically tailored process in place to effectively and efficiently deal with requirements suitable for specifying a software solution that uses machine learning. By analysing the related work from software engineering and artificial intelligence fields, potential contributions have been recognized from agent-based software engineering and goal-oriented requirements engineering research, as well as examples from large product development companies. The challenges have been discussed, with proposals given how and when to tackle them. RE4AI taxonomy has also been outlined, to inform the tailoring of development process.

💡 Research Summary

The paper addresses a critical gap in software engineering: the lack of a dedicated requirements engineering (RE) process for AI‑based complex systems. While machine learning (ML) and deep learning (DL) have become central to modern software, the authors argue that traditional RE—elicitation, analysis, specification, validation, and traceability—does not adequately handle the unique characteristics of AI components. These characteristics include heavy data dependence, model opacity, continuous learning cycles, and intricate interactions between data, models, and the surrounding environment.

Through a systematic literature review, the authors identify three layers of challenges. At the data layer, issues such as large‑scale data acquisition, labeling costs, imbalance, bias, and evolving data distributions create uncertainty in requirement definition. At the model layer, the “black‑box” nature of ML models hampers verification, version control, and impact analysis, while models often act as hidden consumers of other system components, leading to unintended feedback loops. At the system layer, integration with hardware, real‑time performance constraints, and regulatory/ethical requirements (e.g., privacy, fairness) add further complexity.

To address these challenges, the paper draws on two established research streams. First, Agent‑Based Software Engineering (ABSE) treats each subsystem as an autonomous agent, providing a natural way to model interactions, encapsulate decision‑making, and align subsystem behavior with system goals. Agents can be classified from simple reflex agents to goal‑oriented and learning agents, the latter being especially relevant for AI components. Second, Goal‑Oriented Requirements Engineering (GORE) introduces a hierarchical goal model that maps high‑level system objectives to concrete requirements, enabling automated impact analysis when requirements evolve.

The authors synthesize these ideas into a new taxonomy called RE4AI (Requirements Engineering for AI). RE4AI structures AI development into three intertwined dimensions—data, model, and system—and maps each dimension to specific roles (data scientist, data engineer, business analyst, cognitive architect, etc.) and activities (data labeling, pipeline automation, model validation, goal tracking). Table 1 in the paper summarizes the taxonomy, linking challenges such as data entanglement, undeclared consumers, scalability, and debugging to corresponding RE activities and mitigation strategies.

The paper also surveys practical experiences from large organizations that have attempted to embed AI projects into existing agile processes. Common pain points include insufficient end‑to‑end pipeline support, difficulty debugging and interpreting models, and unclear regulatory guidance. The authors demonstrate how RE4AI’s checklist‑driven approach and goal‑based validation can pre‑emptively surface these issues, allowing teams to allocate resources (e.g., dedicated data engineers, automated testing frameworks) more effectively. Additionally, the authors discuss how concepts from Software Product Line Engineering (SPLE) and Model‑Driven RE (MDR‑E) can be adapted for AI, supporting reuse of AI components across product families and systematic management of model‑driven requirements.

In conclusion, the paper makes three key contributions: (1) a comprehensive taxonomy of RE challenges specific to AI‑based complex systems; (2) a synthesis of ABSE and GORE techniques to create a goal‑oriented, agent‑centric RE framework; and (3) practical guidance on integrating RE4AI into existing development lifecycles, highlighting when and how to apply each RE activity. By providing a structured, multi‑layered approach, RE4AI aims to improve requirement traceability, validation, and compliance for AI systems, thereby accelerating their safe and effective deployment in industry.