Toward Fully Autonomous Driving: AI, Challenges, Opportunities, and Needs

Automated driving (AD) is promising, but the transition to fully autonomous driving is, among other things, subject to the real, ever-changing open world and the resulting challenges. However, research in the field of AD demonstrates the ability of artificial intelligence (AI) to outperform classical approaches, handle higher complexities, and reach a new level of autonomy. At the same time, the use of AI raises further questions of safety and transferability. To identify the challenges and opportunities arising from AI concerning autonomous driving functionalities, we have analyzed the current state of AD, outlined limitations, and identified foreseeable technological possibilities. Thereby, various further challenges are examined in the context of prospective developments. In this way, this article reconsiders fully autonomous driving with respect to advancements in the field of AI and carves out the respective needs and resulting research questions.

💡 Research Summary

The paper provides a comprehensive review of the current state of automated driving (AD) systems, focusing on how artificial intelligence (AI) is reshaping the architecture, capabilities, and challenges on the path toward fully autonomous (Level 5) driving. It begins by describing the prevailing modular, service‑oriented software stack in which distinct services—perception, prediction, planning, control—communicate through well‑defined interfaces. Within this framework, AI has been progressively integrated into individual modules, leveraging large public datasets and benchmark challenges to achieve superior performance in object detection, tracking, sensor fusion, and behavior prediction compared to classical rule‑based methods.

The authors then introduce a three‑level model of Situation Awareness (SA) – Level 1 (perception), Level 2 (comprehension), and Level 3 (projection) – and examine how AI currently addresses each level. Level 1 benefits from multimodal sensors and emerging neuromorphic event‑based cameras, yet real‑time processing and resource efficiency remain open problems. Level 2 requires contextual understanding and scene modeling; the paper illustrates with a school‑crossing scenario where identical free‑space maps demand different driving decisions based on nuanced context (time of day, presence of children). Human drivers still outperform AI in such nuanced reasoning, highlighting the need for AI systems capable of rapid, nuanced semantic interpretation. Level 3 involves forecasting the future motions of traffic participants. While learning‑based multimodal joint predictions and occupancy‑flow methods are state‑of‑the‑art, integrating these predictions tightly with planning (rather than treating them as a sequential step) is identified as a critical research direction to avoid overly aggressive or overly conservative behavior.

A major portion of the paper is devoted to safety assurance and explainability. In modular architectures, hard‑coded interfaces allow independent verification of services, but AI components behave as black boxes, making formal verification, runtime monitoring, and fault‑tolerant fallback mechanisms essential. The authors discuss the tension between performance gains and the growing need for transparency, especially as the field moves toward monolithic AI‑only designs that lack clear validation layers.

The paper also surveys emerging AI paradigms—zero‑shot, one‑shot, few‑shot learning, meta‑learning, and large foundation models (FMs) in vision and language—and argues that while these techniques promise better generalization to the open‑world conditions of real traffic, they introduce new challenges: massive data and compute requirements, domain‑shift safety validation, and potential brittleness when deployed on safety‑critical platforms.

In the final sections, the authors propose a research roadmap: (1) maintain a modular service‑oriented backbone while encapsulating AI modules behind formally verified interfaces; (2) develop continuous learning pipelines that span all three SA levels, enabling real‑time adaptation to novel scenarios; (3) combine formal methods, simulation‑based testing, and runtime monitors to create a multi‑layered safety shield; (4) explore model compression, knowledge distillation, and safety‑aware fine‑tuning to make large FMs feasible for automotive hardware; and (5) engage with standards bodies and regulators to define safety certification procedures for AI‑driven AD.

In conclusion, the authors assert that AI is indispensable for achieving fully autonomous driving, yet its deployment must be accompanied by rigorous safety engineering, explainability, robust generalization, and standardized validation frameworks. Only by addressing these intertwined technical and regulatory challenges can the promise of safer, more efficient mobility be realized.