A Survey of Bugs in AI-Generated Code

A Survey of Bugs in AI-Generated Code RUOFAN GAO, School of Mathematical and Computational Sciences, Massey University, New Zealand AMJED TAHIR, School of Mathematical and Computational Sciences, Massey University, New Zealand PENG LIANG, School of Computer Science, Wuhan University, China TEO SUSNJAK, School of Mathematical and Computational Sciences, Massey University, New Zealand FOUTSE KHOMH, SWAT Laboratory, Polytechnique Montréal, Canada Developers are widely using AI code-generation models, aiming to increase productivity and efficiency. However, there are also quality concerns regarding the AI-generated code. The generated code is produced by models trained on publicly available code, which are known to contain bugs and quality issues. Those issues can cause trust and maintenance challenges during the development process. Several quality issues associated with AI-generated code have been reported, including bugs and defects. However, these findings are often scattered and lack a systematic summary. A comprehensive review is currently lacking to reveal the types and distribution of these errors, possible remediation strategies, as well as their correlation with the specific models. In this paper, we systematically analyze the existing AI-generated code literature to establish an overall understanding of bugs and defects in generated code, providing a reference for future model improvement and quality assessment. We aim to understand the nature and extent of bugs in AI-generated code, and provide a classification of bug types and patterns present in code generated by different models. We also discuss possible fixes and mitigation strategies adopted to eliminate bugs from the generated code. CCS Concepts: • Software and its engineering →Software verification and validation; Automatic programming; Software testing and debugging; Empirical software validation. Additional Key Words and Phrases: AI-generated code, code compilation, bugs, software quality ACM Reference Format: Ruofan Gao, Amjed Tahir, Peng Liang, Teo Susnjak, and Foutse Khomh. 2025. A Survey of Bugs in AI-Generated Code. 1, 1 (December 2025), 51 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn 1 Introduction AI-powered code generation tools are revolutionizing software development. Code generation has been a significant beneficiary of the recent advancements in the key AI technology, Large Language Models (LLMs). These models, built using transfer learning to learn from existing code examples [137], can then generate code from natural-language descriptions or other programming contexts. AI code generation tools have become an invaluable resource for developers, enhancing productivity by automating coding tasks, suggesting code snippets for code generation [155], code Authors’ Contact Information: Ruofan Gao, r.gao@massey.ac.nz, School of Mathematical and Computational Sciences, Massey University, Palmerston North, New Zealand; Amjed Tahir, a.tahir@massey.ac.nz, School of Mathematical and Computational Sciences, Massey University, New Zealand; Peng Liang, liangp@whu.edu.cn, School of Computer Science, Wuhan University, China; Teo Susnjak, t.susnjak@massey.ac.nz, School of Mathematical and Computational Sciences, Massey University, New Zealand; Foutse Khomh, foutse.khomh@polymtl.ca, SWAT Laboratory, Polytechnique Montréal, Canada. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM XXXX-XXXX/2025/12-ART https://doi.org/10.1145/nnnnnnn.nnnnnnn , Vol. 1, No. 1, Article . Publication date: December 2025. arXiv:2512.05239v1 [cs.SE] 4 Dec 2025 2 Gao et al. completion [159], code translation [146] and even assisting in program debugging [79, 85] and repairing processes [110, 121]. Recent developer surveys show that the vast majority of developers are widely adopting code generation tools in their development [100]. Several code generation models have emerged, each offering distinct features and capabilities tailored to different stages of software development (such as the GPT family [98], Claude [7], Gemini [132], Llama [113], DeepSeek-Coder [54], among others. While AI code generation models have demonstrated remarkable performance in various code- related tasks, there are still some key challenges [24, 72]. The accuracy and correctness of the AI-generated code are still significant concerns [30], as such code often contains bugs [128, 157

This content is AI-processed based on open access ArXiv data.