Learning to Code with Context: A Study-Based Approach

Learning to Code with Context: A Study-Based Approach Uwe M. Borghoff, Mark Minas and Jannis Schopp Institute for Software Technology, Department of Computer Science, University of the Bundeswehr Munich, Werner-Heisenberg-Weg 39, 85579 Neubiberg, Germany. *Corresponding author(s). E-mail(s): uwe.borghoff@unibw.de; mark.minas@unibw.de; jannis.schopp@unibw.de; Abstract The rapid emergence of generative AI tools is transforming the way software is developed. Consequently, software engineering education must adapt to ensure that students not only learn traditional development methods but also under- stand how to meaningfully and responsibly use these new technologies. In particular, project-based courses offer an effective environment to explore and evaluate the integration of AI assistance into real-world development practices. This paper presents our approach and a user study conducted within a univer- sity programming project in which students collaboratively developed computer games. The study investigates how participants used generative AI tools through- out different phases of the software development process, identifies the types of tasks where such tools were most effective, and analyzes the challenges students encountered. Building on these insights, we further examine a repository-aware, locally deployed large language model (LLM) assistant designed to provide project-contextualized support. The system employs Retrieval-Augmented Gen- eration (RAG) to ground responses in relevant documentation and source code, enabling qualitative analysis of model behavior, parameter sensitivity, and com- mon failure modes. The findings deepen our understanding of context-aware AI support in educational software projects and inform future integration of AI-based assistance into software engineering curricula. Keywords: software development project course, software engineering education, generative AI, repository-aware LLM, retrieval-augmented generation, qualitative analysis 1 arXiv:2512.05242v1 [cs.SE] 4 Dec 2025 1 Introduction The programming project at our university is a software development course in which group work in a software development team is a crucial aspect. Similar courses are probably part of almost all bachelor programs in computer science worldwide. In our programming project, about seven students work together in a team to create a computer game using a structured development process. The main goal is for the par- ticipants to gain experience in software development in a team with all the necessary activities, including planning, programming, testing and documentation, as well as the necessary soft skills such as teamwork and communication. The general availability of generative AI tools, such as ChatGPT and GitHub’s Copilot (Mastropaolo et al. 2023), also has an impact on the way software is devel- oped. So far, such AI tools are not part of the curriculum of our programming project. When asked personally, our students naturally say that they use them frequently, but are often dissatisfied with their answers. To counteract this uncontrolled use of AI and to train our students in its responsible use, we want to provide participants in our programming project with an AI tutor in the coming years. The AI tutor will be chat-based and will support students in software development and, in particular, in the creation of programs. To counteract the much-discussed effect of skill degrada- tion (Crowston and Bolici 2025; Macnamara et al. 2024), however, it will not simply present ready-made solutions, but will provide hints for solutions like a human tutor and guide students to solve problems independently. In recent months, we have followed an empirical approach to plan the implemen- tation and use of the AI tutor. This paper1 describes our multi-stage approach and the results we have achieved with it. In a first step, we empirically collected data on how our students use freely available AI tools. To this end, we conducted a user study among the participants in last year’s programming project from October to December 2024. From the outset, they were asked to understand freely available AI tools as useful aids and to use them as such, e.g., for coding or documenting. At the end of the programming project, we conducted a study in the form of a structured questionnaire that asked about their experiences with AI tools in the various phases and in relation to the various activities. As a result of this user study, we were able to identify several shortcomings of the freely available AI tools in this application domain. Among other things, they are not familiar with the documents and source code of the sample program that are made available to the participants and that they need to familiarize themselves with. The AI tools were therefore almost never able to provide helpful answers when asked questions by students in this context. To counteract this, our own AI tutor must therefore have access to relevant d

This content is AI-processed based on open access ArXiv data.