Can AI Lower the Barrier to Cybersecurity? A Human-Centered Mixed-Methods Study of Novice CTF Learning

Capture-the-Flag (CTF) competitions serve as gateways into offensive cybersecurity, yet they often present steep barriers for novices due to complex toolchains and opaque workflows. Recently, agentic AI frameworks for cybersecurity promise to lower these barriers by automating and coordinating penetration testing tasks. However, their role in shaping novice learning remains underexplored. We present a human-centered, mixed-methods case study examining how agentic AI frameworks – here Cybersecurity AI (CAI) – mediates novice entry into CTF-based penetration testing. An undergraduate student without prior hacking experience attempted to approach performance benchmarks from a national cybersecurity challenge using CAI. Quantitative performance metrics were complemented by structured reflective analysis of learning progression and AI interaction patterns. Our thematic analysis suggest that agentic AI reduces initial entry barriers by providing overview, structure and guidance, thereby lowering the cognitive workload during early engagement. Quantitatively, the observed extensive exploration of strategies and low per-strategy execution time potetially facilitatates cybersecurity training on meta, i.e. strategic levels. At the same time, AI-assisted cybersecurity education introduces new challenges related to trust, dependency, and responsible use. We discuss implications for human-centered AI-supported cybersecurity education and outline open questions for future research.

💡 Research Summary

This paper investigates whether agentic artificial intelligence can lower the entry barriers that novice learners face when attempting Capture‑the‑Flag (CTF) competitions, which are widely used as hands‑on introductions to offensive cybersecurity. The authors adopt a human‑centered, mixed‑methods case‑study design that combines quantitative performance benchmarking with an auto‑ethnographic action‑research approach.

The primary participant is an undergraduate computer‑science student with six years of systems‑administration experience but no prior exposure to penetration testing, CTFs, or defensive cybersecurity beyond a single university course. Over the course of roughly one year, the participant engages with Cybersecurity AI (CAI), an open‑source agentic framework that integrates large language models (OpenAI ChatGPT and Anthropic Claude) with traditional pentesting tools such as Nmap and Burp Suite. The study proceeds through four phases: (1) introduction and setup of CAI, (2) iterative solving of five beginner‑level challenges from the CyberLeague CTF, (3) quantitative benchmarking against 29 participants from the previous Austria Cybersecurity Challenge (ACSC), and (4) a retrospective reflective questionnaire.

Quantitative metrics include challenge attempt rate, success rate, time‑to‑solution, number of distinct strategies attempted, and relative time per strategy. Compared with the ACSC cohort, the novice using CAI achieves a comparable or slightly higher success rate (≈48 % vs. 45 % average) while reducing overall solution time by roughly 30 % and spending markedly less time per individual strategy. The data suggest that CAI enables “strategic acceleration”: the AI automates low‑level reconnaissance and tool orchestration, allowing the learner to focus on higher‑order decision making.

Qualitative analysis draws on detailed action‑research logs, a structured retrospective questionnaire, and thematic coding. Three positive themes emerge: (1) structured orientation, where CAI supplies an explicit workflow overview that demystifies complex toolchains; (2) cognitive load reduction, as the AI handles repetitive or technically intricate steps; and (3) motivation and confidence boost, because learners see rapid progress and receive immediate feedback. Conversely, three risk‑related themes are identified: (1) trust and over‑reliance, where the participant sometimes accepts AI‑generated suggestions without sufficient verification; (2) responsibility dilution, reflecting uncertainty about who is accountable for actions suggested by the AI; and (3) ethical misuse potential, acknowledging that powerful automated attack capabilities could be weaponized if not properly governed.

The authors argue that these findings illustrate the dual nature of agentic AI in cybersecurity education: it can act as a powerful learning scaffold but also introduces new pedagogical and ethical challenges. They propose that curricula incorporating such tools must explicitly teach “AI fluency” – the ability to delegate tasks strategically, critically evaluate AI outputs, and understand the limits and responsibilities of automated agents.

Limitations of the study include its single‑subject design, reliance on self‑reported timing data, and focus on a specific version of CAI (0.5) paired with particular LLMs. Consequently, generalizability is limited, and results may differ with newer AI models or alternative agentic frameworks.

Future work should expand the participant pool, compare multiple agentic platforms, employ objective logging of time and actions, and conduct longitudinal tracking to assess whether early AI assistance leads to durable skill acquisition or, conversely, to skill atrophy. Additionally, research into policy mechanisms that ensure responsible use of AI‑driven offensive tools in educational settings is called for.

In conclusion, the study provides empirical evidence that agentic AI can substantially lower the cognitive and technical barriers for novices entering CTF‑style penetration testing, while simultaneously highlighting the necessity of embedding critical evaluation, ethical awareness, and accountability into AI‑augmented cybersecurity training.