Critical Incidents for Technology Enhanced Learning in Vocational Education and Training - Observations from the field of mechanical engineering

In this study, observations of the Vocational Education and Training (VET) in mechanical engineering companies are carried out. A Learning Management System (LMS) had been developed for the assistance in solving typical task structures, that are used for a period of three and a half years in the apprenticeship. In this study, the Critical Incident Technique (CIT) is applied for the observations. For the subsequent analysis, a classification of incidents is performed. The most important incidents as well as conclusions for Technical Enhanced Learning (TEL) in similar domains are presented.

💡 Research Summary

This paper presents an empirical investigation of a mobile Learning Management System (LMS) designed to support the three‑and‑a‑half‑year apprenticeship in mechanical engineering within the German dual vocational education and training (VET) system. The authors applied the Critical Incident Technique (CIT) to capture decisive events that occurred while apprentices performed typical work tasks in real‑world companies. Over a period from February to October 2017, fifteen task executions were observed across ten training companies, yielding a total of 292 recorded incidents. Each incident was classified by work phase (preparation, documentation, setup, programming, manufacturing, quality control), by its estimated impact on the overall apprenticeship goal (ranging from “strongly negative” to “strongly positive”), and by three overarching dimensions: Process, Artifact, and Social interaction.

The mobile LMS was built to accompany apprentices through six sequential work phases: (1) analysis of mechanical drawings, (2) preparation of machines and cutting tools, (3) documentation via work plans, (4) setup (measuring tools, loading the machine), (5) programming, (6) manufacturing, and (7) quality control. The system integrates domain‑expert curated content—excerpts from the Mechanical and Metal Trades Handbook, 3D visualisations of technical drawings, digital work plans, and searchable high‑quality learning objects—into a task‑oriented interface. Trainers can adapt task hierarchies, sub‑task sequences, and associated resources to local conditions, preserving face‑to‑face communication that trainers consider essential.

The CIT procedure involved (1) defining the general aim (“autonomous production of components and sub‑assemblies”), (2) preparing observation forms that captured context (component specification, CNC vs. conventional manufacturing, LMS usage), (3) recording for each incident the work phase, the observed activity, an estimated positive/negative effect, and optional explanatory notes, and (4) analysing the data using a custom classification model. The model groups incidents into three top‑level categories: Process (temporal sequences and state changes), Artifact (physical or digital objects such as drawings, hardware, software, data streams), and Social (interactions among autonomous subjects, especially trainer‑apprentice dialogue). Incidents could be assigned to multiple categories, allowing the authors to capture the multifaceted nature of real‑world problem solving.

Quantitative results show a clear distribution of incidents across phases. Manufacturing generated the most incidents (89), with a predominance of positive effects (22 strongly/weakly positive) and relatively few negative ones, indicating that the LMS’s support during CNC‑driven production is effective. Conversely, preparation (45 incidents) and documentation (60 incidents) displayed a higher proportion of negative or neutral effects, suggesting that early‑stage support—especially for setting up machines and creating work plans—needs refinement. The setup phase exhibited the fewest incidents (17) and a balanced impact profile, implying that the LMS already provides adequate assistance at this stage.

Qualitative analysis of the Process dimension revealed that many incidents involved transitions between phases (e.g., from programming to manufacturing) where the LMS’s navigation aids and real‑time feedback were crucial. Artifact‑related incidents highlighted the importance of integrating high‑fidelity digital drawings and interactive sheets; apprentices who accessed these resources reported smoother execution and fewer errors. Social incidents underscored the continued relevance of face‑to‑face trainer feedback; even with a sophisticated mobile platform, apprentices relied on immediate clarification and mentorship, especially when encountering ambiguous instructions or unexpected machine behaviour.

From these findings the authors derive three practical design principles for TEL in vocational settings: (1) Alignment of workflow and learning objectives – the LMS must map each work phase to explicit competency goals, ensuring that content is directly applicable to the task at hand; (2) Granular, phase‑specific content – learning objects should be broken down to match the cognitive load of each sub‑task, providing just‑in‑time information without overwhelming the apprentice; (3) Preservation of social support mechanisms – a hybrid model that blends digital self‑directed learning with structured trainer interaction yields the highest perceived usefulness.

The paper also discusses broader implications for digital transformation in VET. Despite the availability of the mobile LMS, traditional paper‑based spreadsheets and textbooks remain entrenched, indicating cultural and infrastructural barriers to full digitisation. The authors recommend systematic change management, including trainer up‑skilling, incremental rollout of digital tools, and continuous feedback loops to adapt the system to evolving workplace practices.

In conclusion, this study demonstrates that the Critical Incident Technique can effectively surface both strengths and weaknesses of a technology‑enhanced learning system in a complex, real‑world apprenticeship environment. The mobile LMS proved particularly valuable during manufacturing and quality‑control phases, while early‑stage phases require further UI/UX refinement and richer instructional scaffolding. The three‑dimensional classification (Process, Artifact, Social) offers a reusable analytical framework for future TEL research across other engineering and manufacturing domains. Policymakers, educators, and developers can leverage these insights to design more integrated, learner‑centred, and socially aware digital learning solutions for vocational education.