On the Influence of Artificial Intelligence on Human Problem-Solving: Empirical Insights for the Third Wave in a Multinational Longitudinal Pilot Study

This article presents the results and their discussion for the third wave (with n=23 participants) within a multinational longitudinal study that investigates the evolving paradigm of human-AI collaboration in problem-solving contexts. Building upon previous waves, our findings reveal the consolidation of a hybrid problem-solving culture characterized by strategic integration of AI tools within structured cognitive workflows. The data demonstrate near-universal AI adoption (95.7% with prior knowledge, 100% ChatGPT usage) primarily deployed through human-led sequences such as “Think, Internet, ChatGPT, Further Processing” (39.1%). However, this collaboration reveals a critical verification deficit that escalates with problem complexity. We empirically identify and quantify two systematic epistemic gaps: a belief-performance gap (up to +80.8 percentage points discrepancy between perceived and actual correctness) and a proof-belief gap (up to -16.8 percentage points between confidence and verification capability). These findings, derived from behavioral data and problem vignettes across complexity levels, indicate that the fundamental constraint on reliable AI-assisted work is solution validation rather than generation. The study concludes that educational and technological interventions must prioritize verification scaffolds (including assumption documentation protocols, adequacy criteria checklists, and triangulation procedures) to fortify the human role as critical validator in this new cognitive ecosystem.

💡 Research Summary

The paper reports findings from the third wave of a multinational longitudinal pilot study on human‑AI collaboration in problem‑solving, involving 23 participants. Near‑universal AI adoption was observed: 95.7 % had prior AI knowledge and 100 % used ChatGPT. The most common workflow, “Think → Internet → ChatGPT → Further Processing,” accounted for 39.1 % of documented sequences, illustrating a hybrid problem‑solving culture where humans retain control over framing, decomposition, and verification while leveraging AI for rapid idea generation.

Crucially, the study uncovers two systematic epistemic gaps that widen with task complexity. The belief‑performance gap shows participants over‑estimate the correctness of AI‑generated answers by up to +80.8 percentage points, reflecting a strong automation bias and susceptibility to LLM “hallucinations.” The proof‑belief gap reveals a mismatch between confidence in an answer and the ability to verify it, with confidence scores falling up to –16.8 percentage points below actual verification capability. These gaps indicate that the primary bottleneck in AI‑assisted work is not generation but validation.

The authors situate their findings within the “extended mind” thesis and cognitive load theory, proposing an AI‑Human Problem‑Solving Interaction Model comprising seven stages: problem framing, adequacy criteria definition, verification planning, resource consultation (including AI), triangulation, documentation/audit, and decision delivery. A “task‑class overlay” adjusts verification intensity according to problem difficulty (minimal checks for simple tasks, dual‑source triangulation for moderate tasks, full replication and sensitivity analysis for complex tasks).

Four competency clusters emerge: (1) decomposition proficiency, (2) strategic resource orchestration, (3) verification discipline, and (4) documentation practices. The study highlights verification as the most critical bottleneck and recommends targeted educational scaffolds—assumption documentation protocols, adequacy‑criteria checklists, and systematic triangulation procedures—as well as technical supports such as verification‑enhanced interfaces and transparent metadata.

In sum, while AI dramatically boosts efficiency in the solution‑generation phase, reliable outcomes depend on robust human verification. Future curricula, organizational policies, and system designs must prioritize verification scaffolding to sustain a safe, effective human‑AI cognitive ecosystem.