The Richest Paradigm You're Not Using: Commercial Videogames at the Intersection of Human-Computer Interaction and Cognitive Science

Synthesizing from Corbett and Munneke (2025), who demonstrated that questions originating in human-computer interaction (HCI) and game design can be answered through the theoretical toolkit of cognitive science, this perspective argues that commercial videogames represent a largely underutilized research environment at the intersection of these two fields. Cognitive science has long relied on carefully controlled laboratory paradigms to study perception, attention, and executive functioning, raising persistent questions about ecological validity. HCI, by contrast, has spent decades developing methods for studying behavior in rich, complex, interactive environments, but has been less concerned with what that behavior reveals about underlying cognitive mechanisms. Commercial videogames sit precisely at this intersection. They are cognitively demanding by design, motivating by nature, and consistent enough across players to support systematic behavioral comparison. The affordance structure of a game does the work that experimental manipulations typically require of the researcher, instantiating cognitive demands that are genuine, sustained, and meaningful to the player. We argue that perception, attention, and executive functioning can be meaningfully studied within commercial games using a minimal observational toolkit of screen recording, eye tracking, and behavioral timing. We propose an affordance-cognition mapping framework as a systematic basis for game selection and research design and offer practical methodological recommendations for researchers wishing to work in this space.

💡 Research Summary

The paper puts forward a compelling argument that commercial video games constitute a rich, under‑exploited research environment situated at the intersection of cognitive science and human‑computer interaction (HCI). Traditional cognitive science relies on tightly controlled laboratory paradigms—flanker tasks, spatial cueing, N‑back—to isolate perceptual, attentional, and executive processes. While these paradigms yield high internal validity, they suffer from limited ecological validity: they strip away the complexity, motivation, and real‑world consequences that shape cognition in everyday life. HCI, on the other hand, has spent decades developing methods for studying behavior in complex, interactive systems—interaction logging, eye tracking, think‑aloud protocols—but has historically been less interested in what those behaviors reveal about underlying cognitive architecture.

Commercial games sit precisely where these two traditions can complement each other. By design, games embed sustained, varied cognitive demands within their affordance structure: visual clutter, time pressure, multitarget tracking, rapid task switching, and long‑term planning are all baked into the gameplay mechanics of titles ranging from fast‑paced first‑person shooters to deep real‑time strategy games. Consequently, a game itself can serve as the experimental manipulation; researchers need not construct artificial stimuli because the game’s design already imposes the desired cognitive load.

The authors propose an “affordance‑cognition mapping” framework that systematically links specific game elements (maps, enemies, items, mission objectives) to cognitive constructs such as exogenous attention, working memory, and prospective planning. This mapping guides the selection of games or levels that naturally elicit the target processes, allowing researchers to study perception, attention, and executive function with a minimal observational toolkit: screen recording, eye tracking, and behavioral timing derived from in‑game events. These tools are non‑invasive, require no access to the game engine’s internal logs, and can be applied across platforms.

Five key properties make commercial games especially suitable for cognitive research: (1) ecological validity—games are intentionally engaging and cognitively demanding; (2) reproducibility—identical levels or maps present the same stimuli to every player; (3) affordance‑driven manipulation—design choices directly instantiate experimental variables; (4) lack of experimental intent—players’ behavior is driven by game logic rather than researcher hypotheses, avoiding demand characteristics; and (5) rich individual‑difference data—variations in expertise, strategy, and play style are continuously expressed and can serve as sensitive indices of cognitive ability.

Methodologically, the paper outlines concrete steps: pilot testing to confirm that a chosen game indeed taxes the intended cognitive function; synchronized screen capture and eye‑tracking to code decision latencies, error patterns, and gaze allocation; extraction of behavioral timestamps aligned with game events (e.g., enemy spawn, objective completion); and optional supplementary measures (self‑report motivation, physiological arousal) to control for affective influences. The authors also discuss practical challenges such as game updates altering experimental conditions and the need for careful version control.

Importantly, the authors differentiate their approach from “gamified” paradigms, where researchers redesign classic cognitive tasks with game‑like elements. In gamification, the experimental intent is baked into the design; in the present proposal, the intent resides solely in the researcher’s interpretation of naturally occurring game demands. This distinction positions commercial games as a “natural experiment” platform that can reveal how cognitive mechanisms operate under authentic, high‑stakes conditions.

The paper concludes by urging the cognitive‑science community to embrace commercial games as a complementary research paradigm. By leveraging HCI’s robust observational methods and cognitive science’s theoretical models, researchers can achieve both the precision of laboratory work and the realism of real‑world behavior. Future directions include expanding to diverse genres and platforms, developing automated coding pipelines using machine learning, and integrating multimodal data (e.g., neurophysiology) to build more comprehensive models of cognition in interactive digital environments.