Exploring Help Facilities in Game-Making Software

Help facilities have been crucial in helping users learn about software for decades. But despite widespread prevalence of game engines and game editors that ship with many of today’s most popular games, there is a lack of empirical evidence on how help facilities impact game-making. For instance, certain types of help facilities may help users more than others. To better understand help facilities, we created game-making software that allowed us to systematically vary the type of help available. We then ran a study of 1646 participants that compared six help facility conditions: 1) Text Help, 2) Interactive Help, 3) Intelligent Agent Help, 4) Video Help, 5) All Help, and 6) No Help. Each participant created their own first-person shooter game level using our game-making software with a randomly assigned help facility condition. Results indicate that Interactive Help has a greater positive impact on time spent, controls learnability, learning motivation, total editor activity, and game level quality. Video Help is a close second across these same measures.

💡 Research Summary

The paper “Exploring Help Facilities in Game‑Making Software” presents a large‑scale empirical investigation of how different types of help resources affect novice users creating a first‑person‑shooter (FPS) level in a custom game‑making environment called GameWorld. The authors begin by reviewing 85 existing game‑making tools and find that text documentation is ubiquitous, video tutorials appear in roughly half of the tools, while interactive tutorials and intelligent agents are rare. This survey informs the design of six experimental conditions: (1) Text Help, (2) Interactive Help, (3) Intelligent Agent Help, (4) Video Help, (5) All Help (a combination of the four), and (6) No Help.

A between‑subjects experiment was conducted on Amazon Mechanical Turk with 1,646 participants who were randomly assigned to one of the six conditions. Each participant was asked to build their own FPS level using the same set of assets and controls, differing only in the help modality presented. The study measured six outcome variables: (i) total time spent in the editor, (ii) self‑reported learnability of the controls, (iii) learning motivation, (iv) cognitive load (NASA‑TLX), (v) total editor actions (e.g., object placements, script edits), and (vi) expert‑rated quality of the final level.

Statistical analysis (ANOVA with Tukey post‑hoc tests) revealed that Interactive Help produced the most consistently positive effects across all metrics. Participants with interactive help spent the most time in the editor, reported the highest control learnability, showed the strongest motivation, experienced the lowest cognitive load, performed the greatest number of editing actions, and created the highest‑rated levels. Video Help performed almost as well, especially excelling in time spent, motivation, cognitive load reduction, and final level quality. Text Help showed a modest benefit only in reducing cognitive load compared with No Help, but did not improve motivation or learnability. Intelligent Agent Help did not differ significantly from No Help on any metric. The All Help condition, which presented all four resources simultaneously, actually increased cognitive load and did not improve other outcomes, suggesting an overload effect.

Usage logs indicated that participants accessed Interactive and Video Help far more frequently than Text or Agent Help, confirming a preference for multimodal, hands‑on assistance. The authors interpret these findings through the lens of multimedia learning theory: interactive tutorials provide immediate, contextual feedback that aligns with the spatial and temporal contiguity principles, while video tutorials combine visual and auditory channels, supporting the multimedia principle. Text‑only help lacks these benefits and can increase extraneous processing. The lack of effect for intelligent agents is attributed to limited conversational depth and possible novelty fatigue.

Design implications are clear: game‑making tools should prioritize built‑in interactive tutorials that guide users step‑by‑step, supplemented by concise video walkthroughs for complex tasks. Overloading users with too many help options can be counter‑productive, raising cognitive load and diminishing learning efficiency. The study also highlights the need for adaptive help systems that present the most appropriate modality based on user expertise and task difficulty.

Limitations include the online, short‑term nature of the experiment, reliance on self‑report measures, and the absence of longitudinal data on skill retention. Future work is suggested to test these help modalities in professional game‑development pipelines, explore adaptive help that changes over time, and examine the impact on collaborative game‑making scenarios.

Overall, this research fills a notable gap in HCI and game‑making literature by providing robust evidence that interactive and video help are the most effective support mechanisms for novice game creators, while excessive or poorly designed help can hinder learning.