Usability of Virtual Reality Application Through the Lens of the User Community: A Case Study

The increasing availability and diversity of virtual reality (VR) applications highlighted the importance of their usability. Function-oriented VR applications posed new challenges that are not well studied in the literature. Moreover, user feedback becomes readily available thanks to modern software engineering tools, such as app stores and open source platforms. Using Firefox Reality as a case study, we explored the major types of VR usability issues raised in these platforms. We found that 77% of usability feedbacks can be mapped to Nielsen’s heuristics while few were mappable to VR-specific heuristics. This result indicates that Nielsen’s heuristics could potentially help developers address the usability of this VR application in its early development stage. This work paves the road for exploring tools leveraging the community effort to promote the usability of function-oriented VR applications.

💡 Research Summary

This paper investigates the usability of function‑oriented virtual‑reality (VR) applications by mining user‑generated feedback from two community‑driven sources: GitHub issue discussions and Oculus app‑store reviews. The authors selected Firefox Reality, an open‑source VR web browser developed by Mozilla, as a case study because it represents a “desktop‑style” application transplanted into a VR environment, thereby exposing novel usability concerns.

Data collection took place in November 2018. Using the GitHub REST API, the researchers retrieved 826 issue threads; they also manually gathered all 20 Oculus store reviews for the product. After removing duplicate issues and those lacking substantive body text, 672 items remained. Through manual inspection, 345 of these (≈51 %) were classified as usability‑related. Open coding of the usability‑related items produced fourteen distinct themes, which the authors grouped under familiar usability dimensions such as flexibility/efficiency, aesthetic/minimalist design, consistency and standards, user control and freedom, visibility of system status, recognition rather than recall, help and documentation, match between system and the real world, error prevention, and error recovery.

To assess how well existing usability frameworks capture these concerns, the authors attempted to map each theme to three sets of heuristics: Sutcliffe & Gault’s (2004) VR‑specific heuristics, Rusu et al.’s (2011) VR‑world heuristics, and Nielsen’s (1995) ten general UI heuristics. The mapping exercise revealed that 267 of the 345 usability‑related items (77 %) aligned with at least one of Nielsen’s heuristics, whereas only a handful could be associated with the VR‑specific sets. The authors attribute this discrepancy to the nature of Firefox Reality, which mirrors a conventional desktop browser rather than an immersive virtual world, making general UI principles more applicable.

Four themes could not be mapped to any of the examined heuristics: security and privacy concerns (22 items), accessibility (9 items), camera control (3 items), and faithful viewpoint (3 items). These unmapped categories highlight issues that are uniquely salient in VR contexts—such as telemetry‑related privacy worries, the need for color‑blind‑friendly UI cues, and the discomfort caused by mismatches between head motion and virtual camera behavior.

The discussion emphasizes that the high Nielsen‑mapping rate is partly a function of the product’s early‑stage, “function‑oriented” design, which still heavily relies on familiar desktop interaction patterns. Nonetheless, the study uncovers VR‑specific usability challenges, notably the inefficiency of point‑and‑click input in head‑mounted displays and the ambiguity of concepts like “full‑screen” when translated into a three‑dimensional space. Users suggested mitigations such as voice‑to‑text dictation and richer haptic feedback, indicating that developers must consider novel interaction modalities to preserve efficiency and comfort.

In conclusion, the authors argue that while Nielsen’s heuristics provide a solid baseline for early‑stage evaluation of function‑oriented VR applications, a comprehensive usability framework for VR must incorporate additional dimensions that address input modality constraints, sensory feedback, privacy, and accessibility. Future work will extend the analysis to other function‑oriented VR apps (e.g., virtual meetings, e‑commerce) to test the generalizability of the findings, and will explore semi‑automated tools that classify user feedback according to both general and VR‑specific heuristics, thereby offering actionable guidance to developers throughout the product lifecycle.