A Comparative Study of Younger and Older Adults Interaction with a Crowdsourcing Android TV App for Detecting Errors in TEDx Video Subtitles

In this paper we report the results of a pilot study comparing the older and younger adults’ interaction with an Android TV application which enables users to detect errors in video subtitles. Overall, the interaction with the TV-mediated crowdsourcing system relying on language profficiency was seen as intuitive, fun and accessible, but also cognitively demanding; more so for younger adults who focused on the task of detecting errors, than for older adults who concentrated more on the meaning and edutainment aspect of the videos. We also discuss participants’ motivations and preliminary recommendations for the design of TV-enabled crowdsourcing tasks and subtitle QA systems.

💡 Research Summary

This paper presents a pilot comparative study of how younger and older adults interact with a crowdsourcing Android TV application—named DreamTV—designed to detect errors in TEDx video subtitles. The motivation stems from the growing volume of video content and the need for high‑quality subtitles, especially for deaf, hard‑of‑hearing, and multilingual audiences. Traditional web‑ or mobile‑based crowdsourcing platforms often exclude older adults due to low ICT skills, complex setups, and lack of perceived personal benefit. By leveraging a familiar Smart TV environment, the authors aimed to lower participation barriers and investigate whether a TV‑mediated approach could engage both age groups effectively.

Methodology
A qualitative comparative design was employed. Seven older adults (ages 60‑79, mean ≈ 71) and seven younger adults (ages 25‑35, mean ≈ 31) were recruited from Warsaw, Poland. All participants possessed at least “basic” digital competence according to the DigComp framework. Each participant received an Android TV set‑top box connected to their home TV, a remote control, and a short (~2 h) protocol consisting of a DigComp questionnaire, a semi‑structured interview, an explanation of subtitle error detection, a demonstration of the app, a hands‑on test, free interaction with five pre‑selected TEDx videos (two Polish, three English), and a post‑session interview.

The DreamTV app displays a TEDx video with volunteer‑generated subtitles retrieved via the Amara API. When a participant spots an error, they pause the video, an overlay appears, and they must select one of four error categories: grammar, meaning, style, or timing. These categories were chosen based on prior research to be more intuitive than professional quality‑assessment schemes. The study also recorded participants’ navigation behavior (e.g., use of the subtitle dialog list) and their verbal feedback.

Key Findings

1. Usability and Cognitive Load – Both groups found the interface intuitive, fun, and well‑suited to remote control operation. However, younger adults treated the task as “work‑like,” focusing intensely on error detection, while older adults approached it as an educational activity, paying more attention to video content. All participants needed to access the full subtitle list to locate the exact line, indicating that a quick‑jump feature is essential for any age group.

2. Error Detection Patterns – Younger adults identified significantly more errors, especially punctuation, synchronization, and fine‑grained grammatical issues. Older adults detected fewer errors, often overlooking punctuation, but were more likely to flag “meaning” problems when a subtitle was unclear. This aligns with literature on aging that suggests lower low‑level memory for details but preserved higher‑order comprehension.

3. Error‑Category Selection – Younger participants frequently complained that categories were “fuzzy” and suggested additional categories such as “punctuation,” “line‑break,” and “technical conventions.” They also preferred the ability to assign multiple categories to a single error. Older participants rarely questioned the categories; when uncertain, they tended to default to “meaning” or deliberated aloud, reflecting lower confidence in critiquing design choices.

4. Motivation, Gamification, and Rewards – Motivation differed sharply. Younger adults were driven by potential monetary rewards, points, leaderboards, and the “fun” of spotting mistakes, viewing the activity as a micro‑task or game. Older adults emphasized personal enrichment: learning new topics, staying mentally active, and contributing to the public good. They were less interested in financial incentives and more in meaningful engagement.

5. Design Recommendations – The authors propose several design adjustments: (a) reorder error categories by importance (meaning → grammar → style → timing); (b) provide an in‑app tutorial and mini‑games to teach category distinctions; (c) implement a “quick‑jump” to the relevant subtitle line; (d) allow customizable font size and contrast for older users; (e) offer a tiered reward system that includes both gamified points and non‑monetary recognitions (e.g., contributor badges).

Conclusion and Future Work
The study demonstrates that a Smart TV‑based crowdsourcing platform can be accessible and enjoyable for both younger and older adults, but the two cohorts exhibit distinct cognitive strategies, error‑detection behaviors, and motivational drivers. Effective design must therefore be age‑aware, balancing gamification for younger users with educational and low‑stress features for older users. Future research should scale up the participant pool, incorporate quantitative performance metrics, and explore long‑term sustainability of such TV‑enabled subtitle QA crowdsourcing, including its impact on subtitle quality and broader accessibility outcomes.