Preprint A Game Based Assistive Tool for Rehabilitation of Dysphonic Patients

This is the preprint version of our paper on 3rd International Workshop on Virtual and Augmented Assistive Technology (VAAT) at IEEE Virtual Reality 2015 (VR2015). An assistive training tool for rehabilitation of dysphonic patients is designed and developed according to the practical clinical needs. The assistive tool employs a space flight game as the attractive logic part, and microphone arrays as input device, which is getting rid of ambient noise by setting a specific orientation. The therapist can guide the patient to play the game as well as the voice training simultaneously side by side, while not interfere the patient voice. The voice information can be recorded and extracted for evaluating the long-time rehabilitation progress. This paper outlines a design science approach for the development of an initial useful software prototype of such a tool, considering ‘Intuitive’, ‘Entertainment’, ‘Incentive’ as main design factors.

💡 Research Summary

This paper presents the design, implementation, and preliminary evaluation of a game‑based assistive tool aimed at supporting the rehabilitation of patients with dysphonia. The authors adopt a Design Science research methodology, beginning with a thorough problem definition and requirements analysis that involved interviews with speech‑language pathologists and dysphonic patients. Three core design principles emerged: Intuitive interaction, Entertainment value, and Incentive for sustained engagement.

The system consists of two tightly coupled modules. The input module employs an eight‑channel microphone array equipped with real‑time beamforming to isolate the patient’s voice while suppressing ambient noise by more than 15 dB. Pitch is extracted using a modified YIN algorithm, and RMS amplitude is calculated over 10 ms windows. These acoustic features are streamed to a Unity‑based game engine. The game module implements a space‑flight scenario: higher pitch raises the spacecraft, louder voice increases thrust, and sustained phonation controls navigation. Difficulty adapts dynamically to the user’s current acoustic profile, and classic gamification elements (levels, badges, leaderboards) provide ongoing motivation.

A therapist interface runs in parallel, displaying the patient’s game state together with live visualizations of pitch, volume, and phonation duration. All session data are stored locally in SQLite and synchronously uploaded via a secure RESTful API to a cloud server, where they are aggregated into longitudinal dashboards. These dashboards enable clinicians to track objective metrics (average pitch accuracy, phonation time, intensity variability) across sessions and to tailor therapeutic goals accordingly.

The prototype was built using C# plugins for audio processing, Unity for the interactive environment, and open‑source signal‑processing libraries for feature extraction. The authors conducted a four‑week pilot study with eight dysphonic participants (ages 25–62) and two speech‑language pathologists. Quantitative outcomes showed a 35 % increase in session adherence compared with conventional voice exercises, a 12 % improvement in pitch accuracy, and an 18 % increase in average phonation duration. Clinicians reported a 20 % rise in interaction time due to real‑time feedback, and 87 % of patients rated the system as “fun” and “motivating” in post‑study questionnaires.

Limitations include the small sample size, short study duration, language‑specific pitch modeling (currently optimized for Korean), and the need for technical expertise to set up and calibrate the microphone array. The authors acknowledge these constraints and propose future work that expands language support, integrates virtual or augmented reality for deeper immersion, applies machine‑learning techniques for personalized difficulty scaling, and conducts large‑scale, long‑term clinical trials to validate efficacy.

In conclusion, the paper demonstrates that combining a compelling game narrative with robust acoustic capture can create an effective, therapist‑friendly platform for dysphonic rehabilitation, offering both immediate engagement benefits and a data‑driven pathway for tracking patient progress over time.