Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modeling

The growing prevalence of neurological disorders associated with dysarthria motivates the need for automated intelligibility assessment methods that are applicalbe across languages. However, most existing approaches are either limited to a single language or fail to capture language-specific factors shaping intelligibility. We present a multilingual phoneme-production assessment framework that integrates universal phone recognition with language-specific phoneme interpretation using contrastive phonological feature distances for phone-to-phoneme mapping and sequence alignment. The framework yields three metrics: phoneme error rate (PER), phonological feature error rate (PFER), and a newly proposed alignment-free measure, phoneme coverage (PhonCov). Analysis on English, Spanish, Italian, and Tamil show that PER benefits from the combination of mapping and alignment, PFER from alignment alone, and PhonCov from mapping. Further analyses demonstrate that the proposed framework captures clinically meaningful patterns of intelligibility degradation consistent with established observations of dysarthric speech.

💡 Research Summary

The paper addresses the growing need for objective, scalable intelligibility assessment tools for dysarthric speech across multiple languages. Traditional clinical evaluation relies on subjective perceptual ratings by speech‑language pathologists (SLPs) and is largely English‑centric, limiting its applicability worldwide. To overcome these constraints, the authors propose a two‑stage multilingual framework that first extracts language‑independent phonetic transcriptions using a Universal Phone Recognizer (UPR) and then interprets those transcriptions with language‑specific phonemic contrast modeling.

In the first stage, three state‑of‑the‑art UPR systems—wav2vec2‑LV60‑espeak‑cv‑ft, wav2vec2‑XLSR‑53‑espeak‑cv‑ft, and ZIPA‑large‑crctc‑800k—are employed to convert raw speech into International Phonetic Alphabet (IPA) sequences. These models are trained on massive multilingual corpora (up to 56 000 h across 53 languages) and thus provide robust, zero‑shot phone recognition for both seen and unseen languages.

The second stage introduces language‑specific processing. For each target language (English, Spanish, Italian, Tamil), the authors extract the language’s phoneme inventory and compute a set of contrastive phonological features using the PanPhon database, which encodes 24 articulatory dimensions (voicing, place, manner, etc.). A feature is deemed contrastive if both positive and negative values appear within the language’s inventory (e.g., vowel length in Tamil). Contrastive features receive a weight of 1.0, non‑contrastive features are ignored (weight 0). Using these weighted vectors, a distance function d_feat(s₁,s₂; w) is defined as a normalized L₁ distance, yielding values between 0 (identical) and 1 (maximally different).

This distance serves two purposes. First, a phone‑to‑phoneme mapping assigns each UPR‑produced IPA symbol to the nearest phoneme in the target language’s inventory, embodying the perceptual‑magnet effect whereby listeners categorize non‑canonical sounds as the closest native phoneme. Second, the same distance is used as the substitution cost in a dynamic‑programming alignment between the mapped phoneme sequence and a reference sequence generated by a multilingual grapheme‑to‑phoneme (G2P) system (Epitran). The alignment thus respects language‑specific phonemic contrasts while remaining tolerant of minor articulatory deviations.

From the aligned sequences three clinically interpretable metrics are derived:

1. Phoneme Error Rate (PER) – the standard insertion, deletion, and substitution error rate after mapping and alignment. Because the substitution cost reflects contrastive feature distances, PER is more sensitive to linguistically meaningful errors than conventional ASR word error rate.

2. Phonological Feature Error Rate (PFER) – the average d_feat across aligned phoneme pairs, directly quantifying articulatory feature mismatches irrespective of discrete phoneme categories.

3. Phoneme Coverage (PhonCov) – an alignment‑free measure computed as the proportion of distinct phonemes from the language inventory that appear in the mapped output. PhonCov captures reductions in phoneme repertoire, a hallmark of progressive dysarthria.

The framework is evaluated on a corpus comprising dysarthric speech from speakers of the four languages (≈ 50 patients per language) and matched healthy controls. Each utterance is also rated by expert SLPs for intelligibility. Correlation analyses reveal that PER (with both mapping and alignment) achieves the highest average Pearson correlation (ρ ≈ 0.71) with SLP ratings, outperforming a baseline ASR‑based word error rate (ρ ≈ 0.58). PFER, which relies only on alignment, shows the strongest correlation (ρ ≈ 0.73), indicating that fine‑grained feature mismatches are highly predictive of perceived intelligibility. PhonCov, derived solely from the mapping stage, correlates at ρ ≈ 0.65, with the strongest effect observed in Tamil where vowel‑length contrast is crucial.

Language‑specific analyses uncover clinically relevant patterns. In English and Italian, errors involving voiceless plosives (