Cross-Linguistic Persona-Driven Data Synthesis for Robust Multimodal Cognitive Decline Detection

Speech-based digital biomarkers represent a scalable, non-invasive frontier for the early identification of Mild Cognitive Impairment (MCI). However, the development of robust diagnostic models remains impeded by acute clinical data scarcity and a lack of interpretable reasoning. Current solutions frequently struggle with cross-lingual generalization and fail to provide the transparent rationales essential for clinical trust. To address these barriers, we introduce SynCog, a novel framework integrating controllable zero-shot multimodal data synthesis with Chain-of-Thought (CoT) deduction fine-tuning. Specifically, SynCog simulates diverse virtual subjects with varying cognitive profiles to effectively alleviate clinical data scarcity. This generative paradigm enables the rapid, zero-shot expansion of clinical corpora across diverse languages, effectively bypassing data bottlenecks in low-resource settings and bolstering the diagnostic performance of Multimodal Large Language Models (MLLMs). Leveraging this synthesized dataset, we fine-tune a foundational multimodal backbone using a CoT deduction strategy, empowering the model to explicitly articulate diagnostic thought processes rather than relying on black-box predictions. Extensive experiments on the ADReSS and ADReSSo benchmarks demonstrate that augmenting limited clinical data with synthetic phenotypes yields competitive diagnostic performance, achieving Macro-F1 scores of 80.67% and 78.46%, respectively, outperforming current baseline models. Furthermore, evaluation on an independent real-world Mandarin cohort (CIR-E) demonstrates robust cross-linguistic generalization, attaining a Macro-F1 of 48.71%. These findings constitute a critical step toward providing clinically trustworthy and linguistically inclusive cognitive assessment tools for global healthcare.

💡 Research Summary

The paper introduces SynCog, a novel framework designed to overcome two major obstacles in speech‑based early detection of Mild Cognitive Impairment (MCI) and Alzheimer’s Disease (AD): severe scarcity of clinically annotated multimodal data and the lack of interpretability in current AI models.
Data synthesis is the first pillar of SynCog. By defining “digital personas” that encode demographic attributes (age, sex, education) and cognitive scores (MMSE, MoCA), the authors prompt a large language model (LLM) to generate picture‑description narratives (the standard “Cookie‑Theft” task) consistent with each persona’s cognitive profile. Advanced voice‑cloning technology then converts these texts into high‑fidelity audio, producing fully paired speech‑text samples that are automatically labeled. Two synthetic corpora are built: SYN‑EN (English) and SYN‑ZH (Mandarin), each containing up to 500 subjects per diagnostic class (AD, MCI, healthy control). Statistical analyses (word count, spatial‑term frequency, filler and vague term usage) and acoustic embedding visualizations (t‑SNE of wav2vec2 features) demonstrate that the synthetic data closely mirrors the distribution of real clinical recordings.

The second pillar is Chain‑of‑Thought (CoT) distillation and fine‑tuning. Instead of training a model solely to output a class label, the authors automatically generate expert‑style reasoning traces for each synthetic sample (e.g., “Reduced total word count and fewer spatial terms indicate lexical retrieval difficulty”). These CoT annotations serve as supervision for a multimodal backbone—Qwen2‑Audio‑7B‑Instruct—using Low‑Rank Adaptation (LoRA). During inference, the fine‑tuned model first produces a step‑by‑step rationale linking acoustic and linguistic cues to clinical markers, then delivers the final diagnosis. This design provides transparent, clinically meaningful explanations, mitigating shortcut learning and increasing trustworthiness.

Experimental evaluation covers three datasets: the English ADReSS (binary) and ADReSSo (audio‑only) benchmarks, and an independently collected Mandarin cohort (CIR‑E) with three classes (HC, MCI, AD). Augmenting the limited real data with synthetic samples yields Macro‑F1 scores of 80.67 % on ADReSS and 78.46 % on ADReSSo, surpassing state‑of‑the‑art baselines. On the Mandarin test set, SynCog achieves a Macro‑F1 of 48.71 %, markedly higher than existing cross‑lingual models, confirming its ability to transfer learned biomarkers across languages.

The authors discuss limitations: (1) synthetic audio may not capture all real‑world noise and dialectal variation; (2) CoT rationales are automatically generated and lack exhaustive expert validation; (3) the framework is currently tied to a single picture‑description task, so broader linguistic generalization remains to be proven. Future work is proposed to expand to multiple speech tasks, incorporate diverse dialects, and involve clinicians in refining CoT annotations, ultimately aiming for deployment in real‑world screening pipelines.

In summary, SynCog combines controllable multimodal data synthesis with reasoning‑oriented fine‑tuning, delivering both performance gains and interpretable outputs. It represents a significant step toward scalable, linguistically inclusive, and clinically trustworthy AI tools for early cognitive decline detection worldwide.