A Dual-Stage Time-Context Network for Speech-Based Alzheimer's Disease Detection

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder that leads to irreversible cognitive decline in memory and communication. Early detection of AD through speech analysis is crucial for delaying disease progression. However, existing methods mainly use pre-trained acoustic models for feature extraction but have limited ability to model both local and global patterns in long-duration speech. In this letter, we introduce a Dual-Stage Time-Context Network (DSTC-Net) for speech-based AD detection, integrating local acoustic features with global conversational context in long-duration recordings.We first partition each long-duration recording into fixed-length segments to reduce computational overhead and preserve local temporal details.Next, we feed these segments into an Intra-Segment Temporal Attention (ISTA) module, where a bidirectional Long Short-Term Memory (BiLSTM) network with frame-level attention extracts enhanced local features.Subsequently, a Cross-Segment Context Attention (CSCA) module applies convolution-based context modeling and adaptive attention to unify global patterns across all segments.Extensive experiments on the ADReSSo dataset show that our DSTC-Net outperforms state-of-the-art models, reaching 83.10% accuracy and 83.15% F1.

💡 Research Summary

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder whose early detection can significantly slow disease progression. Speech‑based screening offers a low‑cost, non‑invasive alternative to imaging or blood tests, especially because early AD manifests as subtle changes in prosody, fluency, pause patterns, and lexical retrieval. Existing approaches typically rely on self‑supervised acoustic models such as Wav2Vec 2.0, HuBERT, or Whisper to extract high‑level representations, but they struggle with long recordings (often >60 seconds) because (i) memory consumption grows with sequence length, (ii) local acoustic details and global conversational context are not simultaneously modeled, and (iii) segment‑wise processing often discards cross‑segment continuity.

The authors propose a Dual‑Stage Time‑Context Network (DSTC‑Net) that explicitly addresses these challenges. First, each recording is divided into fixed‑length overlapping segments (≈10 % overlap) to reduce computational load while preserving boundary information. Segments shorter than the target length are zero‑padded, ensuring uniform tensor shapes. For each segment, a pre‑trained acoustic model (the authors evaluate Wav2Vec 2.0, HuBERT, and Whisper) produces a four‑dimensional feature tensor (layers × segments × time steps × feature‑dim).

The network consists of two complementary modules:

1. Intra‑Segment Temporal Attention (IST​A) – The frame‑level embeddings of a segment are fed into a bidirectional LSTM (128 hidden units per direction, concatenated to 256). The final hidden state serves as a “summary” vector. An attention score αₜ is computed for each time step by measuring similarity between the summary and each hidden state, followed by a softmax normalization. The weighted sum of hidden states emphasizes frames that are most discriminative for AD, and the resulting vector is concatenated with the BiLSTM output and projected through a tanh‑activated fully‑connected layer, yielding a refined segment representation.

2. Cross‑Segment Context Attention (CSCA) – All segment representations from IST​A are passed through a 1‑D convolutional block to capture local inter‑segment patterns, then a second attention mechanism assigns a weight βₘ to each segment. The global feature vector G is obtained by a weighted sum of the convolved segment vectors. This step aggregates discourse‑level cues such as overall speech rate, turn‑taking rhythm, and narrative coherence, which are known to be altered in AD.

The final global vector G is classified by a simple fully‑connected layer into AD vs. healthy control.

Experimental Setup – The model is evaluated on the ADReSSo dataset (237 English speakers, 122 AD, 115 controls) derived from the “Cookie‑Theft” picture description task. Recordings average 62 seconds. The authors implement DSTC‑Net in PyTorch 2.0.1, train with Adam (lr = 1e‑4, batch = 32) for up to 200 epochs, and employ early stopping after 50 epochs without validation improvement. Ten‑fold cross‑validation follows the official split, and a held‑out test set reports final performance.

Results – Using Whisper embeddings, DSTC‑Net achieves 83.10 % accuracy and 83.15 % F1, surpassing all baselines reported in the literature (e.g., Whisper‑only 77.46 % accuracy, Wav2Vec 2.0‑based methods around 78–81 %). Ablation studies show that IST​A alone raises Whisper accuracy to 81.69 % and CSCA alone to 80.12 %; the combination yields the best scores, confirming the complementary nature of local temporal refinement and global context fusion. Additional analysis of segment length versus encoder depth reveals model‑specific optimal windows: Whisper performs best with 10‑second segments, Wav2Vec 2.0 with 5‑second segments, and HuBERT with 15‑second segments, reflecting differing receptive fields and pre‑training objectives.

Discussion and Limitations – The dual‑stage design effectively balances fine‑grained acoustic cues (e.g., micro‑pauses, pitch variation) with macro‑level discourse patterns (e.g., narrative coherence). However, the approach has several constraints: (1) Fixed segment length and overlap ratio may not generalize to highly variable conversational settings; (2) The model relies solely on acoustic information, ignoring potentially valuable lexical or semantic cues from transcripts; (3) The added BiLSTM and attention layers increase model size, which could hinder real‑time deployment on edge devices. Future work could explore dynamic segmentation, multimodal fusion of speech and text, and model compression techniques.

Conclusion – DSTC‑Net introduces a principled way to process long‑duration speech for AD detection by first segmenting recordings, then jointly applying intra‑segment temporal attention and cross‑segment context attention. The architecture achieves state‑of‑the‑art performance on a benchmark dataset, demonstrating that simultaneous modeling of local and global speech patterns is crucial for reliable, early AD screening. This work paves the way for scalable, non‑invasive diagnostic tools that could be integrated into telehealth platforms and routine clinical assessments.