Spoof detection using time-delay shallow neural network and feature switching

Detecting spoofed utterances is a fundamental problem in voice-based biometrics. Spoofing can be performed either by logical accesses like speech synthesis, voice conversion or by physical accesses such as replaying the pre-recorded utterance. Inspired by the state-of-the-art \emph{x}-vector based speaker verification approach, this paper proposes a time-delay shallow neural network (TD-SNN) for spoof detection for both logical and physical access. The novelty of the proposed TD-SNN system vis-a-vis conventional DNN systems is that it can handle variable length utterances during testing. Performance of the proposed TD-SNN systems and the baseline Gaussian mixture models (GMMs) is analyzed on the ASV-spoof-2019 dataset. The performance of the systems is measured in terms of the minimum normalized tandem detection cost function (min-t-DCF). When studied with individual features, the TD-SNN system consistently outperforms the GMM system for physical access. For logical access, GMM surpasses TD-SNN systems for certain individual features. When combined with the decision-level feature switching (DLFS) paradigm, the best TD-SNN system outperforms the best baseline GMM system on evaluation data with a relative improvement of 48.03% and 49.47% for both logical and physical access, respectively.

💡 Research Summary

This paper presents a novel approach for detecting spoofing attacks in voice biometrics, targeting both logical access (LA) attacks like speech synthesis and voice conversion, and physical access (PA) attacks like replaying pre-recorded audio. Inspired by the success of x-vectors in speaker verification, the authors propose a Time-Delay Shallow Neural Network (TD-SNN) for spoof detection.

The core innovation lies in adapting the x-vector architecture, known for handling variable-length utterances, to the binary spoof detection task. Key modifications include: changing the final layer for two-class (bonafide/spoof) classification, replacing the standard cross-entropy loss with a focal loss to better handle class imbalance and hard-to-classify examples, and simplifying the network to four hidden layers (making it “shallow”) considering the binary nature of the problem and data constraints. The architecture comprises two frame-level time-delay layers, a statistics pooling layer that aggregates mean and standard deviation across time to create a fixed-length representation, and a segment-level layer that reduces dimensionality before the final softmax output.

Experiments are conducted on the ASVspoof 2019 dataset, evaluated using the minimum normalized tandem detection cost function (min-t-DCF), which integrates spoof detection performance with that of an automatic speaker verification system. Baseline systems using Gaussian Mixture Models (GMMs) with four different feature sets—Constant Q Cepstral Coefficients (CQCC), Linear Frequency Cepstral Coefficients (LFCC), Inverse Mel-Frequency Cepstral Coefficients (IMFCC), and Linear Filterbank Energies (LFBE)—are established for comparison.

The results show that for PA spoof detection, the proposed TD-SNN systems consistently outperform the GMM baselines across all individual features. For LA detection, however, GMMs surpass TD-SNNs for certain features, indicating no single feature is optimal for all spoofing conditions. To leverage the complementary strengths of different features, the authors employ a Decision-Level Feature Switching (DLFS) paradigm. Instead of simple score fusion, DLFS dynamically selects, for each test trial, the score from the individual feature system that shows maximum discrimination between bonafide and spoof models for that specific trial.

The fusion of the best TD-SNN systems via DLFS yields the most significant improvement. The best TD-SNN DLFS system outperforms the best GMM DLFS system on the evaluation set, achieving a relative improvement of 48.03% in min-t-DCF for LA and 49.47% for PA. Additional analyses demonstrate that the focal loss provides better separation of bonafide and spoof embeddings compared to cross-entropy loss (visualized via t-SNE) and that engineered features like CQCC and LFCC are more effective than raw filterbank energies for this task.

In conclusion, the paper successfully demonstrates the efficacy of a tailored, shallow time-delay neural network for spoof detection and underscores the importance of intelligent feature combination strategies like DLFS to build robust systems capable of countering diverse spoofing attacks.