A high speed unsupervised speaker retrieval using vector quantization and second-order statistics

This paper describes an effective unsupervised method for query-by-example speaker retrieval. We suppose that only one speaker is in each audio file or in audio segment. The audio data are modeled using a common universal codebook. The codebook is based on bag-of-frames (BOF). The features corresponding to the audio frames are extracted from all audio files. These features are grouped into clusters using the K-means algorithm. The individual audio files are modeled by the normalized distribution of the numbers of cluster bins corresponding to this file. In the first level the k-nearest to the query files are retrieved using vector space representation. In the second level the second-order statistical measure is applied to obtained k-nearest files to find the final result of the retrieval. The described method is evaluated on the subset of Ester corpus of French broadcast news.

💡 Research Summary

The paper presents a two‑stage, unsupervised approach for query‑by‑example speaker retrieval that operates without any pre‑labeled speaker models. In the first stage, acoustic features (typically MFCCs) are extracted from every audio file in the collection and clustered using K‑means to build a universal codebook—a bag‑of‑frames representation. Each audio file is then encoded as a normalized histogram over the codebook bins, yielding a fixed‑dimensional vector. Similarity between a query file and all database entries is computed in this vector space using simple distance measures such as cosine similarity or Euclidean distance, and the top‑k most similar files are selected. This stage is computationally cheap and enables real‑time retrieval even on large corpora.

In the second stage, the k‑nearest candidates are re‑ranked using second‑order statistical distances that capture the internal structure of the audio signals. The authors estimate covariance matrices for each file and apply measures such as Kullback‑Leibler divergence, Bhattacharyya distance, or a Bayesian Information Criterion (BIC) based distance. These statistics are sensitive to spectral variance and speaker‑specific nuances that are lost in the histogram representation, allowing a more precise discrimination among the shortlisted files.

The method is evaluated on a subset of the French ESTER broadcast news corpus. Experiments vary the codebook size (e.g., 256, 512, 1024 clusters) and the number of retrieved candidates (k = 5, 10, 20). Results show that the proposed pipeline achieves precision, recall, and F‑measure comparable to or slightly better than conventional Gaussian Mixture Model (GMM) based speaker modeling, while reducing query time by an order of magnitude. The unsupervised nature eliminates the need for speaker‑specific training data, making the system readily adaptable to new speakers, languages, or domains by simply rebuilding the codebook.

The authors argue that this combination of vector quantization for fast coarse filtering and second‑order statistics for fine‑grained re‑ranking offers an effective trade‑off between speed and accuracy. Potential applications include real‑time broadcast monitoring, large‑scale audio archive browsing, and forensic speaker identification. Future work is suggested on handling multi‑speaker segments, extending to multilingual corpora, and integrating deep‑learning based feature extractors to further boost discriminative power.