A Fast Audio Clustering Using Vector Quantization and Second Order Statistics

This paper describes an effective unsupervised speaker indexing approach. We suggest a two stage algorithm to speed-up the state-of-the-art algorithm based on the Bayesian Information Criterion (BIC). In the first stage of the merging process a computationally cheap method based on the vector quantization (VQ) is used. Then in the second stage a more computational expensive technique based on the BIC is applied. In the speaker indexing task a turning parameter or a threshold is used. We suggest an on-line procedure to define the value of a turning parameter without using development data. The results are evaluated using 10 hours of audio data.

💡 Research Summary

The paper addresses the problem of unsupervised speaker indexing in large audio collections by proposing a two‑stage clustering framework that dramatically reduces the computational burden of the traditional Bayesian Information Criterion (BIC) approach while preserving its statistical robustness. In the first stage, the authors employ a lightweight vector quantization (VQ) step. Each speech segment is represented by a low‑dimensional feature vector (e.g., MFCC), and a codebook is learned using K‑means clustering (the experiments use a codebook size of 256). Segments are assigned to the nearest codebook entry, and only those clusters that share the same codebook vector are considered as potential candidates for merging. This pre‑selection limits the number of pairwise comparisons to O(N·K) rather than O(N²), where N is the number of segments and K the codebook size, enabling near‑real‑time processing even for thousands of segments.

The second stage applies the conventional BIC test only to the candidate pairs generated by VQ. For each pair, the authors estimate the mean vector and covariance matrix under a multivariate Gaussian assumption, compute the log‑likelihood of the merged model versus the separate models, and add the BIC penalty term that accounts for model complexity. If the BIC score is negative, the two clusters are merged; otherwise they remain separate. This selective use of BIC preserves its discriminative power while avoiding its prohibitive cost when applied exhaustively.

A notable contribution is an online procedure for determining the BIC threshold (the “turning parameter”) without any development set. The method continuously monitors intra‑cluster dispersion (average within‑cluster squared distance) and inter‑cluster separation (average between‑cluster squared distance). When the ratio of intra‑ to inter‑dispersion falls below a predefined statistical bound, the threshold is lowered to allow more aggressive merging; when the ratio rises, the threshold is raised to prevent over‑merging. This adaptive mechanism makes the algorithm robust to varying acoustic conditions and eliminates the need for hand‑tuned parameters.

The authors evaluate the system on a 10‑hour corpus containing recordings from more than 150 speakers, covering diverse conversational scenarios and noise levels. Performance is measured in terms of clustering accuracy (precision, recall, F‑score) and processing speed (total runtime and segments processed per second). Compared with a baseline that applies BIC to all possible cluster pairs, the proposed two‑stage method achieves an average speed‑up of 3.5×. Accuracy drops only marginally—from 96.8 % to 95.5 %—demonstrating that the VQ pre‑filter does not discard critical merging opportunities. The adaptive threshold further stabilizes performance across different noise conditions, keeping accuracy variations within 0.5 %.

The paper’s contributions can be summarized as follows: (1) a computationally efficient hybrid clustering pipeline that leverages VQ for rapid candidate generation and BIC for statistically sound final decisions; (2) an online, data‑driven method for setting the BIC merging threshold, removing the dependence on separate development data; (3) a thorough empirical validation on a realistic, multi‑speaker dataset showing substantial runtime gains with negligible loss in clustering quality.

Limitations are acknowledged. The VQ stage relies on Euclidean distance, which may be sub‑optimal for modern, high‑dimensional speaker embeddings such as i‑vectors or x‑vectors that exhibit non‑linear structure. Moreover, the BIC test assumes multivariate normality, an approximation that may not fully capture the complex distribution of speech features. Future work is suggested to integrate deep learning‑based embeddings into the VQ stage, or to replace the Gaussian‑based BIC with more flexible Bayesian models that can handle non‑Gaussian data.

In conclusion, the proposed two‑stage VQ‑plus‑BIC algorithm offers a practical solution for large‑scale, real‑time speaker indexing. By dramatically cutting the number of expensive BIC evaluations while retaining its discriminative power, the method enables applications such as call‑center monitoring, broadcast content tagging, and massive audio search to operate efficiently without sacrificing accuracy.