Multilayer bootstrap network for unsupervised speaker recognition

We apply multilayer bootstrap network (MBN), a recent proposed unsupervised learning method, to unsupervised speaker recognition. The proposed method first extracts supervectors from an unsupervised universal background model, then reduces the dimension of the high-dimensional supervectors by multilayer bootstrap network, and finally conducts unsupervised speaker recognition by clustering the low-dimensional data. The comparison results with 2 unsupervised and 1 supervised speaker recognition techniques demonstrate the effectiveness and robustness of the proposed method.

💡 Research Summary

This paper introduces an unsupervised speaker recognition system that leverages the Multilayer Bootstrap Network (MBN), a recently proposed nonlinear dimensionality‑reduction technique. The overall pipeline consists of three stages. First, an unsupervised Universal Background Model (UBM) is trained on raw acoustic features (25‑dimensional MFCCs). For each utterance the UBM yields a high‑dimensional supervector that concatenates mixture occupation counts and centered first‑order statistics, providing a speaker‑ and session‑independent representation. Second, the supervectors are fed into an MBN, which progressively compresses the data through a stack of hidden layers. Each hidden layer contains V independent k‑centers clusterings; each clustering performs random feature selection (a · d dimensions), random sampling of k centroids, and a random reconstruction step (cyclic shift of d₀ selected dimensions). The input is assigned to the nearest centroid, producing a sparse k‑dimensional indicator vector. These indicator vectors from all clusterings are concatenated and passed to the next layer. After L layers, a conventional PCA is applied at the output layer to obtain the final low‑dimensional embedding. The authors adopt a typical hyper‑parameter setting: V = 400, a = 0.5, r = 0.5, and a decreasing sequence of k values (e.g., 3060 → 1530 → 765 → 382 → 191 → 95). The number of hidden layers L is determined by the decay of k, and the final k is chosen to be larger than the expected number of speakers (or a fixed value such as 30 when the number of speakers is unknown).

The low‑dimensional embeddings are finally clustered. When the number of speakers is known, k‑means is applied; otherwise an agglomerative hierarchical clustering is used. The system is evaluated on the Speech Separation Challenge (SSC) corpus, which contains 34 speakers, each with 500 clean utterances. The authors select the first 100 utterances per speaker (3 400 utterances) for testing. MFCCs are extracted with a 25 ms frame length and 10 ms shift. Various UBM configurations are examined: mixture numbers {1, 2, 4, 8, 16, 32, 64} and EM iteration counts {0, 20} to assess robustness to model quality.

Three baseline methods are compared: (1) PCA‑based unsupervised reduction (same UBM, followed by PCA to dimensions {2, 3, 5, 10, 30, 50} and k‑means clustering), (2) direct k‑means clustering on the raw supervectors, and (3) a supervised system that uses UBM → Joint Factor Analysis (unsupervised) → Linear Discriminant Analysis (supervised) with a probabilistic LDA classifier. Performance is measured by Normalized Mutual Information (NMI), which accounts for label permutation and is standard for unsupervised clustering evaluation.

Results show that the MBN‑based approach consistently outperforms both unsupervised baselines across all UBM settings. Even when the UBM is severely under‑parameterized (e.g., 1 mixture) or untrained (0 EM iterations), MBN maintains relatively high NMI, whereas PCA and direct k‑means degrade sharply. Compared with the supervised LDA system, MBN achieves comparable NMI when sufficient output dimensions are used (e.g., 30 or 50). Moreover, MBN’s performance is relatively insensitive to the choice of hyper‑parameters: varying the number of hidden layers, the output dimensionality, or the V and r values leads to only modest changes in NMI. The authors also observe that increasing the number of hidden layers improves accuracy gradually, confirming the benefit of deeper hierarchical representations.

The paper concludes that MBN satisfies three stringent requirements for unsupervised speaker recognition: (i) no need for manually labeled data, (ii) minimal hyper‑parameter tuning, and (iii) robustness to different acoustic modeling conditions. Unlike conventional deep neural networks, MBN’s layers consist of independent clustering units, enabling straightforward parallelization and offering better interpretability of the learned representations. Limitations include the relatively small number of speakers and the use of clean speech; future work should explore scalability to larger, multilingual, and noisy datasets, as well as real‑time deployment considerations. Overall, the study demonstrates that multilayer bootstrap networks provide a powerful and practical tool for unsupervised speaker recognition, bridging the gap between purely linear methods and fully supervised deep learning approaches.