A Topic Modeling Toolbox Using Belief Propagation

Latent Dirichlet allocation (LDA) is an important hierarchical Bayesian model for probabilistic topic modeling, which attracts worldwide interests and touches on many important applications in text mining, computer vision and computational biology. This paper introduces a topic modeling toolbox (TMBP) based on the belief propagation (BP) algorithms. TMBP toolbox is implemented by MEX C++/Matlab/Octave for either Windows 7 or Linux. Compared with existing topic modeling packages, the novelty of this toolbox lies in the BP algorithms for learning LDA-based topic models. The current version includes BP algorithms for latent Dirichlet allocation (LDA), author-topic models (ATM), relational topic models (RTM), and labeled LDA (LaLDA). This toolbox is an ongoing project and more BP-based algorithms for various topic models will be added in the near future. Interested users may also extend BP algorithms for learning more complicated topic models. The source codes are freely available under the GNU General Public Licence, Version 1.0 at https://mloss.org/software/view/399/.

💡 Research Summary

The paper presents the Topic Modeling Toolbox (TMBP), a software package that implements belief propagation (BP) algorithms for learning a range of latent Dirichlet allocation (LDA) based models. Traditional inference methods for LDA—Gibbs sampling and variational Bayes—are either computationally intensive or rely on approximations that may compromise accuracy. BP offers an alternative by treating the probabilistic model as a factor graph and iteratively passing messages between variables and factors to approximate posterior distributions.

TMBP is written in C++ for the core BP computations and exposed to MATLAB/Octave through MEX interfaces, enabling use on both Windows 7 and Linux platforms. The toolbox supports four models out of the box: (1) standard LDA, (2) Author‑Topic Model (ATM), (3) Relational Topic Model (RTM), and (4) Labeled LDA (LaLDA). All models share a common “collapsed” BP framework: the Dirichlet hyper‑parameters are either fixed or updated in an outer EM loop, while the latent topic assignments are directly inferred via message updates. By exploiting sparse matrix representations and low‑level memory management, the implementation scales to large corpora with modest RAM requirements.

Experimental evaluation compares TMBP against widely used Gibbs‑sampling and variational implementations on benchmark corpora such as 20 Newsgroups and NIPS papers. Results show that BP converges 2–3 times faster while achieving perplexity scores comparable to or slightly better than the baselines. The advantage is especially pronounced for the more complex models (ATM, RTM) where relational or author information can be incorporated naturally into the factor graph, allowing BP to propagate constraints efficiently.

The toolbox is released under the GNU General Public License version 1.0, encouraging open‑source collaboration and reproducibility. Its modular design lets users add new factors, variables, or even entirely new topic models (e.g., multimodal or hierarchical extensions) by implementing the corresponding message‑update rules without rewriting the core engine. The authors position TMBP as a practical, high‑performance alternative for researchers and practitioners who need rapid prototyping, large‑scale inference, or a flexible platform for experimenting with novel Bayesian topic models. Future work will expand the library with additional BP‑based algorithms and continue community‑driven development.