Error correcting code using tree-like multilayer perceptron

An error correcting code using a tree-like multilayer perceptron is proposed. An original message $\mbi{s}^0$ is encoded into a codeword $\boldmath{y}_0$ using a tree-like committee machine (committee tree) or a tree-like parity machine (parity tree). Based on these architectures, several schemes featuring monotonic or non-monotonic units are introduced. The codeword $\mbi{y}_0$ is then transmitted via a Binary Asymmetric Channel (BAC) where it is corrupted by noise. The analytical performance of these schemes is investigated using the replica method of statistical mechanics. Under some specific conditions, some of the proposed schemes are shown to saturate the Shannon bound at the infinite codeword length limit. The influence of the monotonicity of the units on the performance is also discussed.

💡 Research Summary

The paper introduces a novel class of error‑correcting codes built on tree‑structured multilayer perceptrons, specifically the Committee Tree and the Parity Tree. An original binary message vector (\mathbf{s}^0) is first processed by one of these neural architectures, producing a high‑dimensional codeword (\mathbf{y}_0). In the Committee Tree, each hidden layer aggregates its inputs by a majority‑vote rule, while the Parity Tree computes the XOR of its hidden units to generate the final bits. The codeword is then transmitted over a Binary Asymmetric Channel (BAC), characterized by distinct flip probabilities (p) (0→1) and (q) (1→0), which reflects realistic non‑symmetric noise.

Four coding schemes are examined: each tree type combined with either monotonic (sigmoidal) or non‑monotonic (asymmetric) activation functions. The statistical‑mechanics replica method is employed to evaluate the average free energy of the system, allowing the authors to derive the relationship between the code rate (R) and the bit‑error probability (P_e) in the thermodynamic limit (infinite codeword length). Crucially, the analysis shows that when non‑monotonic units are used and the tree depth (K) and branching factor (C) are sufficiently large, the replica‑symmetric solution remains stable and the achievable rate saturates the Shannon capacity of the BAC. In other words, the proposed schemes can reach the Shannon bound under these conditions.

Conversely, schemes that rely solely on monotonic units suffer replica‑symmetry breaking, leading to a sub‑optimal free‑energy landscape and a rate strictly below channel capacity. Numerical simulations corroborate the theoretical predictions: non‑monotonic Committee and Parity Trees consistently achieve lower bit‑error rates than their monotonic counterparts, especially when the asymmetry (p \neq q) is pronounced. The tree architecture also offers practical advantages, such as parallelizable decoding and controllable computational complexity.

Overall, the work demonstrates that tree‑like multilayer perceptrons, when equipped with appropriately chosen non‑monotonic activation functions, constitute powerful error‑correcting codes capable of approaching the Shannon limit on asymmetric channels. The paper suggests future directions including finite‑length performance optimization, development of efficient real‑time decoding algorithms, and comparative studies with other nonlinear neural coding schemes.