Automatic Construction of Pattern Classifiers Capable of Continuous Incremental Learning and Unlearning Tasks Based on Compact-Sized Probabilistic Neural Network

This paper proposes a novel approach to pattern classification using a probabilistic neural network model. The strategy is based on a compact-sized probabilistic neural network capable of continuous incremental learning and unlearning tasks. The network is constructed/reconstructed using a simple, one-pass network-growing algorithm with no hyperparameter tuning. Then, given the training dataset, its structure and parameters are automatically determined and can be dynamically varied in continual incremental and decremental learning situations. The algorithm proposed in this work involves no iterative or arduous matrix-based parameter approximations but a simple data-driven updating scheme. Simulation results using nine publicly available databases demonstrate the effectiveness of this approach, showing that compact-sized probabilistic neural networks constructed have a much smaller number of hidden units compared to the original probabilistic neural network model and yet can achieve a similar classification performance to that of multilayer perceptron neural networks in standard classification tasks, while also exhibiting sufficient capability in continuous class incremental learning and unlearning tasks.

💡 Research Summary

The paper introduces a novel pattern‑classification framework based on a compact‑sized Probabilistic Neural Network (CS‑PNN) that can automatically grow and shrink without any hyper‑parameter tuning. Traditional deep neural networks (DNNs) suffer from extensive hyper‑parameter search, catastrophic forgetting during incremental learning, and the need for replay buffers when trying to preserve past knowledge. Probabilistic Neural Networks (PNNs), which consist of a three‑layer architecture (input, radial‑basis‑function hidden layer, linear output layer), avoid many of these issues because each hidden unit directly stores a training sample as a centroid. However, the original PNN requires a hidden unit for every training pattern, leading to large memory footprints and over‑fitting.

The authors address these drawbacks by (i) allowing the radius σ of each RBF unit to vary dynamically according to the current number of classes k and the maximal inter‑sample distance d_max, i.e., σ = d_max / k; (ii) eliminating the heuristic threshold θ that decides when to add a new RBF; instead, a new hidden unit is created only when the incoming sample is mis‑classified. This “error‑driven” growth guarantees that the network expands only when necessary, dramatically reducing the number of hidden units. (iii) For unlearning, the network simply removes the hidden units (or entire class‑specific sub‑nets) associated with the unwanted data, without any additional weight‑adjustment or retraining. Because each hidden unit is linked to a single output class via binary connections, deletion is straightforward.

Four algorithms formalize the process: (1) Construction/Reconstruction for both instance‑wise incremental learning (IIL) and class‑wise incremental learning (CIL); (2) Instance‑wise unlearning; (3) Class‑wise unlearning (class decremental learning, CDL); (4) Testing. During construction, the algorithm scans the training set once. If a sample belongs to a previously unseen class, a new output node and a corresponding hidden unit are added. If the class already exists, the algorithm computes the current d_max, updates σ, evaluates all hidden units, and either adds a new unit (if mis‑classification occurs) or updates the most activated unit’s centroid (if correctly classified). The d_max computation is performed in parallel with the feed‑forward pass, incurring negligible overhead. Unlearning merely deletes the specified hidden units or sub‑nets; the radius is automatically recomputed during the next testing phase.

The experimental study uses nine publicly available datasets (UCI’s abalone, ionosphere, isolet, letter‑recognition, sat, segmentation; plus MNIST, optdigits, pendigits). All features are normalized to