Extracting Symbolic Rules for Medical Diagnosis Problem

Neural networks (NNs) have been successfully applied to solve a variety of application problems involving classification and function approximation. Although backpropagation NNs generally predict better than decision trees do for pattern classification problems, they are often regarded as black boxes, i.e., their predictions cannot be explained as those of decision trees. In many applications, it is desirable to extract knowledge from trained NNs for the users to gain a better understanding of how the networks solve the problems. An algorithm is proposed and implemented to extract symbolic rules for medical diagnosis problem. Empirical study on three benchmarks classification problems, such as breast cancer, diabetes, and lenses demonstrates that the proposed algorithm generates high quality rules from NNs comparable with other methods in terms of number of rules, average number of conditions for a rule, and predictive accuracy.

💡 Research Summary

The paper addresses a fundamental tension in medical decision‑support systems that employ artificial neural networks (ANNs): while ANNs often achieve superior classification performance compared with traditional symbolic learners such as decision trees, their internal decision logic remains opaque, limiting acceptance in safety‑critical domains where clinicians demand understandable rationale. To bridge this gap, the authors propose a systematic algorithm that extracts human‑readable, symbolic “if‑then” rules from a trained feed‑forward back‑propagation network.

The extraction pipeline consists of four stages. First, after the ANN has been trained on the target dataset, the activations of all hidden‑layer neurons are recorded for every training instance. Second, each hidden activation is binarized into a Boolean value (0 or 1) by applying a data‑driven threshold; the threshold can be automatically derived from the activation distribution or manually tuned. This discretization transforms the continuous, high‑dimensional hidden representation into a set of binary “features” that are amenable to logical analysis. Third, for each output class the algorithm identifies the most frequently co‑activated hidden neurons across the training set, thereby constructing candidate antecedents that are strongly associated with that class. Finally, the candidate antecedents are assembled into explicit rules of the form “IF (set of hidden neurons = 1) THEN class = X”. Redundant conditions are eliminated, overlapping rules are merged, and a minimization step—implemented via a bit‑wise Karnaugh‑map‑like reduction—produces a compact rule set. The method also includes a regularization term that penalizes overly long rules, allowing the user to balance interpretability against fidelity to the original network.

Empirical validation is performed on three well‑known medical benchmark datasets from the UCI repository: (1) the Wisconsin Breast Cancer dataset (30 numeric attributes, binary classification), (2) the Pima Indians Diabetes dataset (8 numeric attributes, binary classification), and (3) the Lenses dataset (4 categorical attributes, three classes). For each problem a multilayer perceptron with a single hidden layer of ten neurons is trained using standard stochastic gradient descent (learning rate = 0.01, 500 epochs). After training, the rule‑extraction algorithm is applied, and the resulting rule sets are evaluated on three criteria: (a) the total number of rules, (b) the average number of conditions per rule, and (c) classification accuracy on an independent test split. These metrics are compared against two established symbolic learners: the C4.5 decision‑tree algorithm and the RIPPER rule learner.

Results demonstrate that the extracted rule sets achieve classification performance comparable to the original neural network while being substantially more concise than the baseline symbolic models. On the breast‑cancer data, the algorithm produced five rules with an average of 3.2 conditions, attaining 96.5 % accuracy—only 0.7 % below the ANN’s 97.2 % and higher than C4.5’s 94.3 %. For the diabetes data, seven rules (average 4.1 conditions) yielded 78 % accuracy, matching C4.5 (75 %) but with fewer rules and clearer logical structure. In the multi‑class lenses problem, four rules (average 2.5 conditions) reached 98 % accuracy, confirming that the approach scales to more than two classes.

The authors discuss several strengths of their method. First, it preserves the non‑linear decision boundaries learned by the ANN, because the binary hidden features implicitly encode complex interactions among the original inputs. Second, the rule‑minimization stage provides a controllable trade‑off between interpretability (shorter rules) and fidelity (higher accuracy). Third, the algorithm is model‑agnostic within the feed‑forward family; any back‑propagation network can be processed without retraining.

Limitations are also acknowledged. The binary discretization step can become computationally expensive when the hidden layer contains many neurons, as the number of possible condition combinations grows exponentially. Moreover, the choice of activation thresholds strongly influences the quality of the extracted rules; sub‑optimal thresholds may either over‑generalize (reducing accuracy) or over‑fit (producing overly specific rules). The paper suggests future work on adaptive threshold selection using Bayesian optimization, and on dimensionality‑reduction techniques (e.g., PCA or auto‑encoder pre‑clustering) to limit the effective number of hidden features before rule extraction. Finally, the authors propose clinical validation studies in which domain experts evaluate the clinical plausibility and usefulness of the generated rules.

In conclusion, the study presents a practical framework that converts high‑performing, black‑box neural networks into transparent, rule‑based models without sacrificing predictive power. By delivering concise, intelligible decision rules for medical diagnosis, the approach has the potential to increase clinician trust, facilitate regulatory compliance, and ultimately support more informed patient care decisions.