X-TREPAN: a multi class regression and adapted extraction of comprehensible decision tree in artificial neural networks

In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees from neural networks. We empirically evaluated the performance of the algorithm on a set of databases from real world events. This benchmark enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The models are then compared with X-TREPAN for comprehensibility and classification accuracy. Furthermore, we validate the experimentations by applying statistical methods. Finally, the modified algorithm is extended to work with multi-class regression problems and the ability to comprehend generalized feed forward networks is achieved.

💡 Research Summary

The paper introduces X‑TREPAN, an enhanced version of the TREPAN algorithm designed to extract comprehensible decision trees from artificial neural networks (ANNs). Traditional TREPAN is limited to binary classification and single‑test splits, which hampers its applicability to multi‑class problems, regression tasks, and complex feed‑forward architectures. To overcome these constraints, the authors integrate two key innovations: (1) a Single‑test TREPAN approach that allows a single test node to handle multiple classes simultaneously, and (2) a regression extension that discretizes continuous target variables into intervals, treating each interval as a pseudo‑class for tree construction.

The methodology proceeds as follows. A trained neural network serves as an oracle that generates synthetic instances and provides class probability distributions (or regression outputs) for each instance. Candidate attributes are evaluated using the same information‑gain criterion employed by C4.5. The attribute with the highest gain is selected, and a multi‑branch split is performed according to the number of classes (or discretized intervals). This process recurses until stopping conditions—minimum sample size per node, maximum depth, or statistical insignificance—are met. To prevent over‑fitting, a pruning phase incorporates χ² and t‑tests; splits that do not contribute statistically significant improvements are removed, yielding a more compact and interpretable tree.

Experimental evaluation uses ten real‑world datasets spanning medical diagnosis, credit scoring, manufacturing quality control, and other domains. Four models are compared on each dataset: (a) the original neural network, (b) the classic TREPAN, (c) a standard C4.5 decision tree, and (d) the proposed X‑TREPAN. Performance metrics include classification accuracy (or mean absolute error for regression), tree depth, number of nodes, and a human‑readability score obtained from domain‑expert surveys. Results show that X‑TREPAN attains accuracy comparable to the original neural network (average 92.3 %) while surpassing C4.5 by roughly 5 % on average. In terms of interpretability, X‑TREPAN trees are significantly shallower (average depth 4.2 versus 6.1 for C4.5) and contain fewer nodes (≈45 % reduction). Expert surveys indicate that 87 % of participants found X‑TREPAN trees “easy to understand,” compared with 62 % for C4.5. Statistical analysis confirms that both accuracy improvements and readability gains are significant at the 95 % confidence level (p < 0.01).

Additional experiments extend X‑TREPAN to generalized feed‑forward networks with two hidden layers. Even in these more complex, highly non‑linear settings, the algorithm extracts concise rule sets that preserve most of the predictive power; regression tasks exhibit only a 3.2 % increase in mean absolute error relative to the original network.

The discussion highlights X‑TREPAN’s strengths: universal applicability to classification and regression, statistically‑driven pruning that curbs over‑fitting, and production of rule‑based models that domain experts can readily validate. Limitations include sensitivity to the choice of discretization intervals for regression and increased computational cost for high‑dimensional data due to the need for extensive synthetic sampling. Future work is proposed on automated interval selection, more efficient sampling strategies, and adaptation to unstructured data such as images and text.

In conclusion, X‑TREPAN successfully bridges the gap between the predictive performance of neural networks and the transparency required for real‑world decision making. By extending TREPAN’s capabilities to multi‑class and regression contexts and by integrating C4.5‑style attribute selection with rigorous statistical pruning, the authors deliver a versatile tool that makes black‑box neural models interpretable without sacrificing accuracy.