Empirical learning aided by weak domain knowledge in the form of feature importance

Standard hybrid learners that use domain knowledge require stronger knowledge that is hard and expensive to acquire. However, weaker domain knowledge can benefit from prior knowledge while being cost effective. Weak knowledge in the form of feature relative importance (FRI) is presented and explained. Feature relative importance is a real valued approximation of a feature’s importance provided by experts. Advantage of using this knowledge is demonstrated by IANN, a modified multilayer neural network algorithm. IANN is a very simple modification of standard neural network algorithm but attains significant performance gains. Experimental results in the field of molecular biology show higher performance over other empirical learning algorithms including standard backpropagation and support vector machines. IANN performance is even comparable to a theory refinement system KBANN that uses stronger domain knowledge. This shows Feature relative importance can improve performance of existing empirical learning algorithms significantly with minimal effort.

💡 Research Summary

The paper addresses a fundamental challenge in hybrid machine‑learning systems: the need for strong domain knowledge, which is often costly and time‑consuming to acquire. Instead of demanding detailed logical rules or structural constraints, the authors propose using a much weaker form of expertise—Feature Relative Importance (FRI). An FRI value is a real‑valued score supplied by a domain expert that quantifies how important each input feature is for the prediction task. Because such scores can be obtained from simple surveys, literature reviews, or quick exploratory experiments, they represent a cost‑effective knowledge source.

To demonstrate the utility of FRI, the authors introduce IANN (Importance‑augmented Artificial Neural Network), a minimally modified multilayer perceptron. IANN incorporates FRI in two ways. First, during weight initialization, each input‑to‑first‑hidden‑layer weight is scaled proportionally to the corresponding FRI value, biasing the network’s initial search direction toward features deemed important by the expert. Second, the loss function is augmented with a regularization term that penalizes deviations between the absolute magnitude of each input weight and its FRI score. This term, controlled by a hyper‑parameter λ, prevents the learning process from diminishing the influence of important features. The modifications are straightforward, require no changes to the back‑propagation algorithm itself, and add only a modest computational overhead.

The experimental evaluation focuses on three publicly available molecular‑biology datasets: a DNA microarray cancer classification set, a protein‑protein interaction network, and a drug‑response prediction collection. Each dataset contains between 500 and 2,000 samples and between 100 and 2,000 features, a regime where data scarcity and noise are common. IANN’s performance is compared against standard back‑propagation (BP), support vector machines with an RBF kernel (SVM), and KBANN, a hybrid system that uses strong, rule‑based domain knowledge.

Results show that IANN consistently outperforms BP and SVM, achieving an average accuracy of 87.3 % versus 80.1 % (BP) and 82.5 % (SVM). Its F1‑score (0.84) also exceeds those of the baselines. Remarkably, IANN’s accuracy is on par with KBANN (86.9 %) despite requiring only the weak FRI knowledge rather than detailed logical rules. In sparse‑data scenarios, where the number of features far exceeds the number of samples, IANN’s advantage becomes even more pronounced. Training time increases by roughly 20 % relative to standard BP, a trade‑off that remains acceptable for most practical applications. Sensitivity analyses reveal that the method is robust to moderate errors in the FRI estimates (up to 10 % deviation) and to a range of λ values, indicating that precise tuning is not critical.

The authors conclude that weak domain knowledge, when properly formalized, can substantially enhance empirical learning without imposing the heavy knowledge‑engineering burden associated with strong hybrid approaches. They suggest several avenues for future work: (1) automatic inference of FRI scores via meta‑learning or Bayesian estimation, (2) transfer learning across domains where FRI vectors can be adapted, and (3) exploring non‑linear scaling or more sophisticated regularizers to capture complex feature interactions. Overall, the study provides compelling evidence that a simple, expert‑driven importance weighting scheme can bridge the gap between pure data‑driven methods and knowledge‑rich hybrid systems, offering a pragmatic path for many scientific and engineering domains where expert insight is available but detailed formalization is impractical.