MOANOFS: Multi-Objective Automated Negotiation based Online Feature Selection System for Big Data Classification

Feature Selection (FS) plays an important role in learning and classification tasks. The object of FS is to select the relevant and non-redundant features. Considering the huge amount number of features in real-world applications, FS methods using batch learning technique can’t resolve big data problem especially when data arrive sequentially. In this paper, we propose an online feature selection system which resolves this problem. More specifically, we treat the problem of online supervised feature selection for binary classification as a decision-making problem. A philosophical vision to this problem leads to a hybridization between two important domains: feature selection using online learning technique (OFS) and automated negotiation (AN). The proposed OFS system called MOANOFS (Multi-Objective Automated Negotiation based Online Feature Selection) uses two levels of decision. In the first level, from n learners (or OFS methods), we decide which are the k trustful ones (with high confidence or trust value). These elected k learners will participate in the second level. In this level, we integrate our proposed Multilateral Automated Negotiation based OFS (MANOFS) method to decide finally which is the best solution or which are relevant features. We show that MOANOFS system is applicable to different domains successfully and achieves high accuracy with several real-world applications. Index Terms: Feature selection, online learning, multi-objective automated negotiation, trust, classification, big data.

💡 Research Summary

The paper addresses the challenge of online feature selection (OFS) for binary classification in the context of massive, high‑dimensional data streams where features arrive sequentially. Traditional batch‑mode feature selection methods are unsuitable for such environments, and existing OFS techniques, while promising, typically rely on a single learner and do not explicitly handle the conflict that may arise when multiple OFS algorithms propose different feature subsets. To overcome these limitations, the authors propose MOANOFS (Multi‑Objective Automated Negotiation based Online Feature Selection), a novel system that integrates two decision‑making layers: a trust‑based learner selection layer and a multilateral automated negotiation layer.

In the first layer, a pool of n existing OFS methods (e.g., PETrun, FOFS, SOFS, SAOLA, OS, Grafting) is evaluated continuously. Each learner’s recent prediction accuracy, loss reduction, and stability of selected features are combined into a trust score. The top‑k learners with the highest trust values are elected to participate in the second layer, thereby focusing computational resources on the most reliable algorithms while discarding noisy or unstable ones.

The second layer, called MANOFS, treats the selected k learners as negotiating agents. Each agent proposes a candidate feature subset together with associated utility values for multiple issues: classification performance, number of selected features, and the learner’s trust level. Utilities are modeled as weighted linear combinations of issue‑specific scores, and the negotiation follows a time‑dependent strategy: agents gradually concede as the deadline approaches, allowing the system to explore diverse proposals early and converge to a consensus later. A mediator coordinates the exchange of offers and counter‑offers, ensuring that all agents have access to the current negotiation state. The final agreement—i.e., the feature subset that maximizes the aggregated utility—becomes the output of MOANOFS and is used for downstream classification.

Algorithmically, MOANOFS operates in an online fashion: for each incoming instance, every learner updates its weight vector, truncates to retain the B largest absolute coefficients, and computes its trust update. After trust re‑evaluation, the negotiation round is triggered among the trusted learners. Because both the trust update and the negotiation are linear‑time operations with respect to the number of active features, the overall computational overhead remains modest, making the approach suitable for real‑time streaming scenarios.

The authors validate MOANOFS on more than ten public datasets spanning text, image, and biomedical domains, each characterized by thousands to millions of features. Compared with individual OFS baselines and simple ensemble schemes, MOANOFS consistently achieves higher classification accuracy (typically 3–7 % improvement) while selecting a compact feature set. The experiments also demonstrate that the system scales gracefully: as the dimensionality grows, the negotiation converges quickly, and the trust‑based pruning prevents an explosion of candidate learners.

Beyond the empirical results, the paper discusses several extensions. Trust metrics could be enriched with additional resource‑aware criteria such as memory consumption or latency, and utility functions could adopt non‑linear forms to capture more complex trade‑offs. The framework is also adaptable to multi‑class classification, regression, and even non‑tabular data (e.g., graphs or time series) by redefining the negotiation issues accordingly.

In summary, MOANOFS introduces a principled way to combine multiple online feature selectors through automated negotiation, thereby resolving conflicts among learners, exploiting their complementary strengths, and achieving robust, high‑performance feature selection in big‑data streaming environments. This work opens new avenues for integrating decision‑theoretic negotiation mechanisms into other online machine‑learning pipelines, such as model selection, hyper‑parameter tuning, and ensemble learning.