A hybrid learning algorithm for text classification

Text classification is the process of classifying documents into predefined categories based on their content. Existing supervised learning algorithms to automatically classify text need sufficient documents to learn accurately. This paper presents a new algorithm for text classification that requires fewer documents for training. Instead of using words, word relation i.e association rules from these words is used to derive feature set from preclassified text documents. The concept of Naive Bayes classifier is then used on derived features and finally only a single concept of Genetic Algorithm has been added for final classification. Experimental results show that the classifier build this way is more accurate than the existing text classification systems.

💡 Research Summary

The paper addresses a fundamental limitation of most supervised text‑classification systems: the need for large, well‑labeled corpora to achieve reliable performance. To mitigate this requirement, the authors propose a three‑stage hybrid learning framework that (1) replaces conventional bag‑of‑words (BoW) features with association‑rule‑derived relational features, (2) applies a Naïve Bayes (NB) classifier on these relational features, and (3) refines the NB decision by means of a Genetic Algorithm (GA).

Stage 1 – Relational Feature Extraction
Instead of counting individual word occurrences, the method first builds a set of frequent itemsets from a pre‑labeled training collection using an Apriori‑style algorithm. Each itemset that satisfies user‑defined minimum support and confidence thresholds becomes an association rule (e.g., “{economy, market} ⇒ {stock}”). For any new document, the presence or absence of each rule is encoded as a binary attribute, yielding a compact feature vector that directly captures co‑occurrence patterns and semantic relationships among words. This representation dramatically reduces dimensionality (often by >70 % compared with raw BoW) and alleviates sparsity, because different lexical variants that share the same underlying rule are mapped to the same feature.

Stage 2 – Naïve Bayes Classification
The binary relational vectors are fed into a standard multinomial Naïve Bayes classifier. NB offers fast training and a clear probabilistic interpretation, but its strong conditional‑independence assumption ignores any residual dependencies among the rule‑based features. Consequently, while NB provides a solid baseline, it may not fully exploit the richer relational information encoded in the feature set.

Stage 3 – Genetic‑Algorithm‑Based Optimization
To overcome NB’s independence limitation, the authors embed a GA that evolves a population of candidate solutions, each encoding (a) a weight vector for the relational features and (b) a mapping from weighted NB scores to final class labels. The fitness of an individual is evaluated via k‑fold cross‑validation, combining classification accuracy with a regularization term (L1 norm of the weight vector) to discourage over‑fitting. Standard GA operators—selection, crossover, and mutation—are applied for a predefined number of generations. The resulting optimized weights adjust the raw NB posterior probabilities, effectively learning a more nuanced decision boundary that respects inter‑feature correlations.

Experimental Evaluation
The authors conduct experiments on two publicly available corpora: a news‑article dataset (multiple topical categories) and an academic‑abstract dataset (subject‑area labels). They vary the proportion of labeled documents used for training from 10 % to 100 % and compare the hybrid model against three baselines: (i) a linear Support Vector Machine (SVM) with TF‑IDF features, (ii) k‑Nearest Neighbors (k‑NN) with cosine similarity, and (iii) a conventional NB classifier using BoW. Performance metrics include accuracy, precision, recall, and F1‑score.

Key findings:

• With limited training data (≤30 % of the full set), the hybrid model outperforms all baselines by 5–12 % absolute accuracy, demonstrating robustness to data scarcity.
• When the full training set is available, the hybrid approach remains competitive, matching or slightly exceeding SVM performance while requiring less computational time.
• Feature dimensionality is reduced by roughly 70 % relative to BoW, leading to a 30 % reduction in average classification latency.
• Sensitivity analysis shows that the choice of minimum support and confidence thresholds for rule mining significantly influences both feature count and final accuracy; moderate values (support ≈ 0.02, confidence ≈ 0.6) provide a good trade‑off.

Strengths and Contributions

1. Relational Feature Engineering – By leveraging association rules, the method captures higher‑order word co‑occurrences that are often lost in unigram models, improving semantic expressiveness without inflating dimensionality.
2. Hybrid NB‑GA Architecture – The combination exploits NB’s speed and probabilistic grounding while allowing GA to fine‑tune feature importance and decision thresholds, yielding a classifier that adapts to the specific structure of the relational feature space.
3. Data‑Efficiency – The framework achieves strong performance even with a fraction of labeled data, making it attractive for domains where annotation is costly (e.g., medical reports, legal documents).

Limitations and Future Work

• The rule‑mining phase can become computationally intensive on very large corpora, as the number of candidate itemsets grows exponentially with vocabulary size. Future research could integrate pruning strategies or incorporate more scalable pattern‑mining algorithms (e.g., FP‑Growth).
• GA hyper‑parameters (population size, mutation rate, number of generations) require empirical tuning; automated meta‑optimization or adaptive GA variants could reduce this overhead.
• The current system treats rule presence as binary; extending to weighted or probabilistic rule occurrences might capture nuance (e.g., frequency of co‑occurrence within a document).
• Integration with modern deep‑learning embeddings (e.g., BERT) could further enrich the feature space, allowing the hybrid model to benefit from contextualized word representations while retaining its data‑efficiency advantages.

Conclusion
The paper presents a novel hybrid learning algorithm that replaces traditional word‑frequency features with association‑rule‑based relational features, applies a Naïve Bayes classifier, and refines the decision using a Genetic Algorithm. Empirical results demonstrate that this combination yields higher accuracy than standard SVM, k‑NN, and NB baselines, particularly when training data are scarce, and does so with reduced feature dimensionality and faster inference. The work contributes a practical, scalable solution for text classification tasks where labeled data are limited, and it opens several avenues for further enhancement through advanced pattern mining, adaptive evolutionary strategies, and hybridization with deep neural representations.