Strengthening False Information Propagation Detection: Leveraging SVM and Sophisticated Text Vectorization Techniques in comparison to BERT

The rapid spread of misinformation, particularly through online platforms, underscores the urgent need for reliable detection systems. This study explores the utilization of machine learning and natural language processing, specifically Support Vector Machines (SVM) and BERT, to detect fake news. We employ three distinct text vectorization methods for SVM: Term Frequency Inverse Document Frequency (TF-IDF), Word2Vec, and Bag of Words (BoW), evaluating their effectiveness in distinguishing between genuine and fake news. Additionally, we compare these methods against the transformer large language model, BERT. Our comprehensive approach includes detailed preprocessing steps, rigorous model implementation, and thorough evaluation to determine the most effective techniques. The results demonstrate that while BERT achieves superior accuracy with 99.98% and an F1-score of 0.9998, the SVM model with a linear kernel and BoW vectorization also performs exceptionally well, achieving 99.81% accuracy and an F1-score of 0.9980. These findings highlight that, despite BERT’s superior performance, SVM models with BoW and TF-IDF vectorization methods come remarkably close, offering highly competitive performance with the advantage of lower computational requirements.

💡 Research Summary

The paper addresses the pressing problem of fake‑news proliferation by empirically comparing a classic machine‑learning approach—Support Vector Machines (SVM)—with a modern transformer‑based language model, BERT‑base. The authors use the ISO T “Fake and Real News Dataset”, which contains 21,477 genuine and 23,421 fabricated news articles, split into an 80 % training set (18,796 fake, 17,121 real) and a 20 % test set (4,624 fake, 4,356 real).

Data preprocessing follows a standard pipeline: stop‑word removal (NLTK), tokenization with gensim’s simple_preprocess, and discarding tokens shorter than three characters. For BERT, the same raw texts are tokenized with the bert‑tokenizer from the uncased BERT‑base model.

Three text‑vectorization strategies are applied to the SVM pipeline:

1. TF‑IDF – a sparse representation that weights term frequency by inverse document frequency.
2. Word2Vec (CBOW) – dense 300‑dimensional embeddings trained on the corpus, capturing semantic similarity.
3. Bag‑of‑Words (BoW) – a high‑dimensional count matrix that records raw term frequencies.

Each representation is fed to two SVM variants: a linear kernel and a radial basis function (RBF) kernel. Hyper‑parameters (C for the linear SVM, γ for the RBF) are tuned via cross‑validation.

The BERT‑base model is fine‑tuned for only three epochs with a learning rate of 2e‑5, batch size 16, and evaluated on the same test split.

Results:

• SVM‑Linear – BoW achieves the highest performance (99.81 % accuracy, 0.9980 F1), TF‑IDF follows closely (99.52 % accuracy, 0.9949 F1), while Word2Vec lags (96.54 % accuracy, 0.9644 F1).
• SVM‑RBF – modest gains are observed: BoW‑RBF reaches 99.62 % accuracy (0.9961 F1) and TF‑IDF‑RBF 99.31 % accuracy (0.9928 F1). Word2Vec‑RBF remains the weakest (97.75 % accuracy, 0.9767 F1).
• BERT‑base – after three epochs, it attains 99.98 % accuracy and 0.9998 F1, essentially perfect classification.

The authors note that while BERT’s performance is marginally superior, its computational demands (GPU memory, inference latency) are substantially higher than those of the SVM models, which can run efficiently on CPUs with model sizes in the megabyte range.

Discussion: The study demonstrates that simple, frequency‑based representations (BoW, TF‑IDF) combined with a linear SVM are sufficient to achieve near‑state‑of‑the‑art fake‑news detection on this dataset. The RBF kernel can capture limited non‑linear patterns, offering slight improvements in some cases, but the added complexity does not outweigh the benefits of the linear approach for this task.

Limitations and Future Work: The dataset is English‑only and domain‑specific (political and world news), limiting generalizability to multilingual or cross‑domain scenarios. The comparison excludes larger transformer variants (e.g., BERT‑large, RoBERTa) and lightweight alternatives (DistilBERT, TinyBERT) that could offer better trade‑offs. The authors propose exploring hybrid vectorizations (e.g., concatenating BoW and Word2Vec), ensemble classifiers, and integrating compact transformer models with SVM to further balance accuracy and efficiency.

In conclusion, the paper provides a thorough empirical benchmark, showing that a well‑tuned BoW‑SVM can rival the accuracy of heavyweight transformer models while remaining computationally inexpensive, making it a viable solution for real‑time or resource‑constrained fake‑news detection systems.