Beyond speculation: Measuring the growing presence of LLM-generated texts in multilingual disinformation

Increased sophistication of large language models (LLMs) and the consequent quality of generated multilingual text raises concerns about potential disinformation misuse. While humans struggle to distinguish LLM-generated content from human-written texts, the scholarly debate about their impact remains divided. Some argue that heightened fears are overblown due to natural ecosystem limitations, while others contend that specific “longtail” contexts face overlooked risks. Our study bridges this debate by providing the first empirical evidence of LLM presence in the latest real-world disinformation datasets, documenting the increase of machine-generated content following ChatGPT’s release, and revealing crucial patterns across languages, platforms, and time periods.

💡 Research Summary

The paper “Beyond speculation: Measuring the growing presence of LLM‑generated texts in multilingual disinformation” provides the first empirical evidence that large language model (LLM)‑generated content is already embedded in real‑world disinformation datasets across multiple languages. The authors formulate three research questions: (RQ1) how prevalent are machine‑generated texts in existing disinformation corpora; (RQ2) how do they distribute between “false” and “true” labeled items; and (RQ3) how does prevalence vary across languages, platforms, and time, especially before and after the public release of ChatGPT in November 2022.

To answer these questions, the study builds two binary classifiers for detecting LLM‑generated text. Both are fine‑tuned from the 9‑b parameter Gemma‑2‑it model using QLoRA. The first detector, “Gemma_GenAI”, is trained on the GenAI benchmark (a multilingual mix of 29 generators and 15 languages). The second, “Gemma_MultiDomain”, is trained on the combined MULTITUDE and MultiSocial datasets, which contain real news and social‑media posts together with paraphrased versions generated by several LLMs. Because the two detectors are trained on different distributions, their predictions are combined: a text is labeled machine‑generated if either detector outputs a probability of 1.0 while the other does not output a strict 0.0. For languages where the Gemma_GenAI detector exceeds a 0.1 false‑positive rate (Arabic, German, Italian, Russian), its positive predictions are ignored to keep overall precision high.

The detectors are first validated on five publicly available multilingual benchmarks (MULTITUDE, MultiSocial, SemEval‑2024, GenAI, MIX). Reported performance shows precision ranging from 0.9358 to 0.9985, false‑positive rates (FPR) between 0.0025 and 0.0579, and true‑positive rates (TPR) from 0.4988 to 0.9852. Even in the worst‑case scenario (precision = 0.9358, FPR = 0.0476), at least 93 % of the texts flagged as machine‑generated can be trusted.

Armed with these robust detectors, the authors analyze four real‑world corpora that are likely to contain disinformation:

1. MultiClaim – a fact‑checked multilingual dataset of social‑media posts linked to professional fact‑checkers, covering 2019‑2024.
2. FakeNews – English news articles about the 2023 US election, annotated by GPT‑4 and subsequently verified by humans as “real” or “fake”.
3. USC_X – a sampled multilingual Twitter/X dataset from the 2024 US election period (May–Nov 2024), up to 10 k tweets per month.
4. FIGNEWS – multilingual Facebook posts (5 languages) about the Israel‑Gaza conflict (Oct 2023–Jan 2024).

Temporal trends: In MultiClaim, the mean probability (Mean Score) assigned by the detectors rises sharply after 2022. For Gemma_GenAI the mean score goes from 0.04 in 2021 to 0.36 in 2023; for Gemma_MultiDomain it rises from 0.05 to 0.27 over the same period. Translating these scores into prevalence suggests that at least 1.5 %–15 % of 2023 posts are LLM‑generated. Using the combined‑confidence rule, the proportion of detected machine‑generated posts climbs from 0.93 % (2021) to 1.85 % (2023), a 99 % relative increase. Given the worst‑case precision of 0.93, the authors conservatively estimate that ≥1.7 % of 2023 MultiClaim texts are genuinely machine‑generated or heavily modified by LLMs.

Language‑specific patterns: When aggregating only languages with ≥1 000 samples, Polish (4.7 %) and French (4.2 %) exhibit the highest relative prevalence, despite English and Spanish having the largest absolute counts.

Platform differences: Across the same dataset, Twitter/X shows 0.64 % prevalence, Facebook 1.29 %, and Instagram 1.50 %. Telegram, though showing 1.7 %, contains too few samples for reliable comparison.

Label‑based analysis: “False”, “Misleading”, and “Not categorized” claims together account for >90 % of MultiClaim items; among them, the “False” category has the highest LLM‑generated share (1.36 %). The “True” category, though much smaller (786 items), still contains 0.76 % machine‑generated posts.

Other corpora: The same detection pipeline applied to the additional datasets reveals non‑trivial LLM presence. FIGNEWS shows the highest proportion at 3.16 %; FakeNews contains 2.61 % (human‑labeled “real”) and 3.27 % (human‑labeled “fake”) machine‑generated texts. Notably, over 10 % of French posts in FIGNEWS are flagged as LLM‑generated. In USC_X, the month‑by‑month analysis for 2024 demonstrates a steady upward trajectory, suggesting that election‑related discourse is increasingly being seeded or amplified by LLMs.

Implications: The authors draw three major conclusions. First, LLM‑generated disinformation is not a speculative threat; it is already measurable in real‑world data. Second, the uneven distribution across languages and platforms indicates targeted misuse rather than a uniform background noise, underscoring the need for fine‑grained threat modeling. Third, the presented detection framework, with high precision and demonstrated out‑of‑distribution robustness, is ready for integration into real‑time monitoring, credibility assessment, and platform‑level moderation pipelines.

Future directions: The paper calls for (i) extending detection to LLM‑generated paraphrases and translations, (ii) adversarial training to harden detectors against evolving generation techniques, (iii) policy and legal frameworks that mandate provenance metadata for AI‑generated content, and (iv) cross‑disciplinary collaboration to develop standards for responsible LLM deployment.

In sum, this work bridges the gap between theoretical concerns about LLM‑driven misinformation and concrete empirical measurement, providing the research community, industry practitioners, and policymakers with a data‑driven baseline for assessing and mitigating the growing influence of AI‑generated multilingual disinformation.