Impacts of Racial Bias in Historical Training Data for News AI
📝 Abstract
AI technologies have rapidly moved into business and research applications that involve large text corpora, including computational journalism research and newsroom settings. These models, trained on extant data from various sources, can be conceptualized as historical artifacts that encode decades-old attitudes and stereotypes. This paper investigates one such example trained on the broadly-used New York Times Annotated Corpus to create a multi-label classifier. Our use in research settings surfaced the concerning “blacks” thematic topic label. Through quantitative and qualitative means we investigate this label’s use in the training corpus, what concepts it might be encoding in the trained classifier, and how those concepts impact our model use. Via the application of explainable AI methods, we find that the “blacks” label operates partially as a general “racism detector” across some minoritized groups. However, it performs poorly against expectations on modern examples such as COVID-19 era anti-Asian hate stories, and reporting on the Black Lives Matter movement. This case study of interrogating embedded biases in a model reveals how similar applications in newsroom settings can lead to unexpected outputs that could impact a wide variety of potential uses of any large language model-story discovery, audience targeting, summarization, etc. The fundamental tension this exposes for newsrooms is how to adopt AI-enabled workflow tools while reducing the risk of reproducing historical biases in news coverage.
💡 Analysis
AI technologies have rapidly moved into business and research applications that involve large text corpora, including computational journalism research and newsroom settings. These models, trained on extant data from various sources, can be conceptualized as historical artifacts that encode decades-old attitudes and stereotypes. This paper investigates one such example trained on the broadly-used New York Times Annotated Corpus to create a multi-label classifier. Our use in research settings surfaced the concerning “blacks” thematic topic label. Through quantitative and qualitative means we investigate this label’s use in the training corpus, what concepts it might be encoding in the trained classifier, and how those concepts impact our model use. Via the application of explainable AI methods, we find that the “blacks” label operates partially as a general “racism detector” across some minoritized groups. However, it performs poorly against expectations on modern examples such as COVID-19 era anti-Asian hate stories, and reporting on the Black Lives Matter movement. This case study of interrogating embedded biases in a model reveals how similar applications in newsroom settings can lead to unexpected outputs that could impact a wide variety of potential uses of any large language model-story discovery, audience targeting, summarization, etc. The fundamental tension this exposes for newsrooms is how to adopt AI-enabled workflow tools while reducing the risk of reproducing historical biases in news coverage.
📄 Content
Newsrooms are rapidly integrating a variety of AI tools into their reporting, writing, and publishing workflows. AI tools promise to improve tasks such as story recommendation, story customization, content summarization, and more. At the center of these tools is the representation of word usage probabilities as high-dimension vectors via a large language model (LLM). However, beneath this veneer of technological sophistication lies the conceptualization of these futuristic “things” as historical artifacts that encode attitudes, stereotypes, and language norms of their training data [25].
The field of algorithmic auditing offers techniques to understand this kind of historical bias that can be applied in news-related settings. While definitions of “bias” vary [27], here we engage the particular definition that relates to human prejudices embedded in training data that are encoded into a model. In this paper we explore related concerns through a case study of a multi-label classifier trained on the New York Times Annotated Corpus [23]; the classifier model further used the Google News word2vec [17] as the vectorizer. While situated in a journalism research context as opposed to in a newsroom, the technical task and software architecture are analogous to similar text classifiers used in reporting, editing, recommending, and other publishing areas.
In usage, this classifier revealed a potentially problematic thematic topic label: blacks. To a contemporary reader, this descriptive for African Americans or Black Americans sounds antiquated and othering. Using a variety of auditing methods, we found that the label encodes racial attitudes from previous decades that could systematically distort contemporary news analysis tools. Through this case study we demonstrate two critical insights that have broad implications for AI-based computational journalism: first, that historical training data can fundamentally misrepresent the social categories it claims to classify, and second, that temporal gaps between training and application create systematic oversights that pose risks to newsroom AI systems. Understanding and addressing these are essential for ensuring that journalism’s technological evolution supports rather than undermines its democratic mission. This case study contributes to our growing understanding of this challenge, offering one example of how that can manifest in reproducing historical prejudices, and documents a set of methods one could undertake to assess these potential impacts.
In research domains, scholars have widely adopted and discussed the role classifiers can play in understanding large corpora of news text [1]. Various work has surveyed opportunities and challenges related to newsrooms integration [4,26].
In newsroom domains, regular reports in industry trade publications document applications in newsrooms small and large [2,12].
The domains include fact-checking, summarization, personalization, and beyond. While some work explores potential biases and dangers, the question of concrete impacts on news analysis and production is often unexplored in depth. Some contemporary organizational leaders believe that using AI classifiers on news can help eliminate bias [5,7].
Many ML applications operate as “black boxes” [21] where the implications of misinterpretations can lead to miscommunication or misinformation [9]. “Post-hoc” auditing is one approach that can help understand a model that has already been developed and deployed [14]. These tools attempt to discern the behavior of complex models via proxies that are more understandable to human researchers [10].
Over the last few years there has also been notable attention paid to the question of how LLMs become outdated. Computation scholars have found evidence of “temporal bias” in LLMs, and explored various techniques for adding in contemporary context [28,31]. More relevant for this case study, Mozzherina documented and suggested remedies for temporal inconsistencies in the NYT Annotated Corpus, specifically. Their work reassigned labels based on clustering and iterative evaluation, yielding reduced redundancy in tags and increasing consistency [18].
Our focus on the racially loaded label blacks for this study requires an understanding of the historical norms about labeling and reporting on this minoritized group by U.S. media. While a thorough exposition of these norms is beyond the scope of this paper, negative portrayals of Black Americans in US media have historically exacerbated harmful stereotypes [11,16], and language used in the media to refer to Black people has changed over time [20,24,30]. Recently, many news organizations have revised their style guidelines and reporting practices in response to evolving social norms and calls for more inclusive, accurate representation [6,19]. It is critical that new technologies used in newsrooms allow for journalists to stay current with preferred language and ways of referring to and rep
This content is AI-processed based on ArXiv data.