Gender and Race Bias in Consumer Product Recommendations by Large Language Models

Large Language Models are increasingly employed in generating consumer product recommendations, yet their potential for embedding and amplifying gender and race biases remains underexplored. This paper serves as one of the first attempts to examine these biases within LLM-generated recommendations. We leverage prompt engineering to elicit product suggestions from LLMs for various race and gender groups and employ three analytical methods-Marked Words, Support Vector Machines, and Jensen-Shannon Divergence-to identify and quantify biases. Our findings reveal significant disparities in the recommendations for demographic groups, underscoring the need for more equitable LLM recommendation systems.

💡 Research Summary

This paper investigates whether large language models (LLMs), specifically GPT‑4, embed and amplify gender and race biases when generating consumer product recommendations. The authors design a prompt‑engineering pipeline that asks the model to produce ten product suggestions and a two‑sentence rationale for each, tailored to a given demographic persona (e.g., “Asian female”, “Black male”). The output is forced into a strict JSON format, enabling systematic parsing into two fields: “item text” (the product categories) and “reason text” (the explanatory sentences).

Fifteen demographic groups are defined using the “Marked vs. Unmarked” framework introduced by Cheng et al. (2023): the marked groups consist of four racial minorities (Asian, Black, Latino, Middle‑Eastern) and three gender minorities (woman, non‑binary), while the unmarked reference group is White male. For each group, the model is queried fifteen times, yielding a total of 225 responses. The temperature is set to 1.0 to balance diversity and consistency.

To uncover implicit bias, the study applies three complementary analytical techniques.

1. Marked Words – This method computes weighted log‑odds ratios with a Dirichlet prior and Laplace smoothing for each word, then derives a z‑score to assess statistical significance. An illustrative example shows the word “rice” receiving a log‑odds of 3.77 and a z‑score of 2.28 for the “Asian women” group, indicating a strong association with that demographic.
2. Support Vector Machine (SVM) – After anonymizing the text by removing explicit gender, race, and title terms, a linear SVM is trained to distinguish each marked group from the unmarked baseline. The top‑10 coefficients reveal the most discriminative words: for Black users, terms such as “hair”, “oil”, “balm”, and “conditioner” dominate, whereas for Asian users, “facial”, “cream”, “tea”, “rice”, and “sheet mask” are most salient.
3. Jensen‑Shannon Divergence (JSD) – The authors calculate JSD between the word‑frequency distributions of marked and unmarked groups, providing an information‑theoretic measure of divergence. Words contributing most to the divergence align with the SVM and Marked Words findings, confirming consistent bias signals across methods.

The empirical results demonstrate pronounced disparities. Recommendations for Black personas are heavily weighted toward personal‑care and grooming products, reflecting stereotypical expectations about appearance. Asian personas receive a concentration of skincare and culturally specific items (e.g., tea, rice), echoing common beauty‑routine stereotypes. Middle‑Eastern personas exhibit a mixed pattern of modern technology (“smartphone”) and traditional items (“purifier”, “perfume”), suggesting a dual cultural framing. These patterns indicate that the LLM reproduces societal stereotypes embedded in its training corpus, even when the prompt is ostensibly neutral.

The paper’s contribution lies in adapting three bias‑detection tools—originally devised for political discourse or persona analysis—to the concrete, commercial scenario of product recommendation. By integrating prompt engineering, structured data extraction, and multi‑method analysis, the study provides a reproducible pipeline for auditing LLM‑driven recommendation systems.

In the discussion, the authors argue that such implicit biases can affect user experience, reinforce inequitable access to products, and undermine trust in AI‑mediated commerce. They propose mitigation strategies including refined prompting, post‑generation filtering, and fine‑tuning on debiased corpora. Future work is suggested to incorporate real‑world user feedback, extend the analysis to other LLM architectures, and develop quantitative fairness metrics tailored to recommendation contexts.

Overall, the study highlights that while LLMs offer powerful personalization capabilities, they also risk perpetuating gender and race stereotypes in consumer recommendations. Systematic bias detection and mitigation are essential steps toward building equitable AI‑driven recommendation ecosystems.