Do Chatbots Walk the Talk of Responsible AI?

This study examines whether leading AI chatbot companies implement the responsible AI principles they publicly advocate. The authors used a mixed-methods approach analyzing four major chatbots (ChatGPT, Gemini, DeepSeek, and Grok) across company websites, technical documentation, and direct chatbot evaluations. We found significant gaps between corporate rhetoric and practice.

💡 Research Summary

The paper conducts a systematic audit of four leading conversational AI systems—OpenAI’s ChatGPT, Google DeepMind’s Gemini, China’s DeepSeek, and Grok—to determine whether the responsible AI principles publicly championed by their developers are actually reflected in practice. Using a mixed‑methods design, the authors first performed text‑mining of corporate websites, white‑papers, and ethical guidelines to quantify how often and in what context the five core pillars of responsible AI (fairness, transparency, privacy protection, safety, and sustainability) are mentioned. They then examined technical documentation, API specifications, and any available source code to trace concrete implementation mechanisms such as bias‑mitigation pipelines, explainability modules, data‑minimization strategies, and encryption or deletion policies. Finally, the team executed a controlled evaluation consisting of 200 carefully crafted dialogue scenarios covering sensitive sociopolitical topics, personal‑data requests, and potentially harmful prompts. Each chatbot’s responses were scored on bias, misinformation, risk, transparency (e.g., disclosure of model version or training data), and privacy handling (e.g., data retention and usage). The findings reveal a pronounced gap between rhetoric and reality. While all four firms publish comprehensive, aspirational responsible‑AI statements, the documentation rarely includes measurable metrics, third‑party audit reports, or detailed technical descriptions—especially for explainability and data‑minimization. In practice, ChatGPT showed the highest safety performance but still exhibited cultural and linguistic biases and refused to disclose model provenance. Gemini delivered strong performance yet omitted clear information about user‑data storage periods and encryption standards. DeepSeek generally rejected dangerous requests but sometimes responded with vague deflection, undermining robust risk mitigation. Grok offered the fastest replies but provided no insight into model architecture or training corpora, resulting in the lowest transparency score. Overall, fairness and safety were relatively well‑addressed, whereas transparency, privacy protection, and explainability lagged considerably. The authors recommend mandatory independent audits, integration of responsible‑AI metrics into development pipelines, real‑time disclosure interfaces for model and data provenance, and legally aligned privacy safeguards. They conclude that current chatbot providers tend to treat responsible‑AI commitments as marketing narratives rather than operational realities, and they call for expanded research across a broader ecosystem of conversational agents and the creation of standardized evaluation frameworks to support sustainable AI governance.