Towards Teachable Conversational Agents

The traditional process of building interactive machine learning systems can be viewed as a teacher-learner interaction scenario where the machine-learners are trained by one or more human-teachers. In this work, we explore the idea of using a conversational interface to investigate the interaction between human-teachers and interactive machine-learners. Specifically, we examine whether teachable AI agents can reliably learn from human-teachers through conversational interactions, and how this learning compare with traditional supervised learning algorithms. Results validate the concept of teachable conversational agents and highlight the factors relevant for the development of machine learning systems that intend to learn from conversational interactions.

💡 Research Summary

The paper “Towards Teachable Conversational Agents” investigates whether a conversational interface can serve as an effective medium for human teachers to train a machine‑learning model. The authors build a web‑based chatbot that can operate in two modes: a teaching mode, where users read news articles and interact with the bot by highlighting or typing words that they consider relevant (or irrelevant) for classifying the article, and a testing mode, where the bot classifies new articles based on the knowledge acquired during the teaching dialogue. Three heuristic teaching strategies are defined—externally relevant words, internally relevant words, and internally irrelevant words—mirroring prior work on algorithmic teaching.

The underlying classifier is an enhanced Naïve Bayes model. In addition to the usual word‑frequency probabilities P(w | C), the model incorporates probabilities derived from “conversational keywords” (s) that the teacher supplies during dialogue. To decide whether a keyword is semantically related to a document word, the system computes cosine similarity between 300‑dimensional Word2vec vectors trained on a large news corpus; only similarities above a threshold (≈0.2) are retained. Two inference regimes are explored: (1) without any supervised pre‑training, where the posterior is proportional only to the product of P(s | C) terms, and (2) with supervised pre‑training, where the posterior combines both P(w | C) and P(s | C) multiplicatively. The class with the highest posterior probability is returned.

A formative user study was conducted on Amazon Mechanical Turk with 60 participants (average age 31, 87 % native English speakers). Each participant taught the bot using 20 news articles evenly distributed across four categories (World, Science & Tech, Business, Sports). Participants were required to provide at least one teaching word per article and could freely switch between teaching and testing to monitor the bot’s performance. Demographic data showed moderate prior experience with conversational agents and varied familiarity with the news categories.

Although detailed quantitative results are not reproduced in the excerpt, the authors claim that the conversational teaching approach yields statistically significant improvements over baseline text‑classification methods that rely solely on supervised data. The study also highlights the influence of teacher expertise, the three heuristic guidance types, and the interactive feedback loop on learning efficiency.

In discussion, the paper positions its contribution within the broader literature on interactive machine learning, active learning, and algorithmic teaching, emphasizing that prior work has largely focused on visual or batch feedback, whereas this work demonstrates the feasibility of natural language dialogue as a teaching channel. Limitations include reliance on a relatively simple Naïve Bayes backbone and a fixed similarity threshold, which may not capture deeper contextual nuances or multi‑label scenarios. Future directions suggested include integrating deep neural classifiers, automating dialogue strategy optimization, and modeling long‑term teacher‑learner relationships to support more complex, real‑world applications. Overall, the work provides a compelling proof‑of‑concept that conversational agents can be taught interactively, opening avenues for personalized assistants that continuously adapt through everyday human conversation.