Building a Legal Dialogue System: Development Process, Challenges and Opportunities

This paper presents key principles and solutions to the challenges faced in designing a domain-specific conversational agent for the legal domain. It includes issues of scope, platform, architecture and preparation of input data. It provides functionality in answering user queries and recording user information including contact details and case-related information. It utilises deep learning technology built upon Amazon Web Services (AWS) LEX in combination with AWS Lambda. Due to lack of publicly available data, we identified two methods including crowdsourcing experiments and archived enquiries to develop a number of linguistic resources. This includes a training dataset, set of predetermined responses for the conversational agent, a set of regression test cases and a further conversation test set. We propose a hierarchical bot structure that facilitates multi-level delegation and report model accuracy on the regression test set. Additionally, we highlight features that are added to the bot to improve the conversation flow and overall user experience.

💡 Research Summary

The paper presents a comprehensive account of building a domain‑specific conversational agent for a law‑firm context, leveraging Amazon Web Services (AWS) Lex and Lambda. The authors begin by analysing typical law‑firm workflows and identifying two core use‑cases: a Frequently Asked Questions (FAQ) service that provides single‑turn answers, and a Fact‑Finding (FF) service that collects a user’s contact details and a description of their legal issue through a multi‑turn dialogue. Both services are implemented as a retrieval‑based system, where once an intent is recognized a deterministic response is fetched from a pre‑curated response repository, thereby avoiding the legal risks associated with generative models.

Platform selection is discussed in detail. After comparing Google Dialogflow, Rasa, and AWS Lex, the team chose Lex for its serverless nature, built‑in deep‑learning natural language understanding (NLU), extensive documentation, and tight integration with other AWS services. However, Lex imposes limits on the number of intents and slots per bot, which would be insufficient for the 44 custom intents and numerous slots required for the legal domain. To overcome this, the authors design a hierarchical bot architecture: a parent bot handles generic intents such as greetings and good‑byes, while two child bots—one dedicated to FAQ and the other to FF—receive routed utterances via meta‑intents (“all_faq” and “all_ff”). Each child bot hosts its own classifier, allowing the system to scale beyond Lex’s native limits while keeping maintenance manageable.

Data scarcity, a common obstacle in legal AI, is addressed through three complementary sources. First, a “baseline dataset” of 150 utterances was created by domain experts (lawyers, paralegals, secretaries). Second, a crowdsourcing effort recruited four law‑students who interacted with a prototype bot on Slack, generating diverse paraphrases for a set of hypothetical legal scenarios. Third, three years of real‑world enquiry form submissions from the firm’s website were automatically parsed, yielding additional utterances, new intents (e.g., location queries, urgency, contact method) and slot values. The combined corpus feeds Lex’s NLU training pipeline, improving intent recognition accuracy; performance is validated on a regression test set.

The system architecture is fully serverless. Users access the bot via a web‑embedded chat window or Slack, both feeding text to Lex. Lambda functions orchestrate the conversation flow, validate slot values, and log transcripts to CloudWatch for monitoring. In the FF scenario, once the user supplies their details, another Lambda triggers AWS Simple Email Service (SES) to email the collected information to a designated inbox accessed by the firm’s staff. Deployment is automated with AWS CloudFormation, which provisions all required resources as a single stack.

Results show that the hierarchical design successfully routes user inputs to the appropriate child bot, and the NLU model achieves high intent classification accuracy on the test set. Deterministic response retrieval ensures consistent, legally safe answers. The authors note that while the current implementation supports only FAQ and FF, the modular hierarchy permits straightforward addition of new services (e.g., contract drafting assistance, case status tracking). Future work includes extending the interface to multimodal inputs (voice, images), refining entity extraction with more sophisticated models, and incorporating privacy‑preserving techniques such as encryption and anonymisation to protect sensitive user data. Overall, the paper contributes a practical blueprint for building scalable, compliant, and cost‑effective legal dialogue systems using cloud‑native AI services.