AI Huawei Ireland Research Centre

Chatbots provide a natural dialogue interface to users, simplifying information search and assisting in domain-specific applications. Chatbots are increasingly used in healthcare, ecommerce, public administration, and education; as a “one-stop” fast information hub for end-users. However, the challenges involve (i) domain understanding (e.g., concepts within domains); (ii) anticipating question styles (e.g., linguistic variations, abbreviations and colloquialism); and (iii) multi-linguality.

Traditionally chatbots relied on curated FAQ utterance-responses, but creation of such structured data involve expertise and expensive effort. Thus, NLP techniques like machine reading comprehension (MRC) have been used in chatbots to extract answers for queries from free-flowing unstructured texts (like documents/manuals). Unfortunately, limited efforts exist towards combining the techniques, and separate products are proposed, like Google DialogFlow and knowledge connector, Amazon Lex and Kendra, etc.

We present a hybrid and unified AI based chatbot prototype for integrating both structured and unstructured domain-specific data to seamlessly answer diverse queries, within an online, cloud deployable framework. This would enable the adoption of AI chatbot technology across diverse domains/organizations due to efficient deployment (in cloud), scalability (with multilinguality), and limited effort (from FAQ and documents/manuals). Also, it would immensely benefit the end-users to quickly access pertinent information, across applications and geographical barriers.

We developed an AI based hybrid conversational framework named CAGE, comprising 3 machine learning architectures:

(1) Intent Classification (IC) – This module trains a supervised machine learning model for user question classification based on a pre-defined structured query-response dataset (e.g., FAQ). The model learns to map an end-user’s query to one of the pre-defined questions in the FAQ. Internally, the IC module utilizes several multilingual sentence transformer blocks to map questions into high-dimensional dense vector representations that capture the context and semantics. The vectors obtained from the transformer blocks are concatenated to generate “query meta-embedding” – which is fed to a shallow neural network for classification, thereby understanding question-question similarity to retrieve the appropriate answer.

(2) MRC – This module allows the parsing and understanding of unstructured text for answering the user question. Here a machine learning model is tuned to the applied domain in a self-supervised manner. Specifically, a T5 language model is used to automatically generate possible questions and corresponding answers from domain-specific documents – to form a training dataset for “domain adaptation” of a pre-trained question-answering model.

(3) Checker – The final module drives the seamless transition between IC and MRC modules to extract the best answer– enabling the “hybrid” nature of our system. The appropriate triggering threshold is set based on different confidence scores of both the modules.

Our multilingual CAGE chatbot has been integrated with BotFront dialogue system interface, for deployment as an online cloud based service. The deployed framework is trained on structured FAQ on Mobile Service applications and unstructured text description obtained from the web.

A user query is first passed to IC module to obtain a matching question from FAQ. If the confidence score is greater than a threshold, the matched answer is returned. Otherwise, the query is routed to MRC, to obtain the answer text. If the MRC module is also not confident, the chatbot requests the user to rephrase the query (or flags out-of-scope). This threshold provides a degree of control on the chatbot response, thereby reducing the chances of inappropriate (e.g., irrelevant or offensive) answers being presented to the user.

A typical user interaction involves user greeting the system, to which the chatbot asks how it can help. The user follows with a domain pertinent question, which if present in the FAQ, is matched by our system correctly. For example, the user question “I cannot login to my account?” is matching to the FAQ, and the well-documented answer is returned. Further, open-ended questions are also efficiently answered by the MRC module, wherein information present only in the text are retrieved along with a longer context for readability. For example, for the question “How many languages are supported?”, CAGE reports: … covers 190+ countries …, supporting “70+ languages”. Here, the text in quotes provides the direct answer, while the entire response presents a well-contexted readable answer. Even seemingly objective questions like “Why should I use Maps?” or “What is the benefit of Cloud?” are well answered by CAGE (returning “to find locations, driving directions, …” and “… to back up your data which prevents data loss …” respectively). Finally, for multilingual questions the answer is returned in the same language as that of the query, to enable better readability. A short demo of CAGE can be found in the additional documents and at https://youtu.be/PIzwbrmM4UU.

Such chatbots provide high efficiency, performance and customer reach to organizations and improves user satisfaction and quality-of-experience, facilitating improved human-machine interaction.