7.3.3 How to obtain multilinguality

7.1.3 How to obtain multilinguality

The training data we prepared in the previous section contained both English and Hindi, making the resulting bot multilingual. Multilinguality is not just an added functionality; it depends on the local context. It is sometimes the most logical way. Multilingual chatbots, capable of understanding and responding in multiple languages, hold distinct advantages in certain scenarios. In regions with linguistic diversity or areas where multiple languages are commonly spoken, a single chatbot that accommodates different languages can greatly enhance accessibility and user engagement. Moreover, multilingual bots can facilitate communication in cross-border or multicultural contexts, streamlining interactions and information exchange. In this section, we'll explore how to harness the power of multilinguality for your chatbot, enabling it to serve a wider audience while maintaining a seamless conversational experience.

In terms of training data preparation, one might replicate intents and responses for each language to enable multilingual support in a chatbot. For example:

Intents:

• greet_eng

• greet_swh

• greet_fra

Responses:

• utter_greet_eng

• utter_greet_swh

• utter_greet_fra

While this approach initially seems straightforward, it presents challenges as the number of intents grows. Training the model with a limited number of samples can lead to decreased performance and the potential misclassification of similar intents.

To overcome these challenges, we suggest a robust multilingual architecture that leverages different NLU models for every request. This architecture employs a language classifier, which identifies the language of each incoming query. Based on the detected language, the query is directed to the specific NLU model responsible for intent recognition in that language. Once the intent is recognized, it is passed to the core model, which predicts the appropriate action, such as generating a response or triggering a custom action.

The below diagram shows an example of this that was used in one of CLEAR Global’s chatbot systems:

Multilingual chatbot deployment architecture

By implementing this multilingual architecture, you can ensure accurate intent recognition and provide language-specific responses for a seamless user experience. Throughout this documentation, we will explain the architecture, how to train the models, and how to run the chatbot.