Chatbot Development Methodology: A Melting Pot of Diverse Teams and Frameworks

September 16, 2020

Artificial Intelligence is the driving force behind the creation of innovative products like autonomous vehicles and chatbots. Recent advancements in Natural Language Processing (NLP) have made chatbots, also referred to as virtual assistants, a great option for improving the customer experience. Answering frequently asked questions, filing claims, checking the status of an order and getting feedback from customers are among the most popular use cases for chatbots. Building a chatbot that offers a good experience to customers requires collaboration from an interdisciplinary team of business analysts, service designers, data scientists, machine learning engineers and software developers. The chatbot development methodology blends several modern frameworks and methodologies including design thinking, AI innovation sprints, and agile software development.

Defining the Chatbot’s Purpose and Managing Expectations

Like most artificial intelligence applications, the first step in developing a chatbot is clearly defining its purpose, the problem it is going to solve, and the value it is going to bring to the users. Chatbots should be designed to help users navigate through a very specific business scenario. One must avoid the trap of trying to create a jack of all trades (and master of none) chatbot. For example, some conversational applications are getting a lot of hype that can set unrealistic expectations and contribute to negative user experiences. Defining a narrow scope and a clear chatbot purpose will be key for managing the rest of the chatbot development process.

Understanding the Audience

As expected in any human-centric design approach, it’s important to understand the target audience.  Service designers will conduct interviews to gain an understanding of customers as human beings. Using design thinking techniques, we learn about their pain points, their likes and dislikes, how they communicate and what they value the most. Understanding the human needs of a customer is essential to designing a chatbot experience that delights them.

A Chatbot Design Experience is a great tool to have to guide the team in defining all the key components of a virtual assistant.

Defining the Chatbot’s Personality

Techniques to define user personas should be adapted to create a “chatbot persona”, including its age group, interests, how it acts, how it communicates, its sense of humor and its limitations. Is the chatbot an advisor, an assistant, or a friend? A poorly designed chatbot personality is one of the top reasons why many chatbots fail. A chatbot with personality and empathy toward the customers can drive user engagement and create meaningful experiences. Robotic chatbots, however, have the completely opposite effect.

Designing the User Journey and Conversation

With user persona and chatbot personalities created, a service designer next should define in greater detail the human + machine interaction. He/she must design the user journey and script the actual conversations. What questions might a user answer? If the chatbot does not know an answer, do they redirect the query to a support representative? What information does the chatbot need to know in order to answer a question? The conversational and human-in-the-loop design are the most challenging parts of designing an optimal customer experience.

Developing the Chatbot

Once the chatbot design is complete, the development team can map out the features and create a technical design. Identifying if the chatbot requires artificial intelligence (AI) to fulfill its purpose will be important in defining the chatbot development methodology. Not all chatbots need AI to be fully functional, and if this is the case, the team will likely implement agile development methodologies in the traditional software development lifecycle (SDLC). Beyond building a rule-based chatbot, the software engineering team should define what background tasks and fallback mechanisms need to be carried out in order to properly integrate all the system’s components. Development can be completed through incremental deliveries, adapting design with feedback obtained from the end-users.

If a chatbot requires Natural Language Processing (NLP) components, agile software development and machine learning will need to be blended in a chatbot development methodology. The machine learning process is inherently different from rule-based systems. It requires exploration and experimentation when performing data collection, data preparation (cleaning, wrangling, and merging), feature engineering, model training, and model evaluation. The project team must account for the experimentation that will take part in the chatbot development process. Wovenware has developed a proprietary methodology, the Innovation Sprint, to integrate innovation and discipline in a goal-oriented experimentation process.

Integrating Natural Language Processing

The NLP components in chatbots are mainly used to recognize user intent and extract entities. Each intent can represent a task to be performed. A conversation flow will vary depending on the detected intent. How the chatbot handles intents will define its relationship with the user. Optional components, like sentiment analyzers and language translators, can add value and enhance chatbot experience. Sentiment analysis can help chatbots respond to users in a personalized manner by understanding what makes them happy, while language translation components can be used to give chatbots multilingual capabilities. Data scientists will optimize NLP components in order to provide information about intent, meaning and the context in which the chatbot needs to appropriately respond to a user’s query. Natural Language Processing is constantly evolving so these components must be updated regularly to keep the virtual assistant functioning in optimal conditions.

Testing

Once each system component has been designed and implemented, testing and evaluation should be carried out. For middleware software, unit testing should be performed to test individual components. Machine learning components need to be evaluated differently. Testing data is used to evaluate the trained models and success metrics must be clearly defined in order to measure model performance. Usability testing must be done before and after deployment. These tests will help to understand if the chatbot follows the expected conversational flows and error handling strategies by monitoring user interactions.

From Chatbot Rookie to MVP

Successful chatbots are designed to learn, making maintenance an integral part of the chatbot development methodology. Once the chatbot is interacting with real users it is important to analyze user feedback and sentiment, along with other insights the interactions may produce. Insight analysis may give us an understanding of possible usability issues or areas of improvement, but it also may provide us with possible market opportunities to implement more data-driven solutions. Analytics can help us understand bot performance, user engagement, sentiment and demographics. Fixing bugs and other software defects should be part of the software maintenance strategy, but with evolving customer expectations and technology advancements it is important to also keep the system updated and sustainable. Metrics for the machine learning components should always be monitored in order to address a possible decrease in performance. In the case of chatbots, the ever-evolving nature of natural language may cause the NLP components to worsen over time. Constant retraining, through active learning and evaluation of those components is vital during the maintenance and development process.

Chatbots move from rookie to MVP level through the collaboration of business leaders, service designers, data scientists, machine learning engineers and software developers. Each member of the team needs to stay at the top of his/her game to implement modern frameworks and novel technologies to continue to improve the customer experience. The team will always be on a journey to understand the user, to evolve the application around the things that matter to them the most and make the technology as human as possible, but this is what makes for great chatbots.

 

Leave a Reply

  • (will not be published)