Using AI to Label Contact Centre Contacts and Surveys

What is text classification?

When new queries arrive at the contact centre, machine learning can examine the contents of the query, and determine “what is this customer talking about”. The application of AI in this manner, as with chatbots, is called “text classification”, that is the assignment of different bodies of text into separate labelled buckets. For example, an AI system could be taught that the phrase “I no longer will be going on holidays” as a “cancellation” query for a travel company. 

Contact centres can use text classification results for a variety of applications, including:

• routing particular query types to certain agents or teams
• prioritising queries in certain queues 
• reporting on the breakdown of ticket types received over time
• analysis of surveys retrospectively to discover “what are the broad trends of survey contents”
• removal of agent-based reason code in favour of automatic reason labels

The Simple Approach: Keyword Detection

The simplest method to achieve automated text classification is keyword detection. For example, a contact centre could label all queries with the word ‘cancel’ in them as “cancellation queries”. This method is intuitive, and the first idea that people have when they want to “detect what is happened”.

While fast to set up, and easily understandable, keyword-detection approaches are difficult to maintain, sensitive to slang or misspelled words, typically very challenging to expand to multiple languages, and will not achieve very high accuracy without extensive manual work. For example, the queries “please cancel my booking” and “I don’t want to cancel my booking” both have the terms “cancel” and “cancel my booking” in them.

While not recommended for long-term use over many categories, keyword detection has it’s place to define a “baseline performance”, and failing the availability of a more accurate approach, it can be a useful starting point to demonstrate value.

Ai-Based Text Classification

Artificial intelligence (AI) text classification techniques provide more flexible approaches that can address the shortfalls of the keyword-based approach. AI approaches use statistical language models to classify pieces of text. These models can extrapolate from examples to take word order into account, incorporate slang, counter for common misspellings, and take the context of words into account when making predictions.

AI systems “learn by example”, so most text-classification systems will require a dataset of “training” or “example” data to start off with. The early approaches required many hundreds or even thousands of training samples, typically manually labelled, to produce accurate models. Thankfully, advances in training techniques have dramatically reduced the volumes of training data required (from thousands to tens). As such, AI text-classification models can be fine-tuned to your brand’s and customer’s individual terminology and issues.

Natural-language-based text classification techniques have improved in accuracy over recent years, with accuracies of 95+% obtainable by experienced practitioners. Modern text-classification systems make extensive use of word and sentence embeddings and can work in multilingual environments.

What is Transfer Learning

A huge advance in machine learning techniques that has driven a step-change in text classification accuracy since 2015 has been the invention of “transfer learning” for NLP tasks. Previously, data samples would learn the statistical patterns of word appearances for sentences that trigger text-classification from manually labelled datasets. This was the “bag of words” approach, which used maths operations to identify common and uncommon words for topics of interest. Each time a new model was being built, the data scientist started from scratch with a new data set.

In transfer-learning approaches, a single, much larger model is trained on a gigantic volume of text (for example, the entirety of Wikipedia in English). These models learn how words appear together and their typical order of appearance, for vast quantities of language use. The models are computationally expensive to train, taking many machines multiple days to complete and are often completed by large companies or research institutes (Google / Facebook / Stanford / Microsoft). However, once trained, the resulting model output can be integrated into new text-classification models (i.e. the core language information is “transferred”).

Now, when building a new text-classification model, a data scientist can begin by downloading a pre-existing pre-trained language model, and then layer the domain-specific classification task on top. This approach dramatically increases prediction accuracy with fewer labelled samples, speeds up the development process, and “injects” the statistical learnings from much larger bodies of text into the model workings!

Where to start with text classification

Text classification is a perfect mechanism to “dip your toe” in the machine learning world for customer contact centres, and can be used to show value almost immediately. The classification of received text surveys and inbound customer queries can be performed offline before doing any system integrations, and the results can be used to inform quality improvement initiatives with quantitative “voice of the customer” data.

The key steps to successful application of automated tagging is to focus on the right customer topics and not overreach. The best outcomes will be for customer service topics that are:

• Often encountered (and so lots of training samples),
• Have language that is somewhat different or separable to other topics (models can learn the language used by customers)
• Ideally, will have an associated action that can be automated in later AI initiatives to provide an efficiency gain to the contact centre.

Ideal Outcomes

• Perfect visibility over what topics customers are contacting about or talking about in customer surveys, in real time and historically.
• Higher levels of customer satisfaction through accurate customer service query routing and prioritisation when it is needed.
• Improved automation steps based on accurate text classification labelling of queries as they arrive from customers.
• Best case scenario is a complete removal of the need for manual agent dispositioning of incoming contacts, saving wrap time on every interaction.

Potential Pitfalls

• Inaccurate machine learning models, typically caused by poor training data or classification topics that are too “close” in vocabulary.
• Classification accuracy “drift” time as the phrases used by customers and the topics they are discussing changes over time.
• An initial burst of energy put into the text classification effort to get the project up and running, but a lack of follow-through to expand on topics and check model accuracy afterwards, which will hold back the company from getting the maximum from the technology.