Sentiment Analysis

Speech analytics allows users to analyse and extract information from both live and recorded conversations. It is being used effectively to gather intelligence for security purposes, to enhance
the presentation and utility of rich media applications, and perhaps most significantly, to deliver meaningful and quantitative business intelligence through the analysis of the millions of recorded calls that occur in contact centres across the world.

Sentiment analysis is a branch of speech analytics that focuses specifically on assessing the emotional states displayed in a conversation. One common use of sentiment analysis within contact centres is to provide insight into a customer’s feelings about an organisation, its products, services, customer service processes, as well as its individual agent behaviours. Sentiment analysis data can be used across an organisation to aid in customer relationship management, agent training, and to help identify and resolve troubling issues as they emerge.

The core of automated sentiment analysis centres around creating a model to describe how certain features and content in the audio relate to the sentiments being felt and displayed by the participants in the conversation. At this point, there are two primary methods that have been deployed to predict sentiment within audio: Acoustic and Linguistic modelling.

The Acoustic Approach

The acoustic approach to sentiment analysis relies on measuring specific feature characteristics of the audio. These features—tone of voice, pitch or volume, intensity and rate of speech—can in some circumstances provide basic indicators of sentiment. For example, the speech of a surprised speaker tends to become somewhat faster, louder, and higher in pitch. Sadness and depression are presented as slower, softer, and lower in pitch. An angry caller may speak much faster, much louder, and will increase the pitch of stressed vowels. There are a variety of audio features that can be measured, but some of the most common ones mentioned in the literature

Intensity: energy, sound pressure level
Pitch: variation of fundamental frequency
Jitter: variation in amplitude of vocal fold movements
Shimmer: variation in frequency of vocal fold movements
Glottal pulse: glottal source spectral characteristics
HNR: harmonics-to-noise ratio
Speaking rate: number of phonemes, vowels, syllables, or words per unit of time

When developing an acoustic analysis tool, the system must be built on a model that defines the sentiments being measured, typically along the continuous dimensions of AROUSAL, ranging from passive to active, and VALENCE, ranging from negative to positive (see Batliner et al 2008), or mapped to discrete categories.

The model is based on a database of the audio features as described above, and how their presence may indicate each of the sentiments that are being measured. To create this database, each single-emotion example is pre-selected from a pristine set of recordings, manually reviewed and annotated to identify which sentiment it represents. The final acoustic analysis tools are then statistically trained and a predictive model is validated based on a different set of the same annotated utterances1.

State-of-the art accuracy is under 60%, i.e. less that 60% of pre-selected, single-emotion, noise-free utterances can be correctly classified.

Problems with the acoustic approach

There are some problems inherent in the acoustic modelling approach. First off, because acoustic analysis relies on identifying the audio characteristics of a call, the quality of the audio can significantly impact the ability to identify these features.

In addition, while the acoustic models are built using pristine, full-spectrum and noise free samples of recordings, audio in the real world suffers from background noise, network interference and even background talking that can interfere with the predictive ability of the acoustic model.

Thus, while it can be useful in highly constrained and monitored situations, the acoustic approach to sentiment analysis presents technical challenges for widespread use in unstructured situations like contact centres.

The Linguistic Approach

Conversely, the linguistic approach focuses on the explicit indications of sentiment and context of the spoken content within the audio; linguistic models acknowledge that, when in a charged state, the speaker has a higher probability of using specific words, exclamations or phrases in a particular order. The features that are most often analysed in a linguistic model include:

Lexical: words, phrases, and other linguistic patterns
Disfluencies: filled pauses, hesitation, restarts, and non-verbals such as laughter or breathing
Higher semantics: taxonomy/ontology, dialogue history and pragmatics

As with the acoustic approach, a model is built based on understanding which linguistic elements are predictors of particular sentiments, and this model is then run against a series of recordings to determine the sentiments that are contained therein. The challenge with this approach is in collecting the linguistic information contained in any corpus of audio. This has traditionally been done using a large vocabulary continuous speech recognition (LVCSR) system, often referred to as speech-to-text. However, LVCSR systems are prone to creating significant error in the textual indexes they create, with typically 35% of the words either wrong or missing, even in applications using high quality audio and an extensive dictionary.

A different approach to linguistic analysis is that of phonetic indexing and search, as this allows a high degree of accuracy no matter what the quality of the audio source, and its incorporation of conversational context through the use of structured queries during analysis. (See Word Spotting vs. Phonetic Search vs Speech Recognition for further details of this approach.)

Issues with sentiment detection

The difficulty of basic sentiment analysis is also compounded by the reality that speakers commonly express blended emotions, such as both empathy and annoyance (as in “I do understand madam but I have no miracle solution”, which are tremendously difficult to classify based solely on their acoustic
features. Additionally, acoustic analysis is often incapable of recognising and adjusting for the varied ways different callers express similar sentiment.

Sentiment analysis of any type is often hampered by the fact that multiple events may occur during a call that obscure the true sentiments being displayed. For example, a caller may respond to something environmental (background noise, a side conversation, connection quality) with agitation, but this may not directly relate to the content of the call. Structured queries provide the ability to isolate emotional context from environmental context and produce results that are most relevant to the issue under examination.

Practical Application of Sentiment Analysis in Contact Centres

Most contact centres serve as a customer’s primary interaction point with an organisation, and sentiment analysis can be an invaluable tool for managing and improving the quality of service provided by these centres. Traditionally, contact centres have conducted weekly agent quality reviews with just a handful of the calls, perhaps as few as five selected at random. However, it’s unlikely that this sample set will accurately capture the full range of customer interactions experienced with any particular agent, and it certainly will not provide for a quantitative assessment of customer experience across all agents or across multiple centres.

Rather than taking a traditional “hit or miss” approach with a review of randomly selected calls, a call centre can now leverage automated sentiment analysis to identify and measure broad areas of concern. With this bigger picture, sentiment analysis helps the organisation to uncover repeated patterns that may be indicative of broken systemic processes, or that may be related to the performance of a specific department or representative. This data can then be mapped to understand the relationships between customer sentiment and operational efficiencies.

For example, organisations may see a marked increase of customer dissatisfaction that spans a significant length of time, or spreads across one or multiple contact centres. It is quite common for dissatisfied customers to take a greater amount of agent time during a call, which drives up average handle time (AHT) and increases overall costs. By using sentiment analysis to identify the sources of this dissatisfaction and implementing changes to alleviate them (be it improved agent training, fixing broken back-end processes, etc.) the organisation will see reduced AHT, reduced costs and more satisfied customers.

Following are some real-world results that organisations have achieved using Nexidia’s sentiment analysis approach:

At one contact centre, customer satisfaction surveys featured an opportunity for respondents to leave several seconds of verbal commentary. Analysis found that the company’s direct customers were pleased, but many indirect customers—those who purchased products through a distributor—were having unpleasant experiences and providing negative survey feedback. Because sentiment analysis raised the flag to this situation, the organisation took action to improve the customers’ interactions with the distributor.
Another company found that certain agents were aggressive with callers, so much so that the agents would refuse to transfer customers to a supervisor. So, while the number of escalations remained low, callers were expressing frustration with the confrontational agents. Sentiment analysis quickly identified the offenders, so that management could rectify the situation and minimize the risk of further damaging customer relationships
A technology company had multiple contact centres, each to address a particular type of call. However, after sentiment analysis identified a number of exasperated customers who had been transferred between departments numerous times before finding resolution, the company discovered that one of the contact centres—billing enrolment—was not empowered and, frankly, unnecessary. By identifying this broken process, the company addressed an underlying issue that significantly impacted customer satisfaction.

Not all escalations are related to negative sentiment, however. One call centre discovered that a number of its calls which were transferred to supervisors were in fact positive. Callers were taking time to register pleasure with their agent experiences. So, while the number of escalations initially raised a red flag, sentiment analysis led to a pleasant discovery.

This article has shown that as a branch of speech analytics, sentiment analysis can be a valuable tool allowing companies to monitor the interactions that occur between customers and contact centre representatives. By identifying and correcting situations that are driving negative customer attitudes, organisations win in two ways. Not only can they lower the costs associated with providing quality customer service, but they can also improve overall customer satisfaction which invariably leads to lower customer churn and greater revenues. This two-fold improvement in operations efficiency makes deployment of sentiment analysis a wise investment for an organisation.