NLP-Based Vs. LLM-Powered Sentiment Analysis

In this article John Ortiz at MiaRec compares the latest two approaches to achieving Sentiment Analysis: Natural Language Processing (NLP) or Simple Language Model (SLM)-based versus Generative AI or Large Language Model-based Sentiment Analysis.

In June 2023, we published an article that compared lexicon-based with Machine Learning or NLP-based Sentiment Analysis. Now, a little more than one year later, we have made another huge leap forward with Generative AI-based Sentiment Analysis!

By the end of the article, you will understand the technological differences and which approach is more accurate and easier to use.

What Is Sentiment Analysis?

Sentiment Analysis uses Artificial Intelligence to identify and extract subjective information from interactions between customers and contact centre agents.

Conversation Intelligence solutions often determine Sentiment Analysis by analyzing customer communications’ tone, emotion, and intent, such as phone calls, emails, chat messages, and social media interactions.

Sentiment Analysis uncovers very valuable insights (especially if overlaid with topical analysis):

• Improved Customer Experience: By understanding customer emotions and sentiments, contact centres can tailor responses to better meet customer needs, leading to higher satisfaction.
• Enhanced Agent Performance: Sentiment Analysis gives agents insights into their interactions, helping them improve communication skills and handle customer issues more effectively.
• Proactive Issue Resolution: This approach identifies negative sentiments early, allowing proactive measures to address potential issues before they escalate.
• Data-Driven Decision Making: Enables contact centre managers to make informed decisions based on sentiment trends and patterns.

We specialize in analyzing call recording transcripts. And focus only on what has been said (words in the transcript) and not how it has been said (volume or tone) because this focus/method has proven to be far more accurate.

Please note:

There are three common ways to achieve Sentiment Analysis: lexicon/keyword-based, NLP/ML-based, and LLM/Generative AI-based.

Although a few vendors are still using the lexicon-based approach, this technology is outdated and obsolete compared to the accuracy and ease of setup of the other models. 

NLP-Based Sentiment Analysis (Simple Language Model)

In the late spring of 2023, we introduced a new way to analyze sentiments: Machine Learning-based or Natural Language Processing (NLP)-based Sentiment Analysis.

Rather than providing the AI with a lexicon of keywords to spot in a transcript and that indicated a positive, neutral, and negative sentiment, this Machine Learning-based approach relied on the AI’s ability to learn how to detect sentiment from the context of a text-based input.

This method allows for better handling of context, sarcasm, and complex expressions. ML-based Sentiment Analysis assigned scores to conversations and highlighted specific phrases with positive or negative sentiments.

In other words, we trained a Simple Language Model (SLM) to identify sentiment in texts on its own by giving it enough examples of positive, neutral, and negative sentences.

This model is able to grasp the essence of texts, capturing the subtle nuances of language embedded within them because it uses Natural Language Processing or NLP.

Customers could use it out of the box or customize it by training the SLM based on their company’s specific business use case. However, most organizations relied on the default model because training a new model was time-consuming and cumbersome.

Regardless, this new approach was slightly more accurate and much easier to set up than lexicon-based Sentiment Analysis. However, the downside is that this model still only considers phrases or single sentences and not the entire context of the progression of the conversation.

This sometimes resulted in a wrong assessment. For example, if a customer ranted for five minutes but the agent quickly and efficiently resolved the issue in one minute, the call would be scored negatively on the customer side because there was a lot more negative sentiment than positive, despite the customer being happy and content at the end of the conversation.

Large Language Model (LLM)-Based Sentiment Analysis (Generative AI)

Thanks to the unprecedented speed of AI development, we are replacing the Simple Language Model-based approach with a Large Language Model-based approach.

Rather than relying on Machine Learning and Natural Language Processing alone, we have now developed a Sentiment Analysis that leverages Generative AI. In other words, Sentiment Analysis just got so much more accurate and incredibly easy to customize.

As mentioned above, this new approach is based on a Large Language Model (LLM), which simply is a statistical model of language built on extensive datasets that can comprehend and produce phrases based on the text it has been exposed to. Essentially, a LLM can generate text-based outputs based on probability.

This has massive implications on Sentiment Analysis as the model now understands entire conversations.

Because it can grasp context, it can pick up complex emotions and nuances in the human language, such as humor or sarcasm, making it far more accurate than its two predecessors.

It can also be much more easily customized to your organization’s needs and requirements. Instead of having to train an entire language model, you just tweak a prompt that instructs the AI.

This allows you to define what a positive or negative sentiment looks like for each division. For example, the sentiment of a call is positive for a customer service contact centre if the issue has been resolved to the customer’s satisfaction, while for a sales call centre, it would be assigned as positive if the deal is closed and the customer paid on the spot.

Here is example text of the full prompt written out:

“Based on the submitted call transcript, classify the conversation into neutral, negative, or positive. Score the sentiment score in range from -100 (most negative) to 100 (most positive).

Provide the score separately for agent, customer, and overall conversation.

Score the conversation as positive when the customer was satisfied with the service, all their questions have been answered, and all issues resolved during the call.

Score the conversation as negative when customer remained upset and frustrated at the end of the call because the issues have not been resolved due to agent or company fault.

Explain why you scored the conversation with the resulting score (no more than 50 words).”

To test and improve the accuracy of your prompt changes, you can use AI Prompt Designer.

In the system, this will result in the following sentiment scoring ranges:

• Very Positive (60 to 100)
• Positive (20 to 60)
• Neutral (-20 to 20)
• Negative (-20 to – 60)
• Very Negative (-100 – 60)

I hope this helps you to better understand the differences between NLP-based and Generative AI-based Sentiment Analysis.

This blog post has been re-published by kind permission of MiaRec – View the Original Article

For more information about MiaRec - visit the MiaRec Website

About MiaRec

MiaRec is a global provider of Conversation Intelligence and Auto QA solutions, helping contact centers save time and cost through AI-based automation and customer-driven business intelligence.

Find out more about MiaRec

Call Centre Helper is not responsible for the content of these guest blog posts. The opinions expressed in this article are those of the author, and do not necessarily reflect those of Call Centre Helper.

Author: MiaRec