Speech Analytics And Voice Analytics – The Truth

Speech or voice analytics have been growing in popularity and their use alike for many years. Possibly the first encounter that you will have had with speech analytics was the Microsoft program, Encarta. Encarta was an encyclopedia program developed and put into production in 1993. However, the use of voice recognition in this program did

Speech or voice analytics have been growing in popularity and their use alike for many years. Possibly the first encounter that you will have had with speech analytics was the Microsoft program, Encarta. Encarta was an encyclopedia program developed and put into production in 1993. However, the use of voice recognition in this program did not outlast the competition. Before we get into the article too much, the terms voice analysis and speech analytics are synonymous with each other and used interchangeably throughout the article.

Voice analytics is a common feature in today’s society, with many of the phone calls that you make asking you to say your name or other information. Along with the time, came some extreme advancements in the technology, too. In this article, we will take a look at some of the critical aspects of speech or voice analysis within call centers. Some of those points are:

  • What is speech analytics?
  • How does it work?
  • Typical uses
  • What are the expectations of speech analytics?
  • Benefits of voice analytics
  • Downfalls
  • Are human call centers and QA fading out?

While speech analytics have been available to companies for a long time, there are still questions raised about it, how it works, and if it would benefit your call center. Therefore, we will try to answer those questions here.

What Is Speech Analysis?

Speech analysis is no different from speech recognition or voice analytics. However, speech recognition is often found during the beginning of a call to identify users’ identity, intentions, and security. Speech analysis, on the other hand, is more commonly found useful in the quality assurance realms of call centers.

In the past, if agents had to track what was said, they had to write it down. While that is still common practice for many agents who have large call volumes, the requirement to record everything said is a thing of the past. Almost every phone call that you make to a company today has an initial statement of “this call may be recorded for quality and training purposes.”

It is at that point in which analytics come into play. That will analyze what the caller and agent said, and transform it into an easy to understand, understandable format, most of the time. We will go into how it does what it does, and how good it is at it, during this article.

How Does Voice Analytics Work?

So, the basis of speech analysis is picking up on certain aspects of a call, and making notes of them for you. There are two critical ways in which software does that:


The software will convert your words into phonemes, small, recognizable sounds in speech, of which there are only 44 in the English language. They are the sounds that you make to talk, such as “oo” in “food, or “ea” in easy. One of the main issues with speech to text is that it must be a very sophisticated software to understand your speech.

The main issues with speech to text software are accents and words that are pronounced the same for different meanings:

  • There/Their/They’re
  • Heir/Air
  • Whine/Wine
  • Bare/Bear

Then you have the issues of accent where words sound very similar, such as:

  • Metal/Medal
  • Bazaar/Bizarre
  • Meter/Mitre

Many words sound the same when broken down into phonemes, and even more when there are different accents involved. There are 42 recognized accents in the USA alone, with even more in places like the UK. That fact can lead to many issues within the speech to text software and make it sometimes unreadable after completion.


LVCSR or Large Vocabulary Conversational Speech Recognition is a newer technology that matches whole words instead of the phonemes. It requires a vast database in comparison to the phonetic alternative, and it is slower to process the data. However, it is more accurate. The primary consideration with LVCSR is that it needs to have hundreds of thousands of matches for words instead of sounds.

For example:

“Call centers use voice technology.”

Phonetics will see that sentence with each sound; “ca” “ll” “cen” “ters,” etc.

However, LVCSR uses direct word correlation and puts them into a sentence with an “n” value. The “n” value is how many words the software can process at one time, for example, “n-2” would see the sentence like this:

Call centers

centers use

use voice

voice technology.

If all of the words fit together, it will produce the complete sentence. Therefore, it is a much slower method of voice analytics, but it is a lot more precise as developers enter different accented words into the database.

Voice AnalyticsTypical Uses

There are many uses for voice technology. However, we are mainly concerned with call centers. Even so, there are still many uses for the technology in call centers, such as:

  • Compliance
  • Process improvement
  • Agent coaching and improvement
  • Customer satisfaction
  • Marketing

We will take a look at those sections and how the design of voice technology is to help them.


Compliance is the most significant issue faced by some of the newest regulations, such as GDPR. However, other compliance issues like Do Not Call (DNC) and sensitive personal data such as bank detail recording also fall under compliance. The regulations that face call centers grow almost daily, and some industries like finance and banking, face it even more.

Speech analysis should ensure enforcement of guidelines set out by the TCPA (Telephone Consumer Protection Act) at all times, and also prove compliance should someone make a complaint. Therefore, protecting both the consumer and the company alike. Some of the critical aspects of compliance are:

  • Laws and regulations. – As already discussed, you can monitor TCPA regulations through voice analytics.
  • Documentation verification. – Verification of receipt of essential documentation by customers.
  • Contractual compliance and agreements. – People, unfortunately, can backtrack on contracts
  • Crucial information provided. – These include anything from payment dates to legal information, and call recording statements, etc.

Process Improvement

Being able to see a process in the form of words and numbers allow you to streamline a specific process to maximize agent efficiency. some of the methods that you can improve via voice technology are:

  • Average handling times. – When you can see that agents are asking already answered questions or overcomplicating issues that do not need it, you can address those issues directly.
  • First-call resolution Rates. – Understanding the customer requirements and call center processes together can allow you to amalgamate them into a single method that reduces the number of callbacks, hopefully down to all inquiries dealt within a single call.
  • Improving overall efficiency. – Improving in both of the above will automatically increase your overall productivity. Voice analysis will also allow you to see precisely where processes are broken and need improvement.

Agent Coaching and Improvement

Constant monitoring of a high percentage of calls allows you to identify issues with pinpoint accuracy. When you identify problems that you face with some regularity, you can coach agents in those areas, or for those points with a better understanding between the training staff and the agent.

We have many articles about coaching and how it can help your business. Here are a select few that you may find useful:

Customer Satisfaction

When you have all of the above items working correctly, you will find that your customer satisfaction will improve. Therefore, increasing your retention and acquisition rates.

The only way that you can genuinely measure customer service levels and standards is through customer satisfaction. Speech analysis allows you to see various aspects of customer satisfaction by data. For example:

  • Call cause. – If you have many people calling about a specific fault or issue, you can address that problem directly. Therefore increasing your customer satisfaction, and hopefully reducing the number of future calls about that particular issue.
  • Successful outcomes. – Not only can you see the specific issue driving a customer to call you, but you can also see if the agent resolved that issue or not.


Marketing is the key to business growth. Some large companies can spend millions of dollars on marketing with little to no return on investment. However, when you have access to thousands of hours worth of “the customer’s voice,” you can analyze and use it to your advantage.

Using voice recordings can save you money and increase marketing efficiency without the extra cost. Not only that, though, it will also help you to gauge how well a campaign is working. For example, if you have 100 calls a day about a specific product, then advertise it, you will see if the number of requests increases by an amount that is suitable for the money spent on the marketing.

What Are The Expectations Of Speech Analytics?

In our opinion, the expectations of voice analytics are often too high. While there are some valid expectations, some of them are unfulfilled. Here are some of the categories that people expect from speech or voice analytics:

  • Complete compliance
  • Increase of inbound call to conversion rate
  • 100% accuracy

Let’s have a look at what those expectations are in a little more detail:

Complete Compliance

Complete compliance is a comprehensive statement, and often overused. Gaining full compliance takes much more than voice analytics alone. While you will have better visibility of compliance issues that you face, it will not resolve them for you. As an example, if you have an agent that never states, “this call is recorded,” do you think that they will when you have voice technology? It is highly doubtful.

Therefore, you still need to take control of individual circumstances that are causing you to fail.

Increase Of Call To Conversion Rate

Again, as with compliance, voice analysis will not directly impact your call to conversion rate. It will, however, increase your visibility of where you may be failing. However, that is only possible through a high percentage of accuracy and proper coaching.

100% Accuracy

The accuracy expectation is, by far, the most overrated of them all. There is no possible way of gaining 100% accuracy through voice analytics, yet. As you saw earlier, there are a lot of accents in just one language. Forty-two of those in the USA, 7 in Canada, and 56 in the UK. Furthermore, there are numerous accents for each of the 56 recognized British accents, maybe even hundreds for each.

Some of those accents do not use the same word for the same meaning, either. Therefore, relying on computer-generated analytics alone is NOT going to give you anywhere near 100% accuracy.

Benefits Of Voice Analytics

So, what are the real benefits of voice technology? We have written a similar article about voice technology here.

There are many benefits to any technology, and VT is no different. Before we get into the downfalls of the technology, let’s take a look at what some of those benefits are:

  • Reusable data. – Any data collected through voice technology is easy to find and reusable.
  • More productivity. – You can monitor a much higher percentage of calls through VT, even up to 100%. This higher percentage is a great asset when you are looking at getting better productivity from a QA team.
  • Quicker results. – Instant results delivered from VT monitoring.
  • Smaller space required. – You need much less area for the software to be installed than if you were to analyze 100% of calls with a QA team.


Any technology could be the greatest thing in the world at the time of release. So, let’s say that you buy that latest and greatest voice tech. That will come with consequences:

  • Expense – New technology can cost millions of dollars to develop. Therefore, the cost to your company for the latest investment is going to be substantial. This cost is usually an outright cost that is unlikely to be able to be recovered without a long term plan in place.
  • Complexity – When you get a brand new piece of equipment, is it likely to be significantly more complicated. Even if it becomes easier to use in the long run, it will still mean that you have some of the following issues to deal with:
    • Training – All of the team that has anything to do with the new software will need retraining. This training will incur more costs and time losses.
    • Teething problems – I doubt that many, if any, new technologies that require training will come without any teething difficulties. Again, this will incur more time and money.
  • Expectations – The expectations of new technology can be tremendous, especially when you have paid a lot of money.
  • Uncertainty – All of the above can cause a lot of change. That uncertainty is often unnecessary. However, sometimes, it can be an unfortunate reality of something that you need to deal with. That could also even lead you to revert to your original software, giving you inferior results and a whole lot of time and money wasted.
  • Outdating – As with all technology, there is not a long time before the technology becomes superseded by something bigger and better.

Are Human Call Centers And QA Fading Out?

A simple answer to that question is no. There is no possible way that a machine can replace the human hearing. There are various pros to using voice analysis, of course. However, there are even more cons to relying on computers alone. For example, if you have a small call center, you would be in a position where you can almost solely rely on humans for QA instead of voice analytics.

When you get to more extensive, and more substantial call centers, with say 1,000+ agents, you may require some technological assistance. Although, you will still need to bear in mind that there are some severe downfalls to committing to speech analysis alone:


Validation of anything requires a second check. Companies who rely on voice analytics alone are not in a position to check all of the points within scorecards to validate the false positives and false negatives. While speech or voice analysis is excellent for some aspects of testing for compliance, it is not always the best fit. Validation is a very time-consuming process, but also a required one for higher accuracy rates.

Precise Wording

Many companies fall under an umbrella of stringent guidelines and regulations where words in a specific statement cannot change. As I have already sais, finance, banking, and insurance are some of the more common industries that require such statements. Voice analytics may incorrectly score agents on terms like that because it does not pick us the exact wording due to complications. They can include accents, etc.

The only way to get around that would be to “train” your technology for individual agents who work for you. However, with such a high attrition rate in the call center, that would be completely unacceptable.

Poor Audio Quality

Call centers invest a lot of time and money into phone equipment. Customers, on the other hand, are a lot less reliable in their methods of calling. That can be a real issue, especially for machines, to pick up specific phonemes or whole words. However, humans have evolved as a species to understand many word strings and how they sound even if they cannot hear them properly.

Dialects And Accents

Another one of the most common issues, as we have already spoken about, are accents and dialects. Some of the more pronounced accents that you find will be unable to be picked up by speech analysis. However, humans, on the other hand, can understand much more.


Voice or speech analytics certainly have a place in call centers, and we would not try to say otherwise. However, relying solely on technology is undoubtedly a wrong decision. If you would like to find out more about what Call Criteria and our technology and analysts can do to increase your accuracy, contact us here.

We look forward to hearing from you.