How companies analyze customer feedback

Based on 800 customer questions analyzed

Viable Team

June 11, 2021

A sample of ~800 questions asked in Viable by more than 60 companies told us what companies most want to know from their customers. Based on the types of questions asked, companies are most interested in knowing how to improve their products. They also value prioritized lists and summaries of what customers want, think, or feel.

Below is a breakdown of what our qualitative data analysis says about how companies analyze their customer feedback and how they might use it in their product feedback loops.

Product improvement is top of mind

The most popular type of question businesses ask of their customer feedback is how to improve their product. This indicates that product, user research, and customer support teams want customers input to inform their product development processes.

Whether you’re a product manager, user researcher, or customer experience lead, nothing beats first party data. That is, data directly from your customers. However, it’s rare that a customer will provide such clear and direct feedback as “please improve the product by doing x”—let alone all customers.

In fact, customers rarely use the word “improve” in their feedback. More often than not, you have to comb through a lot of customer feedback and make some interpretations to arrive at clear product improvement opportunities.

A powerful language model can interpret common frustrations, feature requests, or even questions from textual feedback and use that to write useful answers to questions about how to improve your product.

We have found that GPT-3 uses customer feedback such as “I’d like to see...” and “can you add...” but also “xyz feature doesn’t work well” and “I can use xyz feature in other products to do…” to write answers in response to questions about how to improve your product.

Exploring feedback using the search method

The second most popular type of question in our sample dataset is actually not a question at all, but rather search. It’s not surprising, given how we’re all used to searching for information thanks to the prevalence of search engines in our daily lives.

Although it didn’t surprise us that companies use Viable to search for information, we were surprised to find how common it is at 17% of all question types.

Search solves the problem of not knowing where to start. If you don’t know what questions to ask—though here's some inspiration—you can always type in a word or fragment of a topic and see what comes up. The Viable engine is designed to summarize what customers have said about a topic. For example, instead of asking “What do customers think about the calendar?” you could simply type in “calendar” and see what answer is returned. A typical response would look like this:

Our customers are saying that the calendar needs to be improved by adding a Zoom integration and always on Calendar. They also want the info for a contact on the sidebar to be more useful.

Below is just a sample of terms searched in the sample data:

Data loss
Damage
Dispute
Energy
Fees
Flavors
Game tutorial
Maintenance
Memory
Out of stock
Pain point
Problem
Referrals
Tire use
Wallet

In order to improve something, you need to prioritize

After product improvement questions and searches, we found that people most often ask for prioritized lists. Specifically, questions that contain the words “top,” “biggest,” or “most.” For example:
“What are the top customer complaints?”
“What are our customers' 3 biggest concerns?”
“What are the most common order inquiries?”

A prioritized list helps you reduce the number of decisions you have to make. More than that, it gives you clear areas to prioritize backed by direct, first party data.

Companies want customer feedback that's summarized

The fourth most common question type is what we call the topic summary question. Not far behind is the emotion summary question. These are questions such as:

“What are customers saying about membership fees?” (topic summary)
“What do customers find frustrating about order status?” (emotion summary)

These questions seek a summary of customer feedback by topic such as membership fees; or by customer emotion such as frustration—as well as disappointment, confusion, worry, excitement, desire, or delight.

We find people often start with summary-type questions as a way to explore their customers’ feedback. It prompts them to dig further into specific areas. For instance, a line of inquiry on what customers are frustrated about leads to questions about how the company could improve invoices, and then additional questions that hone in on how they could explain charges on invoices more clearly.

Questions reflect how our minds work

There were 11 additional question types that came up frequently in our dataset, including:

Quality assessment (“how helpful do users find our documentation?”)
Count (“how many/much/often…”)
Root-cause analysis (‘why’ or ‘what’ like “what causes shipping delays from Phoenix?”)

There is one question type that came up more frequently than we expected: yes or no questions. For instance, “do customers use our app?” or “do users like our shipping process?” are yes or no questions. These made up a surprising 14% of all questions asked in the sample dataset.

It’s surprising because research best practices generally discourage yes or no questions since they tend to provide limited insight.

However, as humans, we can’t help but want validation about how the world works. Businesses are no different. The intent behind yes or no questions is often to vet a hypothesis. When there's enough data, our model tries to answer yes or no questions with more detail than just simple a yes or a no. For example, for the question “do customers like our financial app?”, an answer might look like this:

Customers like the financial app because it provides a convenient and easy way to access their funds.

Accounting for how humans think and write is the reason GPT-3 has been so groundbreaking. It's more capable of meeting people where they are than previous language models. It helps make Viable not just powerful but also very user-friendly.