Qualitative data adds rich context to any decision-making process, in ways that quantitative data cannot. But first it has to be structured for analysis—a time-consuming task that keeps businesses from being more customer-centric.
The good news is that powerful new language models are emerging that can do such tasks much faster than humans can.
Qualitative data captures observations, feelings, opinions, or ideas that can’t be crunched like numbers but can reveal useful patterns and trends.
Text-based data, in particular, requires interpreting the meaning of words in context. If you break up the words, the context and meaning are lost. That keeps large volumes of text data from being easily sortable in columns and rows the way quantitative data is.
Algorithm-based tools are good at organizing numerical data to predict what comes next. However, they’re not very good at knowing what we humans mean when we write a paragraph of ideas.
Even language models designed specifically to predict what comes next from prose have historically fallen short. The best of these models require a lot of data examples—and computational power—to do this reliably, which gets expensive.
It’s important to note that predicting the next word in a sentence is not the same as understanding meaning. (Artificial intelligence is still nascent on that front. This difference makes humans irreplaceable at certain tasks.)
Reliably predicting what comes next in a string of text is most useful when language models can also write in plain language like a human would.
In a business context, language models that can present information in a human-like way will be able to replace repetitive, mundane tasks.
For instance, automating tedious tasks across large datasets of text—like sorting text to pull out themes (think: search recommendations)—can free up a lot of time. Such tools can decrease time spent typing, copying, pasting, or sorting text-based data.
There are myriad software companies working on automating tedious tasks related to qualitative data. What business leader wouldn’t want their teams spending less time on manual, rote work and more time making decisions that create value?
Creating a good customer experience is one area where language models can be helpful. Every customer-centric company values feedback from their customers. Many have set up ‘voice of the customer’ teams to better understand customer needs and find insights that help them improve their product or service.
Customer support teams are at the center of ‘voice of the customer’ initiatives. They cobble together existing tools to categorize every support ticket, customer satisfaction survey response, live chat log, or customer review. Once they’ve manually organized the data, customer support agents can analyze it for insights.
How can language models automate some of the time-consuming tasks in that workflow?
Many analytics tools use language models to find and categorize phrases for later analysis. Some even do sentiment analysis (labeling each data point as positive, negative, or neutral). But the models can only interpret small amounts of information—say, a keyword—which yields limited insights.
A customer support analyst would still need to read the full text surrounding that keyword to get the context that would improve their product or the customer experience.
Here’s an example of a customer response to an ecommerce retailer’s NPS (net promoter score) survey:
Your online selection is great; browsing is easy. That said, your platform takes work when choosing delivery vs. curbside pick up. I don’t always know what method to choose upfront because I can’t see product availability until checkout. Often if I select pickup, the system tells me that item is not available at the store I selected. So then I have to start all over again to find a replacement, another location, or see if delivery is a better option. Maybe it’s just me, but your checkout makes me jump through many hoops.
A lot of language models could pick out keywords such as delivery or curbside pick up from this text; they may even classify this as a confusing checkout flow.
But identifying themes in keywords is one thing; knowing what to do with them is entirely different.
Going beyond keyword identification is what business users and decision makers need next.
A language model that could write a summary or make recommendations from thousands of support tickets in a way that’s easy to understand would close that gap. It would be like a knowledgeable colleague summarizing takeaways from thousands of support tickets she’s read one by one.
If an analytics tool and a language model could work together to automate time-consuming tasks like summarizing themes in plain language, what would that pairing look like?
It would need to be capable of accurately categorizing customer feedback, summarizing concepts, and even developing recommendations.
It would need to be good at predicting what comes next in a string of text and present it (that is, write it) in a human-like way.
It would need to identify trends and patterns and also discern the relative importance of themes.
The underlying language model would have to be sufficiently lightweight: it couldn't require a lot of data inputs to train it for reliably good text prediction and summarization. That would be impractical and costly.
Businesses need a useful language model that can do a lot with few inputs.
GPT-3, from OpenAI, is a language model that is better at this than any other.
A large dataset—terabytes of data from across the internet—was used to pre-train GPT-3, making it better at predicting language patterns than any other language model available.
Because it’s already pre-trained with a lot of data examples, GPT-3 is ready to use: additional data examples aren’t necessary for the model to accurately predict what comes next in a string of text. All it needs is a handful of data examples for a specific use case, along with a few instructions.
Technically, GPT-3 can be used with zero examples and still generate pretty accurate results. This is far more accessible for businesses than starting from scratch with a lot of data.
It can also generate text like a human. GPT-3 is good at both text prediction and text generation, making it highly compelling for many uses.
A common challenge of many analytics tools is that they’re so broad, with so many features, that users often don’t know where to start or how the tool will solve their problems.
Businesses often rely on research analysts as intermediaries between powerful analytics tools and the decision-makers. While this helps decision-makers, it’s debatable how much time it saves the analysts who must spend hours working in such tools.
Designing analytics tools that are powerful yet easy to use often involves some tradeoffs. It doesn’t get any easier if the tool must also structure qualitative data effectively.
The developers who successfully combine all three elements— powerful analytics, ease of use, and reliable qualitative data structuring—will have an advantage in the market as language models get better and better at providing useful and affordable predictive functionality.
Last Updated: 02/08/21
Your waitlist, onboarding, and free trial are perfect for doing user research
Sales and marketing teams are great at doing user research with future customers. Sales teams use online forms to see if there's a good fit while marketers use demographics and feedback to create better messaging. What about startups without sales or marketing teams? Besides requesting …
How Viable works: productizing GPT-3 and in-house systems for automated text analysis
Viable is designed to address the problem of too much qualitative feedback and not enough time to read through it all. To this end, we built a system that aggregates and organizes text-based customer feedback to answer human questions with human-like responses generated by GPT-3. Below we explain …
How support requests and product feedback work together
Your customers share their feedback with you via support requests, surveys, chatbot interactions, social media engagement, and product reviews. To find insights in qualitative customer feedback, it helps to segment it so you can better identify what types of insights come from which datasets. For …