AI Survey Analyzer
Using AI to organize and prioritize user feedback.
DesignDevelopDeployOrganization and Insights
The IBM product I worked on used an NPS survey to receive user feedback. NPS® is a numerical rating system that is often held up as the gold standard customer experience metric.
We could collect hundreds of surveys a month, and processing responses was time-consuming. I sped up survey feedback review by using IBM Watson’s NLP API to create a NPS survey analyzer.
With this tool, users can quickly see common topics and prioritize the most valuable feedback.
Quantitative to Qualitative
When conducting NPS surveys, Qualtrics suggests you ask related follow-on questions, such as: ‘Please tell us why you gave that rating.’ Asking “why” with a numerical rating can give invaluable insight. However, it also adds to the amount of data to be reviewed by the product team.
A 1% response rate from a product with only 10,000 Daily Active Users would yield 100 instances daily. Left without regular processing, this can result in a mountain of data to parse.
AI Processing
A subfield of artificial intelligence (AI), Natural Language Processing helps machines process and understand the human language so that they can automatically perform repetitive tasks.
We want to present this data so that the user gets the essential information.
This presentation should allow them to make an informed decision
For this specific project, I used IBM Watson’s NLP Sentiment Analyzer. This is an API that takes text and returns categorized data. Watson’s feedback comes as a JSON package that would overwhelm the user.
The JSON has too much data, non-deterministic outputs, and an illegible format. This is why AI outputs can be confusing, or even a mystery. There are three things we can do to overcome these characteristics and provide an ideal user experience.
Provide Transparency
We can explain what affected a decision.
The survey analyzer produced a list of topics identified from user feedback. This list allowed the user to view the instances that lead to its categorization.
Present Uncertainty
We can visualize probability.
Watson provided a probability with the sentiment of user comments. Showing the probability of accuracy gave the user an opportunity to re-examine decisions made by the AI model.
Ask for Feedback
We ask how we did.
If the user found decisions they felt were inaccurate, they could provide immediate feedback in app. This left the power in the hands of the user and helps the tool improve.
Working with the Watson API proved invaluable for me as a programmer. It sharpened my skills by exposing me to cutting-edge artificial intelligence technologies. Leveraging Watson, I significantly improved my NPS survey processing, enabling more accurate and insightful analysis. Moreover, this experience equipped me with the necessary tools and knowledge to thrive in the rapidly expanding AI landscape.