Review Intelligence™
Project Overview
Duration: Mar 2020 - Nov 2020 (9 months)
Role: Design Lead
Team size: 8 people (1 PM, 3 FE, 3 BE)
Project Status: Launched
Keywords: From Idea to Launch, Web Design, Prototyping, Design Workshop
In this project, I led the design from an idea to the final launch (Nov 2020). Review Intelligence is a B2B product that helps employers analyze review sentiments on Glassdoor utilizing NLP technology, reveals insights on 10 workplace topics, including compensation, diversity & inclusion, coworkers, etc.
The problems we are trying to solve
The employee reviews on Glassdoor serve as an important source for employers to understand employee’s voices and opinions. Employers now manually export the reviews, highlight the topics and sentiments in Excel, and analyze the data.
Ideal flow
Previous flow
How might we present the topics mentioned in reviews and employee’s attitudes on them in a digestible and presentable way?
Solution - Key Features
Topic Sentiment Overview
This dashboard summarizes the overall topic sentiment, provides Quick Insights, and a detailed topic-by-topic breakdown for all 10 topics. From here, employers can see where to dive deeper.
Filtered View with Comparison
Different branches and job functions may have very different opinions so the filtering feature is cruicial. All the filtered key metrics are compared with the company average data.
Topic Analysis
Topic Analysis allows employers to dive deep into 1 topic to understand trends over time, read topic snippets from reviews, and a job function or country breakdown of sentiment.
Compare with Competitors
Employer clients can also compare with the competitors that they defined, understanding how their performance compare with the industry.
Export to PDF & Share
All the pages could be exported to printing-friendly PDF files for easy sharing with stakeholders.
Learn more about the product with this 1 min video
Credit to our brilliant B2B Marketing team :D
Design Process
Since I designed this product from 0 to 1 by myself, I do have A LOT of stories to tell about it -- the navigation, the onboarding, and tons of iterations of each 5 pages.
For the sake of your time and confidentiality, I'll only show the iteration process for the overview page.
.png)
Feature prioritization workshop
Before diving directly into the ideating, I held a design workshop with all the key stakeholders of this project, including
-
Leadership (B2B Product Director, B2B Design Director)
-
Product Manager
-
Lead Engineer
-
Data Scientist
-
Product Marketing Manager
-
Customer Success Manager
My purposes were
-
Gather Ideas from different stakeholders
-
Reach high-level idea agreement
-
Since I'm new to the domain, say hi to everyone, catch up myself on previous ideas & knowledge
Information Architecture 1.0
Based on the ideation in the design workshop, I groomed the content and features that we wish to include in this product and developed the IA 1.0 below.
Overview 1.0
Meat - Topic Performance
The meat of the overview page is the individual topic sentiment performance.
In the first version of the taxonomy, there were 6 high-level topics with 19 sub-level topics. I was exploring how to clearly display the hierarchy and to make it easy to scan.
But but, eating meat first might be too chewy...
Bites - Topic Insights
Users don't come here for numbers and graphs or getting lost in the long topic performance list. So I included two sections to reveal some topic insights.
Putting it together - Page Story
Does the story I'm trying to tell make sense for users?
Brown Bag Feedback Sessions
It was back in March 2020 when COVID hit the world. It was very hard to get real employer users to test with. So I reached out to 2 coworkers from our HR department just to get some quick feedback.
"
It's really hard to get the concept in the beginning. I do appreciate the insights on my top strength and weakness, but where are those topics even from?
"
Hierarchy Adjustment
In order to solve the problem of information processing, I adjusted the hierarchy and raised the topic sentiment performance section to the top, and also hid sub-topics so that users can focus on the top-level performance.
Usability Testing with Internal Users
After the iteration, I partnered with our research manager on a usability study with internal users -- Customer Success Managers.
There were 8 sessions of moderated usability testing, conducted by our research manager.
Key Finding 1
The concepts are hard to digest for users, especially the topic hierarchy, which also made the overall product logic very complex.
Key Finding 2
Users were often unsure of the data they were looking at after filtering.
They also raised the concern of unclear data when taking screenshots.
Results Debrief Meeting
After a debrief conversation with my PM, we both agreed the next round of iteration would be a team effort.
So I hosted a results debrief meeting with the key stakeholders (the engineering lead, PMMs, Data Scientists, etc.) and voted on the areas that we should focus on in the next round of iteration.
How could I make it more digestible?
Iterations Based on Research Finding
Taxonomy Adjustments
It was a relatively big team effort to change the taxonomy mapping. Since the team has reached an agreement that this is the key to improve user experience, the process went on pretty smoothly.
Let's do this!
As a result, we were able to eliminate the topic hierarchy, from 6 high-level topics, 19 Sub-Topics to 10 topics.
Redefining Page User Task
Overview
Topic Analysis
Reviews with Topics
Learn overall performance;
Discover topics to dive deeper
Understand the contribution of the topic sentiment performance
Read all original quotes of the topics
Information Architecture 2.0
IA 1.0
Overview 2.0 Page Story
Making it Screenshot-friendly
In order to better serve the using scenario of taking screenshots of the analytics and put them into slides or reports, I introduced the filter label component to design liabrary and added those to the section titles. Now when users take screenshots of any section, they will never lose the filtering or time frame context.
Hand off to engineering team
There are various data statuses and view-ports of different modules. In order to make it clear to engineers, I organized all the possible conditions of the modules into an artboard to highlight possible different treatments.
Here's an example of the topic sentiment performance card.
Impact & Results
-
Successfully launched in Nov. 2020 as planned.
-
Customers gave great feedback for this product, saying it’s great to quickly pull insights out and dive deep easily.
-
Because of the success of this product, I was honored to become the first designer who won the quarterly hammer award of B2B Org (One employee per quarter)
