User onboarding is never finished. Gathering user feedback and improving onboarding isn’t just helpful. It’s necessary to providing value, especially as your user base grows and your product evolves.
While quantitative data can tell you what’s wrong, it doesn’t tell you why. Without knowing why, it’s hard to determine the impact a change could have. Will sprucing up the copy on one in-app message be enough? Or will the entire onboarding flow need to be rebuilt?
Supplementing quantitative data with qualitative feedback will help you better pinpoint what to improve and move the needle on user onboarding with more speed and accuracy.
Here are four ways to get qualitative data to help improve user onboarding.
1) Play back user sessions for true-to-life activity
Session replay tools like FullStory can show how your users currently interact with your product. This raw footage walks you through every click and feature view as users sign up and explore.
Help desk software Help Scout knows the importance of providing immediate value and has a relentless attitude towards improving onboarding. “We always believe we can be more helpful to the user during onboarding,” says Help Scout’s VP of Product Dave Martin.
To find ways to improve, Dave and his team use FullStory to see where people are getting tripped up as they go through the sign-up flow:
“Doing so helps us catch bugs before users report them. It also allows me to observe where there are friction points, or if there is functionality in the flow that the majority of people aren’t using.”
Dave found that people were getting confused by the invited user form. While this probably only caused a second or two of confusion to some users, Dave notes that it’s an “unneeded friction” and plans to improve it.
Pros: FullStory provides a true-to-life vantage point of how users interact with your product. That kind of unsolicited feedback is invaluable to anyone who wants to know how their product actually comes across to real users.
Cons: Can be overwhelming if you don’t have a review system in place or aren’t sure of what you’re looking for.
When to use: It’s worth checking out FullStory whenever you’re working on a particular product engagement issue, or if you’re just curious about user activity.
2) Use live chat to uncover bigger problems
If you use chat for sales or customer support, users may already be providing you with a gold mine of product feedback.
Recruitment software Recruiterbox had been using the live chat tool Olark for sales and support. By reviewing questions that trialers submitted, the support team was able to identify what people struggled with when first using the product. "From there, we tweaked onboarding to help highlight key things people were missing," says Chelsea Stroh, a Customer Happiness Representative at Recruiterbox.
She walks us through a recent improvement:
“For example, folks were creating openings, but not modifying the custom application form, because it wasn't part of the flow. We noticed through chat reviews that this was a common question, so we added it to the workflow when people create their first opening.”
Now, users are prompted to customize their application form after posting an opening:
It’s a small nudge towards a logical next step, but it can be the difference between a user diving deeper and one writing into support, confused and less engaged.
Whenever the support team notices a trend in chat logs, they’re quick to raise the issue to the product team. Chelsea Stroh describes this dynamic:
“Our team has a stellar loop with our product team. We raise ideas to them, and then have a monthly prioritization meeting, to make sure we're on the same page as to where things should be ranked. From there, they build the requests we've submitted.”
Pros: Uncovers organic points of frustration and ideas for solutions that might not come up otherwise. Good combination of qualitative and quantitative data. Allows support teams to help customers in real-time.
Cons: Requires a sophisticated review process to notice trends and a good feedback loop between support and product teams to act on the trends. This could take up an enormous amount of time to set up and may not guarantee results.
When to use: If you’re already using chat, it’s worth checking to see if you have enough volume to spot the same problems over and over again. Requires some coordination between support and product schedules to solidify the feedback and implementation process.
3) Obtain direct feedback during user tests
Traditional user tests remain powerful venues for getting feedback. One of the biggest benefits of user testing is that it’s a very controlled environment. You can ask users to do exactly what you want them to do and guide them towards addressing certain topics. You can also select who to ask to really pinpoint onboarding issues among a particular segment of users.
At Appcues, the entire product team sets aside a full day for user testing every month. Hearing feedback in real-time helps us validate ideas faster and identify opportunities for improvement, with our customers’ help.
Guru Mahendran, an Appcues software engineer, describes how the qualitative feedback helps us build experiences that better resonate with users:
"While data gives us a sense of what is working and what isn't, it doesn't tell the whole story. User tests help us fill in the gaps. They help us understand why there are drops in our funnel and get us closer to what our customers are thinking."
Having user testing days as regular meetings in our product planning cycle has also helped us work towards meaningful deadlines.
Pros: Highly controlled environment. Get to hear people talk in their own voice, which can be especially insightful. Not that time-consuming—many guides say that you just need 5 user tests to find most usability problems.
Cons: Less conducive to spontaneous and organic thoughts.
When to use: It’s always a good idea to get this type of feedback from a select group, before launching something off to your entire user base. You can run a user test anytime you make a major change to your onboarding flow, redesign part of your product, or launch a new feature.
4) Launch feedback surveys right after onboarding
Asking users what they think of onboarding right after they complete the flow can solicit feedback right in the moment. Users are already thinking about onboarding, and likely have some opinions about it.
Digital asset management software Bynder uses Appcues to build an onboarding experience for their self-service demo app. As part of onboarding, they created a simple 2-question survey in a modal window.
When asked “Did you find the tutorial tour helpful?,” 80% of users said yes.
On top of that, the survey had a 60% completion rate, which shows high engagement overall. About 10% of users provided valuable qualitative feedback that led Bynder’s growth team to make the experience shorter and simpler. Ariel Long, Bynder’s growth hacker, says, “UX should always be user-centric. Appcues provides the opportunities to gather both hard and soft data.”
Bynder launched this survey after users played around with a few of the features and had gone through a few product tours. Asking users for their opinion right after one welcome message probably won’t be as effective; give users time to explore and achieve some value to get the most out of surveys.
Pros: Timely and relevant, avoids bugging users out of context. Easy for users to opt in or out.
Cons: More structured feedback due to survey format. Risks asking for too much upfront if the product’s value isn’t yet clear.
When to use: Could be included at the end of any onboarding flow or product tour. Product teams need to set up time to review the data.
Onboarding improvements require holistic feedback loops
It has become easier and easier to make small tweaks to a product’s onboarding experience. With a combination of onboarding tools and analytics platforms, product teams can edit an onboarding flow as easily as marketing teams can whip up an email campaign.
It’s a powerful dual already, but the feedback loop is incomplete without qualitative feedback. There’s little excuse to not do the due diligence when it comes to something as important as onboarding.
Want to become a User Onboarding Master? Check out our free User Onboarding Academy!
