When a product team starts working on a new or existing product, there are a handful of reasons why an engineer may not be looped into the user testing process.
I’ve heard from product teams—and the engineers themselves—that they view user testing as a waste of valuable development time. Or, I see engineers who feel a little uncomfortable interacting with people who use and critique the product they’ve built. I’ve also been in environments where teams fear that an engineer could dilute the product conversation with technical details that aren’t relevant at that stage.
While I agree that these are all valid hesitations for involving engineers in the user testing process, I think these issues can be avoided with the right structure and product culture.
The end result is not wasted time, terrible insults from customers, and unproductive product meetings. Instead, a team can emerge confident with a product that is maximizing its potential value for customers by ingraining empathy into almost every decision the product and engineering team makes.
Getting into the right state of mind for user testing starts with setting the same expectations for your own product that you have for all of the products you and your team use.
Set the same expectations for your own product
Have you ever used software and said, out loud, “why did they build it this way?”
Ask an engineer that same question, and they’ll probably be a lot more passionate about how poorly it’s built—with plenty of profanity.
They’ll look into the source code to examine the toolkit this company is working with and then criticize the tech stack, wondering how anyone could ever build anything on it.
And I’ll be honest—I appreciate the passionate reactions of makers. I think this energy can be channeled into building better products. You bet that I’m guilty of being one of those engineers who complains about other products and sets expectations extremely high.
But what engineers often miss or shy away from is the reality that some of their own users of the product they work on feel the same way.
Your users are probably just like you and lose trust during that five minute period where a feature was unavailable or churn after not finding value out of a product that is difficult to use.
In other words, I’m suggesting that building a great product starts at your team’s engineers setting the same high expectations for themselves as they do with other products.
What’s the best way to organically remind engineers and start this cultural change?
User tests!
User testing should be about getting raw, unbiased feedback that helps an entire product team, including engineers, better understand their users and how they use or get value out of their product. User tests can help engineers incorporate empathy into almost every decision they make, from deciding on what to work on, to when they should deploy.
How to build empathy into (almost) every decision an engineer makes
The best way for engineers to build empathy into every decision is to have them experience first-hand the positive and negative feedback that people give in a user test.
I’ll give you an example.
I clearly remember the impression one of my first support calls left on me. I was a relatively new software engineer in the professional world and needed to diagnose an issue a customer was experiencing with an email tracking tool I was working on.
I remember being pretty nervous hopping on a screen share and audio call with a customer who had been frustrated and unhappy with the product that I had helped build.
Right before the call, I found out this customer was all the way in Germany, which I hadn’t realized before. When we started the call, I immediately set the tone and apologized for any inconvenience our product has caused, explained that I was one of the software engineers working on fixing this issue, and that I needed to recreate the scenario with him.
It was an awesome experience—the customer helped me understand how he used this tool for his business and why the issue was a big deal to him. We recreated the scenario and fixed his account quickly. He appreciated my honesty and was surprised to hear he was talking to one of the software engineers that is building this product.
What he didn’t realize was how this impacted how I think about building product and being involved in user tests—whether they are within structured settings like question/answers and demos or solving a support case with this customer.
The reality is that there is a human on the other end of what we build; we should strive to not let those people down. They may be nearby, or they may be across the world. In any case, we should be building products that solve problems for other people and provide experiences that we expect for ourselves.
Feeling the pain of our users really puts this in perspective, and is a nice reminder for engineers when trying to make decisions like:
- Should I deploy before leaving for lunch or on a Friday night before leaving work?
- Should I quickly revert the bug I shipped or leave it until I ship a fix for it?
- Should I iteratively introduce this feature, or should I try and ship this feature in huge chunks?
If the engineers at your company would have a tough time answering these questions, have them talk to 10, five, or even just one customer, and I bet they’ll establish some strong opinions.
The cultural long tail of engineers in user tests
I’m a strong believer in user tests because I think they are much farther reaching than just helping to iterate on product.
User tests can affect an engineering culture by encouraging a healthy amount of urgency, helping engineers avoid unnecessary complexity, and spreading confidence to the entire team.
If an engineer is in the room when customers repeatedly complain of bugs and downtime, it’s more likely that the process for building, testing, and shipping product will be changed to easily allow engineers to revert bugs that are introduced into production quickly and effectively.
Often times when you look into why a bug was shipped, it’s because several changes were chunked into one, or something was simply missed. It’s just not possible to have other engineers review a huge piece of code, expect them to internalize both the business and technical logic, and then provide a helpful review.
Maybe if engineers trace back customer concerns from user tests to massive pull requests and deploys, they’ll be more likely to ship incremental changes, gated changes, or even changes that avoid migrations and are backwards compatible.
Lastly, hearing from people who use your product can help your team adopt a culture of humility. I’m not asking an engineering team to take action on every feature request, because they may not represent the majority of your users.
User testing can help your team become more self-reflective, and at the end of the day, more confident in the decisions it makes going forward. This self-reflection finds its way into other processes that make up a great engineering team, like peer code review, and constructive feedback on infrastructure and development cycles.
The right way to involve engineers in user tests
User testing is found in many forms because it happens at different product stages. Sometimes they’re used for a brand new product that users have never seen. Other times you are speaking with existing users who may be happy or frustrated.
I like to consider providing support to a user a form of user testing too, because you often learn about the specific concerns of users they have while using your product.
In all of these cases, engineers should be involved early and often.
For example: At Appcues, we allow new engineers to shadow other engineers who have been at the company longer when debugging support tickets. This helps onboard engineers and give them a taste of some of the pain points our customers experience on a daily basis.
I did this in my second week at Appcues, and then worked on a project related to account management that helped reduce the same types of support tickets I had to solve a week before.
Talk about understanding the “why” you are working on something!
All of our product engineers participate in monthly user tests, especially when the tests are related to improving or building products they actively work on. Some of our platform engineers participate, and feedback is always passed along. Since engineers are helping run these user tests, it’s reasonable for them to pass along specific technical limitations our platform may have to other teams.
You don’t want to throw an engineer into a user test, be given mostly negative feedback, and then alienate and blame the engineer for the buggy behaviors or features. A team makes decisions, and the team should feel responsible for the feedback that user tests provide.
An engineer should be in these user tests to not only hear feedback, but to also contribute technical insight to the rest of the product team. If you think this technical insight is going down a rabbit hole, just suggest that ideas are kept high level as the team tries to find usage patterns and user sentiment. Sometime in the near future you may need to dig deeper into the technical abilities or limitations, but now isn’t the time.
Overall, user tests should be:
- A way to discover who your users are, what keeps them up at night, and what makes them happy.
- A view into the way your customers currently use your product as well as how they want to use your product (and potentially why they can’t use your product in that way)
- A positive and constructive experience that spawns new ideas and helps identify areas of product improvement
User tests should not be:
- A way to lead customers to provide answers you want to hear, rather than the answers you should be hearing
- An excuse to have team members and customers bash decisions that product managers and engineers have made
Start user testing with more engineers!
I hope I’ve convinced you to start involving more engineers in your user tests. If you are unblocked from doing this, then get started right now!
If you think that your existing processes or organization will be a blocker to implementing this, here are some points you can use to influence a cultural change.
The best product teams in the world don’t want to let their users down. Departments like customer support, success, and sales all know what it’s like when users are satisfied or unsatisfied because they hear firsthand. Imagine if engineers were exposed to their users the same way? Imagine if they set the same expectations for building their own products as they do with the products they use?
User testing will provide a customer context that should be in almost every decision teams make. You’re forced to start answering questions like “if I was in their situation, would I continue using the product that I’m building?” Imagine if an engineer often put themselves in their users’ shoes before, during, and after they ship product?
You should push your organization to consider these questions. Once these points seem obvious to everyone, then I think you are trending towards true product team alignment.
[Psst: We’re hiring collaborative, user-centric engineers. Check out our open positions.]
