"If you’ve been involved in user research that didn’t hurt at all, then I guarantee you did it wrong." writes Harry Brignull in the "90 Percentof Everything".Reading this quote I immediately remember all the user research I've done over the years and can confirm that this is the truth. The uncomfortable truth about user research. The truth that sometimes drives people away from doing user research.
Most of the stuff we do in business is designed for a positive outcome. Positive = good. New product released = good. The new deal closed = good. New ad campaign released = good. Critical customer support ticket resolved = good. But with user research - it's a different story. User research has validated fully our assumptions = bad. We most likely did the user research wrong. Why is that you'll ask? Because of the confirmation bias mainly.
What is this confirmation bias?
The tendency to interpret new evidence as confirmation of one's existing beliefs or theories.Yes, this is exactly that: "I told you so" kind of statement.
We can see confirmation bias everywhere in business (and in our private life as well). But on the topic of user research, it hits us the hardest. Why?
Let's review the typical user research process:
- Spot a problem
- for example, daily commuters feel bored while taking public transport to work
- Formulate a hypothesis
- for example, daily commuters would appreciate some entertainment on their daily rides to and from work
- Select the research method
- for example, 1-on-1 interviews
- Recruit the representative respondent group
- for example, 5 daily commuters that live in a big city and use different means of public transportation
- Make interviews
- Analyze results
- Validate if knowledge gathered confirms or confronts the hypothesis
On each step of that process we can make mistakes:
- a problem might not exist
- we can wrongly formulate a hypothesis
- we can choose a wrong research method
- the respondent group might not be representative
- we can make mistakes conducting research or when analyzing results
Yet what happens even more often is that we subconsciously will try to confirm our hypothesis with the user research. So we'd make all those mistakes above not to disproof the hypothesis but to confirm it. And the reasons for that is not only personal but also organizational.
Very often our work environment is optimized for delivery. Developers are waiting for some user stories to start coding. Designers waiting for some wireframes to start drawing. Sales waiting for some products to start calling prospects. The entire organization is waiting to start the delivery. Yet when we do user research we often need to say: "wait a bit longer guys, we're not ready yet to jump into delivery".
On a personal level user research also could be uncomfortable. Bad news in user research is good, but it still feels bad to be wrong. And even if you managed to convince yourself that bad is good, it would take significant effort and time to convince your company.
- What? Your hypothesis turned out to be wrong again? You really should take some hypothesis formulation course...
What could help?
Appropriate objectives for user research. Qualitative objectives that reflect the value of vital learning user research, when done right, delivers.The objectives need to reflect that "bad is good".
Not:
- Look, we've been absolutely right and proved our hypothesis from the first attempt.
But rather:
- Look, we've been wrong, and then wrong again and the next couple of times still been wrong. But then... well... we cannot find evidence we're still wrong.
We do user research right when we prove ourselves wrong. When we cannot longer prove ourselves wrong - we can say "well done" and move to the next stages of product development.
Takeaways
- User research could be uncomfortable because it delivers bad news almost always when it's done right.
- In a case of user research: bad = good, which is contradictive to how the rest of an organization is working.
- The biggest enemy of user research is the confirmation bias.
- Look for ways to disprove yourself with a help of user research. When you cannot longer do it - you did user research right.