5 Common Cognitive Biases That Undermine User Research

Cognitive Bias in User Research

The psychological study of cognitive biases focuses on examining the deviation of human thoughts, judgements and rationale from objective fact. Cognitive bias is a symptom of the human condition and is difficult to avoid, as the phenomenon stems from our evolution. So, we tend to frame information in a way that leads us to make incorrect assumptions about the objective truth or reality of a situation.

Why does it matter to user research?

Given that cognitive bias is practically unavoidable, it can easily creep into our research objectives, planning and results. Researcher bias and bias in user responses can influence the reliability of results and can undermine the entire research process. Not ideal.

However, by making yourself aware of their existence and knowing how to recognise them when they unavoidably jump out, you can quickly reduce their impact.

1. Ambiguity Effect

We are programmed to avoid risk. So inevitably, we like to complete an activity when we know the outcome is positive and it feels like a good investment of our time. So, when we come across activities where the result is ambiguous, we tend to avoid it simply because we perceive it as having an increased risk.

Known as the ambiguity effect, it can impact user research significantly. Have you ever been considering exploring a particular behaviour, but been put off by the fact that you didn't fully understand it?

The ambiguity effect causes a blinkered approach to user research, where you can disregard certain areas for research because you are unsure of the final result or getting results that do not match your hypothesis (see below).

Such is the curse of UX research is that you will never be your user. In turn, there will always be a strong presence of ambiguity - the very problem you are trying to remedy. You can avoid falling into this trap by being curious and researching strategically. Nothing is off the table.

2. Confirmation Bias

The Ace of Spades. Confirmation bias is one of the most well-known biases. It drives us to seek information that confirms our own beliefs and preconceptions, even if they are not correct.

Confirmation bias in user research can upend your findings. A specific type of confirmation bias called the observer expectancy effect is common in user research. The bias stems from a researcher manipulating their objectives, questions and hypotheses to steer the results to favour their beliefs.

Be careful as this can even happen subconsciously, so be sure to keep an eye out for it by scrutinising your own beliefs, as well as the ones of others.

3. The Framing Effect

The way we respond to a source of information depends on how it is presented to us. The framing effect is a common researcher bias and can lead to specific prompted responses from testing, especially when using qualitative methods.

The explanation behind the framing effect is that we do not make our decisions based solely on our thoughts but on the delivery of a message. For example, during a user interview, you could frame a question as:

“What do you like about the product?”

While the question seems like an inoffensive approach to collecting information, you are framing the response before an answer is even formulated by the user. That is because the interviewee will likely focus on their likes rather than their possible objections or dislikes. It is better to frame your question like:

"How do you feel about the product?"

Framing the question like so makes it more open-ended and prompts a more well-rounded and truthful response. Another way to critique the framing effect is to use an empathy map to match user behaviour to what they say in interviews.

4. The Social Desirability Effect

As mentioned before, our cognitive biases are caused by thousands of years of evolution; and the social desirability effect is very much a symptom of it. The Social desirability effect often is a bias that manifests in user testing - especially in face-to-face research activities such as user shadowing or focus groups.

People want your approval. Consequently, they will present information to you in a way they think you want to hear or in a way that will please others in a room. If you are not careful, it can manipulate responses.

The best way to avoid the social desirability effect is to be thorough in your questioning. Play the devil's advocate and probe the user from many different angles to get the most truthful and valuable responses. Also, avoid other biases like The Framing Effect that can lead your respondents to give socially desirable answers.

5. Fundamental Attribution Error

Fundamental Attribution Error (FAE) happens when people emphasise personality-based explanations instead of using situational judgements to explain behaviours.

For example, if you were driving and someone overtook you while speeding, your first thoughts are likely some choice swear words or blaming that person for being a dangerous driver. What you don’t know is that the person was stuck in unexpected traffic and late for an important job interview.

Now, we’re not advocating dangerous driving. However, FAE often comes up in user research and testing and is easy to miss if you aren’t paying attention to your user's behaviours. Listen out for “I’ve made a mistake” during testing.

Chances are they haven’t made a mistake and there could be a potential flaw in the product design that they have attributed to their own incompetence. Understanding FAE may lead to your next big user research breakthrough!

*Bonus* The Dunning-Kruger Effect

The Danning-Kruger effect is a bias where highly skilled people tend to undervalue their understanding and people with little very little expertise tend to overvalue their abilities.

The bias can creep into all elements of a UX user researcher's work and product development. Next time someone is giving their “two cents” on your findings, product or ideas remember to be confident in your abilities. Perhaps even more importantly don’t fall victim to this bias and remember to listen to your user researchers and value their know-how and expertise.