I love conducting user interviews and user tests! But I don't really know what I'm doing. Through our education, designers learn how to collect user data. The methods we learn are mainly based on interviews, testing, workshops etc. But we are not trained well enough in how the data we receive must be obtained as cleanly as possible. We need to be more aware of how we potentially "contaminate" the data we collect. We ask leading questions, we suggest what they should do, and we interpret their actions in the best sense.

More than once I have caught myself saying something like "Yes, you probably knew that was where you were supposed to press". And I have probably more than once set up an user interview where I actually already know what I want the user to answer. Give me a UX researcher who can set up good moderated interviews and tests. Or a data analyst who knows how to handle data and how it should come in straight from a production environment, without me having had the opportunity to pollute that data. Because the data I bring in through interviews and tests is tainted by my own biases.

Original illustration made by DeeKay Kwon

It gets messy

I am not asking that anyone else retrieve that data for my own sake, but for everyone else's. I really enjoy conducting user tests, while getting to confirm or disprove my own assumptions. But the data is and remains dirty. It will be like bringing a child to bake buns. It gets messy, and there is always a bit of snot in the dough. It may taste quite good, but there is still a bugger in that bun.

Before an interview or a test, the team should have made themselves aware of which biases they potentially bring with them and how we can be as neutral as possible. Perhaps you can even retrieve the data you are looking for right from a production environment? Then you can rather use interviews and tests to obtain data on things that cannot be obtained directly from prod. The result of a good user interview or a good user test should be a set of data that we have jointly reviewed and checked that we have not mixed our own biases or interpretations with what the user actually said or did.

So let someone else obtain that data

Include me in the interview and the user test, and include me in the process, but there are others who are more qualified to guide the user through. Include me in the work of setting up measurements for production data, but let someone who knows which biases can affect the data take the lead.

I may not be the best at obtaining clean, untainted data, but what we designers are actually good at is analyzing that data and processing it into insights that are understandable to others, and that can be used as a knowledge base for further decisions in the team. Together let's boil it all down to insights that we can use. I think we often get a truer picture of reality by getting good data straight from prod, rather than conducting interviews and user tests all the time. Let someone else obtain that data.

What did you feel while reading? Did it spark something in you? Do you agree or disagree? Join the discussion on UX Norge's slack (Norwegian) or share this article with your colleagues and start a debate! Remember that the goal of Brannfakler is to start discussion!