What are research questions and how to write them? | UX research #3

In this article, we will show how to formulate good research questions in UX and present the differences between open and closed questions. We will also point out in which situation they will work best. Read on to learn what a bad formulation of research questions can result in and to see tips on how to avoid it.

What are research questions and how to write them? – table of contents:

1. What are research questions
2. Open-ended questions in UX research
3. Closed-ended questions in UX research
4. How to form a research question?
5. Summary

What are research questions

UX research, which is a key step in the design process, is unfortunately often conducted poorly – in a way that does not match the respondent or the situation. Incorrectly asked research questions can lead to confusion, misunderstanding as well as wrong analysis of research conclusions. That’s why it’s crucial to ask the right questions in UX research – you’ll gain valuable information and clues on which needs to improve to optimize your product or service and enhance the user’s experience with the product.

Asking the right questions in UX research is challenging. It requires the researcher to have a lot of experience but also to be flexible – to adjust the questions depending on the situation, the respondent, the environment as well as the project itself.

We can describe research questions in UX as a tool for obtaining information about users. There are two types of research questions – open and closed. Both are important in the UX process and can provide us with valuable information. Qualitative research usually mostly consists of open-ended questions, while quantitative research consists of closed questions – but this is not the rule!

Open-ended questions in UX research

Open-ended questions in UX research are qualitative questions that tend to prompt a longer statement, and are less specific (they can hook into many aspects at once), so they allow you to get a lot of information. We can recognize them easily – they are questions starting with “if”, “how” “what”, “describe”, “tell”, etc. Mostly, the reply isn’t “yes” or “no” – they require a longer statement, at least a few sentences long. Open-ended questions deepen a given topic, let the researchers get to know the respondents well and understand their point of view.

Open-ended questions in the context of UX research will find application in:

• usability testing
• individual in-depth interviews (IDI)
• diary studies
• survey person
• task analysis

Open-ended questions have quite a few advantages: they help to elicit detailed personalized answers, connect with the respondent on an emotional level as well as identify users’ pains, frustrations and desires. They also provide the means for researchers to discover more than they expected. However, this form of questioning is often met with reluctance by respondents, who find it easier to answer a simple closed question. A survey conducted using open-ended questions consumes more time, which can make it hard to conduct on a large group of people. Also, analysis of such surveys becomes more difficult and takes more time.

Closed-ended questions in UX research

Closed-ended questions in UX surveys limit potential answers. Respondents can only choose an option from several predetermined ones or answer with a single word (and mostly only yes, no or don’t know). They are easy to recognize as we associate them with test questions, familiar from school exams. While open-ended questions may provide non-obvious answers and the acquisition of interesting information, closed-ended questions are useful for checking assumptions, and hypotheses and obtaining a specific answer. In the context of UX research, open-ended questions will work well:

• When creating and conducting surveys on a large group of participants
• When we care about quantitative indications and survey-supported data to create metrics
• When you are investigating something specified with a limited number of possible answers
• When you plan to repeat this survey in the future and you want to create a universal, repeatable set of questions to assess a certain trend, a change in hindsight.

Examples of closed-ended research questions might be:

• Customer satisfaction (CSAT) surveys – which is a survey with closed-ended questions that checks the level of customer satisfaction by asking the customer to rate the experience on a given scale. This can be a numerical scale such as “on a scale of 1 to 10 rate your satisfaction where 1 is total lack of satisfaction and 10 is satisfaction above expectations” or a simple choice between a sad and smiling icon after using a particular service
• Net Promoter Score (NPS) surveys – is another valuable form of a closed-ended question, asking customers how likely they are to recommend a company/service – ratings are made on a scale, usually, from 1 to 10, This helps to distinguish three categories among customers: promoters (rating 9-10), passives (rating 7-8) and detractors (rating 0-6) (promoters, passives, detractors). Net Promoter Score is a key metric to help assess business growth, track long-term brand success and measure overall customer satisfaction.

Closed questions just like open ones have both advantages and disadvantages. Starting with the advantages: closed-ended surveys provide more measurable data to turn into statistics and metrics as well as increase response rates because they are generally easier to answer. They also enhance survey coordination when the sample involves numerous respondents, enable you to realistically assess the facts, and finally, make it easier for the UX researcher to control the entire process.

The disadvantages of closed-ended questions concern the exposure of participants to bias, the inability to know the in-depth opinion or emotions of respondents, as well as the problem of having the right number of answers to choose from – there may be both too few (so that the participant’s answer may not be included) and too many (which will overwhelm the respondent).

How to form a research question?

To construct an effective question in UX research, it is necessary to first define the research objectives and hypotheses, and then measure them with users during surveys or interviews. When formulating hypotheses, remember that research is as much about discovering new questions as it is about getting answers. The research is not to confirm something we already know, but to learn other opinions, users’ perspectives and discover new unknowns.

With your hypotheses ready, you can proceed to define your UX research objective. The objective should relate to the hypothesis and define what you as a researcher want to achieve. Once you have formulated your hypothesis and research objective, it’s time to create general research questions. These are to define what you would like to discover, to know during the research process – keep in mind, however, that these are not questions that you will ask respondents, but general questions that guide researchers. Here’s an example of the research question: “How can we improve conversions on our site?”

Only with these three elements – the hypothesis, the objective and the research questions – you can begin to plan the research and formulate questions for users (in the form of surveys or research scenarios).

Summary

Formulating and asking the right questions is key in UX research. To create a high-level user experience, we need to deeply understand our users. The goal is to formulate solid, tailored questions during UX research. Keep in mind that UX research is not focused on what the researchers want, their goals and ideas – but on listening to the users and understanding their needs.

If you like our content, join our busy bees community on Facebook, Twitter, LinkedIn, Instagram, YouTube, Pinterest.