Open Ended Questions and Close Ended Questions: What They Are and How They Affect User Research+
- Posted in Conversion Rate Optimization
A complete user research approach involves both the quantitative and qualitative research process.
Seasoned experimenters know that quantitative research without qualitative research leads to an incomplete conversion user research that leaves many questions than answers.
The questions you’re seeking to answer are not how my web visitors interact on my website/app (quantitative research already covers that) but why are they acting the way they do? What are their intentions and motives?
This is where the qualitative research approach comes into play.
Except you’re a mind reader, qualitative research as a conversion rate optimization process is how you get to know the why of your customers and web visitors.
You get to mine unfiltered customer insights by asking them open-ended and closed-ended questions.
As the name suggests, open ended questions are free-form questions that allow customers to air their opinions without restrictions. In contrast, close ended questions have a YES/NO reply format or minimal options.
The mistake I see many marketers and CRO specialists make is the assumption that it is just questions, so they’ll be easy to formulate and ask.
No! You couldn’t be more wrong.
Here’s a scenario that drives home my point.
If you ask the wrong questions, you’ll get the wrong answers, leading you to wrong conversion blockers that you’ll then prioritize, run A/B tests, and get skewed results.
Now, we don’t want this type of monumental error, which is why this article.
We’ll be looking at what’re open ended questions, and close ended questions, what user research is, examples of open ended and close ended questions, different types of qualitative research, how to apply open ended questions and close ended questions, and the impact of open ended questions and close ended questions on user research.
Ready to learn more? Strap in, and let’s get started.
What Are Open Ended Questions and Close Ended Questions?
Open ended questions are questions that don’t restrict the participants in the answers they provide.
What this means is when you ask a participant a question, you expect them to elaborate on their points.
Open ended questions offer a deep pool of insight into your customer’s mind because the interview process is free form, and the participant is expected to give you feedback instead of stock answers from a limited answer pool.
On the other hand, closed ended questions require a yes/no reply or have limited responses that the participants can give.
Close ended questions give limited insight to the research’s conversion rate optimization specialist because the participants are expected to respond from a limited pool of answers.
A significant advantage of close ended questions is that they can be asked to many participants, and you can analyze them for quantitative data (more on this later).
Types of Survey Questions
The beautiful thing about survey questions is that they can be used in open ended questions and close ended questions.
An important point to note is this; before writing your survey questions, ask yourself this question; how do you intend to use the answer from this survey? Do you want quantitative data or insights into the mind of your customer?
This will help you in the framing of your questions.
Below are some popular types of survey questions you can ask for quantitative data and qualitative insights.
- Dropdown questions: It’s an easy way to display a long list of answers to your respondents only when they click on it. This question type makes your survey look clean and not busy.
- Multiple choice questions: This question type offers multiple answers and allows your respondents to select one or more solutions. They’re intuitive and easy to use.
- Open-ended questions: This question type does not present any predefined answer; it allows the respondents to answer however they like without restrictions.
- Slider Questions: The question type is represented by a slider with a figure starting from 0 or 1, and it only moves higher gauging customer satisfaction.
- Demographic questions: This question type is perfect for when you want to gather information on your audience’s background, income level, what interests them, etc.
This section is not meant to be exhaustive; it covers some of the questions you can ask as open-ended and closed-ended questions; for more question types apart from the ones covered here, ask the good old Google and get good responses.
Open Ended Question and Close Ended question; What’s The Difference?
The significant difference between open and close ended questions is that close ended questions provide the respondent with limited answers they must choose from. You can’t air your opinion the way you want, and your response is minimal.
On the other hand, open ended questions allow the participants to respond as they wish, no holds barred. You’re given the opportunity to share your opinion and your thoughts entirely.
Examples Of Open Ended and Close Ended Questions.
Improving the customer experience on your website does not start with an A/B test experiment; it begins with asking the right questions.
Only a mind reader does not need to ask people questions to know how they feel and how they’ll respond.
As a marketer, if you want to know more about your web visitors’ experience on your website and uncover conversion blockers, you need to hear from them by asking the right questions.
Here are some examples of open ended questions and close ended questions we’ve asked our client’s customers that have helped our conversion strategy and informed our hypothesis and A/B tests.
The Importance Of Asking Open Ended Questions.
Open-ended questions are the way to go if you want to mine a lot of information on your web visitors and hear about their experience and what you can do better.
Open ended questions don’t put your respondents in a box. You’ll be surprised to hear answers and strategies you didn’t know about and won’t usually factor in.
At Invesp, we use open ended questions in our Jobs To Be Done (JTBD) process when interviewing customers of clients, and we have uncovered mental models and conversion blockers that otherwise told us we won’t have known.
When To Ask Open Ended Questions.
1. When you’re conducting testimonial interviews.
2. Discovery calls with web visitors who are yet to purchase from you.
3. Informational interviews with customers who have purchased at least once.
4. User persona research to know which set of web visitors to build out your processes for.
5. For design research to know which problems to solve and what solutions are most important to your ideal persona.
6. Use case research.
Tips On Asking Open Ended Questions
If you aim to ask open ended questions, your questions should follow this format;
1. Why, how, describe, tell me, can you explain further, etc. These question types let the respondents know they should air their minds, opinions, and thoughts.
2. When interviewing a participant, aim to collect more information and not one-worded responses because that defeats the purpose of open-ended questions.
3. Don’t make your questions so broad that people get confused.
4. Don’t get carried away with asking open-ended questions that you forget to obtain quantitative data in your customer research stage.
5. Don’t guide your respondents into giving you specific answers; that leads to bias which distorts the results.
The Importance Of Asking Close Ended Questions.
It’s true that close-ended questions limit the answers that respondents can give, but that’s where its power lies.
You can limit unnecessary answers that often happen in open-ended questions with close-ended questions.
Unlike open-ended questions, where you have to book several meetings many times with respondents, with close-ended questions, you get a higher response rate because you don’t require much from the participants.
When To Ask Close-ended questions.
1. In surveys and polls where you’re expecting a large number of responses.
2. When the answers to your questions are limited for specific reasons.
3. When you need to be able to quantitatively measure user responses.
4. You want to convert participants’ opinions into reliable data.
5. You want to collect consistent and reliable data.
Tips On Asking Close Ended Questions
This section is not meant to be exhaustive but to serve as a guide to form your questions.
1. Begin your questions with have, do, will, or did.
2. Write your questions in an easy-to-understand format. Understand that it’s a mixed multitude that’s likely to see your questions, and they’re not the expert; you’re. Write the question in a way that they will understand it.
3. Provide relevant answers to the questions you’re asking. This is a significant advantage of close-ended questions because you can filter the noise.
4. Don’t encourage lengthy answers to your questions. You can follow up with an open-ended question to get a more detailed response.
5. To avoid boring your respondents, switch up your response structure from yes/no to multiple answers, checklists, etc.
6. Don’t use the format of a close-ended question to explore the emotions/feelings of respondents; it’s not meant for that.
7. When crafting the answers to your questions, ensure they cover the range of possibilities.
The Impact Of Open Ended Questions and Close Ended Questions On User Research
The mistake of assumption is why many marketers make decisions not backed by user research and wonder why there’s no improvement in conversions.
Asking your web visitors the relevant questions to get the correct feedback in their voice is critical in understanding your quantitative results (the data you get from Google analytics and other web analytics platforms).
At the end of the day, if you decide to go with open ended questions or close ended questions, make sure your customers have sufficient knowledge of your website/product and the questions you will ask them fit their phase in your funnel so they can give you the correct answers.
You don’t want to be caught asking a first-time visitor about his experience purchasing your product when he’s just at the homepage. This question is meant to be seen by people who just placed their order.
Pro tip: Both open-ended and close-ended questions have their place in the customer research process. With a proper balance of both types of questions asked smartly, one after the other, the goals of JTBD, online surveys, and other qualitative research processes will be met.
Join 25,000+ Marketing Professionals
If you enjoyed this post, please consider subscribing to the Invesp blog feed to have future articles delivered to your feed reader. or,receive weekly updates by email:
Connect with us
The Art and Science of Converting Prospects to Customers
By Khalid Saleh and Ayat Shukairy
- How to Optimize Product Pages on a Shopify Website
- The Role of User Research in E-Commerce Experimentation
- The Role of Usability in eCommerce Experimentation
- Invesp and e-CENS Form a Strategic Partnership
- The Role of Branding in E-Commerce Experimentation
- Shopify Mobile Optimization: Tips and Techniques for Improving Mobile CRO
- The Role of Data Analysis in E-Commerce Experimentation
- How to measure the revenue impact of experimentation
- Top 9 SaaS value proposition examples to learn from in 2023
- How to do Quality Assurance (QA) in a high-velocity testing program