We’re gearing up for another year of brand research and creative, and it seemed like the perfect time to review some market research best practices. Sometimes, knowing what doesn’t work is just as important as knowing what does, so we’ve compiled a list of a few common mistakes to avoid as you design your survey instruments or write focus group and interview scripts.
Mistake #1: Asking Too Much
You have the research dollars, you have a project underway, and so it’s natural to want to uncover as much as you possibly can about your stakeholders. Then you begin seeking feedback from others on campus, and soon you have a list of even more questions they want to ask. The result can be a questionnaire that is too long and too unfocused to be effective. To prevent this mistake, start with a good, solid research plan. What decisions do you need the data to inform? What questions do you need to ask to get that information? Developing a clear plan at the outset can help everyone involved understand the goals of the project, keep research streamlined, and ultimately result in better, more useful data.
Mistake #2: Leading Questions
A leading question is exactly what it sounds like: a question that leads a respondent to a specific or “correct” answer. It implies or assumes that something is true without any evidence to support it. For example, in the question, “How do our university’s world-renowned undergraduate research opportunities affect your interest in attending,” inclusion of the phrase “world-renowned” can influence respondents’ answers, regardless of their own interest in participating in undergraduate research. A better question would exclude the phrase “world-renowned” and result in more reliable data.
Mistake #3: Double-Barreled Questions
Double-barreled questions ask your respondents to answer multiple questions at once. For example, asking respondents to “rate the quality of our academic and student support services” asks them to rate two different attributes, but allows for only one response—not only confusing your respondents, but also resulting in meaningless data. You can’t interpret findings without knowing specifically what the respondent meant. For results to be measurable, stick to one topic per question.
Mistake #4: Order Bias
Order bias happens when you order the questions in a way that the first question influences the response to subsequent questions. For example, if I asked you, “Which of these four television shows is your favorite?” and then, “How much television do you watch in a week?”, you may just focus on that favorite show from the first question instead of considering all the other shows and channels you watch. In some cases, you can avoid order bias by randomizing the order so that respondents don’t all receive the questions and/or response options in the same order.
Mistake #5: Respondent Fatigue
When surveys start getting lengthy, it can leave respondents frustrated, especially if they were only planning to spend a few minutes filling out your survey. This leads to respondents rushing to fill out the survey and skipping optional open-ended questions, which ultimately skews your data.
The same goes for qualitative research. Qualitative research is built around a moderator guide to lead the conversation, but sometimes it’s hard to estimate how long the conversation will actually take, because it depends on how much any given participant elaborates. This is why it is so important to limit the number of questions to allow participants enough time to do the storytelling and describing the “why.” If an interview gets too long, the participant might start giving shorter responses or even end the interview early.
Mistake #6: Asking Difficult Questions First
When designing research studies, you should carefully control the direction and flow of the conversation. Starting with broader, more general questions helps make the participant feel more comfortable. Then, by the time respondents get to the more difficult or sensitive questions, they have already thought about some of the basic, broad concepts of the topic that help provide context. Think of it this way: You wouldn’t ask a major donor for a half-million-dollar gift at your first lunch meeting, right? Similarly, you want to ease into the (research) relationship and build a foundation of comfort and trust.
