|
Marketing Viewpoint by Ruth Winett
Why Professionals Should Use Cognitive Interviewing
(Survey Tips Pt 2)*
Our professional lives revolve around asking questions either verbally or in writing. Think of the pages of questions your doctor or your tax attorney or your lawyer asks before an appointment. Consider the questions salespeople ask to identify customers' needs and preferences or the test questions teachers pose.
Yet, all too often, the questions miss the mark because the questioner and the survey respondent/client/customer/patient/student are not "on the same page." If you want better results, pretest questions using the cognitive interviewing techniques described below.
What Is Cognitive Interviewing?
Linda Naiditch says cognitive interviewing is a "type of pretest." Cognitive interviewing, "actively delves into how [people] interpret the meaning of questions and possible responses, what they think about when they are considering how to answer, how they decide on their answers and what their answers mean."
Why People Misunderstand Your Questions
People misunderstand questions of all types for three major reasons:
Different reference points. Each member of a movie audience will describe a movie they have just seen in different ways. People know different things. They have different life experiences and different perspectives. And, they value different things.
Poor questions. You may have used language that is vague, ambiguous, contradictory, offensive, or beyond the understanding of the audience.
The none-of-the-above phenomenon. You cannot anticipate all the ways people would like to answer a question. When the responses provided in multiple choice questions don't suit the respondents, the respondents often choose the best of the worst. This skews survey results.
How To Use Cognitive Interviewing
Gordon B. Willis, author of Cognitive Interviewing: A Tool For Improving Questionnaire Design, did the original research on cognitive interviewing. Naiditch has outlined five ways to ferret out problematic questions, pinpoint faulty assumptions, identify omitted but essential information, and identify ambiguity:
Use your own words. Ask respondents to restate the question or the possible responses in their own words. This will help reveal misunderstandings.
Think-aloud. Ask respondents to report what they are thinking as they answer a particular question. This approach could lead to highly productive new lines of questioning.
Define key words. Ask respondents to define ambiguous key words. This will reveal words that researchers should define for other respondents to avoid confusion. Naiditch found that "sustainability" meant widely different things to different respondents.
Identify difficult questions. Ask which questions are hard to understand or answer. Once you have probed to find out why these questions are problematic, you can fix the problem.
Identify other signs of difficulty, such as hesitating to answer a question or acting confused because some questions "conflict" with each other or fail to consider respondents' situations. Naiditch says one respondent couldn't answer a multiple choice question about whether he had normal blood pressure. His blood pressure was normal-when he took medication. They added another choice - normal blood pressure when taking medication.
Once you have identified problem questions, you should revise them or eliminate them.
You know when direct questions to a respondent/customer/client/patient/or student have missed the mark because of the person's puzzled expression. But, when you use phone, written, or online surveys and questionnaires, you can't see people's facial expressions. To avoid confusion, some market researchers and even medical researchers are now using cognitive interviewing to pretest questions. These pretest techniques have potential for lawyers, teachers, psychologists, and other professionals, as well.
*Click here to read Ten Tips for Creating Online Surveys that Get Results (Pt 1).
Market Intelligence for Growing Companies © August, 2013
articles index |
home | back |
