Sheâs wearing high heels but manages to glide her way down the street as she chases the man. As she gets closer to him, the camera zooms in and her questions start to fly. She almost knocks his nose with her microphone. He tries to get into his car but sheâs faster than him.
She pushes the door closed and repeats her jabbing questions such as…
âDo you agree that you were WRONG to steal their money?â and
âDonât you think these people DESERVEÂ their money back?â
This is a common scenario on television and one that Iâm sure youâve witnessed.
Although at the time you probably thought âYeah, you tell him, Miss!â these questions are considered very, very bias⊠especially when you use them as survey questions.
Why? Because questions like these could influence the answer you receive.
How?
Well, in the first case…
“Do you agree that you were WRONG to steal their money?â
…you could easily hope for a âYes, OF COURSE! Iâm sorry!â
Otherwise, whatâs the point asking that question? It might be hard for the guy to say âNo!â as he knows it could make him look bad. He’d probably keep quiet, though.
With the next question,
âDonât you think these people DESERVE their money back?â  …you could probably hope for the guy to do one of the following:
- Remain silent and drive off
- Pull out a mobile phone and call his lawyer
- Shout âNo comment!â and drive off
- Put on a sad face and respond with âI havenât done anything wrong!â
- Use violence and push her aside⊠jump in his car and drive off
But why would anyone ask these questions? Is it to further prove a point that this man is a bad person?
Who knows?
But letâs go back to survey questions.
Why… or maybe ‘how’ do questions like these sneak into surveys?
I haven’t met anyone who has deliberately added bias questions in their surveys but I have been handed questions like these to add to surveys.
They were âaccidentallyâ worded in a way that they easily confuse or lead the respondent. The questions would have forced you (the respondent) to believe that weâre expecting you to answer in a particular way. With questions like these, I would have ended up with bias responses.
Itâs like if people were to ask you: âDonât you think you should find a job?â
You feel that youâre expected to agree with them.
We caught these questions well ahead of the surveys going out and reworded them.
Itâs possible that some survey designers donât know how to properly design survey questions.
Others probably donât understand the topic well enough. Or they didnât test the questions with a focus group.
Not testing a survey is considered a âsinâ in marketing research. ALWAYS test your questions with a focus group so that you know which of the questions should be reworded or ditched.
Another reason that bias questions sneak into surveys is the writer didnât consider other peopleâs perspective on the topic of the survey. Had they tested their questionnaire to see how well it works, they would have known that they would need to reword.
Whatâs the problem with using misleading survey questions?
Adding bias questions to surveys is rarely intentional but is always a big problem:
- They mislead the respondent â weâve already established that.
- They mislead your data â so it would appear that people agree to your line of thought.
- The misled data will be used to make important decisions for your business. Yikes!
- The âevidenceâ can easily be disproven by competitors if they were to conduct a âproperâ survey.
- The âevidenceâ leads customers and prospective customers to believe in you⊠and they could end up very disappointed later on.
Now⊠whether or not the survey analyst realises this after collecting data is a whole different story and maybe one for another day.
If you would like to see some examples of leading survey questions used in market research and marketing research, join the mailing list below to receive notification of the next post that offers a FREE list to check your questions against.
If you’re developing a survey for your brand and are not quite sure whether you’ve got things right, I’m able to assist with both your marketing research and market research activities. Get in touch today and we can discuss your requirements.
Woman courtesy of Michal Marcol, cartoons by The Hiking Artist
Reader Interactions