Bad Surveys: Asking What You Want to Hear

It’s so tempting, and probably just part of our human nature – but the tendency to send surveys that lack the potential for real insight leads companies down paths of their own creation to their eventual demise.

It Sounds Good

It usually starts with one noble soul who brings up to the group the idea of doing a survey. This survey is the beacon of hope for bringing some outside information on a customer, an experience, or a product back to the group. It’s done in the hope of getting some “real” insight into the overall success of the group’s efforts, so that they can get a direction for the future.

The problems start when that same group creates the survey questions. Things start innocently enough. But slowly and certainly, the desire to to justify the group’s effort, hopes, and dreams starts to creep in.

It starts out with questions like, “Do you enjoy your new widget?”

Tired of marketing that does not get the results you need? Stop trying and  start doing. See How ››

But the slippery slope starts to slide into questions that narrow the scope and bring more leading words into the fray. This is how you end up with questions like,”What frequency do you use your widget? Daily, Weekly, or Monthly?”. It’s how you can totally ignore the options that exist outside of your assumptions – answers like, “None”, or, “I no longer have the widget.”

These are a few common pitfalls in survey creation: leading questions and limited answer pools.

The point is, I don’t fault anyone for creating a bad survey. The intention is usually good. But I do think, as an organization, you need to avoid these situations.

Hopefully, the following can help.

Things to Watch For

Leading Questions

This can be a subtle killer of your survey – when you start using leading questions that help guide the participant toward the answer you desire. This can be done by assuming a feeling or emotion, or by using assumptive language. Assumptive language refers to phrasing that create assumptions in the overall statements. For example: “Most people like fish when they dine paired with red wine. Would you like red or white?” It doesn’t take long to see how this impacts results.

Limited Answer Pool

The next common error is to have a limited answer pool. Most of us who want to get hard data, or typed data, like to use limited answers for many questions. Options like “Yes” and “No,” or scales of one to five. But in doing so, sometimes we can lock ourselves into the answers we are looking for and lock ourselves out of getting real insight.

The better suggestion is to think around all the possible answers that could exist and then work on your pool of answers. Allow for negative and positive, for very high or for very low. Don’t limit the pool, or it will lack depth.

All Quantitative Questions

Another issue is using too many (or only using) quantitative questions. That’s fancy term for answers that are contained in a pool of possible answers or numbers. Quantitative questions contain a quantity or an integer that can help you remember.

For example: “How many times did you go to the bathroom today?” The answer there is a quantity.’

An example of limiting responses via qualitative questioning: “What color of hat do you like? Red, blue, or black?” In this case, a qualitative question could have provided more insight.

Stop Directing. Start Listening.

There are many other details to the making of a great survey, but taking your time and making sure you aren’t leading the participant with your questions is a key factor in getting good results.

Tired of marketing that does not get the results you need? Stop trying and  start doing. See How ››

Want to increase your survey skills? Let’s talk about working together to help bring your survey to completion.

New Call-to-action