Archive for the ‘Omnibus Surveys’ Category

Your Margin of Error Is Probably Wrong

Thursday, December 29th, 2011

Even if you are not involved in political polling, it is worth paying attention to the methods and best practices of political pollsters.  One reason is that few other areas of research offer a way to completely validate one’s methods.  Pollsters are using sampling and survey methods to predict the behaviors of a much larger population.  Then in just one day that population behaves, we get a near-perfect count of exactly how they behaved, and we know whether the methods worked.

Several industry colleagues have recently been debating the merits of calculating and reporting “margins of error” in political polling, and pointed us to some surprising data from The New York Times: (more…)

Best Practices for Using Statistics in PR

Wednesday, November 9th, 2011

One powerful way to gain visibility and credibility in your marketplace is by sponsoring survey research that documents problems and solutions in areas where you have expertise.  To be successful, it requires (1) rigorous research carefully designed to uncover the right topics, and (2) savvy PR work that uses data to tell a credible and compelling story.

The Public Relations Society of America (PRSA) and the American Statistical Association have just published a handy guide for PR professionals that outlines best practices for using, interpreting and reporting statistics in press releases and other PR materials.  Some of those best practices include the following: (more…)

Internet Surveys and the Associated Press (AP)

Thursday, September 29th, 2011

Is it true that the Associated Press refuses to carry stories from online surveys?  Yes, as odd as that seems nowadays.  But news media face a difficult problem given how easy it is to conduct biased public opinion polling, especially now with online panels and social networks.  So some news organizations like the Associated Press (AP), The New York Times, and ABC News have developed guidelines that specify for a survey or public opinion poll to be valid and reliable, it must be conducted by telephone.

(more…)

An Interactive Graph for Choosing Sample Size

Thursday, June 9th, 2011

A good chart is the best way to understand the law of diminishing returns when it comes to sample size.  So for our June 2011 newsletter we built an interactive graph for choosing sample size.  It’s cool, educational, and useful.  Moreover, it will show you just how mind boggling the numbers behind sampling can be.  It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.

(more…)

Advice for PR Surveys: Avoid Numeric Scales

Thursday, April 28th, 2011

As much as we love numbers, we find ourselves often advising clients against using numeric scales in their surveys.  A numeric scale is any response format that asks people to give a number within a certain range to indicate the strength of their feeling or opinion.  The insanely popular survey question used to calculate Net Promoter Scores is a good example:

“How likely is it that you would recommend Acme Solutions to a friend or colleague?  Please answer on a scale from zero to ten, where zero means not at all likely, five is a neutral score, and ten means extremely likely.”

There are many good reasons to use numeric scales and many types of research for which numeric scales are optimal.  The NPS scale is good because it has eleven points with meaningful endpoints  and a meaningful midpoint.  Research shows that scales like this can be highly reliable and valid, with sufficient variability to allow for sophisticated statistical modeling.

But if your objective is to use survey data for marketing materials, public relations, news releases, or white papers, numeric scales make things difficult.  They are not easy to summarize in words, and if you want to use charts that tell quick, compelling stories, you will end up having to do something like this:

A Poor Fit: Pie Charts and Numeric Scales

(more…)

How to Stop Fraudulent Polls

Friday, September 10th, 2010

With the sad proliferation of silly surveys, non-scientific interest polls, and downright fraudulent polls, the research industry is stepping up with a number of key initiatives to combat the trend.  Versta Research is part of that effort, and in August announced that we are part of the Transparency Initiative being developed by the American Association for Public Opinion Research (AAPOR).

As of this writing, Versta is one of 67 prominent survey and polling organizations currently supporting the initiative. The initiative is designed to create protocols and recognition for regular disclosure of methods when survey organizations conduct public polls. (more…)

Writing Successful Omnibus Survey Questions

Friday, May 14th, 2010

Choosing an omnibus survey can be a simple approach to getting survey data, because it is usually inexpensive and fast, and involves asking just a few questions.  But there is sometimes a downside to simplicity:  You have just a few questions to get that nugget of data you’re hoping to use as a news hook or to provide insight to your client.  If your key questions are off target, you can’t turn to other content in your survey to find something usable.

Here are four tips for writing omnibus survey questions to ensure that your effort is successful: (more…)

When to Choose an Omnibus Survey over a Custom Survey

Monday, January 25th, 2010

Omnibus surveys evolved as a way to share costs by combining survey questions from multiple clients.  Because the biggest cost in conducting a survey is getting it set up and then finding a statistically representative sample of respondents to answer the questions, for some surveys it makes sense to pool resources and conduct a larger, shared survey.

But new technologies have made data collection more efficient and less expensive, so omnibus surveys are not always the best option.  For many clients, a fully customized survey might be less expensive than an omnibus survey. (more…)

What Is An Omnibus Survey?

Saturday, January 16th, 2010

An omnibus survey is a shared-cost survey that can be a good way for you to save money on surveys.  It works by combining survey questions from multiple clients and then collecting responses to all questions from the same group of respondents.  After that, each client is given the data (the survey responses) to their specific questions.  Omnibus surveys are usually fun and engaging for respondents because instead of a 15-minute survey about just one topic, the survey will cover four or five different topics, which adds interest and variety. (more…)