In the old days (decades ago), phone surveys had limited utility because many people had no phone service in their homes. When that changed, phone surveys became ubiquitous because they allowed researchers better control over the process. Data quality improved. Now increasing numbers of people have moved to cell phones only, which has been a significant challenge for the survey industry. The numbers are staggering:
The number of people without home access to landline telephones is increasing.
Almost half of adults under age 30 live in a household with only wireless telephone service.
Choosing an omnibus survey can be a simple approach to getting survey data, because it is usually inexpensive and fast, and involves asking just a few questions. But there is sometimes a downside to simplicity: You have just a few questions to get that nugget of data you’re hoping to use as a news hook or to provide insight to your client. If your key questions are off target, you can’t turn to other content in your survey to find something usable.
Here are four tips for writing omnibus survey questions to ensure that your effort is successful: (more…)
If something works and it keeps on working but you don’t know exactly why it works, what would you do? Our view is that you should keep doing it. Not everyone agrees with us. The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report last month (we posted a summary of findings last week). To us, the most jarring statement in the report was this:
“There currently is no generally accepted theoretical basis from which to claim that survey results using samples from nonprobability online panels are projectable to the general population.”
Even with careful statistical weighting based on demographics, known biases, propensity to be online and partake in surveys, and so on, the report concludes that online panels should not be used to estimate population parameters. Why? Not because this method doesn’t work (in many cases it does) but because there is no statistical theory to explain why it works, in contrast to probability sampling, for which there is solid theory explaining why it works. (more…)
Telephone surveys are still considered the gold standard for rigorous public opinion polling and market research. The reason is that virtually every household in the U.S. can be reached by telephone, and therefore we have careful methods of determining the probability that any individual person is included in a sample to be surveyed. Knowing this probability is at the core of statistical inference, which makes mathematical purists very happy.
Here are the steps involved in conducting a rigorous “gold standard” telephone survey of the U.S. population: (more…)
Bad pollsters give the market research industry a bad name, so it is encouraging when smart people figure out clever ways of ratting them out. What is a bad pollster? One who makes up data to support an agenda, or who asks biased questions to get preferred answers. The only good reason for doing research or public opinion polling is to learn or share something new. All else is suspect.
Two researchers recently came up with methods of testing whether polling data is legitimate in a case where a research firm is accused of falsifying publicly released data. (more…)
Versta Research fields omnibus surveys, and we often recommend such surveys for our clients. But omnibus surveys do no not always save money or offer the insights that are needed, and so we often recommend inexpensive alternatives as well. (more…)
Market researchers complain a lot when they get data back from surveys and see that people have been “speeding” through their surveys or that people are not giving thoughtful responses.
But the problem is rarely “bad respondents” – instead the problem is lazy researchers. When people discover that the survey they just agreed to take is boring, tedious, repetitive, or too long, they either quit altogether or they stop providing good answers.