Posts Tagged ‘Public Polls’
Thursday, October 14th, 2010
Should you state a survey’s margin of error in your press release when pitching a story to the media? In our view, the answer is no. Why not? Because margins of error refer to sampling error only, not about the overall accuracy or error of the survey itself.
But how many readers of your news story understand what sampling error is? For that matter, how many researchers understand what sampling error is? It was only after four college semesters of advanced mathematical statistics that I finally “got it.” More importantly, how many readers understand that there are many other potential sources of survey error? Most undoubtedly assume that all error is somehow accounted for when you confidently proclaim the margin of error being ±4%. (Or, more absurdly, ±3.6% or even ±3.57% — examples of phony accuracy like this are all too easy to find.)
In short, margins of error are misleading because they deal with only one source of error. They convey a false sense of accuracy. And they should not be used. This idea is not always popular among colleagues and clients. But take comfort: One of the giants of public polling, Harris Interactive, refuses to report margins of error in its work, for precisely the reasons outlined above. Here we quote their methodological statement that accompanies every press release and report they issue: (more…)
Tags: ethics, media, Public Polls, Public Relations, Sampling, statistics
Posted in Presenting Research, Public Polls, Public Relations | No Comments »
Friday, September 17th, 2010
In an op-ed column two days ago, Stuart Rothenberg, a prominent political (and non-partisan) commentator argued that partisan pollsters (those who work directly for either Democratic or Republican candidates) do a better job than presumably objective third party pollsters. Why? Because they have to get it right. Their campaign strategies depend on it. Quoting Mr. Rothenberg: (more…)
Tags: data quality, Market Research, media, Online Surveys, Public Polls
Posted in Data Collection, Market Research, Online Surveys, Public Polls | No Comments »
Friday, September 10th, 2010
With the sad proliferation of silly surveys, non-scientific interest polls, and downright fraudulent polls, the research industry is stepping up with a number of key initiatives to combat the trend. Versta Research is part of that effort, and in August announced that we are part of the Transparency Initiative being developed by the American Association for Public Opinion Research (AAPOR).
As of this writing, Versta is one of 67 prominent survey and polling organizations currently supporting the initiative. The initiative is designed to create protocols and recognition for regular disclosure of methods when survey organizations conduct public polls. (more…)
Tags: data quality, ethics, omnibus, Public Polls, research, trust
Posted in Future Trends, Omnibus Surveys, Public Polls, Public Relations | No Comments »
Wednesday, June 9th, 2010
The idea that online panel surveys can replace telephone surveys ruffles feathers among my colleagues at the American Association of Public Opinion Research (AAPOR). So what would they think of using Twitter posts as a substitute for phone surveys?
The idea seems crazy, but as reported in Science, researchers at Carnegie Mellon University have found that certain kinds of twitter data can give them a good read on public sentiment. (more…)
Tags: phone surveys, public opinion, Public Polls, Sampling, social media
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls | No Comments »
Thursday, June 3rd, 2010
In the old days (decades ago), phone surveys had limited utility because many people had no phone service in their homes. When that changed, phone surveys became ubiquitous because they allowed researchers better control over the process. Data quality improved. Now increasing numbers of people have moved to cell phones only, which has been a significant challenge for the survey industry. The numbers are staggering:
The number of people without home access to landline telephones is increasing.
Almost half of adults under age 30 live in a household with only wireless telephone service.
(more…)
Tags: mobile surveys, phone surveys, Public Polls
Posted in Data Collection, Future Trends, Methods & Tools, Public Polls | No Comments »
Friday, April 9th, 2010
If something works and it keeps on working but you don’t know exactly why it works, what would you do? Our view is that you should keep doing it. Not everyone agrees with us. The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report last month (we posted a summary of findings last week). To us, the most jarring statement in the report was this:
“There currently is no generally accepted theoretical basis from which to claim that survey results using samples from nonprobability online panels are projectable to the general population.”
Even with careful statistical weighting based on demographics, known biases, propensity to be online and partake in surveys, and so on, the report concludes that online panels should not be used to estimate population parameters. Why? Not because this method doesn’t work (in many cases it does) but because there is no statistical theory to explain why it works, in contrast to probability sampling, for which there is solid theory explaining why it works. (more…)
Tags: Online Surveys, panels, Public Polls, Sampling, statistics, survey respondents
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Sampling | No Comments »
Thursday, February 25th, 2010
Telephone surveys are still considered the gold standard for rigorous public opinion polling and market research. The reason is that virtually every household in the U.S. can be reached by telephone, and therefore we have careful methods of determining the probability that any individual person is included in a sample to be surveyed. Knowing this probability is at the core of statistical inference, which makes mathematical purists very happy.
Here are the steps involved in conducting a rigorous “gold standard” telephone survey of the U.S. population: (more…)
Tags: Data Collection, Internet, Market Research, population, public opinion, Public Polls, Sampling
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls, Sampling | No Comments »
Thursday, December 31st, 2009
As an industry driven by data and information, market research and public opinion polling has seen dramatic changes in the last ten years and will no doubt change quickly and in big ways during the next ten.
Looking back, here are what we consider to be the five biggest changes that shaped current challenges faced by market research and opinion polling: (more…)
Tags: communication, data, Data Collection, data quality, Market Research, mathematics, media, online, Online Surveys, public opinion, Public Polls, Public Relations, research, Sampling, statistics, stories, survey, survey respondents, survey technology
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Presenting Research, Public Polls, Public Relations, Sampling, Turning Data into Stories, Uncategorized | No Comments »
Thursday, December 17th, 2009
Bad pollsters give the market research industry a bad name, so it is encouraging when smart people figure out clever ways of ratting them out. What is a bad pollster? One who makes up data to support an agenda, or who asks biased questions to get preferred answers. The only good reason for doing research or public opinion polling is to learn or share something new. All else is suspect.
Two researchers recently came up with methods of testing whether polling data is legitimate in a case where a research firm is accused of falsifying publicly released data. (more…)
Tags: bias, data quality, ethics, mathematics, Public Polls, statistics, trust
Posted in Future Trends, Public Polls, Public Relations | No Comments »
Friday, November 20th, 2009
Five of Chicago’s PR leaders gathered at a PRSA forum this week to discuss current trends and the future of public relations. The discussion was striking in how fully it echoed the trends and challenges facing the polling and research industry, and what we need to do to keep our eye on the ball. Here are a few take-away ideas from that forum that apply to both PR professionals and their research partners: (more…)
Tags: communication, data, insight, journalism, media, Public Polls, Public Relations, research, stories, survey, value
Posted in Future Trends, Presenting Research, Public Relations, Turning Data into Stories, Uncategorized | 1 Comment »