Posts Tagged ‘Public Polls’
Thursday, June 13th, 2013
Even if you don’t care about political polling, or the fact that Gallup consistently overestimates support for Republican candidates, it is worth paying attention to how Gallup is trying to fix its problems with surveys and polling.
They are not happy with how poorly their polls have fared (who would be?), and they have teams of smart people trying to figure out what is wrong. Given their high profile, they are making the process and findings of their investigations public, and we have much to learn from that.
Last week they released their findings from an extensive review, which involved outside experts as well as internal ones. It is fascinating to read, because they identify 13 suspects in their survey process that all companies who do survey research should always be thinking about: (more…)
Tags: phone surveys, Public Polls, reliability, research design, validity
Posted in Data Analysis & Analytics, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Sampling | No Comments »
Friday, April 26th, 2013
It is hard to find an appropriate use for Google Surveys, because, as we outlined in a review article last fall, its capabilities are limited. But last week we needed a quick incidence test of how many U.S. adults own a certain type of investment product. Google Surveys seemed perfect. It was not fast, by the way. It took five days to collect data from 200 respondents. Google says this is because we asked a screening question before asking about product ownership. Even so, this survey took longer than a standard omnibus.
But what struck me most about my trial run with Google Surveys was the Creepy Factor. It made me realize in a most uncomfortable way that Google tracks everything I do. I knew this already, and I follow ongoing discussions about online privacy. I have a personal g-mail account, a G+ page, and I use Google as the starting point for almost everything I do on the Internet. I know that they track everything I do. But it was never so creepy and apparent until I fielded a Google survey. How was it creepy? (more…)
Tags: ethics, Market Research, omnibus, Online Surveys, privacy, public opinion, Public Polls, survey technology
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | Comments Off
Thursday, December 27th, 2012
With all our excitement over the last few months about the accuracy of online polling during the election season—substantially outperforming “gold standard” telephone research—there was not time to share ESOMAR’s September 2012 updated guide to purchasing online sample. The guide consists of 28 questions all purveyors of online sample should answer, publish, and make available to every buyer of its products and services. The guide has been updated to reflect rapid changes in online sampling over the last couple of years, including use of routers, real-time sampling, and blended sample from multiple sources.
Before purchasing online sample for your next research survey, be sure that you know the answers to these 28 questions: (more…)
Tags: online, Online Panels, Online Surveys, panels, Public Polls, Sampling
Posted in Data Collection, Market Research, Online Surveys, Public Polls, Resources and Recommendations, Sampling | Comments Off
Thursday, November 29th, 2012
If there is one super important lesson to be learned from this year’s round of election polling, it is that online surveys work. Google Consumer Surveys, which use non-probability online samples, predicted the election far better than Gallup did. And online surveys, overall, outperformed telephone surveys.
The New York Times’ Nate Silver compiled polling results from 23 organizations that conducted at least five surveys in the final three weeks of the campaign. He calculated how far their projections were from the actual outcome of the presidential race. Google (a fully automated, online solution) came in second place, predicting the actual outcome within 1.6 percentage points. Gallup (using “gold standard” telephone methods) came in last, predicting the outcome within 7.2 percentage points.
(more…)
Tags: Online Surveys, phone surveys, Public Polls
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Public Relations | Comments Off
Wednesday, November 14th, 2012
I have never been a media poll watcher or politics junkie who tracks every new poll saying which candidate is favored to win. It is hard to see the point of spending so much time and money predicting the outcome of an event that will be known with certainty within a matter of days, weeks, or months. But elections are an amazing way to see survey research methods in action, and there are few opportunities to have those methods validated so quickly, accurately, and unforgivingly than political polling.
What did we learn from the polls this election season? (more…)
Tags: methods, models, Online Surveys, phone surveys, Public Polls, research, statistics, survey
Posted in Future Trends, Market Research, Public Polls | Comments Off
Wednesday, October 31st, 2012
By elephants, we mean Republicans. Or maybe you have too many Democrats. Maybe it keeps going back and forth, which is the problem that Gallup sometimes has. In the spirit of learning all we can from election season polling, this week we focus on whom to include (or exclude) in your research, analysis, and market projections.
The issue is showcased right now as political polls attempt to measure voter preference and predict the election outcome. Is voter preference really as volatile and open to persuasion as the polls sometimes suggest? Probably not. A 2004 research article in Public Opinion Quarterly carefully documented that much of the volatility in Gallup’s polls results from how they screen respondents and weight their data. (more…)
Tags: bias, concept testing, data analysis, Market Research, Public Polls, Sampling, statistics
Posted in Data Analysis & Analytics, Market Research, Methods & Tools, New Products and Innovation, Sampling, Survey Design | Comments Off
Thursday, October 18th, 2012
Six months ago Google launched an intriguing new way to conduct cost-effective surveys that offers an alternative to omnibus surveys. One or two easy survey questions are presented to online users as they seek access to high quality media sites. They gain free access in exchange for answering the survey questions. Google tracks how many people are answering each question and manages the process to ensure that the sample of respondents answering each question closely matches the overall U.S. population (based on Census data for those who have Internet access).
Here are some of the more interesting aspects of their approach: (more…)
Tags: omnibus, Online Panels, Online Surveys, Public Polls
Posted in Data Collection, Future Trends, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | Comments Off
Friday, September 21st, 2012
During a presidential election year there is no escaping the flurry of public opinion polling and the intense scrutiny that surveys get from the media. But love it or hate it, there are excellent reasons to pay close attention to this year’s political polling.
(more…)
Tags: best practices, journalism, media, methods, news, phone surveys, public opinion, Public Polls, Public Relations, Sampling, survey, Survey Design
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls, Public Relations, Sampling, Survey Design | Comments Off
Thursday, July 19th, 2012
Survey response rates are now staggeringly low—in the single digits. A typical response rate for a relatively high-budget, carefully executed phone survey is merely 9%, down from 36% just fifteen years ago. Here are the numbers from research conducted earlier this year by the Pew Center:
Survey Response Rates Continue to Decline
If you want to throw money at a survey and try really hard to boost your response rate (the high-effort survey shown in the chart above), you can likely get up to 20% to 25%. But you will need to:
(more…)
Tags: phone surveys, public opinion, Public Polls, response rates, Sampling, survey
Posted in Data Collection, Public Polls, Sampling | Comments Off
Friday, May 18th, 2012
I recently saw a press release about a study showing that only 19.5% of news release headlines are optimized for SEO. It brought to mind all kinds of issues about how best to report numbers in press releases. In particular it highlighted the important issue of whether specific numbers are meaningful and whether they communicate a misleading sense of precision.
For example, when a survey reports a margin of error to any decimal place, it suggests a level of precision that is misleading. Do a quick search, and you’ll find press releases reporting margins of sampling error such as +/- 4.8%, +/- 10.5%, or +/- 1.85%. These numbers are based on sample size formulas that assume perfect random sampling and one hundred percent response rates, which are almost never achieved. (more…)
Tags: public opinion, Public Polls, Public Relations, survey
Posted in Presenting Research, Public Polls, Public Relations, Turning Data into Stories | Comments Off