Posts Tagged ‘Sampling’
Wednesday, January 18th, 2012
Many of us in marketing research have been deploying web surveys for over ten years, and web surveys are, by far, the dominant mode of data collection in our industry nowadays. But our techniques and methods are an amalgam of practices adapted from other data collection modes, learned in part through trial and error, and taught to others through channels more akin to oral traditions. So it is helpful when our academic colleagues manage to document and codify the art and science of what we do. (more…)
Tags: bias, Internet, Market Research, Online Surveys, Sampling, statistics, Survey Design
Posted in Data Analysis & Analytics, Data Collection, Market Research, Online Surveys, Resources and Recommendations, Sampling, Survey Design | No Comments »
Thursday, December 29th, 2011
Even if you are not involved in political polling, it is worth paying attention to the methods and best practices of political pollsters. One reason is that few other areas of research offer a way to completely validate one’s methods. Pollsters are using sampling and survey methods to predict the behaviors of a much larger population. Then in just one day that population behaves, we get a near-perfect count of exactly how they behaved, and we know whether the methods worked.
Several industry colleagues have recently been debating the merits of calculating and reporting “margins of error” in political polling, and pointed us to some surprising data from The New York Times: (more…)
Tags: journalism, public opinion, Public Polls, Sampling, statistics, survey
Posted in Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Sampling | No Comments »
Wednesday, November 23rd, 2011
Versta Research is a strong advocate for using online panels for surveys. As telephone usage and technology have changed, phone surveys are increasingly difficult and expensive, and they are not necessarily more rigorous than other methods.
But that doesn’t mean “anything goes” when it comes to fielding market research surveys and public opinion polls through online panels. Many panels are poorly managed and overused, and some have high proportions of fraudulent respondents. While conducting good research through online panels is possible, it requires a great deal of effort and oversight from smart people who know what they are doing.
I was reminded of this recently as we worked with a newer panel provider that recruits respondents through not-for-profit organizations. (more…)
Tags: Internet, Online Panels, Online Surveys, Sampling, social media
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Sampling | No Comments »
Thursday, September 29th, 2011
Is it true that the Associated Press refuses to carry stories from online surveys? Yes, as odd as that seems nowadays. But news media face a difficult problem given how easy it is to conduct biased public opinion polling, especially now with online panels and social networks. So some news organizations like the Associated Press (AP), The New York Times, and ABC News have developed guidelines that specify for a survey or public opinion poll to be valid and reliable, it must be conducted by telephone.
(more…)
Tags: Data Collection, Internet, journalism, omnibus, Online Surveys, phone surveys, public opinion, Public Polls, Public Relations, Sampling
Posted in Data Collection, Omnibus Surveys, Online Surveys, Public Polls, Public Relations, Sampling | No Comments »
Wednesday, August 31st, 2011
Monty Hall in Let's Make A Deal
The hardest part of quantitative market research is not that it involves numbers, math, or even statistics, but that it involves complex problems in probability.
Over the past several years, psychologists have been documenting how difficult it is for us humans to solve even “simple” probability problems. One fascinating example is a puzzle known as the Monty Hall dilemma based on the 1960’s game show Let’s Make A Deal. Monty would offer his contestants three doors to choose from, one of which had a valuable prize behind it. After the contestant chose, Monty would open one of the other two doors, deliberately choosing one that had no prize behind it. Then he offered the contestant an option of staying with the original choice, or switching to the other unopened door. Which should the contestant do? (more…)
Tags: conjoint, data, Market Research, mathematics, Sampling, statistics, stories
Posted in Data Analysis & Analytics, Market Research, Presenting Research, Sampling, Turning Data into Stories | No Comments »
Friday, August 19th, 2011
The most recent government estimates of cell phone usage among U.S. households were released a few weeks back, and the pace at which landline usage is disappearing is astonishing. Here are just some of the numbers:
- Thirty percent of U.S. households do not have a landline telephone
- An additional 16% have a landline telephone, but never or rarely use it to receive calls
- The percentage of households without landlines is increasing by about five to six percentage points each year
- Half of young adults under age 30 have no landline in their homes
- Half of adult renters have no landline in their homes
- Nearly four out of ten Hispanic adults have no landline in their homes
(more…)
Tags: consumer behavior, Data Collection, Internet, Market Research, phone surveys, Public Polls, Sampling, social media, survey technology
Posted in Data Collection, Future Trends, Market Research, Public Polls, Sampling | No Comments »
Tuesday, August 9th, 2011
These days most researchers agree that if you want to do a random sample phone survey of the U.S. population, you ought to include cell phones. More than one-quarter of the population do not have landline telephones at home. Those who do have landline telephones are less likely than ever to answer them, and less likely than ever to participate in surveys.
But it is not easy to include cell phones. The sampling protocols and the post-stratification weighting become more complicated. You need to account for a higher probability of cell phone owners being in your sample, because most of them also have landlines. You can’t use automated or predictive dialing to call cell phone numbers. You can’t target geography as well, because area codes and exchanges have become mobile. And people get mad at you if they have to pay for incoming calls, so you need to offer cash.
What’s the bottom line effect on costs for a survey that includes cell phones? A recent study sponsored by the American Association for Public Opinion Research (AAPOR) documents the following: (more…)
Tags: bias, Data Collection, Market Research, phone surveys, population, public opinion, Public Polls, research, Sampling
Posted in Data Collection, Future Trends, Market Research, Public Polls, Sampling | No Comments »
Thursday, June 9th, 2011
A good chart is the best way to understand the law of diminishing returns when it comes to sample size. So for our June 2011 newsletter we built an interactive graph for choosing sample size. It’s cool, educational, and useful. Moreover, it will show you just how mind boggling the numbers behind sampling can be. It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.
(more…)
Tags: charts, Market Research, population, public opinion, Public Polls, Public Relations, Sampling, survey respondents, visualizing data
Posted in Charts and Data Visualization, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Public Relations, Sampling | No Comments »
Thursday, May 26th, 2011
A new study presented by two professors from Harvard University and the University of Massachusetts at Amherst was probably one of the liveliest and potentially disruptive presentations at least week’s annual meeting of the American Association of Public Opinion Research (AAPOR) in Phoenix.
Why? Because their research challenges the beliefs of many AAPOR-ites who disregard most online research as being theoretically indefensible since it is not based on probability sampling. The research presented was based on parallel surveys conducted last year, designed to allow careful comparison of three survey modes: (more…)
Tags: bias, census, Data Collection, Internet, journalism, media, news, Online Surveys, phone surveys, public opinion, Public Polls, Public Relations, Sampling
Posted in Data Collection, Methods & Tools, Online Surveys, Public Polls, Public Relations, Sampling | No Comments »
Thursday, December 30th, 2010
The latest data from the CDC’s National Health Interview Survey show that one quarter (25%) of U.S. adults do not have land-line telephones in their homes. So if you conduct a traditional random-digit-dial (RDD) phone survey, you will automatically be excluding one quarter of the population. Does it matter, given that surveys rarely interview everyone anyway? Probably. If those 25% are different from the remaining 75% in important ways, then excluding them will skew your survey findings.
(more…)
Tags: bias, Data Collection, data quality, ethics, Market Research, phone surveys, public opinion, Public Polls, research, Sampling, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Public Polls, Sampling | No Comments »
