Posts Tagged ‘panels’

28 Questions to Ask before Buying Online Sample

Thursday, December 27th, 2012

With all our excitement over the last few months about the accuracy of online polling during the election season—substantially outperforming “gold standard” telephone research—there was not time to share ESOMAR’s September 2012 updated guide to purchasing online sample.  The guide consists of 28 questions all purveyors of online sample should answer, publish, and make available to every buyer of its products and services.  The guide has been updated to reflect rapid changes in online sampling over the last couple of years, including use of routers, real-time sampling, and blended sample from multiple sources.

Before purchasing online sample for your next research survey, be sure that you know the answers to these 28 questions: (more…)

Tags: , , , , ,
Posted in Data Collection, Market Research, Online Surveys, Public Polls, Resources and Recommendations, Sampling | No Comments »

Why Cost Matters with Online Panel Surveys

Wednesday, May 30th, 2012

Cost matters when you choose a sample or panel provider for your survey because there are good panels and bad panels.  Bad panels provide survey respondents at cheap prices.  But they do a lousy job managing and screening their members.  Not surprisingly, a good portion of the data you get from bad panels will likely be lousy.

A recent study entitled “Dirty Little Secrets of Online Panel Research” by one of our industry colleagues described and documented lousy panel management practices of some companies.  Mystery shoppers joined and participated in online surveys offered by nearly all of the leading panel companies that most of us rely on.  Here are some of the “worst practices” they uncovered: (more…)

Tags: , , ,
Posted in Data Collection, Methods & Tools, Online Surveys, Sampling | No Comments »

Practical Statistics vs. Theoretical Statistics

Friday, April 9th, 2010

If something works and it keeps on working but you don’t know exactly why it works, what would you do?  Our view is that you should keep doing it.  Not everyone agrees with us.  The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report last month (we posted a summary of findings last week).  To us, the most jarring statement in the report was this:

“There currently is no generally accepted theoretical basis from which to claim that survey results using samples from nonprobability online panels are projectable to the general population.”

Even with careful statistical weighting based on demographics, known biases, propensity to be online and partake in surveys, and so on, the report concludes that online panels should not be used to estimate population parameters.  Why?  Not because this method doesn’t work (in many cases it does) but because there is no statistical theory to explain why it works, in contrast to probability sampling, for which there is solid theory explaining why it works. (more…)

Tags: , , , , ,
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Sampling | No Comments »