The technology to field surveys via mobile devices has been around for a while, but has not yet gained much traction (and for good reasons). But with smartphones now proliferating at a remarkable pace, we may be in for a change. Deloitte released their 2011 IT and technology predictions last week, arguing that smartphones are likely to account for almost half of computer spending during the year. (more…)
Posts Tagged ‘Online Surveys’
Tips for Surveys on Smartphones
Thursday, January 27th, 2011Getting Respondents to Love Your Survey
Thursday, December 23rd, 2010Good data from a survey requires, among other things, thoughtful feedback from respondents. Low-quality data most often comes not from fraudulent or lazy respondents, but from well-meaning survey takers who are bored or irritated with surveys that are not user-friendly.
In this article we focus not on the unengaged respondents who may be offering up poor-quality data (which is usually our own fault) but instead ask what we can learn from survey takers who say they loved a particular survey. Why do they love good surveys, and what about them do they love? (more…)
Why You Need a Partisan Pollster
Friday, September 17th, 2010
In an op-ed column two days ago, Stuart Rothenberg, a prominent political (and non-partisan) commentator argued that partisan pollsters (those who work directly for either Democratic or Republican candidates) do a better job than presumably objective third party pollsters. Why? Because they have to get it right. Their campaign strategies depend on it. Quoting Mr. Rothenberg: (more…)
More Research on Phone vs. Online Surveys
Friday, May 7th, 2010Another article was just published in the Spring 2010 issue of Public Opinion Quarterly exploring data quality differences between online surveys and phone surveys. The findings were based on a lab experiment in which subjects completed survey questions either on a computer or over an intercom system with an interviewer. Doing the study in a laboratory isolated the mode effect of computerized self-administered data collection vs. an interview conducted by a human. (more…)
Practical Statistics vs. Theoretical Statistics
Friday, April 9th, 2010If something works and it keeps on working but you don’t know exactly why it works, what would you do? Our view is that you should keep doing it. Not everyone agrees with us. The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report last month (we posted a summary of findings last week). To us, the most jarring statement in the report was this:
“There currently is no generally accepted theoretical basis from which to claim that survey results using samples from nonprobability online panels are projectable to the general population.”
Even with careful statistical weighting based on demographics, known biases, propensity to be online and partake in surveys, and so on, the report concludes that online panels should not be used to estimate population parameters. Why? Not because this method doesn’t work (in many cases it does) but because there is no statistical theory to explain why it works, in contrast to probability sampling, for which there is solid theory explaining why it works. (more…)
How Good Are Online Survey Panels?
Thursday, April 1st, 2010Ten years ago, surveys through online panels were rare. Mostly we conducted telephone surveys. Today it is the other way around. With online survey panels being a $2 billion industry in the U.S., there is now a wealth of comparative data to measure and assess the implications of using online surveys as a substitute for phone surveys.
The American Association of Public Opinion Research (AAPOR) convened a task force in the fall of 2008 to study online survey panels, and they have just released their report. It summarizes issues related to recruitment, panel maintenance, post-survey statistical adjustments, validity, and reliability.
Here is a summary of their conclusions and recommendations (quoted verbatim): (more…)
Recent Findings on Phone vs. Online Surveys
Friday, February 5th, 2010A research article the Winter 2009 issue of Public Opinion Quarterly provided some useful comparisons of data quality between phone surveys and various types of online surveys. The findings are based on an experiment that fielded identical questionnaires via three survey modes, and, not surprisingly, there are strengths and weaknesses to each type of survey.
Phone vs. Online Surveys: Strengths & Weaknesses
As reflected in the table above, the article is rather technical. But there are two key summary points worth learning from the study: (more…)
What Is An Omnibus Survey?
Saturday, January 16th, 2010An omnibus survey is a shared-cost survey that can be a good way for you to save money on surveys. It works by combining survey questions from multiple clients and then collecting responses to all questions from the same group of respondents. After that, each client is given the data (the survey responses) to their specific questions. Omnibus surveys are usually fun and engaging for respondents because instead of a 15-minute survey about just one topic, the survey will cover four or five different topics, which adds interest and variety. (more…)
People Don’t Lie on Surveys
Thursday, December 24th, 2009For some reason I’m always a little surprised that people tell the truth on surveys. I like to think of my “healthy skepticism” as a professional asset, because it forces us to check and double check, corroborate and triangulate. Before we commit to the findings of a research effort, we need to feel 100% sure we’re right and that our findings are based on solid data. (more…)