Another article was just published in the Spring 2010 issue of Public Opinion Quarterly exploring data quality differences between online surveys and phone surveys. The findings were based on a lab experiment in which subjects completed survey questions either on a computer or over an intercom system with an interviewer. Doing the study in a laboratory isolated the mode effect of computerized self-administered data collection vs. an interview conducted by a human. (more…)
Posts Tagged ‘data quality’
More Research on Phone vs. Online Surveys
Friday, May 7th, 2010How Long Should a Survey Be?
Friday, April 23rd, 2010Asking people to fill out long, tiresome, and boring surveys is a scourge of the research, polling, and survey industry. (Another is asking them to fill out a survey every time they interact with you — see There Are Too Many Surveys.) Asking people to fill out long surveys teaches them to avoid surveys in the future, and indeed we see survey participation rates continuing to decline. But more importantly if you are the one who needs to rely on survey data, long surveys result in measurably lower data quality. (more…)
Recent Findings on Phone vs. Online Surveys
Friday, February 5th, 2010A research article the Winter 2009 issue of Public Opinion Quarterly provided some useful comparisons of data quality between phone surveys and various types of online surveys. The findings are based on an experiment that fielded identical questionnaires via three survey modes, and, not surprisingly, there are strengths and weaknesses to each type of survey.
Phone vs. Online Surveys: Strengths & Weaknesses
As reflected in the table above, the article is rather technical. But there are two key summary points worth learning from the study: (more…)
People Don’t Lie on Surveys
Thursday, December 24th, 2009For some reason I’m always a little surprised that people tell the truth on surveys. I like to think of my “healthy skepticism” as a professional asset, because it forces us to check and double check, corroborate and triangulate. Before we commit to the findings of a research effort, we need to feel 100% sure we’re right and that our findings are based on solid data. (more…)
Forensic Polling Analysis
Thursday, December 17th, 2009Bad pollsters give the market research industry a bad name, so it is encouraging when smart people figure out clever ways of ratting them out. What is a bad pollster? One who makes up data to support an agenda, or who asks biased questions to get preferred answers. The only good reason for doing research or public opinion polling is to learn or share something new. All else is suspect.
Two researchers recently came up with methods of testing whether polling data is legitimate in a case where a research firm is accused of falsifying publicly released data. (more…)
Better Data through Better Survey Design
Monday, September 14th, 2009Market researchers complain a lot when they get data back from surveys and see that people have been “speeding” through their surveys or that people are not giving thoughtful responses.
But the problem is rarely “bad respondents” – instead the problem is lazy researchers. When people discover that the survey they just agreed to take is boring, tedious, repetitive, or too long, they either quit altogether or they stop providing good answers.