Archive for the ‘Data Collection’ Category
Friday, July 30th, 2010
This past Monday I moderated a panel of thought leaders in market research to ponder the question: “How Will Social Media Change Customer Satisfaction and Loyalty Research?” The event was sponsored by the American Marketing Association, and included participants from GfK, Maritz, MARC, SAS, Market Tools, and Versta Research.
One of the fascinating insights to emerge from our discussion was that social media is not only a new channel of information and data, but that it is fundamentally different from previous channels of data. (more…)
Tags: Market Research, networks, satisfaction research, social media
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Sampling | 2 Comments »
Wednesday, June 9th, 2010
The idea that online panel surveys can replace telephone surveys ruffles feathers among my colleagues at the American Association of Public Opinion Research (AAPOR). So what would they think of using Twitter posts as a substitute for phone surveys?
The idea seems crazy, but as reported in Science, researchers at Carnegie Mellon University have found that certain kinds of twitter data can give them a good read on public sentiment. (more…)
Tags: phone surveys, public opinion, Public Polls, Sampling, social media
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls | No Comments »
Thursday, June 3rd, 2010
In the old days (decades ago), phone surveys had limited utility because many people had no phone service in their homes. When that changed, phone surveys became ubiquitous because they allowed researchers better control over the process. Data quality improved. Now increasing numbers of people have moved to cell phones only, which has been a significant challenge for the survey industry. The numbers are staggering:
The number of people without home access to landline telephones is increasing.
Almost half of adults under age 30 live in a household with only wireless telephone service.
(more…)
Tags: mobile surveys, phone surveys, Public Polls
Posted in Data Collection, Future Trends, Methods & Tools, Public Polls | No Comments »
Friday, May 28th, 2010
We saw an ad today for a downloadable survey app similar to Survey Monkey or Zoomerang that was pitched as a tool for actionable insights. Wow! Download, install, run . . . click again, and there they are, sitting on your desktop or smart phone: actionable insights. (more…)
Tags: Data Collection, insight, Market Research, survey, survey technology, tools
Posted in Data Collection, Methods & Tools, Turning Data into Stories | No Comments »
Friday, May 7th, 2010
Another article was just published in the Spring 2010 issue of Public Opinion Quarterly exploring data quality differences between online surveys and phone surveys. The findings were based on a lab experiment in which subjects completed survey questions either on a computer or over an intercom system with an interviewer. Doing the study in a laboratory isolated the mode effect of computerized self-administered data collection vs. an interview conducted by a human. (more…)
Tags: bias, Data Collection, data quality, Online Surveys, phone surveys, validity
Posted in Data Collection, Methods & Tools, Online Surveys | No Comments »
Friday, April 23rd, 2010
Asking people to fill out long, tiresome, and boring surveys is a scourge of the research, polling, and survey industry. (Another is asking them to fill out a survey every time they interact with you — see There Are Too Many Surveys.) Asking people to fill out long surveys teaches them to avoid surveys in the future, and indeed we see survey participation rates continuing to decline. But more importantly if you are the one who needs to rely on survey data, long surveys result in measurably lower data quality. (more…)
Tags: Data Collection, data quality, open-ends, survey, Survey Design, survey respondents
Posted in Data Collection, Online Surveys, Survey Design, Survey Tips | No Comments »
Friday, April 9th, 2010
If something works and it keeps on working but you don’t know exactly why it works, what would you do? Our view is that you should keep doing it. Not everyone agrees with us. The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report last month (we posted a summary of findings last week). To us, the most jarring statement in the report was this:
“There currently is no generally accepted theoretical basis from which to claim that survey results using samples from nonprobability online panels are projectable to the general population.”
Even with careful statistical weighting based on demographics, known biases, propensity to be online and partake in surveys, and so on, the report concludes that online panels should not be used to estimate population parameters. Why? Not because this method doesn’t work (in many cases it does) but because there is no statistical theory to explain why it works, in contrast to probability sampling, for which there is solid theory explaining why it works. (more…)
Tags: Online Surveys, panels, Public Polls, Sampling, statistics, survey respondents
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Sampling | No Comments »
Thursday, April 1st, 2010
Ten years ago, surveys through online panels were rare. Mostly we conducted telephone surveys. Today it is the other way around. With online survey panels being a $2 billion industry in the U.S., there is now a wealth of comparative data to measure and assess the implications of using online surveys as a substitute for phone surveys.
The American Association of Public Opinion Research (AAPOR) convened a task force in the fall of 2008 to study online survey panels, and they have just released their report. It summarizes issues related to recruitment, panel maintenance, post-survey statistical adjustments, validity, and reliability.
Here is a summary of their conclusions and recommendations (quoted verbatim): (more…)
Tags: Data Collection, Internet, Online Panels, Online Surveys, Sampling
Posted in Data Collection, Methods & Tools, Online Surveys, Sampling | No Comments »
Thursday, March 4th, 2010
Nearly every survey begins with screening questions to ensure that only the people you are trying to reach are included in the survey. For example, if you are conducting a survey of women, you need to ask about gender and kick out the men. And because every question costs money, you want to qualify respondents quickly and terminate those who do not belong.
Here is a helpful hint: Do not actually terminate respondents until after you have asked all screening questions. (more…)
Tags: Data Collection, demographics, Survey Design, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Survey Design, Survey Tips | No Comments »
Thursday, February 25th, 2010
Telephone surveys are still considered the gold standard for rigorous public opinion polling and market research. The reason is that virtually every household in the U.S. can be reached by telephone, and therefore we have careful methods of determining the probability that any individual person is included in a sample to be surveyed. Knowing this probability is at the core of statistical inference, which makes mathematical purists very happy.
Here are the steps involved in conducting a rigorous “gold standard” telephone survey of the U.S. population: (more…)
Tags: Data Collection, Internet, Market Research, population, public opinion, Public Polls, Sampling
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls, Sampling | No Comments »