Posts Tagged ‘data quality’
Tuesday, January 24th, 2012
Election years are a perfect time to learn about (and brush up on) the fundamentals of survey research. Not only are the airwaves inundated with public opinion polling, but methodological experts are called upon to talk about developments and current best practices as new technologies and methods become central to measuring consumer and public opinion and behavior.
This week the Poynter Institute is offering a webinar of particular interest for PR professionals, whether or not you care about political polls. We also recommend it for any marketing professional because this type of polling is an exemplar of what all marketing research tries to achieve in measuring what people think and what they are likely to do. (more…)
Tags: data quality, journalism, Market Research, media, news, omnibus, public opinion, Public Polls, Public Relations
Posted in Public Polls, Public Relations, Resources and Recommendations | No Comments »
Wednesday, October 5th, 2011
In response to last week’s newsletter, Is Your Research Good Enough for The New York Times?, which discussed hurdles of getting online survey research reported by some news organizations, a customer reminded us that online surveys can be difficult to sell internally as well. Too many people have been burned by junk data from online surveys.
One problem with online panels is that some respondents (a small minority) participate only to get paid in cash or redeemable credits. If these respondents are not providing thoughtful answers, the data are suspect. All panels have the problem, though some are worse than others; reputable sample providers work hard to identify and remove fraudulent respondents from their panels.
But we should not rely on panel providers alone to ensure valid data. Buyers of panel surveys should always look at the data case by case to identify and remove suspicious cases. Here are typical indicators of potentially bad data:
(more…)
Tags: data quality, Online Surveys, open-ends, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Tips | No Comments »
Thursday, May 12th, 2011
We are often surprised by the number of senior researchers in the market research industry who never touch raw data. Often they don’t even have the tools, since “data processing” is outsourced to lower levels or other countries. It is surprising because we almost always engage in work where getting into the data and puzzling over anomalies or hypotheses yields much deeper insight.
Here is an example of how critical it can be to look closely at your data, and in this case, very early in the data collection process. We launched an online survey last week and got reports back from our sample supplier that incidence was just one-third of what we expected, which would have serious feasibility and cost implications.
But once we looked at their report portal, we saw that for every qualified respondent completing the survey, two qualified respondents quit before finishing. That’s an unusually high ratio of “suspends” as we call them. So what was the problem? Were we just getting lousy respondents who did not want to seriously participate in a survey? Was the survey was too difficult, tedious, boring, or confusing? One source of answers (rarely examined) is to look at the data question by question to identify where in the survey people are quitting.
The story in this data: Something is wrong with your survey
(more…)
Tags: data, Data Collection, data quality, insight, research, stories, survey, survey respondents
Posted in Charts and Data Visualization, Data Analysis & Analytics, Data Collection, Methods & Tools, Survey Design | No Comments »
Saturday, April 16th, 2011
Among the many sources of potential error that can affect surveys are respondents themselves. They sometimes misinterpret questions, respond in socially acceptable ways, or give “easy” answers in hopes that a more interesting question is just around the corner.
This is not to say they are bad or fraudulent respondents. Research shows that the vast majority of survey respondents are careful, thoughtful, and truthful in how they answer survey questions. The problem with respondent error, it turns out, is poor survey design, which may involve biased or ambiguous questions, tasks that are too complicated or boring, surveys that are too long, and so on.
Recent research shows that grid-style questions that look like this:
or this: (more…)
Tags: bias, Data Collection, data quality, Market Research, Online Surveys, Survey Design, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Wednesday, March 30th, 2011
One resource that we give far too little thought in market research are the people who offer their time and thoughts about the stuff we are researching. They are truly the lifeblood of market research.
An article in the spring 2011 issue of Public Opinion Quarterly looks at trends over the last thirty years in Americans’ view of polling and market research surveys. The news is mixed. While the vast majority feel that public opinion polling is generally a good thing, fewer feel that market research surveys serve a useful purpose, and trust in the industry is not so great. The most worrisome news (but not surprising, given the number of truly bad surveys flooding our lives nowadays) is the steady decline in people saying that participating in research is interesting and in their best interest:
Declining Satisfaction with Surveys
In addition to the chart above, key statistics outlined in the article include: (more…)
Tags: Data Collection, data quality, Market Research, public opinion, Public Polls, Survey Design, survey respondents
Posted in Data Collection, Future Trends, Market Research, Public Polls, Public Relations, Survey Design | No Comments »
Thursday, December 30th, 2010
The latest data from the CDC’s National Health Interview Survey show that one quarter (25%) of U.S. adults do not have land-line telephones in their homes. So if you conduct a traditional random-digit-dial (RDD) phone survey, you will automatically be excluding one quarter of the population. Does it matter, given that surveys rarely interview everyone anyway? Probably. If those 25% are different from the remaining 75% in important ways, then excluding them will skew your survey findings.
(more…)
Tags: bias, Data Collection, data quality, ethics, Market Research, phone surveys, public opinion, Public Polls, research, Sampling, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Public Polls, Sampling | No Comments »
Thursday, December 23rd, 2010
Good data from a survey requires, among other things, thoughtful feedback from respondents. Low-quality data most often comes not from fraudulent or lazy respondents, but from well-meaning survey takers who are bored or irritated with surveys that are not user-friendly.
In this article we focus not on the unengaged respondents who may be offering up poor-quality data (which is usually our own fault) but instead ask what we can learn from survey takers who say they loved a particular survey. Why do they love good surveys, and what about them do they love? (more…)
Tags: best practices, Data Collection, data quality, Market Research, Online Surveys, Survey Design, survey respondents
Posted in Data Collection, Online Surveys, Survey Design, Survey Tips | No Comments »
Friday, September 17th, 2010
In an op-ed column two days ago, Stuart Rothenberg, a prominent political (and non-partisan) commentator argued that partisan pollsters (those who work directly for either Democratic or Republican candidates) do a better job than presumably objective third party pollsters. Why? Because they have to get it right. Their campaign strategies depend on it. Quoting Mr. Rothenberg: (more…)
Tags: data quality, Market Research, media, Online Surveys, Public Polls
Posted in Data Collection, Market Research, Online Surveys, Public Polls | No Comments »
Friday, September 10th, 2010
With the sad proliferation of silly surveys, non-scientific interest polls, and downright fraudulent polls, the research industry is stepping up with a number of key initiatives to combat the trend. Versta Research is part of that effort, and in August announced that we are part of the Transparency Initiative being developed by the American Association for Public Opinion Research (AAPOR).
As of this writing, Versta is one of 67 prominent survey and polling organizations currently supporting the initiative. The initiative is designed to create protocols and recognition for regular disclosure of methods when survey organizations conduct public polls. (more…)
Tags: data quality, ethics, omnibus, Public Polls, research, trust
Posted in Future Trends, Omnibus Surveys, Public Polls, Public Relations | No Comments »
Friday, May 7th, 2010
Another article was just published in the Spring 2010 issue of Public Opinion Quarterly exploring data quality differences between online surveys and phone surveys. The findings were based on a lab experiment in which subjects completed survey questions either on a computer or over an intercom system with an interviewer. Doing the study in a laboratory isolated the mode effect of computerized self-administered data collection vs. an interview conducted by a human. (more…)
Tags: bias, Data Collection, data quality, Online Surveys, phone surveys, validity
Posted in Data Collection, Methods & Tools, Online Surveys | No Comments »
