Posts Tagged ‘survey respondents’
Wednesday, October 5th, 2011
In response to last week’s newsletter, Is Your Research Good Enough for The New York Times?, which discussed hurdles of getting online survey research reported by some news organizations, a customer reminded us that online surveys can be difficult to sell internally as well. Too many people have been burned by junk data from online surveys.
One problem with online panels is that some respondents (a small minority) participate only to get paid in cash or redeemable credits. If these respondents are not providing thoughtful answers, the data are suspect. All panels have the problem, though some are worse than others; reputable sample providers work hard to identify and remove fraudulent respondents from their panels.
But we should not rely on panel providers alone to ensure valid data. Buyers of panel surveys should always look at the data case by case to identify and remove suspicious cases. Here are typical indicators of potentially bad data:
(more…)
Tags: data quality, Online Surveys, open-ends, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Tips | No Comments »
Wednesday, June 29th, 2011
One of the surprises of multi-mode research is that offering people a choice of how they want to complete a survey (online, by mail, by phone, etc.) does not necessarily boost response rates. An article in the most recent issue of Public Opinion Quarterly provides new evidence of this. The study showed that even in a population with full access to both mail and Internet options (and full literacy in both modes), a full mail survey achieves a higher response rate than a web-based survey. It also achieves a higher response rate than a choice of either mail or web. So much for our recent article on The Myth of Too Many Choices!
But the study went further to explore some ways of sequentially deploying multiple survey options and multiple modes of information that can substantially boost web-based survey response rates. Here are some key takeaways from the research: (more…)
Tags: Data Collection, Internet, Online Surveys, response rates, survey, survey respondents
Posted in Data Collection, Online Surveys, Survey Tips | No Comments »
Thursday, June 9th, 2011
A good chart is the best way to understand the law of diminishing returns when it comes to sample size. So for our June 2011 newsletter we built an interactive graph for choosing sample size. It’s cool, educational, and useful. Moreover, it will show you just how mind boggling the numbers behind sampling can be. It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.
(more…)
Tags: charts, Market Research, population, public opinion, Public Polls, Public Relations, Sampling, survey respondents, visualizing data
Posted in Charts and Data Visualization, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Public Relations, Sampling | No Comments »
Thursday, May 12th, 2011
We are often surprised by the number of senior researchers in the market research industry who never touch raw data. Often they don’t even have the tools, since “data processing” is outsourced to lower levels or other countries. It is surprising because we almost always engage in work where getting into the data and puzzling over anomalies or hypotheses yields much deeper insight.
Here is an example of how critical it can be to look closely at your data, and in this case, very early in the data collection process. We launched an online survey last week and got reports back from our sample supplier that incidence was just one-third of what we expected, which would have serious feasibility and cost implications.
But once we looked at their report portal, we saw that for every qualified respondent completing the survey, two qualified respondents quit before finishing. That’s an unusually high ratio of “suspends” as we call them. So what was the problem? Were we just getting lousy respondents who did not want to seriously participate in a survey? Was the survey was too difficult, tedious, boring, or confusing? One source of answers (rarely examined) is to look at the data question by question to identify where in the survey people are quitting.
The story in this data: Something is wrong with your survey
(more…)
Tags: data, Data Collection, data quality, insight, research, stories, survey, survey respondents
Posted in Charts and Data Visualization, Data Analysis & Analytics, Data Collection, Methods & Tools, Survey Design | No Comments »
Wednesday, May 4th, 2011
One reason that some people dislike surveys (okay, I may be projecting) is that too many surveys have the Me Generation attitude:
Enough about you, the customer, and what you need. What about ME? Do you like me? How much do you like me? Would you recommend me to your friend? Please let me know, because we need to track our satisfaction scores. It helps us build our metrics and our dashboards. And if you like me enough, I get a big bonus. Hurray for me!
Who are your customer satisfaction surveys really about?
(more…)
Tags: best practices, marketing, satisfaction research, survey respondents
Posted in Market Research, Methods & Tools, Survey Tips | No Comments »
Saturday, April 16th, 2011
Among the many sources of potential error that can affect surveys are respondents themselves. They sometimes misinterpret questions, respond in socially acceptable ways, or give “easy” answers in hopes that a more interesting question is just around the corner.
This is not to say they are bad or fraudulent respondents. Research shows that the vast majority of survey respondents are careful, thoughtful, and truthful in how they answer survey questions. The problem with respondent error, it turns out, is poor survey design, which may involve biased or ambiguous questions, tasks that are too complicated or boring, surveys that are too long, and so on.
Recent research shows that grid-style questions that look like this:
or this: (more…)
Tags: bias, Data Collection, data quality, Market Research, Online Surveys, Survey Design, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Wednesday, March 30th, 2011
One resource that we give far too little thought in market research are the people who offer their time and thoughts about the stuff we are researching. They are truly the lifeblood of market research.
An article in the spring 2011 issue of Public Opinion Quarterly looks at trends over the last thirty years in Americans’ view of polling and market research surveys. The news is mixed. While the vast majority feel that public opinion polling is generally a good thing, fewer feel that market research surveys serve a useful purpose, and trust in the industry is not so great. The most worrisome news (but not surprising, given the number of truly bad surveys flooding our lives nowadays) is the steady decline in people saying that participating in research is interesting and in their best interest:
Declining Satisfaction with Surveys
In addition to the chart above, key statistics outlined in the article include: (more…)
Tags: Data Collection, data quality, Market Research, public opinion, Public Polls, Survey Design, survey respondents
Posted in Data Collection, Future Trends, Market Research, Public Polls, Public Relations, Survey Design | No Comments »
Thursday, January 13th, 2011
In a recent survey we fielded among B2B decision-makers, respondents told us how much they liked participating in the study compared to other research studies they have done. They said it was “real” and interesting because it was confronting them with questions that reflect the kinds of decisions and trade-offs they make every day in their work.
The technique we used for that study is called Adaptive Conjoint. If you want to know how people in your target audience make decisions—how they weigh the pros and cons of your product or service versus others— adaptive conjoint can be a powerful technique that provides robust and insightful data at the same time it really engages the participants. (more…)
Tags: conjoint, mathematics, survey respondents, survey technology
Posted in Data Analysis & Analytics, Data Collection, Methods & Tools, Online Surveys | No Comments »
Thursday, December 30th, 2010
The latest data from the CDC’s National Health Interview Survey show that one quarter (25%) of U.S. adults do not have land-line telephones in their homes. So if you conduct a traditional random-digit-dial (RDD) phone survey, you will automatically be excluding one quarter of the population. Does it matter, given that surveys rarely interview everyone anyway? Probably. If those 25% are different from the remaining 75% in important ways, then excluding them will skew your survey findings.
(more…)
Tags: bias, Data Collection, data quality, ethics, Market Research, phone surveys, public opinion, Public Polls, research, Sampling, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Public Polls, Sampling | No Comments »
Thursday, December 23rd, 2010
Good data from a survey requires, among other things, thoughtful feedback from respondents. Low-quality data most often comes not from fraudulent or lazy respondents, but from well-meaning survey takers who are bored or irritated with surveys that are not user-friendly.
In this article we focus not on the unengaged respondents who may be offering up poor-quality data (which is usually our own fault) but instead ask what we can learn from survey takers who say they loved a particular survey. Why do they love good surveys, and what about them do they love? (more…)
Tags: best practices, Data Collection, data quality, Market Research, Online Surveys, Survey Design, survey respondents
Posted in Data Collection, Online Surveys, Survey Design, Survey Tips | No Comments »
