Posts Tagged ‘survey respondents’
Friday, May 17th, 2013
If you have ever been called to participate in a phone survey, you probably know the routine where you hear a question and then jump in with an answer. A good interviewer will remind you that she needs to read the entire question and all the answer options just to be sure that you offered the best response option.
We can’t do this with self-administered online surveys, but there is a way to minimize error associated with respondents jumping to conclusions: Put all clarifying instructions before asking the question, not after. A recent study published in Public Opinion Quarterly documented that if you put instructions before the question, respondents spend more time answering because they are reading the instructions and answering more carefully. Not surprisingly, their answers are more accurate.
Here is an example of a typical question that has clarifying instructions after the question: (more…)
Tags: Online Surveys, research, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Design, Survey Tips | Comments Off
Wednesday, September 26th, 2012
Figure 1. The U.S. Census Bureau's current questions about race and ethnicity. Click for full-size image.
Fifteen years ago the U.S. Census Bureau removed “Hispanic” from its question about race and instead created a stand-alone question to assess Hispanic ethnicity. It’s the reason we often advise our clients to ask about race and ethnicity separately, using questions similar to the Census Bureau’s, which are shown in Figure 1.
Well, chances are good that this will be soon be revised back to include Hispanic in a single question about race. The Census Bureau just concluded a large-scale study to assess the reliability and validity of its race and ethnicity measures, along with several potential alternatives. It involved questionnaires and re-interviews with a sample of nearly half a million households, plus 67 focus groups with nearly 800 people. (more…)
Tags: census, Data Collection, survey, Survey Design, survey respondents
Posted in Data Collection, Future Trends, Market Research, Survey Design | Comments Off
Thursday, August 2nd, 2012
Is this YOUR distracted survey respondent?
A couple weeks ago we presented new data showing that response rates continue to decline. You can now expect that a typical, rigorously executed phone survey will yield a response rate in the single digits.
Scientific evidence over the last decade has shown that high response rates do not necessarily yield more accurate surveys. In fact, it turns out that high response rates can actually hurt the accuracy of surveys.
(more…)
Tags: Online Surveys, phone surveys, response rates, Sampling, survey, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Public Polls | Comments Off
Wednesday, April 25th, 2012
Most people who take surveys want to share their opinions, which is important for researchers hoping to get a few nuggets of data from willing respondents. The trouble is, if a survey it not written carefully, a respondent’s urgent desire to share their feelings may bias their answers to other questions.
Two marketing professors at Northwestern University’s Kellogg School of Management recently published a paper in the AMA’s Journal of Marketing Research identifying and documenting this unique kind of bias. They call it “response substitution.” (more…)
Tags: bias, customer satisfaction, satisfaction research, survey, survey respondents
Posted in Market Research, Methods & Tools, Survey Design, Survey Tips | Comments Off
Wednesday, April 11th, 2012
How big of a sample size do you really need? A recent article in the New York Times cited the following statistics:
- A small Voice of the Customer (VoC) research company called Mindshare Technologies collects satisfaction data from 175,000 respondents every day. That’s 60 million in a year.
- ForeSee, a small customer experience analytics firm fielded 15 million surveys in 2011.
These numbers are believable. I get a pop-up survey from ForeSee at least two or three times a week.
And it is absurd. Granted, these companies (and hundreds of other similar firms) are collecting surveys for multiple clients. But almost certainly, nobody needs to collect that much survey data from that many survey respondents. Why not? (more…)
Tags: bias, data, Data Collection, data mining, insight, Market Research, response rates, Sampling, satisfaction research, survey respondents, survey technology
Posted in Data Analysis & Analytics, Data Collection, Market Research, Methods & Tools, Sampling | Comments Off
Wednesday, February 15th, 2012
Data quality will suffer if respondents are bored with long surveys
It is hard to resist the temptation of asking “just one more question” when you’ve got an engaged respondent answering your survey questions online or on the phone. But it is crucial to do so because plenty of research shows that longer surveys result in bad data. Survey respondents may be willing to answer just one more question, but at some point the quality of information you get from them declines. Survey respondents become inattentive and offer lazy answers, or worse, they offer quick random answers just to get the survey over with.
At Versta Research we have a few rules of thumb for survey length based on (1) academic and industry research measuring data quality, (2) conversations with colleagues and suppliers throughout the industry, and (3) our ongoing experience of what works and what does not work. The maximum survey lengths we typically recommend are: (more…)
Tags: data, Data Collection, data quality, Market Research, Online Surveys, phone surveys, social media, survey, Survey Design, survey respondents, survey technology
Posted in Data Collection, Survey Design, Survey Tips | Comments Off
Wednesday, October 5th, 2011
In response to last week’s newsletter, Is Your Research Good Enough for The New York Times?, which discussed hurdles of getting online survey research reported by some news organizations, a customer reminded us that online surveys can be difficult to sell internally as well. Too many people have been burned by junk data from online surveys.
One problem with online panels is that some respondents (a small minority) participate only to get paid in cash or redeemable credits. If these respondents are not providing thoughtful answers, the data are suspect. All panels have the problem, though some are worse than others; reputable sample providers work hard to identify and remove fraudulent respondents from their panels.
But we should not rely on panel providers alone to ensure valid data. Buyers of panel surveys should always look at the data case by case to identify and remove suspicious cases. Here are typical indicators of potentially bad data:
(more…)
Tags: data quality, Online Surveys, open-ends, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Tips | Comments Off
Wednesday, June 29th, 2011
One of the surprises of multi-mode research is that offering people a choice of how they want to complete a survey (online, by mail, by phone, etc.) does not necessarily boost response rates. An article in the most recent issue of Public Opinion Quarterly provides new evidence of this. The study showed that even in a population with full access to both mail and Internet options (and full literacy in both modes), a full mail survey achieves a higher response rate than a web-based survey. It also achieves a higher response rate than a choice of either mail or web. So much for our recent article on The Myth of Too Many Choices!
But the study went further to explore some ways of sequentially deploying multiple survey options and multiple modes of information that can substantially boost web-based survey response rates. Here are some key takeaways from the research: (more…)
Tags: Data Collection, Internet, Online Surveys, response rates, survey, survey respondents
Posted in Data Collection, Online Surveys, Survey Tips | Comments Off
Thursday, June 9th, 2011
A good chart is the best way to understand the law of diminishing returns when it comes to sample size. So for our June 2011 newsletter we built an interactive graph for choosing sample size. It’s cool, educational, and useful. Moreover, it will show you just how mind boggling the numbers behind sampling can be. It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.
(more…)
Tags: charts, Market Research, population, public opinion, Public Polls, Public Relations, Sampling, survey respondents, visualizing data
Posted in Charts and Data Visualization, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Public Relations, Sampling | Comments Off
Thursday, May 12th, 2011
We are often surprised by the number of senior researchers in the market research industry who never touch raw data. Often they don’t even have the tools, since “data processing” is outsourced to lower levels or other countries. It is surprising because we almost always engage in work where getting into the data and puzzling over anomalies or hypotheses yields much deeper insight.
Here is an example of how critical it can be to look closely at your data, and in this case, very early in the data collection process. We launched an online survey last week and got reports back from our sample supplier that incidence was just one-third of what we expected, which would have serious feasibility and cost implications.
But once we looked at their report portal, we saw that for every qualified respondent completing the survey, two qualified respondents quit before finishing. That’s an unusually high ratio of “suspends” as we call them. So what was the problem? Were we just getting lousy respondents who did not want to seriously participate in a survey? Was the survey was too difficult, tedious, boring, or confusing? One source of answers (rarely examined) is to look at the data question by question to identify where in the survey people are quitting.
The story in this data: Something is wrong with your survey
(more…)
Tags: data, Data Collection, data quality, insight, research, stories, survey, survey respondents
Posted in Charts and Data Visualization, Data Analysis & Analytics, Data Collection, Methods & Tools, Survey Design | Comments Off
