Posts Tagged ‘survey’
Thursday, December 29th, 2011
Even if you are not involved in political polling, it is worth paying attention to the methods and best practices of political pollsters. One reason is that few other areas of research offer a way to completely validate one’s methods. Pollsters are using sampling and survey methods to predict the behaviors of a much larger population. Then in just one day that population behaves, we get a near-perfect count of exactly how they behaved, and we know whether the methods worked.
Several industry colleagues have recently been debating the merits of calculating and reporting “margins of error” in political polling, and pointed us to some surprising data from The New York Times: (more…)
Tags: journalism, public opinion, Public Polls, Sampling, statistics, survey
Posted in Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Sampling | No Comments »
Wednesday, June 29th, 2011
One of the surprises of multi-mode research is that offering people a choice of how they want to complete a survey (online, by mail, by phone, etc.) does not necessarily boost response rates. An article in the most recent issue of Public Opinion Quarterly provides new evidence of this. The study showed that even in a population with full access to both mail and Internet options (and full literacy in both modes), a full mail survey achieves a higher response rate than a web-based survey. It also achieves a higher response rate than a choice of either mail or web. So much for our recent article on The Myth of Too Many Choices!
But the study went further to explore some ways of sequentially deploying multiple survey options and multiple modes of information that can substantially boost web-based survey response rates. Here are some key takeaways from the research: (more…)
Tags: Data Collection, Internet, Online Surveys, response rates, survey, survey respondents
Posted in Data Collection, Online Surveys, Survey Tips | No Comments »
Thursday, May 12th, 2011
We are often surprised by the number of senior researchers in the market research industry who never touch raw data. Often they don’t even have the tools, since “data processing” is outsourced to lower levels or other countries. It is surprising because we almost always engage in work where getting into the data and puzzling over anomalies or hypotheses yields much deeper insight.
Here is an example of how critical it can be to look closely at your data, and in this case, very early in the data collection process. We launched an online survey last week and got reports back from our sample supplier that incidence was just one-third of what we expected, which would have serious feasibility and cost implications.
But once we looked at their report portal, we saw that for every qualified respondent completing the survey, two qualified respondents quit before finishing. That’s an unusually high ratio of “suspends” as we call them. So what was the problem? Were we just getting lousy respondents who did not want to seriously participate in a survey? Was the survey was too difficult, tedious, boring, or confusing? One source of answers (rarely examined) is to look at the data question by question to identify where in the survey people are quitting.
The story in this data: Something is wrong with your survey
(more…)
Tags: data, Data Collection, data quality, insight, research, stories, survey, survey respondents
Posted in Charts and Data Visualization, Data Analysis & Analytics, Data Collection, Methods & Tools, Survey Design | No Comments »
Wednesday, February 16th, 2011
After spending $65 million and previewing the show over sixty times since the end of last year, the producers of the new Spider-Man musical in New York are turning to focus groups and surveys in hopes that market research can do something…anything…to save the amazing Spider-Man from destruction.
Last week, theater critics roundly panned the show, calling it among the worst Broadway shows ever. The New York Times described is as “so grievously broken in every respect that it is beyond repair.” But days after these negative reviews, a market research firm was brought in, soliciting volunteers to help fix the show: (more…)
Tags: concept testing, Focus Groups, insight, Market Research, media, qualitative research., survey
Posted in Focus Groups, Market Research, New Products and Innovation, Survey Design, Survey Tips | No Comments »
Friday, December 3rd, 2010
Surveys matter only if people are willing to fill them out – and getting people to fill them out is not an easy task. There are many interesting distractions in the world! And there many silly surveys and junk surveys competing for people’s attention. When the time comes that you need important feedback via a survey, getting a sufficient response rate can be challenging.
Though relatively low response rates can yield statistically sound data (see our article, Do Response Rates Really Matter?) you need to have some data to work with. One organization we recently advised found itself with just two respondents for a do-it-yourself survey they launched, and unfortunately they lost the opportunity to go back into the field for another shot.
Based on our experience and what we have learned from the ever-evolving scientific literature on survey methods, here are three important steps you can take to improve your survey response rates: (more…)
Tags: Data Collection, response rates, survey, survey respondents
Posted in Data Collection, Market Research, Survey Tips | No Comments »
Friday, May 28th, 2010
We saw an ad today for a downloadable survey app similar to Survey Monkey or Zoomerang that was pitched as a tool for actionable insights. Wow! Download, install, run . . . click again, and there they are, sitting on your desktop or smart phone: actionable insights. (more…)
Tags: Data Collection, insight, Market Research, survey, survey technology, tools
Posted in Data Collection, Methods & Tools, Turning Data into Stories | No Comments »
Friday, April 23rd, 2010
Asking people to fill out long, tiresome, and boring surveys is a scourge of the research, polling, and survey industry. (Another is asking them to fill out a survey every time they interact with you — see There Are Too Many Surveys.) Asking people to fill out long surveys teaches them to avoid surveys in the future, and indeed we see survey participation rates continuing to decline. But more importantly if you are the one who needs to rely on survey data, long surveys result in measurably lower data quality. (more…)
Tags: Data Collection, data quality, open-ends, survey, Survey Design, survey respondents
Posted in Data Collection, Online Surveys, Survey Design, Survey Tips | No Comments »
Wednesday, March 10th, 2010
When multiple surveys about the same topic give different results, consider yourself lucky. It provides an opportunity to dissect and understand the question you are trying to answer in a way you might not get otherwise. A recent New York Times article provides a nice example when it comes to polls about health care. (more…)
Tags: insight, survey, Survey Design
Posted in Survey Design, Turning Data into Stories | No Comments »
Wednesday, February 24th, 2010
One problem with new survey technologies that make it easy (and inexpensive) to design and field surveys is that we now have an explosion of poorly done and ill-intentioned surveys. Have you ever suspected that a survey isn’t quite legit?
Keeping your survey efforts honest is critical. Sincere efforts to document issues through rigorous surveys succeed, and when done correctly, people are influenced by numbers. But trying to shock or manipulate with sensationalist, biased, and phony data gets you only so far. For a great survey, keep it honest, rigorous, and valid, and carefully control for bias. We, at Versta, can help you succeed with this.
-Joe Hopper, Ph.D.
Tags: bias, survey, trust
Posted in Funnies, Presenting Research | No Comments »
Friday, February 5th, 2010
A research article the Winter 2009 issue of Public Opinion Quarterly provided some useful comparisons of data quality between phone surveys and various types of online surveys. The findings are based on an experiment that fielded identical questionnaires via three survey modes, and, not surprisingly, there are strengths and weaknesses to each type of survey.
Phone vs. Online Surveys: Strengths & Weaknesses
As reflected in the table above, the article is rather technical. But there are two key summary points worth learning from the study: (more…)
Tags: bias, data quality, Internet, Online Surveys, phone surveys, population, Sampling, survey, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Sampling | No Comments »
