Archive for the ‘Online Surveys’ Category
Friday, May 17th, 2013
If you have ever been called to participate in a phone survey, you probably know the routine where you hear a question and then jump in with an answer. A good interviewer will remind you that she needs to read the entire question and all the answer options just to be sure that you offered the best response option.
We can’t do this with self-administered online surveys, but there is a way to minimize error associated with respondents jumping to conclusions: Put all clarifying instructions before asking the question, not after. A recent study published in Public Opinion Quarterly documented that if you put instructions before the question, respondents spend more time answering because they are reading the instructions and answering more carefully. Not surprisingly, their answers are more accurate.
Here is an example of a typical question that has clarifying instructions after the question: (more…)
Tags: Online Surveys, research, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Friday, April 26th, 2013
It is hard to find an appropriate use for Google Surveys, because, as we outlined in a review article last fall, its capabilities are limited. But last week we needed a quick incidence test of how many U.S. adults own a certain type of investment product. Google Surveys seemed perfect. It was not fast, by the way. It took five days to collect data from 200 respondents. Google says this is because we asked a screening question before asking about product ownership. Even so, this survey took longer than a standard omnibus.
But what struck me most about my trial run with Google Surveys was the Creepy Factor. It made me realize in a most uncomfortable way that Google tracks everything I do. I knew this already, and I follow ongoing discussions about online privacy. I have a personal g-mail account, a G+ page, and I use Google as the starting point for almost everything I do on the Internet. I know that they track everything I do. But it was never so creepy and apparent until I fielded a Google survey. How was it creepy? (more…)
Tags: ethics, Market Research, omnibus, Online Surveys, privacy, public opinion, Public Polls, survey technology
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | No Comments »
Thursday, December 27th, 2012
With all our excitement over the last few months about the accuracy of online polling during the election season—substantially outperforming “gold standard” telephone research—there was not time to share ESOMAR’s September 2012 updated guide to purchasing online sample. The guide consists of 28 questions all purveyors of online sample should answer, publish, and make available to every buyer of its products and services. The guide has been updated to reflect rapid changes in online sampling over the last couple of years, including use of routers, real-time sampling, and blended sample from multiple sources.
Before purchasing online sample for your next research survey, be sure that you know the answers to these 28 questions: (more…)
Tags: online, Online Panels, Online Surveys, panels, Public Polls, Sampling
Posted in Data Collection, Market Research, Online Surveys, Public Polls, Resources and Recommendations, Sampling | No Comments »
Thursday, November 29th, 2012
If there is one super important lesson to be learned from this year’s round of election polling, it is that online surveys work. Google Consumer Surveys, which use non-probability online samples, predicted the election far better than Gallup did. And online surveys, overall, outperformed telephone surveys.
The New York Times’ Nate Silver compiled polling results from 23 organizations that conducted at least five surveys in the final three weeks of the campaign. He calculated how far their projections were from the actual outcome of the presidential race. Google (a fully automated, online solution) came in second place, predicting the actual outcome within 1.6 percentage points. Gallup (using “gold standard” telephone methods) came in last, predicting the outcome within 7.2 percentage points.
(more…)
Tags: Online Surveys, phone surveys, Public Polls
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Public Relations | No Comments »
Wednesday, November 7th, 2012
What IS a credibility interval, you ask? It is a term making its way into mainstream market research just as Bayesian statistics are making their way into market research. A credibility interval provides a range of values, calculated using Bayesian statistical techniques, within which a statistical estimate is likely to fall. It is analogous to a confidence interval, which is the traditional and commonly used measure of sampling error in survey research and statistical estimation.
As with a confidence interval, a credibility interval can be a legitimate, compelling, and mathematically rigorous way of expressing the certainty of statistical estimates. Unfortunately it is being used in the same sloppy, inappropriate, and misleading ways that confidence interval and “margins of error” are being used. (more…)
Tags: confidence intervals, margin of error, statistics
Posted in Data Analysis & Analytics, Future Trends, Market Research, Online Surveys, Public Polls | No Comments »
Thursday, October 18th, 2012
Six months ago Google launched an intriguing new way to conduct cost-effective surveys that offers an alternative to omnibus surveys. One or two easy survey questions are presented to online users as they seek access to high quality media sites. They gain free access in exchange for answering the survey questions. Google tracks how many people are answering each question and manages the process to ensure that the sample of respondents answering each question closely matches the overall U.S. population (based on Census data for those who have Internet access).
Here are some of the more interesting aspects of their approach: (more…)
Tags: omnibus, Online Panels, Online Surveys, Public Polls
Posted in Data Collection, Future Trends, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | No Comments »
Thursday, October 11th, 2012
Figure 1. Example of a Daily Field Report (Click Image)
A mistake often made by both professional and do-it-yourself researchers is letting a survey sit in the field without actively monitoring it. Once we design a survey and put it out there for people to respond, we just wait patiently (or get busy on another project) until we have data for analysis, right? But collecting data is never straightforward. It nearly always requires daily adjustments and decisions from the most senior members of a research team.
So at Versta Research, all fieldwork we conduct or oversee requires a daily and detailed fieldwork report that gives us visibility into all kinds of technical and conceptual issues that might affect the quality and outcomes of research. Figure 1 shows an example of a report; nothing fancy, but full of crucial data. As we review these reports, we watch for several warning signs and intervene where needed:
(more…)
Tags: data, Data Collection, data quality, Online Surveys, Sampling
Posted in Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Survey Tips | No Comments »
Thursday, August 2nd, 2012
Is this YOUR distracted survey respondent?
A couple weeks ago we presented new data showing that response rates continue to decline. You can now expect that a typical, rigorously executed phone survey will yield a response rate in the single digits.
Scientific evidence over the last decade has shown that high response rates do not necessarily yield more accurate surveys. In fact, it turns out that high response rates can actually hurt the accuracy of surveys.
(more…)
Tags: Online Surveys, phone surveys, response rates, Sampling, survey, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Public Polls | No Comments »
Wednesday, May 30th, 2012
Cost matters when you choose a sample or panel provider for your survey because there are good panels and bad panels. Bad panels provide survey respondents at cheap prices. But they do a lousy job managing and screening their members. Not surprisingly, a good portion of the data you get from bad panels will likely be lousy.
A recent study entitled “Dirty Little Secrets of Online Panel Research” by one of our industry colleagues described and documented lousy panel management practices of some companies. Mystery shoppers joined and participated in online surveys offered by nearly all of the leading panel companies that most of us rely on. Here are some of the “worst practices” they uncovered: (more…)
Tags: data quality, Online Panels, Online Surveys, panels
Posted in Data Collection, Methods & Tools, Online Surveys, Sampling | No Comments »
Thursday, May 24th, 2012
Bad data is worse than no data. With no data, at least you have intuition and experience to rely on. Bad data, on the other hand, will result in “findings” that are likely to spoil important decisions.
With most surveys fielded online nowadays, we have lost the opportunity for phone (or in-person) interviewers to assess the quality of a respondent’s answers. On the other hand, with online surveys there are all sorts of useful indicators to help assess whether respondents are providing truthful responses. This is because almost nobody takes a survey purposefully to give wrong or misleading data. Instead, they are bored, lazy, or irritated by your less-than-optimal survey design. Or they are trying to get through a survey quickly to collect an incentive payment or sweepstakes entry.
There are tell-tale signs of when respondents are giving bad data. We look for them in almost every survey we conduct or dataset we analyze. Here are six ways we suggest to identify bad survey data: (more…)
Tags: data quality, Online Surveys
Posted in Data Analysis & Analytics, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys | No Comments »
