Posts Tagged ‘Online Surveys’
Friday, May 17th, 2013
If you have ever been called to participate in a phone survey, you probably know the routine where you hear a question and then jump in with an answer. A good interviewer will remind you that she needs to read the entire question and all the answer options just to be sure that you offered the best response option.
We can’t do this with self-administered online surveys, but there is a way to minimize error associated with respondents jumping to conclusions: Put all clarifying instructions before asking the question, not after. A recent study published in Public Opinion Quarterly documented that if you put instructions before the question, respondents spend more time answering because they are reading the instructions and answering more carefully. Not surprisingly, their answers are more accurate.
Here is an example of a typical question that has clarifying instructions after the question: (more…)
Tags: Online Surveys, research, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Friday, April 26th, 2013
It is hard to find an appropriate use for Google Surveys, because, as we outlined in a review article last fall, its capabilities are limited. But last week we needed a quick incidence test of how many U.S. adults own a certain type of investment product. Google Surveys seemed perfect. It was not fast, by the way. It took five days to collect data from 200 respondents. Google says this is because we asked a screening question before asking about product ownership. Even so, this survey took longer than a standard omnibus.
But what struck me most about my trial run with Google Surveys was the Creepy Factor. It made me realize in a most uncomfortable way that Google tracks everything I do. I knew this already, and I follow ongoing discussions about online privacy. I have a personal g-mail account, a G+ page, and I use Google as the starting point for almost everything I do on the Internet. I know that they track everything I do. But it was never so creepy and apparent until I fielded a Google survey. How was it creepy? (more…)
Tags: ethics, Market Research, omnibus, Online Surveys, privacy, public opinion, Public Polls, survey technology
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | No Comments »
Thursday, December 27th, 2012
With all our excitement over the last few months about the accuracy of online polling during the election season—substantially outperforming “gold standard” telephone research—there was not time to share ESOMAR’s September 2012 updated guide to purchasing online sample. The guide consists of 28 questions all purveyors of online sample should answer, publish, and make available to every buyer of its products and services. The guide has been updated to reflect rapid changes in online sampling over the last couple of years, including use of routers, real-time sampling, and blended sample from multiple sources.
Before purchasing online sample for your next research survey, be sure that you know the answers to these 28 questions: (more…)
Tags: online, Online Panels, Online Surveys, panels, Public Polls, Sampling
Posted in Data Collection, Market Research, Online Surveys, Public Polls, Resources and Recommendations, Sampling | No Comments »
Thursday, November 29th, 2012
If there is one super important lesson to be learned from this year’s round of election polling, it is that online surveys work. Google Consumer Surveys, which use non-probability online samples, predicted the election far better than Gallup did. And online surveys, overall, outperformed telephone surveys.
The New York Times’ Nate Silver compiled polling results from 23 organizations that conducted at least five surveys in the final three weeks of the campaign. He calculated how far their projections were from the actual outcome of the presidential race. Google (a fully automated, online solution) came in second place, predicting the actual outcome within 1.6 percentage points. Gallup (using “gold standard” telephone methods) came in last, predicting the outcome within 7.2 percentage points.
(more…)
Tags: Online Surveys, phone surveys, Public Polls
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Public Relations | No Comments »
Wednesday, November 14th, 2012
I have never been a media poll watcher or politics junkie who tracks every new poll saying which candidate is favored to win. It is hard to see the point of spending so much time and money predicting the outcome of an event that will be known with certainty within a matter of days, weeks, or months. But elections are an amazing way to see survey research methods in action, and there are few opportunities to have those methods validated so quickly, accurately, and unforgivingly than political polling.
What did we learn from the polls this election season? (more…)
Tags: methods, models, Online Surveys, phone surveys, Public Polls, research, statistics, survey
Posted in Future Trends, Market Research, Public Polls | No Comments »
Thursday, October 18th, 2012
Six months ago Google launched an intriguing new way to conduct cost-effective surveys that offers an alternative to omnibus surveys. One or two easy survey questions are presented to online users as they seek access to high quality media sites. They gain free access in exchange for answering the survey questions. Google tracks how many people are answering each question and manages the process to ensure that the sample of respondents answering each question closely matches the overall U.S. population (based on Census data for those who have Internet access).
Here are some of the more interesting aspects of their approach: (more…)
Tags: omnibus, Online Panels, Online Surveys, Public Polls
Posted in Data Collection, Future Trends, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | No Comments »
Thursday, October 11th, 2012
Figure 1. Example of a Daily Field Report (Click Image)
A mistake often made by both professional and do-it-yourself researchers is letting a survey sit in the field without actively monitoring it. Once we design a survey and put it out there for people to respond, we just wait patiently (or get busy on another project) until we have data for analysis, right? But collecting data is never straightforward. It nearly always requires daily adjustments and decisions from the most senior members of a research team.
So at Versta Research, all fieldwork we conduct or oversee requires a daily and detailed fieldwork report that gives us visibility into all kinds of technical and conceptual issues that might affect the quality and outcomes of research. Figure 1 shows an example of a report; nothing fancy, but full of crucial data. As we review these reports, we watch for several warning signs and intervene where needed:
(more…)
Tags: data, Data Collection, data quality, Online Surveys, Sampling
Posted in Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Survey Tips | No Comments »
Thursday, August 23rd, 2012
Way back in 1944, Edwards Deming published an article in the American Sociological Review that could be required reading for anybody who does research today. He outlined all potential (and unfortunately, common) sources of error in survey research.
Apparently our contemporary obsession with sample sizes, random samples, response rates, and margins of error is not so new. In outlining all sources of error, Demining wanted to emphasize that “sampling errors, even for small samples, are often the least of the errors present.”
So despite some old-fashioned language and defunct technologies (Versta Research has never fielded a survey via telegraph!) we feel it is worth reproducing here what Deming called the thirteen factors “affecting the ultimate usefulness of a survey” as all of them apply as much today as they did 68 years ago:
(more…)
Tags: bias, margin of error, Online Surveys, response rates, Sampling, survey, Survey Design
Posted in Data Analysis & Analytics, Data Collection, Market Research, Public Polls, Sampling, Survey Design | No Comments »
Thursday, August 2nd, 2012
Is this YOUR distracted survey respondent?
A couple weeks ago we presented new data showing that response rates continue to decline. You can now expect that a typical, rigorously executed phone survey will yield a response rate in the single digits.
Scientific evidence over the last decade has shown that high response rates do not necessarily yield more accurate surveys. In fact, it turns out that high response rates can actually hurt the accuracy of surveys.
(more…)
Tags: Online Surveys, phone surveys, response rates, Sampling, survey, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Public Polls | No Comments »
Wednesday, May 30th, 2012
Cost matters when you choose a sample or panel provider for your survey because there are good panels and bad panels. Bad panels provide survey respondents at cheap prices. But they do a lousy job managing and screening their members. Not surprisingly, a good portion of the data you get from bad panels will likely be lousy.
A recent study entitled “Dirty Little Secrets of Online Panel Research” by one of our industry colleagues described and documented lousy panel management practices of some companies. Mystery shoppers joined and participated in online surveys offered by nearly all of the leading panel companies that most of us rely on. Here are some of the “worst practices” they uncovered: (more…)
Tags: data quality, Online Panels, Online Surveys, panels
Posted in Data Collection, Methods & Tools, Online Surveys, Sampling | No Comments »
