Archive for the ‘Data Collection’ Category
Thursday, June 13th, 2013
Even if you don’t care about political polling, or the fact that Gallup consistently overestimates support for Republican candidates, it is worth paying attention to how Gallup is trying to fix its problems with surveys and polling.
They are not happy with how poorly their polls have fared (who would be?), and they have teams of smart people trying to figure out what is wrong. Given their high profile, they are making the process and findings of their investigations public, and we have much to learn from that.
Last week they released their findings from an extensive review, which involved outside experts as well as internal ones. It is fascinating to read, because they identify 13 suspects in their survey process that all companies who do survey research should always be thinking about: (more…)
Tags: phone surveys, Public Polls, reliability, research design, validity
Posted in Data Analysis & Analytics, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Sampling | No Comments »
Thursday, June 6th, 2013
The Excel coding error, courtesy of The Roosevelt Institute
Or maybe we should say: why vendors SHOULD do everything twice BEFORE their work hits your desk and you send it back because you found errors. When it comes to something as complex and exacting as market research or public opinion polling, there are almost certainly mistakes the first time around. If a company does not have processes to validate data and deliverables, unfortunately those mistakes end up with you.
Recent errors in an economics paper that laid the foundation for Europe’s austerity programs provides a dramatic and painful example, as outlined by Paul Krugman of the New York Times: (more…)
Tags: data analysis, data quality, Market Research, statistics, tools
Posted in Data Analysis & Analytics, Data Collection, Market Research, Methods & Tools | No Comments »
Friday, May 17th, 2013
If you have ever been called to participate in a phone survey, you probably know the routine where you hear a question and then jump in with an answer. A good interviewer will remind you that she needs to read the entire question and all the answer options just to be sure that you offered the best response option.
We can’t do this with self-administered online surveys, but there is a way to minimize error associated with respondents jumping to conclusions: Put all clarifying instructions before asking the question, not after. A recent study published in Public Opinion Quarterly documented that if you put instructions before the question, respondents spend more time answering because they are reading the instructions and answering more carefully. Not surprisingly, their answers are more accurate.
Here is an example of a typical question that has clarifying instructions after the question: (more…)
Tags: Online Surveys, research, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Design, Survey Tips | Comments Off
Friday, April 26th, 2013
It is hard to find an appropriate use for Google Surveys, because, as we outlined in a review article last fall, its capabilities are limited. But last week we needed a quick incidence test of how many U.S. adults own a certain type of investment product. Google Surveys seemed perfect. It was not fast, by the way. It took five days to collect data from 200 respondents. Google says this is because we asked a screening question before asking about product ownership. Even so, this survey took longer than a standard omnibus.
But what struck me most about my trial run with Google Surveys was the Creepy Factor. It made me realize in a most uncomfortable way that Google tracks everything I do. I knew this already, and I follow ongoing discussions about online privacy. I have a personal g-mail account, a G+ page, and I use Google as the starting point for almost everything I do on the Internet. I know that they track everything I do. But it was never so creepy and apparent until I fielded a Google survey. How was it creepy? (more…)
Tags: ethics, Market Research, omnibus, Online Surveys, privacy, public opinion, Public Polls, survey technology
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | Comments Off
Thursday, April 4th, 2013
Some business managers and marketing executives mistakenly believe that “big data” will deliver better insight because of the sheer volume of data now at our disposal. Now we just need the statisticians, the computing power, and the analytics software to sift through it all, right? Not so. The truth is, for most purposes you don’t need a lot of data. You need a small random sample of data. (more…)
Tags: data, margin of error, Market Research, methods, Sampling
Posted in Data Analysis & Analytics, Data Collection, Future Trends, Market Research, Sampling | Comments Off
Wednesday, March 6th, 2013
We are in the midst of a couple projects that require in-depth telephone interviews (IDIs) with senior executives for B2B research. With every new project, we start wondering whether new technologies and tools can make the work better. For IDIs, however, I am always struck by how perfectly our old tools work. If you ever struggle with figuring out how to record a telephone research interview in a super simple way that creates an audio file direct to your computer, here are a few “old” tools that work extremely well.
Dictation Buddy is a simple and small software package with no bells and whistles, but it works perfectly. It records voices cleanly, allows us to adjust the recording saturation, has easy play-back, and allows us to choose various file formats for saving. It is also an excellent tool for transcribing, allowing use of a foot pedal for playback. It costs less than $50. (more…)
Tags: ethics, Market Research, tools
Posted in Data Collection, Market Research, Methods & Tools, Resources and Recommendations | Comments Off
Thursday, December 27th, 2012
With all our excitement over the last few months about the accuracy of online polling during the election season—substantially outperforming “gold standard” telephone research—there was not time to share ESOMAR’s September 2012 updated guide to purchasing online sample. The guide consists of 28 questions all purveyors of online sample should answer, publish, and make available to every buyer of its products and services. The guide has been updated to reflect rapid changes in online sampling over the last couple of years, including use of routers, real-time sampling, and blended sample from multiple sources.
Before purchasing online sample for your next research survey, be sure that you know the answers to these 28 questions: (more…)
Tags: online, Online Panels, Online Surveys, panels, Public Polls, Sampling
Posted in Data Collection, Market Research, Online Surveys, Public Polls, Resources and Recommendations, Sampling | Comments Off
Thursday, November 29th, 2012
If there is one super important lesson to be learned from this year’s round of election polling, it is that online surveys work. Google Consumer Surveys, which use non-probability online samples, predicted the election far better than Gallup did. And online surveys, overall, outperformed telephone surveys.
The New York Times’ Nate Silver compiled polling results from 23 organizations that conducted at least five surveys in the final three weeks of the campaign. He calculated how far their projections were from the actual outcome of the presidential race. Google (a fully automated, online solution) came in second place, predicting the actual outcome within 1.6 percentage points. Gallup (using “gold standard” telephone methods) came in last, predicting the outcome within 7.2 percentage points.
(more…)
Tags: Online Surveys, phone surveys, Public Polls
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Online Surveys, Public Polls, Public Relations | Comments Off
Thursday, October 18th, 2012
Six months ago Google launched an intriguing new way to conduct cost-effective surveys that offers an alternative to omnibus surveys. One or two easy survey questions are presented to online users as they seek access to high quality media sites. They gain free access in exchange for answering the survey questions. Google tracks how many people are answering each question and manages the process to ensure that the sample of respondents answering each question closely matches the overall U.S. population (based on Census data for those who have Internet access).
Here are some of the more interesting aspects of their approach: (more…)
Tags: omnibus, Online Panels, Online Surveys, Public Polls
Posted in Data Collection, Future Trends, Methods & Tools, Omnibus Surveys, Online Surveys, Public Polls | Comments Off
Thursday, October 11th, 2012
Figure 1. Example of a Daily Field Report (Click Image)
A mistake often made by both professional and do-it-yourself researchers is letting a survey sit in the field without actively monitoring it. Once we design a survey and put it out there for people to respond, we just wait patiently (or get busy on another project) until we have data for analysis, right? But collecting data is never straightforward. It nearly always requires daily adjustments and decisions from the most senior members of a research team.
So at Versta Research, all fieldwork we conduct or oversee requires a daily and detailed fieldwork report that gives us visibility into all kinds of technical and conceptual issues that might affect the quality and outcomes of research. Figure 1 shows an example of a report; nothing fancy, but full of crucial data. As we review these reports, we watch for several warning signs and intervene where needed:
(more…)
Tags: data, Data Collection, data quality, Online Surveys, Sampling
Posted in Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Online Surveys, Survey Tips | Comments Off
