Posts Tagged ‘bias’
Wednesday, January 18th, 2012
Many of us in marketing research have been deploying web surveys for over ten years, and web surveys are, by far, the dominant mode of data collection in our industry nowadays. But our techniques and methods are an amalgam of practices adapted from other data collection modes, learned in part through trial and error, and taught to others through channels more akin to oral traditions. So it is helpful when our academic colleagues manage to document and codify the art and science of what we do. (more…)
Tags: bias, Internet, Market Research, Online Surveys, Sampling, statistics, Survey Design
Posted in Data Analysis & Analytics, Data Collection, Market Research, Online Surveys, Resources and Recommendations, Sampling, Survey Design | No Comments »
Tuesday, August 9th, 2011
These days most researchers agree that if you want to do a random sample phone survey of the U.S. population, you ought to include cell phones. More than one-quarter of the population do not have landline telephones at home. Those who do have landline telephones are less likely than ever to answer them, and less likely than ever to participate in surveys.
But it is not easy to include cell phones. The sampling protocols and the post-stratification weighting become more complicated. You need to account for a higher probability of cell phone owners being in your sample, because most of them also have landlines. You can’t use automated or predictive dialing to call cell phone numbers. You can’t target geography as well, because area codes and exchanges have become mobile. And people get mad at you if they have to pay for incoming calls, so you need to offer cash.
What’s the bottom line effect on costs for a survey that includes cell phones? A recent study sponsored by the American Association for Public Opinion Research (AAPOR) documents the following: (more…)
Tags: bias, Data Collection, Market Research, phone surveys, population, public opinion, Public Polls, research, Sampling
Posted in Data Collection, Future Trends, Market Research, Public Polls, Sampling | No Comments »
Thursday, May 26th, 2011
A new study presented by two professors from Harvard University and the University of Massachusetts at Amherst was probably one of the liveliest and potentially disruptive presentations at least week’s annual meeting of the American Association of Public Opinion Research (AAPOR) in Phoenix.
Why? Because their research challenges the beliefs of many AAPOR-ites who disregard most online research as being theoretically indefensible since it is not based on probability sampling. The research presented was based on parallel surveys conducted last year, designed to allow careful comparison of three survey modes: (more…)
Tags: bias, census, Data Collection, Internet, journalism, media, news, Online Surveys, phone surveys, public opinion, Public Polls, Public Relations, Sampling
Posted in Data Collection, Methods & Tools, Online Surveys, Public Polls, Public Relations, Sampling | No Comments »
Saturday, April 16th, 2011
Among the many sources of potential error that can affect surveys are respondents themselves. They sometimes misinterpret questions, respond in socially acceptable ways, or give “easy” answers in hopes that a more interesting question is just around the corner.
This is not to say they are bad or fraudulent respondents. Research shows that the vast majority of survey respondents are careful, thoughtful, and truthful in how they answer survey questions. The problem with respondent error, it turns out, is poor survey design, which may involve biased or ambiguous questions, tasks that are too complicated or boring, surveys that are too long, and so on.
Recent research shows that grid-style questions that look like this:
or this: (more…)
Tags: bias, Data Collection, data quality, Market Research, Online Surveys, Survey Design, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Thursday, December 30th, 2010
The latest data from the CDC’s National Health Interview Survey show that one quarter (25%) of U.S. adults do not have land-line telephones in their homes. So if you conduct a traditional random-digit-dial (RDD) phone survey, you will automatically be excluding one quarter of the population. Does it matter, given that surveys rarely interview everyone anyway? Probably. If those 25% are different from the remaining 75% in important ways, then excluding them will skew your survey findings.
(more…)
Tags: bias, Data Collection, data quality, ethics, Market Research, phone surveys, public opinion, Public Polls, research, Sampling, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Public Polls, Sampling | No Comments »
Wednesday, September 22nd, 2010
Despite having been on the front lines of social research for twenty five years, interviewing respondents personally and eliciting data through surveys, I still feel somewhat surprised and disbelieving that people really want to participate in research. But they do. Sometimes eagerly. Almost always truthfully. Surely, my surprise stems from my own reluctance to fill out surveys.
It turns out that I may just lack the survey-taking gene. No joke. New research of genetic and fraternal twins shows that our willingness to participate in research is shaped in part by our genes. The research was led by Lori Foster Thompson, an associate professor of psychology at North Carolina State University, and is soon to be published in the Journal of Organizational Behavior. As reported on Science Daily: (more…)
Tags: bias, Sampling, survey respondents
Posted in Data Collection, Market Research, Sampling | No Comments »
Friday, August 13th, 2010
Does data displayed in charts and graphs, rather than tables, lead to better decisions? Not according to the latest research reported in this month’s Journal of Marketing Research.
The authors looked at various types of biases that creep into business managers’ decisions when based on data presented to them. They did this by conducting experiments with business school students and managers who are members of the American Marketing Association. Some were presented with numeric data in tables, while others were presented with data in charts or graphs. All tables, charts, and graphs were clear and well-designed. (more…)
Tags: bias, charts, communication, data, Market Research, statistics, stories, visualizing data
Posted in Charts and Data Visualization, Market Research, Presenting Research, Turning Data into Stories | No Comments »
Wednesday, June 16th, 2010
We are currently engaged in a research project for a client in South Korea, so issues of cross-cultural communication are top-of-mind for us right now. Whether we rely on translations, or whether we speak the same language as our clients and respondents, it is important for researchers to understand differences in how people think and respond to research questions because data is always context sensitive. (more…)
Tags: bias, culture, international, Survey Design
Posted in Methods & Tools, Survey Design | No Comments »
Friday, May 7th, 2010
Another article was just published in the Spring 2010 issue of Public Opinion Quarterly exploring data quality differences between online surveys and phone surveys. The findings were based on a lab experiment in which subjects completed survey questions either on a computer or over an intercom system with an interviewer. Doing the study in a laboratory isolated the mode effect of computerized self-administered data collection vs. an interview conducted by a human. (more…)
Tags: bias, Data Collection, data quality, Online Surveys, phone surveys, validity
Posted in Data Collection, Methods & Tools, Online Surveys | No Comments »
Wednesday, February 24th, 2010
One problem with new survey technologies that make it easy (and inexpensive) to design and field surveys is that we now have an explosion of poorly done and ill-intentioned surveys. Have you ever suspected that a survey isn’t quite legit?
Keeping your survey efforts honest is critical. Sincere efforts to document issues through rigorous surveys succeed, and when done correctly, people are influenced by numbers. But trying to shock or manipulate with sensationalist, biased, and phony data gets you only so far. For a great survey, keep it honest, rigorous, and valid, and carefully control for bias. We, at Versta, can help you succeed with this.
-Joe Hopper, Ph.D.
Tags: bias, survey, trust
Posted in Funnies, Presenting Research | No Comments »