Archive for the ‘Survey Design’ Category
Friday, May 17th, 2013
If you have ever been called to participate in a phone survey, you probably know the routine where you hear a question and then jump in with an answer. A good interviewer will remind you that she needs to read the entire question and all the answer options just to be sure that you offered the best response option.
We can’t do this with self-administered online surveys, but there is a way to minimize error associated with respondents jumping to conclusions: Put all clarifying instructions before asking the question, not after. A recent study published in Public Opinion Quarterly documented that if you put instructions before the question, respondents spend more time answering because they are reading the instructions and answering more carefully. Not surprisingly, their answers are more accurate.
Here is an example of a typical question that has clarifying instructions after the question: (more…)
Tags: Online Surveys, research, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Design, Survey Tips | Comments Off
Thursday, February 28th, 2013
I was the kid who skipped recess to help grade quizzes; the graduate student who delayed getting a degree because it meant the end of school; the professor who told students that he was now in the 33rd grade and still loved school.
Even now I can’t resist reviewing great opportunities for more coursework and learning that can help Versta and its clients do smarter work.
A number of top universities offer condensed summer coursework and seminars on topics critical to market research. Knowledge and innovation in these areas advance quickly, so staying on top of this learning is essential. Here are some that we highly recommend: (more…)
Tags: innovation, Market Research, methods, statistics, Survey Design
Posted in Market Research, Methods & Tools, Resources and Recommendations, Sampling, Survey Design | Comments Off
Wednesday, January 30th, 2013
It is good to use multiple survey questions to measure a particular attitude or opinion, and for many years experts on survey design recommended reversing the wording on some questions to get a stronger and more valid measure. For example, if we want to measure retirement confidence we might ask respondents how much they agree with the statement “I am confident that I will be able to live comfortably in retirement” and “I worry about being able to make ends meet in retirement.”
A recent meta-analysis in the Journal of Marketing Research shows that fewer and fewer researchers are using reverse wordings. Why? Because reversed wordings introduce ambiguities, make surveys more difficult for respondents, and often lead to increased levels of measurement error. Still, survey item reversals are considered a best practice because (1) they ensure fuller measurement of an attitude or opinion, (2) they keep respondents from answering carelessly, and (3) help correct for agreement bias. For this reason, the authors advocate “the continued use of reversed items in survey research” but they also caution researchers to “use them with care.”
Here are several of their specific recommendations we found most useful: (more…)
Tags: best practices, measurement, response scales, Survey Design
Posted in Market Research, Survey Design, Survey Tips | Comments Off
Thursday, December 20th, 2012
An article not too long ago in the Journal of Consumer Psychology summed up current research on brand extensions thus:
What factors determine whether or not a brand extension will be successful? The most important factor identified by prior research is perceived fit. Consumers respond more favorably if they are able to perceive a fit between the extension and the parent brand. . . . Perceived fit, no matter how it is defined, is the most important determinant of brand extension success—more important than marketing support, retailer acceptance, and quality of the parent brand.
The last sentence is worth reading again! Marketing support, effective distribution, and even strength of the parent brand matter less than whether buyers think the extension makes sense. (more…)
Tags: brand equity, Market Research, new product development, product innovation, product marketing
Posted in Market Research, New Products and Innovation, Survey Design, Topics in Marketing | Comments Off
Wednesday, October 31st, 2012
By elephants, we mean Republicans. Or maybe you have too many Democrats. Maybe it keeps going back and forth, which is the problem that Gallup sometimes has. In the spirit of learning all we can from election season polling, this week we focus on whom to include (or exclude) in your research, analysis, and market projections.
The issue is showcased right now as political polls attempt to measure voter preference and predict the election outcome. Is voter preference really as volatile and open to persuasion as the polls sometimes suggest? Probably not. A 2004 research article in Public Opinion Quarterly carefully documented that much of the volatility in Gallup’s polls results from how they screen respondents and weight their data. (more…)
Tags: bias, concept testing, data analysis, Market Research, Public Polls, Sampling, statistics
Posted in Data Analysis & Analytics, Market Research, Methods & Tools, New Products and Innovation, Sampling, Survey Design | Comments Off
Wednesday, September 26th, 2012
Figure 1. The U.S. Census Bureau's current questions about race and ethnicity. Click for full-size image.
Fifteen years ago the U.S. Census Bureau removed “Hispanic” from its question about race and instead created a stand-alone question to assess Hispanic ethnicity. It’s the reason we often advise our clients to ask about race and ethnicity separately, using questions similar to the Census Bureau’s, which are shown in Figure 1.
Well, chances are good that this will be soon be revised back to include Hispanic in a single question about race. The Census Bureau just concluded a large-scale study to assess the reliability and validity of its race and ethnicity measures, along with several potential alternatives. It involved questionnaires and re-interviews with a sample of nearly half a million households, plus 67 focus groups with nearly 800 people. (more…)
Tags: census, Data Collection, survey, Survey Design, survey respondents
Posted in Data Collection, Future Trends, Market Research, Survey Design | Comments Off
Friday, September 21st, 2012
During a presidential election year there is no escaping the flurry of public opinion polling and the intense scrutiny that surveys get from the media. But love it or hate it, there are excellent reasons to pay close attention to this year’s political polling.
(more…)
Tags: best practices, journalism, media, methods, news, phone surveys, public opinion, Public Polls, Public Relations, Sampling, survey, Survey Design
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls, Public Relations, Sampling, Survey Design | Comments Off
Thursday, August 23rd, 2012
Way back in 1944, Edwards Deming published an article in the American Sociological Review that could be required reading for anybody who does research today. He outlined all potential (and unfortunately, common) sources of error in survey research.
Apparently our contemporary obsession with sample sizes, random samples, response rates, and margins of error is not so new. In outlining all sources of error, Demining wanted to emphasize that “sampling errors, even for small samples, are often the least of the errors present.”
So despite some old-fashioned language and defunct technologies (Versta Research has never fielded a survey via telegraph!) we feel it is worth reproducing here what Deming called the thirteen factors “affecting the ultimate usefulness of a survey” as all of them apply as much today as they did 68 years ago:
(more…)
Tags: bias, margin of error, Online Surveys, response rates, Sampling, survey, Survey Design
Posted in Data Analysis & Analytics, Data Collection, Market Research, Public Polls, Sampling, Survey Design | Comments Off
Thursday, July 12th, 2012
In a typical graduate-level research course on survey methods, students are taught to ask demographic questions about race, ethnicity, age, gender, income, etc. at the end of a survey. The questions are typically off-topic, and they sometimes scare away respondents who feel such questions are intrusive.
But with most surveys now fielded through online survey panels and with researchers trying to reach increasingly narrow targets, putting demographic questions at the beginning of surveys is common. Often we have strict demographic quotas and screening criteria, so demographics need to be assessed immediately to decide whether potential respondents qualify for inclusion in a survey.
(more…)
Tags: best practices, demographics, Market Research, Survey Design
Posted in Data Collection, Market Research, Survey Design, Survey Tips | Comments Off
Wednesday, June 6th, 2012
A colleague at the University of Chicago is planning to teach a summer course on survey methods in North Korea. He asked a large group of researchers who belong to AAPOR (the American Association of Public Opinion Research) for recommendations on 10 books to bring that he could leave in the library.
We get similar questions from clients and research groups who want to have a small library of essential books and reference works on research methods for their organizations. Here are ten culled from our list that we often suggest, and from the list that others nominated as well: (more…)
Tags: methods, research design, Survey Design
Posted in Market Research, Methods & Tools, Resources and Recommendations, Survey Design | Comments Off
