Posts Tagged ‘Survey Design’
Friday, May 17th, 2013
If you have ever been called to participate in a phone survey, you probably know the routine where you hear a question and then jump in with an answer. A good interviewer will remind you that she needs to read the entire question and all the answer options just to be sure that you offered the best response option.
We can’t do this with self-administered online surveys, but there is a way to minimize error associated with respondents jumping to conclusions: Put all clarifying instructions before asking the question, not after. A recent study published in Public Opinion Quarterly documented that if you put instructions before the question, respondents spend more time answering because they are reading the instructions and answering more carefully. Not surprisingly, their answers are more accurate.
Here is an example of a typical question that has clarifying instructions after the question: (more…)
Tags: Online Surveys, research, Survey Design, survey respondents
Posted in Data Collection, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Thursday, February 28th, 2013
I was the kid who skipped recess to help grade quizzes; the graduate student who delayed getting a degree because it meant the end of school; the professor who told students that he was now in the 33rd grade and still loved school.
Even now I can’t resist reviewing great opportunities for more coursework and learning that can help Versta and its clients do smarter work.
A number of top universities offer condensed summer coursework and seminars on topics critical to market research. Knowledge and innovation in these areas advance quickly, so staying on top of this learning is essential. Here are some that we highly recommend: (more…)
Tags: innovation, Market Research, methods, statistics, Survey Design
Posted in Market Research, Methods & Tools, Resources and Recommendations, Sampling, Survey Design | No Comments »
Wednesday, January 30th, 2013
It is good to use multiple survey questions to measure a particular attitude or opinion, and for many years experts on survey design recommended reversing the wording on some questions to get a stronger and more valid measure. For example, if we want to measure retirement confidence we might ask respondents how much they agree with the statement “I am confident that I will be able to live comfortably in retirement” and “I worry about being able to make ends meet in retirement.”
A recent meta-analysis in the Journal of Marketing Research shows that fewer and fewer researchers are using reverse wordings. Why? Because reversed wordings introduce ambiguities, make surveys more difficult for respondents, and often lead to increased levels of measurement error. Still, survey item reversals are considered a best practice because (1) they ensure fuller measurement of an attitude or opinion, (2) they keep respondents from answering carelessly, and (3) help correct for agreement bias. For this reason, the authors advocate “the continued use of reversed items in survey research” but they also caution researchers to “use them with care.”
Here are several of their specific recommendations we found most useful: (more…)
Tags: best practices, measurement, response scales, Survey Design
Posted in Market Research, Survey Design, Survey Tips | No Comments »
Thursday, October 4th, 2012
If you ever feel like strangling the next research consultant who promises to deliver “actionable insights” then you can join me and the Dilbert team in making fun of it instead. Not only is it a horrid phrase, but it is unfortunately meaningless now that every research firm claims to be different from others by virtue of their actionable insights, magically produced by actionable insight dashboards, actionable insight survey tools, or special actionable insight “methodologies.”
With so much actionable insight around, why is so much research ignored? (more…)
Tags: customer satisfaction, insight, Market Research, marketing, satisfaction research, Survey Design
Posted in Funnies, Market Research | No Comments »
Wednesday, September 26th, 2012
Figure 1. The U.S. Census Bureau's current questions about race and ethnicity. Click for full-size image.
Fifteen years ago the U.S. Census Bureau removed “Hispanic” from its question about race and instead created a stand-alone question to assess Hispanic ethnicity. It’s the reason we often advise our clients to ask about race and ethnicity separately, using questions similar to the Census Bureau’s, which are shown in Figure 1.
Well, chances are good that this will be soon be revised back to include Hispanic in a single question about race. The Census Bureau just concluded a large-scale study to assess the reliability and validity of its race and ethnicity measures, along with several potential alternatives. It involved questionnaires and re-interviews with a sample of nearly half a million households, plus 67 focus groups with nearly 800 people. (more…)
Tags: census, Data Collection, survey, Survey Design, survey respondents
Posted in Data Collection, Future Trends, Market Research, Survey Design | No Comments »
Friday, September 21st, 2012
During a presidential election year there is no escaping the flurry of public opinion polling and the intense scrutiny that surveys get from the media. But love it or hate it, there are excellent reasons to pay close attention to this year’s political polling.
(more…)
Tags: best practices, journalism, media, methods, news, phone surveys, public opinion, Public Polls, Public Relations, Sampling, survey, Survey Design
Posted in Data Collection, Future Trends, Market Research, Methods & Tools, Public Polls, Public Relations, Sampling, Survey Design | No Comments »
Thursday, August 23rd, 2012
Way back in 1944, Edwards Deming published an article in the American Sociological Review that could be required reading for anybody who does research today. He outlined all potential (and unfortunately, common) sources of error in survey research.
Apparently our contemporary obsession with sample sizes, random samples, response rates, and margins of error is not so new. In outlining all sources of error, Demining wanted to emphasize that “sampling errors, even for small samples, are often the least of the errors present.”
So despite some old-fashioned language and defunct technologies (Versta Research has never fielded a survey via telegraph!) we feel it is worth reproducing here what Deming called the thirteen factors “affecting the ultimate usefulness of a survey” as all of them apply as much today as they did 68 years ago:
(more…)
Tags: bias, margin of error, Online Surveys, response rates, Sampling, survey, Survey Design
Posted in Data Analysis & Analytics, Data Collection, Market Research, Public Polls, Sampling, Survey Design | No Comments »
Thursday, July 12th, 2012
In a typical graduate-level research course on survey methods, students are taught to ask demographic questions about race, ethnicity, age, gender, income, etc. at the end of a survey. The questions are typically off-topic, and they sometimes scare away respondents who feel such questions are intrusive.
But with most surveys now fielded through online survey panels and with researchers trying to reach increasingly narrow targets, putting demographic questions at the beginning of surveys is common. Often we have strict demographic quotas and screening criteria, so demographics need to be assessed immediately to decide whether potential respondents qualify for inclusion in a survey.
(more…)
Tags: best practices, demographics, Market Research, Survey Design
Posted in Data Collection, Market Research, Survey Design, Survey Tips | No Comments »
Wednesday, June 6th, 2012
A colleague at the University of Chicago is planning to teach a summer course on survey methods in North Korea. He asked a large group of researchers who belong to AAPOR (the American Association of Public Opinion Research) for recommendations on 10 books to bring that he could leave in the library.
We get similar questions from clients and research groups who want to have a small library of essential books and reference works on research methods for their organizations. Here are ten culled from our list that we often suggest, and from the list that others nominated as well: (more…)
Tags: methods, research design, Survey Design
Posted in Market Research, Methods & Tools, Resources and Recommendations, Survey Design | No Comments »
Wednesday, February 15th, 2012
Data quality will suffer if respondents are bored with long surveys
It is hard to resist the temptation of asking “just one more question” when you’ve got an engaged respondent answering your survey questions online or on the phone. But it is crucial to do so because plenty of research shows that longer surveys result in bad data. Survey respondents may be willing to answer just one more question, but at some point the quality of information you get from them declines. Survey respondents become inattentive and offer lazy answers, or worse, they offer quick random answers just to get the survey over with.
At Versta Research we have a few rules of thumb for survey length based on (1) academic and industry research measuring data quality, (2) conversations with colleagues and suppliers throughout the industry, and (3) our ongoing experience of what works and what does not work. The maximum survey lengths we typically recommend are: (more…)
Tags: data, Data Collection, data quality, Market Research, Online Surveys, phone surveys, social media, survey, Survey Design, survey respondents, survey technology
Posted in Data Collection, Survey Design, Survey Tips | No Comments »
