Archive for the ‘Methods & Tools’ Category
Thursday, July 28th, 2011
An issue we continually struggle with at Versta Research is how to automate the research process and leverage new technologies without losing the essence of what good research does. Good research does not report data, build charts, or generate dashboards. It learns, answers new questions, interprets data, and helps users focus on information and findings that are relevant to their needs.
The last couple of weeks we have been working with a group that specializes in coding and tabulating text responses to open-ended questions on surveys. They have tools and technology that undoubtedly make the process easier and more efficient (we have used those tools, and they are impressive). They are also have a singular focus and expertise that is supposed to help streamline the process, cut costs, and improve speed and efficiency.
The results have been mediocre at best, even with human coders working the technology and making the critical decisions. (more…)
Tags: data, insight, Market Research, open-ends, qualitative research., survey technology
Posted in Data Analysis & Analytics, Future Trends, Market Research, Methods & Tools | 2 Comments »
Thursday, July 7th, 2011
Versta Research just hit a magic number: 100. That’s the number of articles we have written to help our clients and their colleagues keep abreast of important trends in market research. If your market research supplier is not providing ongoing thought leadership in design, methods, and analytics, then what are the chances they are bringing ongoing and deep insight to your specific research needs?
To celebrate, we’re serving up a sampler of our five best articles. How did we decide they are the best? Our clients told us. These are the articles that they write to us about, forward to their colleagues, and for which they return to our website time and again. These are also the articles for which we get requests for print-ready PDF versions. (Just let us know if you want one!) (more…)
Tags: communication, conjoint, Market Research, product innovation, research, statistics, stories
Posted in Data Analysis & Analytics, Future Trends, Market Research, Methods & Tools, New Products and Innovation, Turning Data into Stories | No Comments »
Thursday, June 16th, 2011
Executives who lead entrepreneurial firms have dramatically different attitudes about market research from their counterparts at larger established firms, according to a recent study from Saras Sarasvathy, an associate professor of business administration at the University of Virginia.
The study suggests that entrepreneurs are more focused on immediate and practical questions that will help them get their products into the hands of customers, and that traditional market research may not be the best way to get the right data and answers. That makes sense.
But according to an article in the February issue of Inc. magazine, “when asked what kind of market research they would conduct for [a] hypothetical start-up, most of Sarasvathy’s subjects responded with variations on the following: (more…)
Tags: data, insight, Market Research, product innovation, research, satisfaction research, segmentation, tracking studies, value
Posted in Future Trends, Market Research, Methods & Tools, New Products and Innovation | No Comments »
Thursday, June 9th, 2011
A good chart is the best way to understand the law of diminishing returns when it comes to sample size. So for our June 2011 newsletter we built an interactive graph for choosing sample size. It’s cool, educational, and useful. Moreover, it will show you just how mind boggling the numbers behind sampling can be. It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.
(more…)
Tags: charts, Market Research, population, public opinion, Public Polls, Public Relations, Sampling, survey respondents, visualizing data
Posted in Charts and Data Visualization, Data Collection, Market Research, Methods & Tools, Omnibus Surveys, Public Polls, Public Relations, Sampling | No Comments »
Thursday, June 2nd, 2011
We have always been big fans of the Net Promoter Score (NPS) metric because it has convinced many firms to begin using customer satisfaction measurement scales that work better and that are tied to what people do rather than what people think. Eleven point scales (with points zero to ten) allow for optimal variation. They are intuitive and appealing: people quickly grasp the idea of rating something on a zero to ten scale, and are familiar with the idea from grade school. They also have a neutral mid-point, which is important for many customer satisfaction and loyalty studies.
But NPS questions do not make sense in many situations. Here’s one we saw last week—it’s a survey sent by Amazon to sellers who call regarding complicated issues with how their products are being displayed on the website or how payments are being transferred:
A Poor Use of the Net Promoter Question
(more…)
Tags: measurement, satisfaction research, Survey Design, tracking studies
Posted in Market Research, Methods & Tools, Survey Design, Survey Tips | No Comments »
Thursday, May 26th, 2011
A new study presented by two professors from Harvard University and the University of Massachusetts at Amherst was probably one of the liveliest and potentially disruptive presentations at least week’s annual meeting of the American Association of Public Opinion Research (AAPOR) in Phoenix.
Why? Because their research challenges the beliefs of many AAPOR-ites who disregard most online research as being theoretically indefensible since it is not based on probability sampling. The research presented was based on parallel surveys conducted last year, designed to allow careful comparison of three survey modes: (more…)
Tags: bias, census, Data Collection, Internet, journalism, media, news, Online Surveys, phone surveys, public opinion, Public Polls, Public Relations, Sampling
Posted in Data Collection, Methods & Tools, Online Surveys, Public Polls, Public Relations, Sampling | No Comments »
Thursday, May 12th, 2011
We are often surprised by the number of senior researchers in the market research industry who never touch raw data. Often they don’t even have the tools, since “data processing” is outsourced to lower levels or other countries. It is surprising because we almost always engage in work where getting into the data and puzzling over anomalies or hypotheses yields much deeper insight.
Here is an example of how critical it can be to look closely at your data, and in this case, very early in the data collection process. We launched an online survey last week and got reports back from our sample supplier that incidence was just one-third of what we expected, which would have serious feasibility and cost implications.
But once we looked at their report portal, we saw that for every qualified respondent completing the survey, two qualified respondents quit before finishing. That’s an unusually high ratio of “suspends” as we call them. So what was the problem? Were we just getting lousy respondents who did not want to seriously participate in a survey? Was the survey was too difficult, tedious, boring, or confusing? One source of answers (rarely examined) is to look at the data question by question to identify where in the survey people are quitting.
The story in this data: Something is wrong with your survey
(more…)
Tags: data, Data Collection, data quality, insight, research, stories, survey, survey respondents
Posted in Charts and Data Visualization, Data Analysis & Analytics, Data Collection, Methods & Tools, Survey Design | No Comments »
Wednesday, May 4th, 2011
One reason that some people dislike surveys (okay, I may be projecting) is that too many surveys have the Me Generation attitude:
Enough about you, the customer, and what you need. What about ME? Do you like me? How much do you like me? Would you recommend me to your friend? Please let me know, because we need to track our satisfaction scores. It helps us build our metrics and our dashboards. And if you like me enough, I get a big bonus. Hurray for me!
Who are your customer satisfaction surveys really about?
(more…)
Tags: best practices, marketing, satisfaction research, survey respondents
Posted in Market Research, Methods & Tools, Survey Tips | No Comments »
Saturday, April 16th, 2011
Among the many sources of potential error that can affect surveys are respondents themselves. They sometimes misinterpret questions, respond in socially acceptable ways, or give “easy” answers in hopes that a more interesting question is just around the corner.
This is not to say they are bad or fraudulent respondents. Research shows that the vast majority of survey respondents are careful, thoughtful, and truthful in how they answer survey questions. The problem with respondent error, it turns out, is poor survey design, which may involve biased or ambiguous questions, tasks that are too complicated or boring, surveys that are too long, and so on.
Recent research shows that grid-style questions that look like this:
or this: (more…)
Tags: bias, Data Collection, data quality, Market Research, Online Surveys, Survey Design, survey respondents
Posted in Data Collection, Market Research, Methods & Tools, Online Surveys, Survey Design, Survey Tips | No Comments »
Friday, March 18th, 2011
Learning the math behind market research is not easy because there is no programmatic way to master it as a body of learning. It is not like algebra, geometry, calculus, or statistics in high school or college. It is complex and multifaceted and draws upon nearly every area of theoretical mathematics, but it must be continually adapted to the needs and practical problems of measuring and predicting customer behaviors and attitudes.
So it requires both (1) a rigorous foundation in mathematics and (2) years of experience to understand how it gets re-worked and applied to the real-life questions of market research. And even the rigorous foundation needs to be continually reinforced and expanded as the scope of our capabilities expands. Twenty years ago, who would have thought that Bayesian statistics and Monte Carlo simulations would become so central to our work? (more…)
Tags: Market Research, mathematics, statistics
Posted in Data Analysis & Analytics, Market Research, Methods & Tools, Survey Design | No Comments »
