In last year’s cliffhanger episode of “Can a Focus Group Save Spider-Man?” we pondered whether market research was powerful enough to save a Broadway show from doom and destruction.  After crushing reviews from theater critics, the producers hired a market research firm to help them rewrite the show.

Guess what?  It worked.  Since the show re-opened in June, it is regularly among the top five earners among Broadway shows, and The Hollywood Reporter reports that in December it “set a new record for a single-week box office gross, raking in $2,941,794 for the week ending January 1.”

In a year when Steve Jobs’ quip that “it’s not the consumers’ job to know what they want” has been used to bludgeon market research, we see research doing a pretty good job identifying buyers’ needs and helping decision makers address those needs in smarter ways.

It is true that consumers can’t answer questions like, “What show do want us to produce next?” or “What great technological innovation would be most useful to you?”  But they certainly can tell you want they want, and the art of research is to ask just the right questions to give you enough insight about what to do next.  Here are three things you can ask about, and that audiences, customers, and prospects will happily tell you:

1.  What is important to them and what they care about.  Insights about what matters to buyers will help you design your product or service and will provide deep insight about how to market it.

2.  What their frustrations are and what is not working.  Most consumers are eager to critique products and services that fail to meet their needs, which highlights the opportunities for new solutions.

3.  How good or bad your idea is.  There are lots of ways to test concepts, products, and ideas, and survey research is remarkably good at predicting winners versus losers.

The lesson from Spider-Man is that it does not take superhuman powers to fix a flawed strategy nor off-the-charts creative genius to make a good product sell.  A thoughtful effort to ask questions and to listen to your customers is sometimes all it takes to turn harrowing encounters with goblins into success.

–Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/focus-groups-save-spider-man/feed/ 0 http://www.verstaresearch.com/blog/using-avatars-robots-for-survey-research/ http://www.verstaresearch.com/blog/using-avatars-robots-for-survey-research/#comments Thu, 22 Sep 2011 18:13:13 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=1609

Two researchers at the U.S. Census Bureau recently outlined an emerging innovation in survey research that could reverse the trend towards passive, boring, self-administered surveys that characterizes much online research.  The idea is to use internet avatars in real-time interviewing with survey respondents.

Beyond just the heightened interest of having an animated survey, the avatars would be programmed to register and interpret respondents’ verbal answers, facial expressions, and body language through webcams.

Suppose, for example, that a respondent answers a question with detailed information that answers a follow-up question as well.  The avatar would use natural language processing to insert that data into the subsequent question, and then avoid asking the follow-up.  Or if the respondent looks away from the screen and pauses for time longer than is typical, the avatar can offer a rephrased question or a reassuring comment to re-engage the participant and to put him or her at ease.  This type of innovation  could bring many of the advantages of live interviewing back into the realm of internet surveys, which are far more efficient in terms of time and cost.

The use of effective avatar interviewers is at least several years away, however, because it involves not only evolving internet technologies, but also advanced linguistic processing, facial and voice recognition technologies, and so on.  In fact, the sheer technological difficulty of truly replacing human interviewers reminds us of how absurd it is for research companies to make claims about technology replacing higher-order activities in the research process, such as providing analysis and insight.

At least for now, software and services with “actionable insight” buttons generate yet more mountains of data in need of human synthesis and interpretation.  If anything, the role for smart and experienced researchers who can turn all that data into a story is growing.  It is growing for researchers who work on the client side and who have direct accountability to the executives who need data-driven insights.  And it is growing for firms like Versta Research where the highest levels of intellectual and human capital are central to our work.

–Joe Hopper, Ph.D.

One way to analyze the data is to use a simple counting technique.  It works pretty well, by the way.  But the more common way these days is to use advanced statistical modeling that allows for stronger individual-level estimates of importance scores.  This affords better opportunities for segmenting the data and analyzing subgroups.

We typically see analysts transforming the importance scores (or utilities) to a 0—100 scale.  It is a compelling scale because all importance scores add up to 100, which mimics the technique of having respondents allocate 100 importance points to each of the features they care about.

But if you do a lot of MaxDiff studies and frequently present them to your management team, you lose the ability to provide a rule of thumb as to what counts as a “high” score vs. a “low” score.  If one MaxDiff exercise uses fifteen attributes, an importance score of 20 is quite high; if another MaxDiff exercise uses just five attributes, and importance score of 20 is just average.

The solution?  Instead, we typically transform scores to a 0—100*N scale, where N=the number of attributes tested.  If you tested fifteen attributes, transform to a 0—1500 scale.  If you tested five attributes, transform to a 0—500 scale.  With this method, a score of 100 is always the average, scores above 100 are always above average, and scores below 100 are always below average.  Sometimes we transform this into a chart showing “percent above/below  average” as shown in the third example below, but this is not always necessary because anchoring the average at 100 makes it simple to calculate those percentages mentally.

Example 1: Versta's Recommended Scaling of MaxDiff Scores

Example 2: The Typical Way of Scaling MaxDiff Scores

Example 3: Another (Sometimes Useful) Way to Scale MaxDiff Scores

Need help with the smartest ways to design, implement, and report MaxDiff studies?  Call us at (312) 348-6089 for assistance and further information.

–Joe Hopper, Ph.D.

one shortcut is to become reckless: to act impulsively instead of expending the energy to first think through the consequences. . . .  The other shortcut is the ultimate energy saver: do nothing. Instead of agonizing over decisions, avoid any choice.

But give the brain a hit of glucose (the basic fuel that runs cell functioning), and our willpower and rational decision-making are restored.

The findings are from multiple experiments over the past decade that relied on a variety of scenarios that both academic researchers and marketing people care a great deal about: selecting (and paying for) options on new car purchases, buying computers, shopping in malls or grocery stores, selecting fabrics for customized products, and making critical financial decisions that involve trade-offs between short-term rewards and long-term gains.

The findings also have implications for all kinds of marketing decisions and strategies.  How many choices should consumers be offered?  How difficult should those choices be?  Should you offer a default or recommended option?  Where in the decision process should the recommended option be offered?

One area in which Versta Research has done a great deal of work is in consumer attitudes and behaviors when it comes to saving, spending, and investing, particularly for retirement.  We all know that few consumers are saving enough.  Most consumers know this, too, but they struggle to make the most effective decisions.  Bombarding them with education and tools seems to go only so far.

Fortunately, many financial services companies are doing a better and better job helping people save and invest more wisely.  In part, they are doing it by taking advantage of our proclivities to “do nothing” when fatigued by choice, though nobody has understood until now why this works.  The recent research cited here offers new insights that may help configure better choices and contexts for financial decision making, in part because it suggests an intriguing link to brain physiology:

Apparently ego depletion causes activity to rise in some parts of the brain and to decline in others. Your brain does not stop working when glucose is low. It stops doing some things and starts doing others. It responds more strongly to immediate rewards and pays less attention to long-term prospects.

So before you are faced with the task of configuring your 401k or making other long-term financial and investing decisions, here are a few tips at least suggested by the research cited here: Be well rested.  Set aside time early in the day when your decision-making abilities are fresh.  Avoid times right after you have made other significant choices or decisions, either at work, when shopping, or in your personal life.  And the best suggestion of all?  Have a cookie with your 401(k).

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/have-a-cookie-with-your-401k/feed/ 0 http://www.verstaresearch.com/blog/top-5-picks-best-articles-on-market-research/ http://www.verstaresearch.com/blog/top-5-picks-best-articles-on-market-research/#comments Thu, 07 Jul 2011 18:59:16 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=1484 Versta Research just hit a magic number: 100.  That’s the number of articles we have written to help our clients and their colleagues keep abreast of important trends in market research.  If your market research supplier is not providing ongoing thought leadership in design, methods, and analytics, then what are the chances they are bringing ongoing and deep insight to your specific research needs?

To celebrate, we’re serving up a sampler of our five best articles.  How did we decide they are the best?  Our clients told us.  These are the articles that they write to us about, forward to their colleagues, and for which they return to our website time and again.  These are also the articles for which we get requests for print-ready PDF versions.  (Just let us know if you want one!)

1. Turning Data into Stories

There are two critical elements to top notch research.  First, it has to be right, which means focusing on the rigors of research design, data collection, and statistical analysis.  Second, it has to be heard, understood, and used, and in our view that means turning data into stories.  In this article we focus on what it means to turn data into stories, and we outline what you gain by doing so.

2. The Art of Asking Questions

Somewhere along the way to research becoming central to how businesses learn about their customers, the art of asking questions was lost. As a result, there is a lot of research for research’s sake, data in search of answers, and findings in search of questions.  The first thing you should do before starting research is figure out what question your research must answer.  Here’s how.

3. Magic Numbers in Market Research

Researchers cling to a handful of “magic numbers” that guide the decisions they make.  There are magic numbers for sample size, the optimal number of points on a scale, thresholds for statistical significance, and how big a focus group should be.  This article demonstrates and explains.

4. The ABCs of CBC: Understanding Conjoint for Market Research

This article focuses on the basic ideas, advantages, and uses of conjoint research. What is conjoint? How and why is it used? What insights can it give you? Furthermore, what are some of the pros and cons of fielding research using a conjoint method vs. other methods you might use?

5. Game Changing Product Innovation

A lot of research supporting new product development is a like machine that ends up creating NON-innovation because of over-benchmarking.  We suggest an alternative.

We write these articles because even though we are in the business of doing rigorous research for our clients, research only matters if it is thoughtfully communicated, understood, and used.  We hope that our efforts help your organization better design and deploy research to make smarter business decisions.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/top-5-picks-best-articles-on-market-research/feed/ 0 http://www.verstaresearch.com/blog/fifteen-basics-of-brand-smart-research/ http://www.verstaresearch.com/blog/fifteen-basics-of-brand-smart-research/#comments Sat, 25 Jun 2011 22:49:45 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=1451 This past week the American Marketing Association in Chicago held its 2011 annual BrandSmart conference, bringing together top-level marketers from companies such as Groupon, Motorola, Allscripts, Cars.com, Deloitte, Coldwell Banker, Accenture, Hospira, Walgreens, Blue Cross Blue Shield, Grainger, Morningstar, and many others, all of whom shared the newest strategies and case studies for brand building and successful marketing.

As director of the market research group within the AMA I was surprised that many of my research colleagues at other firms did not attend ostensibly because there were no sessions devoted specifically to brand research.  It was surprising because the other marketing professionals who were attending never doubted for a moment the essential role of research in effective brand marketing.  Indeed, important lessons about the role of research permeated every presentation.

Out of 36 Branding Basics outlined in the conference program, fifteen were clearly about research even though none actually used the word research.  These specific Branding Basics were:

1. Understand the totality of your brand image
2. Understand the likes and dislikes of the customer
3. Understand the associations people draw from your brand
4. Understand what your product stands for with the customer
5. Understand the real desire of your customer
6. Understand the value of the customer
7. Study why a customer buys your product
8. Assess if your brand appeals to customers
9. Understand as a user the imagery of your brand
10. What does your brand personify?
11. Understand the customer’s perception of price
12. Understand the value created for the customer
13. Identify the correct vision and relevant message
14. Understand how the brand compares to its competitors
15. Measure brand equity

In fact, one thing I appreciate about this list is that the word research never appears.  The focus is on understanding (the word from which Versta Research derives its name) and on answering questions.  In our experience, when marketers are focused on the questions they need to have answered rather than saying “We need research on X” or “We need to survey our customers about Y,” the research outcomes are far more useful.

Market researchers should be immersing themselves in conversations like yesterday’s BrandSmart conference. While we need to build deep expertise in the methods, minutia, innovations and trends in research itself, we also need deep immersion in the world of marketing.  This is how and where our research will get used, and it will ensure that we’re doing smart research to help our clients be Brand Smart.

—Joe Hopper, Ph.D.

The study suggests that entrepreneurs are more focused on immediate and practical questions that will help them get their products into the hands of customers, and that traditional market research may not be the best way to get the right data and answers.  That makes sense.

But according to an article in the February issue of Inc. magazine, “when asked what kind of market research they would conduct for [a] hypothetical start-up, most of Sarasvathy’s subjects responded with variations on the following:

“OK, I need to know which of their various groups of students, trainees, and individuals would be most interested so I can target the audience a little bit more. What other information… I’ve never done consumer marketing, so I don’t really know. I think probably…I think mostly I’d just try to…I would…I wouldn’t do all this, actually. I’d just go sell it. I don’t believe in market research. Somebody once told me the only thing you need is a customer. Instead of asking all the questions, I’d try and make some sales. I’d learn a lot, you know: which people, what were the obstacles, what were the questions, which prices work better. Even before I started production. So my market research would actually be hands-on actual selling.”

We heartily agree that sometimes you should get out there and sell rather than conducting more market research (see our article What You May Need Is Marketing, Not Market Research).  But the problem with taking this quote at face value is that a really good market researcher would never say “How would you use market research?”  She would say, “What do you need to know?   What answers to questions would help you achieve your most critical business objectives?” (See our article The Art of Asking Questions.)  Then she would decide whether and how market research can best be used.

At Versta Research we believe that no matter what size company you are, you should be thinking about research more like an entrepreneur.  Great entrepreneurs are using, gathering, analyzing, and interpreting data all the time to help them make decisions.  But their data might be coming from reports on sales calls rather than standard satisfaction or tracking surveys or another new product benchmarking study.  That is a good thing, and a smarter way to approach research even if you are an established firm with a substantial research budget.

So when somebody offers you the shiny new market research tool which is now the “best practice” or the “benchmarked metric,” set it aside.  Instead, outline your questions.  Describe the data and information that would help you achieve your most critical business objectives.  Ask whether research is the most efficient and insightful tool to deliver answers.

If you need help with that process, we are happy to advise.

—Joe Hopper, Ph.D.

How Many Choices Are Too Many?

But there have been a number of studies and a good deal of retail research documenting the opposite as well.  Indeed, it makes sense to think that offering more choice will maximize the chance of meeting individual consumers’ needs.

A recent article published in the Journal of Consumer Research suggests that idea of choice overload may be overblown.  The authors analyzed results from fifty published and unpublished experiments on the topic.  They concluded:

Although strong instances of choice overload have been reported in the past, direct replications and the results of our meta-analysis indicated that adverse effects due to an increase in the number of choice options are not very robust: The overall effect size in the meta-analysis was virtually zero. While the distribution of effect sizes could not be explained solely by chance, presumably much of the variance between studies was due to a few experiments reporting large positive and large negative effect sizes. The metaanalysis further confirmed that “more choice is better” with regard to consumption quantity and if decision makers had well-defined preferences prior to choice. There was also a slight publication bias such that unpublished and more recent experiments were somewhat less likely to support the  choice overload hypothesis.

So the evidence that too many choices hurts sales is generally weak.  Moreover, there appear to be two well-established situations in which more choice is clearly better:

(1) Consumers are more likely to buy if they have more choices about how much to buy (size, quantity, etc.)

(2) Consumers who already know what they want are more likely to buy the more choices they have.

If you are struggling with the question of “How many choices?” the article is worth reading because it outlines the many dimensions and factors that affect decision-making.  It is also worth calling Versta Research because we keep current on the latest scholarship, and because are eager to help you think about the smartest approaches to answering your questions.

—Joe Hopper, Ph.D.

Last week, theater critics roundly panned the show, calling it among the worst Broadway shows ever. The New York Times described is as “so grievously broken in every respect that it is beyond repair.”  But days after these negative reviews, a market research firm was brought in, soliciting volunteers to help fix the show:

If you love Broadway shows, we would like to invite you to be a test audience member and participate in either Act 1 (first part of show) or Act 2 (second part of show) of the all new Broadway musical “Spider-Man: Turn Off the Dark.”

We will be asking you to fill out a survey after the performance, plus stay for a 15 minute discussion in the Spider-Man V.I.P. room. In turn for your feedback and participation you will receive a Spider-Man goodie bag worth over $60.00, on us.

Can market research really rescue the show?  Possibly.  High quality research, both qualitative (such as focus groups) and quantitative (such as surveys) can provide deep insights into how customers and audiences think, perceive, react, and respond to products, services, and opportunities.

If market research can help rescue the show, here are two suggestions we offer that will be critical to keep in mind:

1.  Keep the focus groups focused. Too often clients or managers want their customers to tell them what would make them love their product.  That’s not something customers can reliably tell you.  They can tell you what they care about, what products they purchase and why, what needs and problems they are trying to solve, and what frustrations they face.  Our advice generally: Keep the focus group participants focused on the things that matter to them, then connect the insights gained to the products being offered.

2.  Ask only about things that can be fixed. For all the talk about “actionable” research, there is a great deal of research that offers supposedly actionable ideas that clients or managers simply cannot and will not act upon.  How does the audience feel about Bono’s music for the show?  Unless management is prepared to dump Bono or have him re-write (again), don’t ask.  The key is to determine during the research design which specific decisions can and cannot be made, and then to focus the research on collecting data that lends specific insight to those decisions.

Asking market research to help rescue a $65 million investment at the last minute seems like a tall order, but fortunately great research does not take super-human powers.  At Versta Research we rely on brains, experience, thoughtfulness, and a commitment to working closely with clients to ensure that we deliver insightful stories that are understood and acted upon.

So what will become of Spider-Man?  Will he die under the crushing weight of debt and harsh reviews?  Or will the critics groan in agony as Americans happily pay for the worst show on earth?  Stay tuned for the next exciting episode of “Can a Focus Group Save Spider-Man?”

—Joe Hopper, Ph.D.

The Role of Social Media in Customer Satisfaction and Loyalty Research

A partial transcript of our panel’s deliberations was just published in the October 2010 issue of Marketing News, the AMA’s monthly magazine.  Here is a quick summary of key points highlighted in the article:

1. There is no “one” social media, but many channels with different users, objectives, and purposes for which customers post their comments.  Data need to be interpreted within the context of the channel from which that data comes.
2. Most social media is used for listening and interacting with customers as part of a customer service and customer relations effort, rather than for research per se.  But whatever its purpose, social media can offer rich data for mining insights, which can inform a well-rounded CS&L research effort.
3. Social media channels are unlikely to replace traditional modes of CS&L research, especially survey-based research using carefully drawn samples.
4. Triangulating data from social media with findings from other sources, including survey research, in-bound call data, purchase history, and so on, can help pinpoint the most critical customer satisfaction issues that a company needs to address.
5. Social media data can help pinpoint the range and meaning of issues that need to be explored with traditional quantitative research.
6. Social media can be mined for quotes and richness to provide a deeper and more compelling understanding of quantitative data.

One fascinating topic we discussed that was not covered in the article was the potential for network analysis informing CS&L research.  In my view, this is where a seismic shift in how we do research will happen.  Not everyone agreed.  In fact, the final question I posed to the panel was whether they see a seismic shift happening in CS&L research as a result of social media.  Here is my tally of the answers:

• Yes it will create a seismic shift – 2 votes
• No it will not create a seismic shift – 1 vote
• Maybe not seismic, but definitely some shake-ups – 2 votes
• Too soon to tell – 1 vote

Feel free to contact us at Versta Research if you would like us to send you a copy of the published article.  And of course give us a call if you need to discuss your CS&L research needs, including how to integrate new technologies and data streams into your analysis.

—Joe Hopper, Ph.D.