The last couple of weeks we have been working with a group that specializes in coding and tabulating text responses to open-ended questions on surveys.  They have tools and technology that undoubtedly make the process easier and more efficient (we have used those tools, and they are impressive).  They are also have a singular focus and expertise that is supposed to help streamline the process, cut costs, and improve speed and efficiency.

The results have been mediocre at best, even with human coders working the technology and making the critical decisions. They efficiently and accurately coded each response into one or more  buckets.  But they created buckets that give hardly any insight into what the client wants to know.  “What do you love most about this product?”  The coders accurately identified all consumers who mentioned the physical size of the product.  But they lost critical distinctions not only about big versus small, but also about size being a constraint (not enough room for a larger product) versus a preference for how consumers wanted the product to look (an aesthetic choice).  They got the topic right, but did not answer the question in a meaningful way.  So what good was all that coding?

I asked a colleague (highly paid, less efficient) to fix and re-code the data, and I asked her to think not in terms of topics but in terms of answers to questions.  She did, and remarked, “You have an advantage, because you know more about this product and what is relevant to the research than the people (and machines) who did the coding.”

Yes, that’s the point, and that’s the struggle for technology and automation.  Smart researchers who know the right questions and continually think about how best to answer them with data will always have an advantage.  Even when pitted against the best and most efficient technologies, we will always win the insight contest.

There are lots of places where technology is helping us do our work faster, smarter, and at a lower cost.  But no matter what the innovators in tools and technology tell you, it makes a huge difference to have smart people with expertise slogging through the data, deciding how to analyze and present it, and transforming that into a story you can use.  We, at Versta Research, consistently and substantially outperform machines and outsourced labor, which means that you, the client, win as well.

—Joe Hopper, Ph.D.

Last week, theater critics roundly panned the show, calling it among the worst Broadway shows ever. The New York Times described is as “so grievously broken in every respect that it is beyond repair.”  But days after these negative reviews, a market research firm was brought in, soliciting volunteers to help fix the show:

If you love Broadway shows, we would like to invite you to be a test audience member and participate in either Act 1 (first part of show) or Act 2 (second part of show) of the all new Broadway musical “Spider-Man: Turn Off the Dark.”

We will be asking you to fill out a survey after the performance, plus stay for a 15 minute discussion in the Spider-Man V.I.P. room. In turn for your feedback and participation you will receive a Spider-Man goodie bag worth over $60.00, on us.

Can market research really rescue the show?  Possibly.  High quality research, both qualitative (such as focus groups) and quantitative (such as surveys) can provide deep insights into how customers and audiences think, perceive, react, and respond to products, services, and opportunities.

If market research can help rescue the show, here are two suggestions we offer that will be critical to keep in mind:

1.  Keep the focus groups focused. Too often clients or managers want their customers to tell them what would make them love their product.  That’s not something customers can reliably tell you.  They can tell you what they care about, what products they purchase and why, what needs and problems they are trying to solve, and what frustrations they face.  Our advice generally: Keep the focus group participants focused on the things that matter to them, then connect the insights gained to the products being offered.

2.  Ask only about things that can be fixed. For all the talk about “actionable” research, there is a great deal of research that offers supposedly actionable ideas that clients or managers simply cannot and will not act upon.  How does the audience feel about Bono’s music for the show?  Unless management is prepared to dump Bono or have him re-write (again), don’t ask.  The key is to determine during the research design which specific decisions can and cannot be made, and then to focus the research on collecting data that lends specific insight to those decisions.

Asking market research to help rescue a $65 million investment at the last minute seems like a tall order, but fortunately great research does not take super-human powers.  At Versta Research we rely on brains, experience, thoughtfulness, and a commitment to working closely with clients to ensure that we deliver insightful stories that are understood and acted upon.

So what will become of Spider-Man?  Will he die under the crushing weight of debt and harsh reviews?  Or will the critics groan in agony as Americans happily pay for the worst show on earth?  Stay tuned for the next exciting episode of “Can a Focus Group Save Spider-Man?”

—Joe Hopper, Ph.D.

Editing is key, however.  Who wants to read something in the halting, choppy, in-eloquent speech that most of us use in talking?  A killer quote is one that is short, direct, pithy, and on point.  Achieving this requires a journalistic standard for presenting quotes.  What does that mean?  Here are some steps:

• Transform the long run-on streams of words into sentences
• Remove the um’s, ah’s, like’s, you-know’s and other verbal fillers
• Cut out false starts and small tangents that do not contribute to intent of the sentence
• Do not change the words they use, or the order in which they use them

The goal is to focus on the intent and meaning of what they were saying.  Then clean it up so that it says exactly what they meant, and really did say.  As a check, imagine going back to your respondent a few minutes after they spoke and asking “Here’s what I’ve written down as what you said.  Is this right?”  If they would likely agree, then you’ve got it.

Here is an example from a project we completed this week:

Before: From what I’ve, you know, I’ve understood, that with Acme it’s very easy and very easy to navigate. That’s what the differences are between the companies is how easy it is to get in there and actually find what you’re looking for, and from what I’m . . . my feedback I’ve gotten about Acme, is that it is very easy and people feel comfortable getting in there and going through the process online.

After:  From what I’ve understood, Acme is very easy to navigate.  That’s the difference between companies: How easy it is to get in there and actually find what you are looking for.  That’s the feedback I’ve gotten about Acme.

If you are curious about whether our “journalistic standard” is appropriate, here’s a fascinating radio story that ran several years ago, which NPR re-ran this week in memory of John Solomon, the reporter who wrote it.

Radio interviews, it turns out, are similarly edited.  And according to NPR, they have never had a person interviewed on their show complain about how they were edited.

Need help grabbing your audience and telling a compelling research story?  Give us a call.  We focus on the rigors of research, and then turning data into stories so that your research gets heard and used, and so that it really makes a difference.

—Joe Hopper, Ph.D.

The Role of Social Media in Customer Satisfaction and Loyalty Research

A partial transcript of our panel’s deliberations was just published in the October 2010 issue of Marketing News, the AMA’s monthly magazine.  Here is a quick summary of key points highlighted in the article:

1. There is no “one” social media, but many channels with different users, objectives, and purposes for which customers post their comments.  Data need to be interpreted within the context of the channel from which that data comes.
2. Most social media is used for listening and interacting with customers as part of a customer service and customer relations effort, rather than for research per se.  But whatever its purpose, social media can offer rich data for mining insights, which can inform a well-rounded CS&L research effort.
3. Social media channels are unlikely to replace traditional modes of CS&L research, especially survey-based research using carefully drawn samples.
4. Triangulating data from social media with findings from other sources, including survey research, in-bound call data, purchase history, and so on, can help pinpoint the most critical customer satisfaction issues that a company needs to address.
5. Social media data can help pinpoint the range and meaning of issues that need to be explored with traditional quantitative research.
6. Social media can be mined for quotes and richness to provide a deeper and more compelling understanding of quantitative data.

One fascinating topic we discussed that was not covered in the article was the potential for network analysis informing CS&L research.  In my view, this is where a seismic shift in how we do research will happen.  Not everyone agreed.  In fact, the final question I posed to the panel was whether they see a seismic shift happening in CS&L research as a result of social media.  Here is my tally of the answers:

• Yes it will create a seismic shift – 2 votes
• No it will not create a seismic shift – 1 vote
• Maybe not seismic, but definitely some shake-ups – 2 votes
• Too soon to tell – 1 vote

Feel free to contact us at Versta Research if you would like us to send you a copy of the published article.  And of course give us a call if you need to discuss your CS&L research needs, including how to integrate new technologies and data streams into your analysis.

—Joe Hopper, Ph.D.

We were reminded of the importance of communication and writing research for multiple audiences from Eric Zorn’s recent column in the Chicago Tribune, from which we quote:

How many of us feel this way? I asked [the researcher] for percentages and she responded by e-mail that her research was qualitative, not quantitative.

I asked then in what sense is it “research” any more than choosing anecdotes out of random interviews? Journalists do love numbers.

“I don’t write my research for journalists,” she wrote back. “I write it for the community of scholars who conduct this type of research, and who have since the 1930s when the Chicago School of Sociology began at the University of Chicago. Honestly, if you are going to interact with researchers who conduct perfectly legitimate qualitative research, I strongly suggest that you gain some knowledge of that type of tradition before you throw around words like `random interviews’ and `anecdotes.’”

Ouch.

Mr. Zorn’s question was a good one. It could and should have been answered. It’s really no different from the kinds of questions you and we often hear from the executives to whom we are presenting our work. “How do you know?” “Why does this matter?” “How does this help me?” We nearly always have multiple audiences. Good researchers can and must communicate beyond their own communities.

We at Versta Research have done a great deal of research and survey work to support newspaper stories and communications materials, so we definitely do write for journalists. They love numbers, but they also love stories, so we give them headlines and storylines and numbers to support it all. At the same time, we write for communities of scholars and see our work published in academic journals. The two need not be at odds.

We also write for top level executives (we give them a three page deck focused on implications with supporting evidence) mid-level managers (we give them a ten page deck focused on problems, diagnoses, and solutions), and market researchers (we give them a thirty page deck and a clear story supported with data).

Each audience requires a slightly different approach, answers pitched at different levels, varying degrees of detail, and so on. Need help? Our clients say that our experience and skill at turning data into stories adds significant value to the work they do, all the way from design to implementation, reporting and presentation. Research can make a difference in back offices and boardrooms, and with the right reporting and follow up it can make a difference among journalists asking tough questions as well.

—Joe Hopper, Ph.D.

How do you bring the two together?  In our view, you bring them together by investing in people who are trained in both methods and who have deep experience in both.  The newsletter article outlines:

• Why it is best to be fluent in both
• The strengths and promises of each method
• The qualities and training needed to excel at both
• The false promise of technology
• Practical ideas for incorporating the strengths of each into a research effort

Ultimately, research is all about asking questions, finding information and data, thinking about it, probing deeper, assimilating and synthesizing it, and then turning data into stories so that the research gets heard and understood.  Whether you use qualitative or quantitative methods, your data needs to tell a compelling human (business) story, and the best stories draw upon both.

Give us a call, and we would be happy to share with you examples and stories of our work that have successfully bridged the gap.  In the meantime, take a look at our June 2010 Newsletter for a deeper understanding of our approach, and what you can expect from our people.

—Joe Hopper, Ph.D.

You need to find out the who, what, where, when, and why they hate you.  Probe deeply with focus groups, in-depth interviews, social media monitoring, and survey research.  Rigorous research done right (our specialty) will give you valuable insight.  And if it is communicated expertly to management (our specialty) you can probably fix the problems.  Then, the next time you listen to your customers, you won’t need to duck and cover.

-Joe Hopper, Ph.D.

Why would a market research firm use such a primitive tool for its own feedback?  Are we like the cobbler who can’t afford shoes for our own children?

No.  The reason is that our feedback survey is exceedingly simple.  We ask just two questions:  What was valuable about our work?  What do we need to improve?  (Actually, there is a third question – we’ll leave you in suspense until we complete our first engagement with you.)  We do not ask rating questions because we have no need to quantify the information, and we do not need to quantify because we do not have tens of thousands of clients.  All we need to know – from each client, personally and individually — is what we should keep doing, and what we need to fix.

Versta always uses and recommends the tools that get the job done most effectively.  While there are a lot of terrible do-it-yourself surveys on Survey Monkey, it can be a decent tool if you know how to use it and your needs are simple.

If you’re not sure about your needs, or would like some help with your own DIY project on Survey Monkey, give us a call and we would be happy to lend our expertise.

–Joe Hopper, Ph.D.

The reason?  Many clients have been using focus groups not for deep understanding of consumer needs and preferences, or to generate creative group dynamics among participants.  They were using them to get reactions and “votes” on package designs or product concepts.  Do you like the blue package, or the red package, and why?  Current technologies make it much easier and faster to answer simple questions like this through online surveys.  Since there is not a lot of research design or analysis involved, clients can do this work themselves.

There will always be a role for the expertise that a good focus group moderator or analyst offers, but helping clients pick between the red box and the blue box is not it.  When technologies make our work easier, faster, and cheaper, it makes sense that we bring key elements of this work back in house, while we work with partners and experts to tackle the tougher research challenges we face.

–Joe Hopper, Ph.D.