Earlier this month Arthur C. Nielsen, Jr. died.  He left behind a giant and reputable market research company and a brand name recognized throughout the world.  The A.C. Nielsen company was started by his father and in its early years tracked the sales of goods through grocery and drug stores.  The company then moved into media tracking and became the authoritative source for measuring audience size and demographics.  Nearly every company with an advertising budget continues to rely on Nielsen data to determine where to advertise and how much to spend.

Nielsen’s legacy is that he demonstrated the value of collecting and tracking data, and lots of it.  Every item we purchase is now logged, counted, and tracked.  Every television and radio show is tracked for how many viewers it has and in what markets they live.  And of course everything we do on the Internet is recorded and tracked.  Even our bodily locations are tracked via GPS or cell phone signals.  Most market research firms today generate the bulk of their revenue simply by collecting, tracking, tabulating, and reporting data.

This important legacy has left us with tons of data, growing at an exponential rate,  and a monumental challenge of how to synthesize it and move beyond mere tabulation and reporting.  The question is, how do we meet that challenge and take Nielsen’s legacy to the next frontier?  In our view, it will involve two key efforts:

1.  Understanding data in much deeper ways and analyzing it with data mining tools, new algorithms, and new approaches that go beyond traditional statistics, including Bayesian analysis, neural networks, and machine-learning techniques.

2. Interpreting and communicating data in ways that are more practical, relevant, meaningful, and useful.  In other words, turning data into stories that real people, real managers, and real businesses understand and can use.

To be sure, much of the research industry is still (appropriately) focused on implementing technologies to better manage, tabulate, and report volumes of data.  But automated tables and charts with ever-expanding levels of detail are reaching their limits of utility.  Versta Research is proud to be on the next frontier, where better interpretation and understanding of data is key.

–Joe Hopper, Ph.D.

The study suggests that entrepreneurs are more focused on immediate and practical questions that will help them get their products into the hands of customers, and that traditional market research may not be the best way to get the right data and answers.  That makes sense.

But according to an article in the February issue of Inc. magazine, “when asked what kind of market research they would conduct for [a] hypothetical start-up, most of Sarasvathy’s subjects responded with variations on the following:

“OK, I need to know which of their various groups of students, trainees, and individuals would be most interested so I can target the audience a little bit more. What other information… I’ve never done consumer marketing, so I don’t really know. I think probably…I think mostly I’d just try to…I would…I wouldn’t do all this, actually. I’d just go sell it. I don’t believe in market research. Somebody once told me the only thing you need is a customer. Instead of asking all the questions, I’d try and make some sales. I’d learn a lot, you know: which people, what were the obstacles, what were the questions, which prices work better. Even before I started production. So my market research would actually be hands-on actual selling.”

We heartily agree that sometimes you should get out there and sell rather than conducting more market research (see our article What You May Need Is Marketing, Not Market Research).  But the problem with taking this quote at face value is that a really good market researcher would never say “How would you use market research?”  She would say, “What do you need to know?   What answers to questions would help you achieve your most critical business objectives?” (See our article The Art of Asking Questions.)  Then she would decide whether and how market research can best be used.

At Versta Research we believe that no matter what size company you are, you should be thinking about research more like an entrepreneur.  Great entrepreneurs are using, gathering, analyzing, and interpreting data all the time to help them make decisions.  But their data might be coming from reports on sales calls rather than standard satisfaction or tracking surveys or another new product benchmarking study.  That is a good thing, and a smarter way to approach research even if you are an established firm with a substantial research budget.

So when somebody offers you the shiny new market research tool which is now the “best practice” or the “benchmarked metric,” set it aside.  Instead, outline your questions.  Describe the data and information that would help you achieve your most critical business objectives.  Ask whether research is the most efficient and insightful tool to deliver answers.

If you need help with that process, we are happy to advise.

—Joe Hopper, Ph.D.

But NPS questions do not make sense in many situations.  Here’s one we saw last week—it’s a survey sent by Amazon to sellers who call regarding complicated issues with how their products are being displayed on the website or how payments are being transferred:

A Poor Use of the Net Promoter Question

The problem with this question is that any honest respondent will almost certainly answer “very unlikely” and give a score of zero.  The support representatives are anonymous, so it is impossible to recommend them.  If you call or email Amazon, you cannot request a specific representative, making the issue moot.  And even if you could recommend a specific representative, how often would you have the opportunity to make such a recommendation?  Few sellers talk to other sellers about the display and payment problems they are having.  So all in all, how likely do you think most sellers are to recommend the specific person they talked with?  If we have any faith in the reliability and validity of survey data, it should be about zero.

The hapless support representative likely got a lousy score (there goes her bonus!) not because she did a poor job, but because whoever is in charge of this tracking survey didn’t think very hard about NPS, how to use it and when to use it.  More generally, somebody didn’t think very hard about how to do smart research.

Staying on top of recent innovations, important industry trends, and best practices is critical.  We can help you with that, with suggestions about the right ways to apply them so that you get valid insights.  Your customers and your service reps will thank you.  And the next time your manager and executive team rate you, they will likely recommend you for the next important research project or satisfaction tracking study (score 9 or 10).

—Joe Hopper, Ph.D.

The cartoon brings to mind three lessons worth pondering:

1. Customer satisfaction research is often “not fun”—but it can be
2. Internal data can be a goldmine of insight and there is often a lot of it lying around
3. Ethical considerations dictate that just because research can be done does not mean it should be done

1. Customer satisfaction research is often “not fun.” Why?  In our view it is because so many customer satisfaction surveys are done for the wrong reasons, focus on the wrong issues, and ask about the wrong people.  What could be more boring—and more unenlightening—than tracking the percentage of customers who like (or dislike) you month after month and year after year?

To make customer satisfaction research far more interesting and useful, focus not on your products and not on you, but instead on your customers, what they need, and what will delight them.  Then assemble a team that is committed to thinking hard and adding value to every report they deliver.  Tracking studies can be fun if focused on the challenge of consistently providing an insightful story.

2. Internal data can be a goldmine of insight.  Organizations often send out survey after survey without ever taking stock of what they have already learned, and without considering the wealth of data they have internally.  Yet, the volume of useful, untapped internal data that most organizations have is astonishing.  You can learn a great deal about your customers and satisfaction, for example, simply by analyzing historical data.  Who are you losing or gaining as customers?  What segments, industries, or geographies characterize them?  You don’t always need surveys to answer these questions.

3. Ethical approaches to research are essential.  Data should never be collected and archived without the people supplying that data knowing about it and knowing how it will be used.  This is a fundamental ethical requirement all social scientific research, and it extends to all market research and public opinion polling as well.  We recommend that research teams adhere strictly to the strictest standards of ethics and privacy as outlined by The Council of American Survey Research Organizations (CASRO) and the American Association of Public Opinion Research (AAPOR).

Need help on any or all of these fronts?  Definitely stay away from Mordac the Preventer of Information Services, and give us a call instead.

—Joe Hopper, Ph.D.

The Role of Social Media in Customer Satisfaction and Loyalty Research

A partial transcript of our panel’s deliberations was just published in the October 2010 issue of Marketing News, the AMA’s monthly magazine.  Here is a quick summary of key points highlighted in the article:

1. There is no “one” social media, but many channels with different users, objectives, and purposes for which customers post their comments.  Data need to be interpreted within the context of the channel from which that data comes.
2. Most social media is used for listening and interacting with customers as part of a customer service and customer relations effort, rather than for research per se.  But whatever its purpose, social media can offer rich data for mining insights, which can inform a well-rounded CS&L research effort.
3. Social media channels are unlikely to replace traditional modes of CS&L research, especially survey-based research using carefully drawn samples.
4. Triangulating data from social media with findings from other sources, including survey research, in-bound call data, purchase history, and so on, can help pinpoint the most critical customer satisfaction issues that a company needs to address.
5. Social media data can help pinpoint the range and meaning of issues that need to be explored with traditional quantitative research.
6. Social media can be mined for quotes and richness to provide a deeper and more compelling understanding of quantitative data.

One fascinating topic we discussed that was not covered in the article was the potential for network analysis informing CS&L research.  In my view, this is where a seismic shift in how we do research will happen.  Not everyone agreed.  In fact, the final question I posed to the panel was whether they see a seismic shift happening in CS&L research as a result of social media.  Here is my tally of the answers:

• Yes it will create a seismic shift – 2 votes
• No it will not create a seismic shift – 1 vote
• Maybe not seismic, but definitely some shake-ups – 2 votes
• Too soon to tell – 1 vote

Feel free to contact us at Versta Research if you would like us to send you a copy of the published article.  And of course give us a call if you need to discuss your CS&L research needs, including how to integrate new technologies and data streams into your analysis.

—Joe Hopper, Ph.D.

Today, an industry colleague (another owner of a market research firm) said to me, “We all lust after those big tracking studies.”  For most market research firms, tracking studies are attractive because they involve big samples, multiple ongoing deliverables, and multi-year commitments, all of which means predictable, ongoing (and usually substantial) revenue.

At Versta Research, we lust after them for a different reason: they are the true test of whether our key personnel can really add value and insight to the work that we do.  How do you take something routine and predictable, and turn it into an effort that delivers an “Aha” every week?  Here’s how we do it:

• We find smart, methodologically-rigorous, and business-savvy associates to run the study instead of “capable” research people who are tracking-study-care-takers.

• We clear off our desks and close our doors and examine the data for new angles and insights with every deliverable.

• We look at the data collection process for ways to streamline so that our time is spent on thinking rather than compiling data and putting numbers into tables and charts.

• We advise you on whether the money spent is worth the insight gained, and if collecting and tabulating data is your only goal, whether there are better ways to automate and/or bring the process in-house.

Too often the smartest people avoid tracking studies because they can be boring.  What you need, however, is smart people who are committed and passionate about adding significant value to every project they do for you.  There is nothing boring about finding the story in your data and helping you communicate it to your managers or internal clients.  That is the challenge of tracking studies that we love.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/of-lust-and-tracking-studies/feed/ 0 http://www.verstaresearch.com/blog/don%e2%80%99t-do-research-in-your-sleep/ http://www.verstaresearch.com/blog/don%e2%80%99t-do-research-in-your-sleep/#comments Mon, 03 May 2010 00:41:22 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=482 A colleague in market research once complained to me that he felt bored and unchallenged by all the client satisfaction and loyalty research he was doing, claiming he had mastered it to the point that he could do satisfaction and loyalty research in his sleep. I was struck because I could not think of any market research that I found boring or unchallenging, and certainly none that I could do in my sleep.  On the contrary, my experience is that doing great research requires intellectual work and waking thoughtfulness no matter how many times it is done and for how many clients.

Contrast this with a client for whom we were generating weekly data reports from a tracking study who said that her directive (and hence our directive) was that “nothing goes to my internal clients without insight.”  And so every week we cleared off our desks, closed the door, and examined the data for new angles and insights.  If we were unable to come up with a compelling, non-boring, challenging story, then surely our client’s client would not find it either.  And if nobody can find a compelling story in the data, why is the client spending money on it, and why are we being paid to do it?

In our view, high quality market research requires smart, thinking people who are:

• Curious about your problems
• Passionate about getting the right data to answer the right questions
• Diligent and detailed in turning that data into stories that give you insight

Nobody can do these things in their sleep.  The next time your research partner starts nodding off mid-sentence, “Why, yes, we have done this kind of work so many times we can do this project in our sleep . . .” take another look at the work they are doing for you and ask yourself whether everything they do delivers thoughtful insights you can use.  If it doesn’t, give us a call for a different perspective on how to do research.

—Joe Hopper, Ph.D.