One of the fascinating insights to emerge from our discussion was that social media is not only a new channel of information and data, but that it is fundamentally different from previous channels of data. As such, it opens up new areas of inquiry for our efforts. What is that fundamental difference? It is the networked nature of social media. As we code and tabulate people’s comments on social media as part of our CS&L research, potentially we have access to the networks of each person whose comments we are analyzing. We can know how many people are reading each comment, we can measure how strong and extensive the network of influence is, where it overlaps important segments of customers, and so on.

The implications of this are huge. The impact of loyalty among one’s best customers can now be defined not only in terms of how much they buy and how “sticky” they are, but also in terms of their influence among other customers and prospects. Smart CS&L research will not count everyone’s opinion the same, but will give more weight to those occupying central nodes of critical networks. There are implications for sampling as well. Is true random sampling necessary, or can “networked” sampling provided sufficient entry points that give visibility into the full population of customers?

In my view, this sort of network analysis will bring about a seismic shift in CS&L research, though not all panelists agreed. We were split about 50/50 on whether social media constitutes just one more channel of data to integrate, bringing greater precision to what we’ve always done, or whether it represents a more radical departure. A full report of our panel’s deliberations will be presented in the October 2010 issue of Marketing News, the AMA’s monthly magazine.

The stimulating discussion among panelists also highlighted the importance of remembering that good CS&L research requires ongoing thoughtfulness, intelligence, and curiosity. New tools and technologies are often easy to install, but their value is in offering new opportunities to bring deeper understanding and analysis to research.

Stay tuned – we’ll provide a closer look at some of those opportunities for better research (and reprints upon request) when the AMA publishes its report this fall.

—Joe Hopper, Ph.D.

For those of us at Versta Research, he is well known for his focus on understanding versus research:

“Basically, I’m not interested in doing research and I never have been,” he said. “I’m interested in understanding, which is quite a different thing. And often to understand something you have to work it out yourself because no one else has done it.”  (From an interview cited in the New York Times)

Why are we inspired by this?  Because even though we are interested in doing research, we do it for one reason:  To understand things.  In fact, we named our company from the social science concept (and German word) “verstehen” which means to understand.

Fundamentally, we define our work not by proprietary methods, or the specific tools that we use or develop, or the statistical procedures we implement.  We define it by our interest in and approach to understanding.  And that, of course, means using, learning, developing, and inventing whatever tools, processes, data collection techniques, and analysis that will get good answers.  In short, to achieve understanding, we do research.

Understanding is the outcome.  That’s what we really care about (and what you really care about) and what motivates our work.  Research is the means to that end.

If you have not done so, take a look at the homepage of our website.  It’s all about understanding.  We are committed to helping you understand:

• Your clients
• Your prospects
• The public
• Consumers
• Your competition
• Your customers
• Your data
• Your products
• Your image
• The world

If and when research can help you understand, we are here to help.

—Joe Hopper, Ph.D.

We were reminded of the importance of communication and writing research for multiple audiences from Eric Zorn’s recent column in the Chicago Tribune, from which we quote:

How many of us feel this way? I asked [the researcher] for percentages and she responded by e-mail that her research was qualitative, not quantitative.

I asked then in what sense is it “research” any more than choosing anecdotes out of random interviews? Journalists do love numbers.

“I don’t write my research for journalists,” she wrote back. “I write it for the community of scholars who conduct this type of research, and who have since the 1930s when the Chicago School of Sociology began at the University of Chicago. Honestly, if you are going to interact with researchers who conduct perfectly legitimate qualitative research, I strongly suggest that you gain some knowledge of that type of tradition before you throw around words like `random interviews’ and `anecdotes.’”

Ouch.

Mr. Zorn’s question was a good one. It could and should have been answered. It’s really no different from the kinds of questions you and we often hear from the executives to whom we are presenting our work. “How do you know?” “Why does this matter?” “How does this help me?” We nearly always have multiple audiences. Good researchers can and must communicate beyond their own communities.

We at Versta Research have done a great deal of research and survey work to support newspaper stories and communications materials, so we definitely do write for journalists. They love numbers, but they also love stories, so we give them headlines and storylines and numbers to support it all. At the same time, we write for communities of scholars and see our work published in academic journals. The two need not be at odds.

We also write for top level executives (we give them a three page deck focused on implications with supporting evidence) mid-level managers (we give them a ten page deck focused on problems, diagnoses, and solutions), and market researchers (we give them a thirty page deck and a clear story supported with data).

Each audience requires a slightly different approach, answers pitched at different levels, varying degrees of detail, and so on. Need help? Our clients say that our experience and skill at turning data into stories adds significant value to the work they do, all the way from design to implementation, reporting and presentation. Research can make a difference in back offices and boardrooms, and with the right reporting and follow up it can make a difference among journalists asking tough questions as well.

—Joe Hopper, Ph.D.

We highly recommend learning about theories of presentation, perception, and data visualization, and we are big fans of Edward Tufte’s approach to visual explanations.  At the same time, we recommend learning the basics of using simple tools, like pie charts, bar charts, line graphs, and so on.  One good source for developing a mastery of the basics is a book called Graphing Statistics & Data: Creating Better Charts, from which we have learned a few tips about using bar charts:

• Use bar charts to show variables with distinct (non-continuous) values
• Bar charts are good at showing both proportion and quantity (unlike pie charts, which are good at showing proportions only)
• Adjust the chart settings so that the bars are wider than the gaps between the bars.  They need to be wide enough to invite visual comparisons among them, but not so wide that they resemble a histogram (see the examples below)
• If your variable has many values you want to show, or if labeling the values requires a lot of text, use a horizontal bar chart
• For horizontal bar charts, rank order the bars so that long bars (high values) are at the top of the chart, and short bars (low values) are at the bottom

This bar chart is flawed because the bars are too skinny relative to the spaces between them. They do not invite visual comparison.

The bar chart is flawed because the spaces between the bars are too narrow. It makes the chart look like a histogram, which is appropriate only for continuous variables.

This bar chart is just right.

As most of us know from having tried to disentangle one too many indecipherable charts in murky research reports, these basics seem easy, but they rarely are.  They require skill, art, and expertise developed through years of day-to-day efforts to understand, synthesize, and communicate data.  When done well, good charts tell the story of the data.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/visualizing-data-five-tips-to-using-a-bar-chart/feed/ 0 http://www.verstaresearch.com/blog/using-mobile-technologies-in-research/ http://www.verstaresearch.com/blog/using-mobile-technologies-in-research/#comments Fri, 02 Jul 2010 13:17:39 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=650

One critical trend that is transforming the research industry is the rapid adoption of mobile technologies, especially smartphones.  It transforms research in multiple ways.  It means that people disconnect their landlines, so we have to adjust our methods for phone polling.  But more importantly, it means people are accessible in multiple ways (text, phone, email) all the time, and wherever they are.  We can ask them for feedback instantaneously in the stores where they shop, or as they are making a decision we care about.  We can talk to them, send messages, ask them to respond to survey questions, ask them to take pictures or videos for us, and so on.  There are amazing opportunities for rich, immediate data for research that we could only dream about a few years back.

Quirk’s magazine recently ran a set of articles about trends in mobile research, focusing in particular on survey research.  We thought it is was worth repeating some of the insights highlighted in those articles:

• Surveys via mobile devices are increasingly possible, both SMS-based and web-based surveys
• Achieving representative samples for mobile surveys can be a challenge, however
• Attention spans for mobile surveys are short; ask no more than five or six simple questions
• Design surveys with small screens in mind
• Avoid long questions that need explanation
• Avoid long lists of answer options
• Avoid grid questions
• Take advantage of open-ended questions; people with mobile devices are capable (and often prefer) typing out responses on their phones
• Be sure to cover any costs that respondents might incur

Although nobody should be doing mobile research just because they can, keeping up with new tools, capabilities, and opportunities for rich data is one piece of doing smart and rigorous research.  Versta Research can advise, and help you take advantage of new technologies and methods when, and if, you need them.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/using-mobile-technologies-in-research/feed/ 0 http://www.verstaresearch.com/blog/bridging-the-quantitative-qualitative-gap/ http://www.verstaresearch.com/blog/bridging-the-quantitative-qualitative-gap/#comments Fri, 25 Jun 2010 12:56:49 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=639 The summer 2010 newsletter from Versta Research focuses on how to bridge the gap between quantitative research and qualitative research, whether it be market research or academic research.  Both methods give rich insights, and both offer compelling ways to summarize and communicate data.  But rarely does each method draw upon the strengths of the other.

How do you bring the two together?  In our view, you bring them together by investing in people who are trained in both methods and who have deep experience in both.  The newsletter article outlines:

• Why it is best to be fluent in both
• The strengths and promises of each method
• The qualities and training needed to excel at both
• The false promise of technology
• Practical ideas for incorporating the strengths of each into a research effort

Ultimately, research is all about asking questions, finding information and data, thinking about it, probing deeper, assimilating and synthesizing it, and then turning data into stories so that the research gets heard and understood.  Whether you use qualitative or quantitative methods, your data needs to tell a compelling human (business) story, and the best stories draw upon both.

Give us a call, and we would be happy to share with you examples and stories of our work that have successfully bridged the gap.  In the meantime, take a look at our June 2010 Newsletter for a deeper understanding of our approach, and what you can expect from our people.

—Joe Hopper, Ph.D.

For example, in some cultures people are especially reluctant to give negative answers, which exaggerates the positive-response bias we are accustomed to seeing in the U.S.  Even seemingly “factual” questions can be subject to measurement error and biases that make cross-cultural comparisons potentially difficult.

To help, a team of academic researchers led by the University of Michigan’s Survey Research Center and the University of Nebraska’s Survey Research and Methodology Program have just released a new set of guidelines for best practices in cross-cultural surveys.  They address a full range of topics, including questionnaire design, translation, adaptation and quality.  The guidelines are excellent (and voluminous – 638 pages) and provide an essential primer on problems and solutions for comparative survey research across cultures and countries.

The guidelines are also a good reminder that researchers should always be sensitive to cultural issues, biases, and variations even within their own cultures.  Statistically significant gender variation may reflect important differences between men and women, or it may reflect basic cultural differences in how men and women answer survey questions.  Remember to keep both possibilities in mind, and think about ways to query your data to uncover the most plausible answer.

If you’re not sure about how culture and context affect your plan for research or the data you are analyzing, we would be happy to offer you our best thinking.  Please don’t hesitate to call us at 312-348-6089.

—Joe Hopper, Ph.D.

The idea seems crazy, but as reported in Science, researchers at Carnegie Mellon University have found that certain kinds of twitter data can give them a good read on public sentiment. The looked at things like positive or negative comments about President Obama and found results that aligned with traditional polls. Comments about finances and savings aligned with consumer confidence polls. What does the polling industry think? Here’s what one colleague says: “I believe that I am now going to run to a dark corner of the house and cower in fear of what may come.”

To be sure, we are not recommending that you rely on social media to accurately measure overall public opinion. No way. But in our view, these findings may lead to new insights about how people, individually and collectively, behave and think, and how research — both academic and practical — can harness these new forms of data to measure markets and the social world.  The findings may also force us to re-think our theories of statistical inference that rely on random sampling. There is much research to be done before we will know, but possibilities for social media someday offering insights that are as statistically valid as our current methods is intriguing and nothing to cower about.

Thinking about measuring social media as part of your research plan? We would be happy to advise you. We can help you explore new options in research while ensuring that your research and findings remain rigorous and defensible.

—Joe Hopper, Ph.D.

The number of people without home access to landline telephones is increasing.

Almost half of adults under age 30 live in a household with only wireless telephone service.

In response, our industry has developed newer (and complicated) methods to include cell-only households along with land-based phone sampling.  But do-not-call rules, the cost to respondents of receiving calls on wireless phones, and the fact that exchanges no longer map to geographic regions have been significant challenges.

Now there is a new challenge:  Even though the number cell phones and the number of people who carry them is increasing exponentially, people are not talking on them.  More than nine out of ten households now has cell phone service, but recent data indicate that voice usage is not increasing, and for the first time “the amount of data in text, e-mail messages, streaming video, music and other services on mobile devices in 2009 surpassed the amount of voice data in cellphone calls” (see NYT article).

As people use their phones more and more in multiple other ways, the opportunity for public polling and market research is to find new ways of engaging people who are willing to share data and opinions.  There are now surveys designed for mobile devices and real-time enthnographies using video, photography, and voice from cell phones.  Mobile devices are also increasingly used for purchases, data monitoring, and loyalty programs, all of which can be rich sources of insight for market research.

Need help thinking about the best way to conduct your survey or research?  The best way will depend on your specific questions and the group of people you want to understand.  Give us a call (we like to talk on the phone); we will help you sort out your options for an optimal approach.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/trouble-for-phone-surveys-nobody-talks/feed/ 0 http://www.verstaresearch.com/blog/click-here-for-actionable-insights/ http://www.verstaresearch.com/blog/click-here-for-actionable-insights/#comments Fri, 28 May 2010 14:22:45 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=577 We saw an ad today for a downloadable survey app similar to Survey Monkey or Zoomerang that was pitched as a tool for actionable insights.  Wow!  Download, install, run . . . click again,  and there they are, sitting on your desktop or smart phone: actionable insights.

Is this possible?  No.  It unfortunately confuses the tools of market research and public opinion polling with the interpretation and outcomes of research.  To be sure, better tools and technology help us do our work faster, smarter, and cheaper.  They bring sophisticated tools into the hands of smaller organizations who can then help businesses and media outlets with nimble and cost efficient solutions.  Market researchers are benefiting enormously from these new technologies and tools.

But ultimately these easy-to-use applications that deliver real time data and pie-charts give us … well, just data and pie-charts.  They don’t give us insights.  For insights, we need smart people who bring expertise, brainpower, and thoughtful creativity to the research effort — people who know how to design and implement studies, and then interpret and communicate information to answer critical questions.  When managers and clients see their own desktops stuffed with “auto-alerts-sent-to-key-stakeholders-enterprise-wide” from our newest suite of tools, they need people to answer challenging questions like, “So what?  Is this really true?  What does it mean?  How should I proceed?”

These are the kinds of questions we can help you with.  We can help you sort through the latest tools and can help you implement them within  your organization.  But don’t forget that tools are a means something bigger and more fundamental, like a story with a context, a puzzle, a question that needs to be answered.

—Joe Hopper, Ph.D.