Here is an example from our recent work.  We had a working data set with 107 variables.  We wanted to know whether we could reduce these down into a smaller set of variables using techniques like factor analysis and scaling.  We were planning to use linear regression techniques as well, so we wanted to be aware of any collinearity issues.

Our first step in getting familiar with the data was to run a bivariate correlation matrix.  This resulted in a matrix of 11,449 coefficients.  High correlations would suggest potential collinearity and opportunities for data reduction.  Not so long ago, it was not feasible to review so many coefficients in a single matrix.  But now we can use color rules in Excel to create a “heat map” that makes this possible:

107x107 Correlation Matrix with Excel Color Rules Used to Flag Higher Correlations

Clusters of orange show logical groups of variables that are highly correlated, making them candidates for data reduction through factor analysis.  Long strings of orange (in rows or columns) point to specific variables that are highly correlated with many items in the survey, suggesting redundancy, collinearity, or maybe even a valuable finding.

Here are some tips to creating this type of visualized map:

• Keep your data set in a logical order (probably the order in which survey questions were asked) so that clusters represent logical relationships rather than random ones
• Run a full matrix of bivariate correlations (we used SPSS syntax because 107 variables exceeds the limits of the SPSS Windows interface)
• Export the output to Excel
• In Excel, remove rows showing n-counts and p-values
• Select the entire sheet and create a cell fill rule (our rule was to fill the cell with orange if the cell value was greater than .1)

Data visualization is not just about fancy charts (or purchasing expensive software to make them) but about simple graphical techniques to work smarter and more effectively.  In this case it was also just a first step towards advanced statistical techniques, which included factor analysis, reliability analysis, linear regression, logistic regression, CHAID, and cluster analysis.  And advanced statistical techniques are, of course, just an interim step in making sense of the data and turning it all into clear and compelling story.

Need help sorting through all your data, making sense of it, and turning it into a story?  That’s what we do, so feel free to reach out.

—Joe Hopper, Ph.D.

Algorithms are central to the work that we do in business and market research, and they are top of mind for us at Versta Research because we have been involved in several data-intensive projects that involve either (a) developing new algorithms for clients, or (b) tools that apply sophisticated algorithms to data in new and exciting ways.

What is an algorithm?  It is “a mechanical or recursive computational procedure” (American Heritage Dictionary).  New technologies, data capacities, data collection techniques, and speed have indeed made this the age of algorithms.  Here are some interesting examples, including a few we’ve seen in the news lately, and a few that relate to our work:

• The best example (of course) is Google.  Good search engines are amazingly fast, efficient, and smart algorithms that find relevant materials based on just a few words that you supply.

• Yahoo is using search algorithms not just to find materials, but to make decisions about what to cover in their own news reporting. It used to be that editors made decisions about what is newsworthy; now algorithms do.

• IBM is perfecting a machine that can play (and win) the game of Jeopardy against humans.  It involves algorithms that search for multiple meanings of words and facts, and then matches them against an enormous database of information.  It assigns probabilities to each possible answer (or question, as the case may be) and responds accordingly.

• Versta Research completed a project developing a matching algorithm (matching people with products) based on survey data that had thousands of data points on each person and each potential product.  The process involved mining the data for ways of matching that optimized customer satisfaction.

• We also just completed a project that used convergent cluster and ensemble analysis (CCEA) to explore patient segments using patient chart data.  The technique relies on multiple clustering algorithms, generating hundreds of potential solutions and then selecting the optimal one based on a reproducibility algorithm.

What is interesting is that the “basics” of inferential and descriptive statistics are becoming a smaller and smaller piece of data analysis.  More than ever, we combine them with procedures that involve data mining, predictive analytics, Monte Carlo simulations, and so on.

What are the implications for research firms and the clients who employ us?  The value of having extremely well-trained researchers who are fluent not only in statistics, but in mathematics, modeling, logic, and yes, even algorithms, is essential.  And if, on top of that, you have smart people who know how to use, interpret, and tell a story with that data, the value of your market research will really shine.

—Joe Hopper, Ph.D.

One of the fascinating insights to emerge from our discussion was that social media is not only a new channel of information and data, but that it is fundamentally different from previous channels of data. As such, it opens up new areas of inquiry for our efforts. What is that fundamental difference? It is the networked nature of social media. As we code and tabulate people’s comments on social media as part of our CS&L research, potentially we have access to the networks of each person whose comments we are analyzing. We can know how many people are reading each comment, we can measure how strong and extensive the network of influence is, where it overlaps important segments of customers, and so on.

The implications of this are huge. The impact of loyalty among one’s best customers can now be defined not only in terms of how much they buy and how “sticky” they are, but also in terms of their influence among other customers and prospects. Smart CS&L research will not count everyone’s opinion the same, but will give more weight to those occupying central nodes of critical networks. There are implications for sampling as well. Is true random sampling necessary, or can “networked” sampling provided sufficient entry points that give visibility into the full population of customers?

In my view, this sort of network analysis will bring about a seismic shift in CS&L research, though not all panelists agreed. We were split about 50/50 on whether social media constitutes just one more channel of data to integrate, bringing greater precision to what we’ve always done, or whether it represents a more radical departure. A full report of our panel’s deliberations will be presented in the October 2010 issue of Marketing News, the AMA’s monthly magazine.

The stimulating discussion among panelists also highlighted the importance of remembering that good CS&L research requires ongoing thoughtfulness, intelligence, and curiosity. New tools and technologies are often easy to install, but their value is in offering new opportunities to bring deeper understanding and analysis to research.

Stay tuned – we’ll provide a closer look at some of those opportunities for better research (and reprints upon request) when the AMA publishes its report this fall.

—Joe Hopper, Ph.D.

For those of us at Versta Research, he is well known for his focus on understanding versus research:

“Basically, I’m not interested in doing research and I never have been,” he said. “I’m interested in understanding, which is quite a different thing. And often to understand something you have to work it out yourself because no one else has done it.”  (From an interview cited in the New York Times)

Why are we inspired by this?  Because even though we are interested in doing research, we do it for one reason:  To understand things.  In fact, we named our company from the social science concept (and German word) “verstehen” which means to understand.

Fundamentally, we define our work not by proprietary methods, or the specific tools that we use or develop, or the statistical procedures we implement.  We define it by our interest in and approach to understanding.  And that, of course, means using, learning, developing, and inventing whatever tools, processes, data collection techniques, and analysis that will get good answers.  In short, to achieve understanding, we do research.

Understanding is the outcome.  That’s what we really care about (and what you really care about) and what motivates our work.  Research is the means to that end.

If you have not done so, take a look at the homepage of our website.  It’s all about understanding.  We are committed to helping you understand:

• Your clients
• Your prospects
• The public
• Consumers
• Your competition
• Your customers
• Your data
• Your products
• Your image
• The world

If and when research can help you understand, we are here to help.

—Joe Hopper, Ph.D.

We highly recommend learning about theories of presentation, perception, and data visualization, and we are big fans of Edward Tufte’s approach to visual explanations.  At the same time, we recommend learning the basics of using simple tools, like pie charts, bar charts, line graphs, and so on.  One good source for developing a mastery of the basics is a book called Graphing Statistics & Data: Creating Better Charts, from which we have learned a few tips about using bar charts:

• Use bar charts to show variables with distinct (non-continuous) values
• Bar charts are good at showing both proportion and quantity (unlike pie charts, which are good at showing proportions only)
• Adjust the chart settings so that the bars are wider than the gaps between the bars.  They need to be wide enough to invite visual comparisons among them, but not so wide that they resemble a histogram (see the examples below)
• If your variable has many values you want to show, or if labeling the values requires a lot of text, use a horizontal bar chart
• For horizontal bar charts, rank order the bars so that long bars (high values) are at the top of the chart, and short bars (low values) are at the bottom

This bar chart is flawed because the bars are too skinny relative to the spaces between them. They do not invite visual comparison.

The bar chart is flawed because the spaces between the bars are too narrow. It makes the chart look like a histogram, which is appropriate only for continuous variables.

This bar chart is just right.

As most of us know from having tried to disentangle one too many indecipherable charts in murky research reports, these basics seem easy, but they rarely are.  They require skill, art, and expertise developed through years of day-to-day efforts to understand, synthesize, and communicate data.  When done well, good charts tell the story of the data.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/visualizing-data-five-tips-to-using-a-bar-chart/feed/ 0 http://www.verstaresearch.com/blog/bridging-the-quantitative-qualitative-gap/ http://www.verstaresearch.com/blog/bridging-the-quantitative-qualitative-gap/#comments Fri, 25 Jun 2010 12:56:49 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=639 The summer 2010 newsletter from Versta Research focuses on how to bridge the gap between quantitative research and qualitative research, whether it be market research or academic research.  Both methods give rich insights, and both offer compelling ways to summarize and communicate data.  But rarely does each method draw upon the strengths of the other.

How do you bring the two together?  In our view, you bring them together by investing in people who are trained in both methods and who have deep experience in both.  The newsletter article outlines:

• Why it is best to be fluent in both
• The strengths and promises of each method
• The qualities and training needed to excel at both
• The false promise of technology
• Practical ideas for incorporating the strengths of each into a research effort

Ultimately, research is all about asking questions, finding information and data, thinking about it, probing deeper, assimilating and synthesizing it, and then turning data into stories so that the research gets heard and understood.  Whether you use qualitative or quantitative methods, your data needs to tell a compelling human (business) story, and the best stories draw upon both.

Give us a call, and we would be happy to share with you examples and stories of our work that have successfully bridged the gap.  In the meantime, take a look at our June 2010 Newsletter for a deeper understanding of our approach, and what you can expect from our people.

—Joe Hopper, Ph.D.

For example, in some cultures people are especially reluctant to give negative answers, which exaggerates the positive-response bias we are accustomed to seeing in the U.S.  Even seemingly “factual” questions can be subject to measurement error and biases that make cross-cultural comparisons potentially difficult.

To help, a team of academic researchers led by the University of Michigan’s Survey Research Center and the University of Nebraska’s Survey Research and Methodology Program have just released a new set of guidelines for best practices in cross-cultural surveys.  They address a full range of topics, including questionnaire design, translation, adaptation and quality.  The guidelines are excellent (and voluminous – 638 pages) and provide an essential primer on problems and solutions for comparative survey research across cultures and countries.

The guidelines are also a good reminder that researchers should always be sensitive to cultural issues, biases, and variations even within their own cultures.  Statistically significant gender variation may reflect important differences between men and women, or it may reflect basic cultural differences in how men and women answer survey questions.  Remember to keep both possibilities in mind, and think about ways to query your data to uncover the most plausible answer.

If you’re not sure about how culture and context affect your plan for research or the data you are analyzing, we would be happy to offer you our best thinking.  Please don’t hesitate to call us at 312-348-6089.

—Joe Hopper, Ph.D.

The idea seems crazy, but as reported in Science, researchers at Carnegie Mellon University have found that certain kinds of twitter data can give them a good read on public sentiment. The looked at things like positive or negative comments about President Obama and found results that aligned with traditional polls. Comments about finances and savings aligned with consumer confidence polls. What does the polling industry think? Here’s what one colleague says: “I believe that I am now going to run to a dark corner of the house and cower in fear of what may come.”

To be sure, we are not recommending that you rely on social media to accurately measure overall public opinion. No way. But in our view, these findings may lead to new insights about how people, individually and collectively, behave and think, and how research — both academic and practical — can harness these new forms of data to measure markets and the social world.  The findings may also force us to re-think our theories of statistical inference that rely on random sampling. There is much research to be done before we will know, but possibilities for social media someday offering insights that are as statistically valid as our current methods is intriguing and nothing to cower about.

Thinking about measuring social media as part of your research plan? We would be happy to advise you. We can help you explore new options in research while ensuring that your research and findings remain rigorous and defensible.

—Joe Hopper, Ph.D.

The number of people without home access to landline telephones is increasing.

Almost half of adults under age 30 live in a household with only wireless telephone service.

In response, our industry has developed newer (and complicated) methods to include cell-only households along with land-based phone sampling.  But do-not-call rules, the cost to respondents of receiving calls on wireless phones, and the fact that exchanges no longer map to geographic regions have been significant challenges.

Now there is a new challenge:  Even though the number cell phones and the number of people who carry them is increasing exponentially, people are not talking on them.  More than nine out of ten households now has cell phone service, but recent data indicate that voice usage is not increasing, and for the first time “the amount of data in text, e-mail messages, streaming video, music and other services on mobile devices in 2009 surpassed the amount of voice data in cellphone calls” (see NYT article).

As people use their phones more and more in multiple other ways, the opportunity for public polling and market research is to find new ways of engaging people who are willing to share data and opinions.  There are now surveys designed for mobile devices and real-time enthnographies using video, photography, and voice from cell phones.  Mobile devices are also increasingly used for purchases, data monitoring, and loyalty programs, all of which can be rich sources of insight for market research.

Need help thinking about the best way to conduct your survey or research?  The best way will depend on your specific questions and the group of people you want to understand.  Give us a call (we like to talk on the phone); we will help you sort out your options for an optimal approach.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/trouble-for-phone-surveys-nobody-talks/feed/ 0 http://www.verstaresearch.com/blog/click-here-for-actionable-insights/ http://www.verstaresearch.com/blog/click-here-for-actionable-insights/#comments Fri, 28 May 2010 14:22:45 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=577 We saw an ad today for a downloadable survey app similar to Survey Monkey or Zoomerang that was pitched as a tool for actionable insights.  Wow!  Download, install, run . . . click again,  and there they are, sitting on your desktop or smart phone: actionable insights.

Is this possible?  No.  It unfortunately confuses the tools of market research and public opinion polling with the interpretation and outcomes of research.  To be sure, better tools and technology help us do our work faster, smarter, and cheaper.  They bring sophisticated tools into the hands of smaller organizations who can then help businesses and media outlets with nimble and cost efficient solutions.  Market researchers are benefiting enormously from these new technologies and tools.

But ultimately these easy-to-use applications that deliver real time data and pie-charts give us … well, just data and pie-charts.  They don’t give us insights.  For insights, we need smart people who bring expertise, brainpower, and thoughtful creativity to the research effort — people who know how to design and implement studies, and then interpret and communicate information to answer critical questions.  When managers and clients see their own desktops stuffed with “auto-alerts-sent-to-key-stakeholders-enterprise-wide” from our newest suite of tools, they need people to answer challenging questions like, “So what?  Is this really true?  What does it mean?  How should I proceed?”

These are the kinds of questions we can help you with.  We can help you sort through the latest tools and can help you implement them within  your organization.  But don’t forget that tools are a means something bigger and more fundamental, like a story with a context, a puzzle, a question that needs to be answered.

—Joe Hopper, Ph.D.