The authors looked at various types of biases that creep into business managers’ decisions when based on data presented to them.  They did this by conducting experiments with business school students and managers who are members of the American Marketing Association.  Some were presented with numeric data in tables, while others were presented with data in charts or graphs.  All tables, charts, and graphs were clear and well-designed.

The results?  When compared to an optimal decision based on a purely rational assessment of the data, decisions were typically biased, and “graphical formats that followed existing recommendation for the appropriate display of data did not reduce . . . biases compared with data presented in tables.”  Moreover, “neither real-world experience nor explicit training reduced these biases.”

In some ways this is surprising, because graphing data can often help us better (and more quickly) grasp its meaning.  On the other hand it is not surprising.  Graphs can be so visually compelling that they might hinder purely rational assessments.

In our view (and the authors’ as well) the research does not invalidate the need for effective data visualization.  A good chart can be a compelling piece of story.  But it is not the story, any more than a table of numbers can be the story.  A good chart is a communication tool.  So are good tables, and so are good sentences that weave together a compelling story.

At Versta Research we use chart and tables in about equal proportions (sometimes tables work better) and we always integrate them into a clear story with appropriate statistical modeling to lend support.  The solution to overcoming bias is not fancier graphics, but rather a clear presentation of information with a compelling synthesis and assessment of what it means.

—Joe Hopper, Ph.D.

We were reminded of the importance of communication and writing research for multiple audiences from Eric Zorn’s recent column in the Chicago Tribune, from which we quote:

How many of us feel this way? I asked [the researcher] for percentages and she responded by e-mail that her research was qualitative, not quantitative.

I asked then in what sense is it “research” any more than choosing anecdotes out of random interviews? Journalists do love numbers.

“I don’t write my research for journalists,” she wrote back. “I write it for the community of scholars who conduct this type of research, and who have since the 1930s when the Chicago School of Sociology began at the University of Chicago. Honestly, if you are going to interact with researchers who conduct perfectly legitimate qualitative research, I strongly suggest that you gain some knowledge of that type of tradition before you throw around words like `random interviews’ and `anecdotes.’”

Ouch.

Mr. Zorn’s question was a good one. It could and should have been answered. It’s really no different from the kinds of questions you and we often hear from the executives to whom we are presenting our work. “How do you know?” “Why does this matter?” “How does this help me?” We nearly always have multiple audiences. Good researchers can and must communicate beyond their own communities.

We at Versta Research have done a great deal of research and survey work to support newspaper stories and communications materials, so we definitely do write for journalists. They love numbers, but they also love stories, so we give them headlines and storylines and numbers to support it all. At the same time, we write for communities of scholars and see our work published in academic journals. The two need not be at odds.

We also write for top level executives (we give them a three page deck focused on implications with supporting evidence) mid-level managers (we give them a ten page deck focused on problems, diagnoses, and solutions), and market researchers (we give them a thirty page deck and a clear story supported with data).

Each audience requires a slightly different approach, answers pitched at different levels, varying degrees of detail, and so on. Need help? Our clients say that our experience and skill at turning data into stories adds significant value to the work they do, all the way from design to implementation, reporting and presentation. Research can make a difference in back offices and boardrooms, and with the right reporting and follow up it can make a difference among journalists asking tough questions as well.

—Joe Hopper, Ph.D.

We highly recommend learning about theories of presentation, perception, and data visualization, and we are big fans of Edward Tufte’s approach to visual explanations.  At the same time, we recommend learning the basics of using simple tools, like pie charts, bar charts, line graphs, and so on.  One good source for developing a mastery of the basics is a book called Graphing Statistics & Data: Creating Better Charts, from which we have learned a few tips about using bar charts:

• Use bar charts to show variables with distinct (non-continuous) values
• Bar charts are good at showing both proportion and quantity (unlike pie charts, which are good at showing proportions only)
• Adjust the chart settings so that the bars are wider than the gaps between the bars.  They need to be wide enough to invite visual comparisons among them, but not so wide that they resemble a histogram (see the examples below)
• If your variable has many values you want to show, or if labeling the values requires a lot of text, use a horizontal bar chart
• For horizontal bar charts, rank order the bars so that long bars (high values) are at the top of the chart, and short bars (low values) are at the bottom

This bar chart is flawed because the bars are too skinny relative to the spaces between them. They do not invite visual comparison.

The bar chart is flawed because the spaces between the bars are too narrow. It makes the chart look like a histogram, which is appropriate only for continuous variables.

This bar chart is just right.

As most of us know from having tried to disentangle one too many indecipherable charts in murky research reports, these basics seem easy, but they rarely are.  They require skill, art, and expertise developed through years of day-to-day efforts to understand, synthesize, and communicate data.  When done well, good charts tell the story of the data.

—Joe Hopper, Ph.D.

]]> http://www.verstaresearch.com/blog/visualizing-data-five-tips-to-using-a-bar-chart/feed/ 0 http://www.verstaresearch.com/blog/two-keys-to-writing-great-research-reports/ http://www.verstaresearch.com/blog/two-keys-to-writing-great-research-reports/#comments Thu, 25 Mar 2010 13:36:29 +0000 Joe Hopper http://www.verstaresearch.com/blog/?p=437 A truly effective research report is both parsimonious and richly nuanced.  In other words, (1) it is short and to the point, and (2) it captures the complexity of reality.  But how do you do both?

The importance of the first was highlighted in Sunday’s “Corner Office” interview in the New York Times business section.  Guy Kawasaki, co-founder of the Alltop news aggregation site, noted the importance of brief and pithy reports:

Q. What should business schools teach more of, or less of?

A. They should teach students how to communicate in five-sentence e-mails and with 10-slide PowerPoint presentations. If they just taught every student that, American business would be much better off.

Q. Why?

A. Because no one wants to read “War and Peace” e-mails. Who has the time? Ditto with 60 PowerPoint slides for a one-hour meeting.  What you learn in school is the opposite of what happens in the real world. In school, you’re always worried about minimums. You have to reach 20 pages or you have to have so many slides or whatever. Then you get out in the real world and you think, “I have to have a minimum of 20 pages and 50 slides.”

However, the importance of complexity was highlighted the next day in an article about Edward Tufte, a contemporary champion of presenting rich data via simple and compelling graphics.  Over-simplification can lead to bad decisions:

Mr. Tufte devotes a section of one of his books to explaining how clearer graphics could have persuaded NASA officials to postpone the takeoff [that resulted in the space shuttle Challenger disaster] because of cold weather. One of his conclusions is that presentations before the explosion, and even after, were too simplified. For simplicity, information was left out about the many missions during warmer weather that were uneventful. But the absence of that information meant that it was easy to overlook the larger pattern, that cold weather was dangerous to the O-ring.

In an interview, Mr. Tufte emphasized the need to enlist “the clarity of intense information.”

That’s the thing about transparency: you know it when you don’t see it. It some cases, it can mean more information. But other times, the reader can be overwhelmed by too much irrelevant information or, in one of Mr. Tufte’s favorite terms, “chartjunk.”

The goal in writing and presenting research should be to keep it brief and concise, but at the same time, communicate “intense” and relevant information.  A clear story will do this for you because good research stories are laden with meaning, complexity, and nuance, but they can be communicated succinctly, clearly, and with unambiguous implications.  Need help?  Versta Research specializes in rigorous research and communicating intense information (without all the chartjunk) to internal and external audiences.  We help you turn data into stories, and would be pleased to assist on your next research project.

—Joe Hopper, Ph.D.

What do we know about the kinds of stories and research reports that can achieve this?  A recent study reported in The New York Times provides some clues.  Researchers looked at which types of NYT articles get e-mailed by readers the most.  One surprising result was that science articles (stories that convey research findings!) get e-mailed the most.  The reason is not that readers like science more than other topics – it’s that science articles, at least those in The New York Times, have content characteristics that people find compelling and want to share.   The researchers analyzed thousands of articles (both science and non-science) and noted the characteristics of those most likely to go viral.  They tend to be articles that:

• Elicit strong emotions
• Have positive themes rather than negative themes
• Inspire awe, or a sense of discovery and grandeur beyond oneself
• Challenge the reader intellectually
• Are surprising in some way

When it comes to market research, this does not mean that all your research presentations need to be awe-inspiring and emotional calls to action.  But it does mean that your research should tell a story that relates to people and what they do, feel, and care about.  It should show them opportunities as well as barriers.  It should provide context.  The numbers do not need to be easy, but remember they help tell the story rather than being the story.  If you’re writing a story for media placement and would love to see your story leveraged beyond the initial hit, incorporate as many of these characteristics as you can.

A lot of surveys and market research reports today sit buried in PowerPoint charts that nobody has bothered to read – a sorry state of affairs that inspires Versta’s mission to help you turn data into stories.  With the right approach, your managers and audiences will not only listen to your findings and act on them, but they will be eager to share them with their managers and their friends and family as well.

Joe Hopper, Ph.D.

Looking back, here are what we consider to be the five biggest changes that shaped current challenges faced by market research and opinion polling:

1. Online data collection using sample panels became the dominant mode of surveying, displacing a huge industry in telephone surveying.  Plus there has been a dramatic decline in households owning landlines, which makes the ideal of probability sampling extremely difficult to achieve.
2. Survey technology made data collection easy and cheap which means that a lot of market research became a commodity, resulting in industry consolidation and need to find new ways of adding value.
3. An explosion of too many surveys, which are everywhere, powered by simple tools like Survey Monkey.  Along with this there has been a dramatic decline in response rates, bringing the issue of quality to the fore like never before.
4. Statistical and mathematical expertise grew in demand.  There is so much data and information, and it is so easy to get.  But who knows what to do with it all?
5. “Audience-ready” reports became more valuable.  Newspapers laid off staff and market research departments downsized, demanding a level of professional involvement to ensure that research reports are easy to read, interpret, and ready to deliver to the audiences who need them most.

One product of these five trends is Versta Research’s focus on high-level expertise and academic brainpower so that you get smart, creative, and flexible market research.  Another is the skill and commitment we have in helping you communicate research to managers, clients, reporters, and the audiences you need to reach.

In our next post we’ll highlight what we predict will be the five biggest trends that will shape market research challenges in the decade to come.

–Joe Hopper, Ph.D.

1.  Statistics are essential in understanding the world.

Quoting from the article: “Malcolm Gladwell recently said that if he were trying to break into journalism today, he would start by getting a master’s degree in statistics. The Harvard psychologist Steven Pinker . . . might second this advice. Asked via e-mail what is the most important scientific concept that lay people fail to understand, he responded: “Statistical reasoning. A difficulty in grasping probability underlies fallacies from medical quackery and stock-market scams to misinterpreting sex differences and the theory of evolution.”

2.  Statisticians often fail to communicate in ways that help.

Steven Pinker continued: “Academics lack perspective. In a debate on whether the world is round, they would argue ‘no,’ because it’s an oblate spheroid.  They suffer from ‘the curse of knowledge’: the inability to imagine what it’s like not to know something that they know.”

These two ideas resonated with us because they address the two critical components of what we try to achieve for our clients:  answering critical questions with high level expertise and then communicating those answers to the managers who need them most.  Market research is not delivering on its promise until it helps you with both of these components.

–Joe Hopper, Ph.D.

There will always be a need for good storytelling. This point was eloquently made by Susan Howe, President of Weber Shandwick Chicago.  The media through which stories are told is shifting, but the fundamentals of good PR remain.  Likewise for research, the tools we use are changing every day, but turning data into stories is a constant that underlies change.

You know the story better than anyone. Rich Jernstedt, EVP of Fleishman-Hillard argued that good PR professionals own and communicate the client’s story at every phase because a reporter will never know that story as deeply.   Likewise, an effective research partner helps a client communicate data from beginning to end, long after the “report” is delivered.  Nobody knows the research as thoroughly as we do.

We need to add value. Janet Cabot, Co-President of the Chicago Office of Edelman, highlighted the importance of research and intellectual capital because effective tactics are now a “given” and clients are looking for more.   The same goes for research itself.  Focusing on data collection and tabulation no longer adds value.  We need to leverage our considerable intellectual capital by helping clients interpret, grasp and communicate the research.

It was an optimistic and inspiring forum that was consistent with our thinking at Versta Research:    Change is an opportunity to focus on (1) the fundamentals of what we do (designing rigorous research to answer critical questions) and (2) the value we add (turning data into stories), both of which make for strong and lasting partnerships with our clients.

–Joe Hopper, Ph.D.

The first approach is to mine data.  Find data that you already have, analyze that data, use it to answer interesting questions, and then tell the story.  For example, financial services, HR, benefits, and other outsourcing companies like Hewitt Associates track a wealth of data about employees and employers as part of their business.  They can then use that data to write interesting stories about how many people cash out their 401(k) plans when leaving a job, or how many companies are switching healthcare plans as costs continue to rise.

The second approach is to generate new data by commissioning a survey that answers interesting questions to drive your story.  For example, one client I worked for got significant story placements by surveying people about skin cancer myths and reasons for not taking simple precautions like wearing sunscreen.  Another gets ongoing coverage for a survey of IT professionals about future trends in the marketplace.

With both approaches, the trick is to (1) build a credible foundation with expert research and data analysis, and (2) turn data into stories. Versta Research can help you with both pieces.  We can help you answers questions with expertise, turn data into stories, and ultimately help you  communicate those stories to the audiences who need them most.

–Joe Hopper, Ph.D.

Pie charts can be challenging because they are so common and so commonly misused.  When done poorly, they force us to think hard about what we are seeing and why.  When done well, they tell the story of our data.

Here are six tips on effectively designing and using pie charts:

• Pie charts are good for showing proportions, not quantities
• Use pie charts for category variables (like gender, or region) but not for variables that measure levels of things (like satisfaction)
• Pie charts should begin at 12 o’clock, and are read clockwise
• The maximum number of slices to the pie should be five or six
• Always show the numerical values on the chart
• Avoid three-dimensional “tilting” of the chart because it distorts true proportions

Programs that create charts automatically can be terrific tools, but they will never replace the need for you to think about your story and how to portray it in a chart.

–Joe Hopper, Ph.D.