Archive for the ‘Presenting Research’ Category
Wednesday, September 7th, 2011
It's pretty, but it's chartjunk
Turning data into stories involves not just words, but pictures as well. In the world of quantitative market research, that usually means charts, graphs, and tables. Moreover, just like poorly written sentences that often complicate rather than clarify data, charts and graphs in market research too often suffer from “chartjunk,” as Edward Tufte calls it. Any superfluous details, design elements, or decorations that do not tell the viewer something new about the data are chartjunk.
At Versta Research we write a lot of reports. We also revise others’ reports to help our clients find and more clearly present research stories to their management teams. Here are three of the more common chart design mistakes we see and help our clients avoid: (more…)
Tags: charts, Market Research, satisfaction research, stories, visualizing data
Posted in Charts and Data Visualization, Market Research, Methods & Tools, Presenting Research, Turning Data into Stories | No Comments »
Wednesday, August 31st, 2011
Monty Hall in Let's Make A Deal
The hardest part of quantitative market research is not that it involves numbers, math, or even statistics, but that it involves complex problems in probability.
Over the past several years, psychologists have been documenting how difficult it is for us humans to solve even “simple” probability problems. One fascinating example is a puzzle known as the Monty Hall dilemma based on the 1960’s game show Let’s Make A Deal. Monty would offer his contestants three doors to choose from, one of which had a valuable prize behind it. After the contestant chose, Monty would open one of the other two doors, deliberately choosing one that had no prize behind it. Then he offered the contestant an option of staying with the original choice, or switching to the other unopened door. Which should the contestant do? (more…)
Tags: conjoint, data, Market Research, mathematics, Sampling, statistics, stories
Posted in Data Analysis & Analytics, Market Research, Presenting Research, Sampling, Turning Data into Stories | No Comments »
Thursday, July 14th, 2011
How statistics are calculated and presented has a huge effect on how audiences interpret information and make decisions. A recent study about medical decisions based on drug efficacy data highlights the critical importance of how you turn your data into stories, no matter what industry. The research shows that different stories, all of them true and all of them based on the same data will lead to sharply different assessments and decisions. An article in the New York Times summarized one scenario tested by the researchers:
If your doctor tells you that highly reliable studies have shown that taking a certain pill will cut your risk of getting a serious disease in half, would you take it?
Suppose he adds that the risk is 2 percent for people who do not take the pill, but your risk will be reduced to 1 percent if you do. Would you still take it? And what would you do if he told you that only one of every 100 patients who take the drug will actually benefit from it?
The doctor could have said any of these things, all truthfully, because they are just different ways of describing the same data. (more…)
Tags: communication, data, Market Research, research, stories, visualizing data
Posted in Data Analysis & Analytics, Market Research, Presenting Research, Turning Data into Stories | No Comments »
Thursday, April 28th, 2011
As much as we love numbers, we find ourselves often advising clients against using numeric scales in their surveys. A numeric scale is any response format that asks people to give a number within a certain range to indicate the strength of their feeling or opinion. The insanely popular survey question used to calculate Net Promoter Scores is a good example:
“How likely is it that you would recommend Acme Solutions to a friend or colleague? Please answer on a scale from zero to ten, where zero means not at all likely, five is a neutral score, and ten means extremely likely.”
There are many good reasons to use numeric scales and many types of research for which numeric scales are optimal. The NPS scale is good because it has eleven points with meaningful endpoints and a meaningful midpoint. Research shows that scales like this can be highly reliable and valid, with sufficient variability to allow for sophisticated statistical modeling.
But if your objective is to use survey data for marketing materials, public relations, news releases, or white papers, numeric scales make things difficult. They are not easy to summarize in words, and if you want to use charts that tell quick, compelling stories, you will end up having to do something like this:
A Poor Fit: Pie Charts and Numeric Scales
(more…)
Tags: charts, communication, Data Collection, journalism, media, news, omnibus, Public Relations, stories, Survey Design, visualizing data
Posted in Charts and Data Visualization, Data Collection, Omnibus Surveys, Presenting Research, Public Relations, Survey Design, Survey Tips, Turning Data into Stories | No Comments »
Saturday, January 22nd, 2011
Unless your company has a department dedicated to it, market research can be a hard sell because higher level executives may not believe in the value of research. At Versta, we have a certain sympathy with these executives. In our view, market research in and of itself has little value; it is the outcomes of research—the answers to questions—that can have value.
It is important to distinguish the two because unfortunately there is plenty of research not designed to answer important questions. Research is frequently done because someone has a nagging feeling that they need more information to make better decisions or because doing research is considered a “best practice.” Such research generates lots of data that marketing managers wonder what to do with. Not surprisingly, they and their bosses start to question the value of research.
Our advice is to do market research only after formulating specific questions and information needs and only after you have a clear idea (in writing) of what you will do with the answers to those questions. We have produced a whitepaper entitled The Art of Asking Questions (you can download it by clicking on the image) that outlines a useful process to help you formulate those questions.
The Art of Asking Questions: A White Paper from Versta Research
The key is to articulate five types of questions, including: (more…)
Tags: insight, Market Research, research, Survey Design, value
Posted in Market Research, Presenting Research, Survey Design | No Comments »
Friday, December 17th, 2010
With the magic of the holidays upon us, we got to thinking about “magic” in market research, or the lack thereof. So our just- published quarterly newsletter focuses on magic numbers in market research, arguing that certain “magical” numbers seem to guide much of what we do, whether we realize it or not. The best researchers know the difference between the real magic of numbers and the not-so-real. We hope you find our commentary useful.
There is also, of course, the magic of story-telling, which market research can embrace, but rarely does. We were inspired by a recent New York Times interview with Aaron Levie, CEO of Box.net. What could be more boring than online file storage? But as Levie said: (more…)
Tags: confidence intervals, Market Research, mathematics, response scales, Sampling, statistics, stories
Posted in Market Research, Presenting Research, Public Polls, Sampling, Turning Data into Stories | No Comments »
Thursday, November 11th, 2010
One way you can make a research report really pop is to carefully select and edit quotes from the people involved in the study. For qualitative research, that means pulling quotes from transcripts of in-depth interviews, focus groups, online bulletin boards, social media, etc. For survey research, it means pulling quotes from open-ended questions that were recorded verbatim.
Editing is key, however. Who wants to read something in the halting, choppy, in-eloquent speech that most of us use in talking? A killer quote is one that is short, direct, pithy, and on point. Achieving this requires a journalistic standard for presenting quotes. What does that mean? Here are some steps: (more…)
Tags: Focus Groups, journalism, Market Research, media, open-ends, qualitative research, stories
Posted in Focus Groups, Presenting Research, Turning Data into Stories | No Comments »
Thursday, October 14th, 2010
Should you state a survey’s margin of error in your press release when pitching a story to the media? In our view, the answer is no. Why not? Because margins of error refer to sampling error only, not about the overall accuracy or error of the survey itself.
But how many readers of your news story understand what sampling error is? For that matter, how many researchers understand what sampling error is? It was only after four college semesters of advanced mathematical statistics that I finally “got it.” More importantly, how many readers understand that there are many other potential sources of survey error? Most undoubtedly assume that all error is somehow accounted for when you confidently proclaim the margin of error being ±4%. (Or, more absurdly, ±3.6% or even ±3.57% — examples of phony accuracy like this are all too easy to find.)
In short, margins of error are misleading because they deal with only one source of error. They convey a false sense of accuracy. And they should not be used. This idea is not always popular among colleagues and clients. But take comfort: One of the giants of public polling, Harris Interactive, refuses to report margins of error in its work, for precisely the reasons outlined above. Here we quote their methodological statement that accompanies every press release and report they issue: (more…)
Tags: ethics, media, Public Polls, Public Relations, Sampling, statistics
Posted in Presenting Research, Public Polls, Public Relations | No Comments »
Friday, October 1st, 2010
An essential component to Versta Research’s strength in turning data into stories is communicating those stories to the audiences who need them most. Those audiences might be researchers, internal clients, brand management teams, PR specialists, or reporters. Or the audience might be the top level executives at the companies or organizations we are working with. It is always exciting when research gets presented to the very top levels of management because it is an opportunity to see how (and whether) research really matters.
Our Fall 2010 newsletter focuses on presenting to the top levels of executive management. How do you “wow” your CEO with research, instead of putting her to sleep? We focus on three components: (more…)
Tags: communication, Market Research, stories
Posted in Market Research, Presenting Research, Turning Data into Stories | No Comments »
Friday, August 13th, 2010
Does data displayed in charts and graphs, rather than tables, lead to better decisions? Not according to the latest research reported in this month’s Journal of Marketing Research.
The authors looked at various types of biases that creep into business managers’ decisions when based on data presented to them. They did this by conducting experiments with business school students and managers who are members of the American Marketing Association. Some were presented with numeric data in tables, while others were presented with data in charts or graphs. All tables, charts, and graphs were clear and well-designed. (more…)
Tags: bias, charts, communication, data, Market Research, statistics, stories, visualizing data
Posted in Charts and Data Visualization, Market Research, Presenting Research, Turning Data into Stories | No Comments »
