Election years are a perfect time to learn about (and brush up on) the fundamentals of survey research.  Not only are the airwaves inundated with public opinion polling, but methodological experts are called upon to talk about developments and current best practices as new technologies and methods become central to measuring consumer and public opinion and behavior.

This week the Poynter Institute is offering a webinar of particular interest for PR professionals, whether or not you care about political polls.  We also recommend it for any marketing professional because this type of polling is an exemplar of what all marketing research tries to achieve in measuring what people think and what they are likely to do.

The webinar (scheduled for January 26, 2012 with archived replay available afterwards) is being co-sponsored by the American Association of Public Opinion Research, and will be led by Claudia Deane, associate director for public opinion and survey research at the Kaiser Family Foundation.  It is designed for non-researchers, and will focus on:

• The science of how polling and survey research works
• How to determine the legitimacy of a poll and the rigor of survey research
• How to assess the quality and usefulness of survey questions

If you miss the webinar or hunger for more, Versta Research has (and can point you toward) additional resources to help PR professionals, marketing professionals, and market researchers understand, keep abreast of, and communicate the importance of research.  We are one of the few research firms in the industry with a background in university teaching and academic research striving to bring that expertise to the practical worlds of communications campaigns and marketing insight.  Please give us a call and we would be happy to help you further.

–Joe Hopper, Ph.D.

One powerful way to gain visibility and credibility in your marketplace is by sponsoring survey research that documents problems and solutions in areas where you have expertise.  To be successful, it requires (1) rigorous research carefully designed to uncover the right topics, and (2) savvy PR work that uses data to tell a credible and compelling story.

The Public Relations Society of America (PRSA) and the American Statistical Association have just published a handy guide for PR professionals that outlines best practices for using, interpreting and reporting statistics in press releases and other PR materials.  Some of those best practices include the following:

• Disclose who paid for the work, and who did the research
• Clearly describe how the research was conducted
• Describe the sample of the survey, and whether it was truly random
• Remember that almost any survey can suffer from bias
• Rely on descriptive statistics to report the data
• Remember that all statistical research includes some level of uncertainty
• Clearly describe trends and effects
• Avoid making claims about the future based on recent history
• Use causal statements cautiously, as they are very hard to prove
• Run your insights by the person who did the research to be sure the data support it

Turning data into stories is never easy, but as this partnership between PRSA and the ASA makes clear, it is becoming more and more important for PR professionals to have a working knowledge of statistics.  And if Shonali Burke’s report from the 2011 PRSA conference is any indication, PR professionals are embracing the challenge.

Versta Research can help you with this challenge.  We are experts in research.  We know how to design surveys and report statistics that get your message heard.  Give us a call at (312) 348-6089 when you need a compelling custom survey or omnibus survey to help you tell your story.

–Joe Hopper, Ph.D.

Why?  Because their research challenges the beliefs of many AAPOR-ites who disregard most online research as being theoretically indefensible since it is not based on probability sampling.  The research presented was based on parallel surveys conducted last year, designed to allow careful comparison of three survey modes:  a “gold standard” phone survey that included both landline and cell phone RDD sampling, an opt-in Internet panel survey that used careful weighting techniques to ensure a comparably representative sample of the U.S. population, and a traditional paper-based mail survey.  The questionnaires for each mode were nearly identical and elicited data on variety of topics including several that could be validated against the most rigorous U.S. Census and NIH data.

Here is how the authors summarized their results:

Comparing the findings from the modes to each other and the validated benchmarks, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates. (download full paper)

Their research adds to a growing body of research showing that rigorously done online surveys can be as useful and effective as rigorously done phone surveys.

True, most online surveys are not rigorously executed (but they can be), and the practical successes of online surveys have outpaced our understanding of the statistical, theoretical, and social underpinnings of why they work.  But it is both intellectually closed-minded and pragmatically foolish to dismiss all online surveys as “invalid” and “not newsworthy” as some of our colleagues in AAPOR do.

Our approach at Versta Research is rooted in our training and continuing engagement in academic research, but we are also rigorously focused on doing what works to get the urgent answers and compelling business stories that our clients need.  Online survey research is one crucial piece of that.

—Joe Hopper, Ph.D.

“How likely is it that you would recommend Acme Solutions to a friend or colleague?  Please answer on a scale from zero to ten, where zero means not at all likely, five is a neutral score, and ten means extremely likely.”

There are many good reasons to use numeric scales and many types of research for which numeric scales are optimal.  The NPS scale is good because it has eleven points with meaningful endpoints  and a meaningful midpoint.  Research shows that scales like this can be highly reliable and valid, with sufficient variability to allow for sophisticated statistical modeling.

But if your objective is to use survey data for marketing materials, public relations, news releases, or white papers, numeric scales make things difficult.  They are not easy to summarize in words, and if you want to use charts that tell quick, compelling stories, you will end up having to do something like this:

A Poor Fit: Pie Charts and Numeric Scales

The problem with this graphic is that the numbers inside the pie chart are confusing, and the words highly willing, not willing, and neutral were never actually used or selected by most respondents.  Somebody wrote the questionnaire and used a numeric scale without first considering how they were going to use and present the data.

Here is the question that was used: “When thinking of your financial investments, how willing are you to take risks? Please use a 10-point scale, where 1 means Not At All Willing, and 10 means Very Willing.” Had this organization been working with us, we would have advised using a scale like this:

• Not at all willing
• Not very willing
• Somewhat willing
• Very willing

Depending on their objectives and the story they wanted to tell, we may have advised including a “Neutral” category as well.  A scale based on words rather than numbers would have been much more useful in talking about how investors are willing or not willing to take risks.

There are no “magic scales” or response categories that should always be used.  If you find a research professional arguing otherwise, chances are they are not listening carefully to what you need, nor are they thinking much about how the data they collect will deliver on the core objectives of your research.  Telling a story with data requires thinking about the very last endpoint (presentation of data to the audiences you want to reach) from the very beginning (conceptualizing and designing the research).

—Joe Hopper, Ph.D.

There are three tiers of survey research common in public relations, only two of which can truly optimize your PR:

Serious Surveys.  Beyond providing rich material for news releases, serious surveys are designed to establish true thought leadership.  They are leveraged for publication through white papers, conference presentations, or peer-reviewed journals.  The surveys we conducted and published about ulcerative colitis are excellent examples, and for the last three years have been used by the client to demonstrate a deep understanding of the issues faced by patients and physicians.

Solid Surveys. Solid surveys answer interesting questions or document issues that people care about, with solutions that people can use.  The primary goal is media placement, but a solid survey can be leveraged further.  For example, one client we worked for got significant story placements by surveying people about skin cancer myths and reasons for not taking simple precautions like wearing sunscreen.  Another gets ongoing coverage for a survey of IT professionals about future trends in the marketplace.

Silly Surveys. These are unfortunately not designed to answer authentic questions, but to grab attention in ways that may not even be relevant to your business.  We saw one survey not long ago that measured which song titles, from among five or six listed, captured people’s optimism or pessimism about the decade ahead.  The client?  An office products company.  Another recent survey announced the percentage of women who would give up TV, cell phones, computers, or sex in exchange for losing 10 pounds.  Who cares?  These are not meaningful comparisons; they’re just silly.

To be fair, silly surveys can attract good media, but there are costs.  First, they diminish your (and our) long-term credibility as journalists start to ask whether they’re reading yet another gimmick survey or whether your latest effort offers something of real value.  Second, there is an opportunity cost.  You could be investing in information that not only gets media attention, but is leveraged in multiple ways and for many years.  We have a client for whom we did a solid survey back in 2004, which they are still using on their website to engage consumers in 2010.

If you’re in the business of PR, we suggest leaving the silly surveys to Facebook polls.  Instead, focus on building a credible foundation with expert research and data analysis.  Versta Research can help you with this.  We can help you answer questions with expertise, turn data into stories, and ultimately help you communicate those stories to the audiences you need to reach.

—Joe Hopper, Ph.D.

What do we know about the kinds of stories and research reports that can achieve this?  A recent study reported in The New York Times provides some clues.  Researchers looked at which types of NYT articles get e-mailed by readers the most.  One surprising result was that science articles (stories that convey research findings!) get e-mailed the most.  The reason is not that readers like science more than other topics – it’s that science articles, at least those in The New York Times, have content characteristics that people find compelling and want to share.   The researchers analyzed thousands of articles (both science and non-science) and noted the characteristics of those most likely to go viral.  They tend to be articles that:

• Elicit strong emotions
• Have positive themes rather than negative themes
• Inspire awe, or a sense of discovery and grandeur beyond oneself
• Challenge the reader intellectually
• Are surprising in some way

When it comes to market research, this does not mean that all your research presentations need to be awe-inspiring and emotional calls to action.  But it does mean that your research should tell a story that relates to people and what they do, feel, and care about.  It should show them opportunities as well as barriers.  It should provide context.  The numbers do not need to be easy, but remember they help tell the story rather than being the story.  If you’re writing a story for media placement and would love to see your story leveraged beyond the initial hit, incorporate as many of these characteristics as you can.

A lot of surveys and market research reports today sit buried in PowerPoint charts that nobody has bothered to read – a sorry state of affairs that inspires Versta’s mission to help you turn data into stories.  With the right approach, your managers and audiences will not only listen to your findings and act on them, but they will be eager to share them with their managers and their friends and family as well.

Joe Hopper, Ph.D.

The first approach is to mine data.  Find data that you already have, analyze that data, use it to answer interesting questions, and then tell the story.  For example, financial services, HR, benefits, and other outsourcing companies like Hewitt Associates track a wealth of data about employees and employers as part of their business.  They can then use that data to write interesting stories about how many people cash out their 401(k) plans when leaving a job, or how many companies are switching healthcare plans as costs continue to rise.

The second approach is to generate new data by commissioning a survey that answers interesting questions to drive your story.  For example, one client I worked for got significant story placements by surveying people about skin cancer myths and reasons for not taking simple precautions like wearing sunscreen.  Another gets ongoing coverage for a survey of IT professionals about future trends in the marketplace.

With both approaches, the trick is to (1) build a credible foundation with expert research and data analysis, and (2) turn data into stories. Versta Research can help you with both pieces.  We can help you answers questions with expertise, turn data into stories, and ultimately help you  communicate those stories to the audiences who need them most.

–Joe Hopper, Ph.D.