Election years are a perfect time to learn about (and brush up on) the fundamentals of survey research.  Not only are the airwaves inundated with public opinion polling, but methodological experts are called upon to talk about developments and current best practices as new technologies and methods become central to measuring consumer and public opinion and behavior.

This week the Poynter Institute is offering a webinar of particular interest for PR professionals, whether or not you care about political polls.  We also recommend it for any marketing professional because this type of polling is an exemplar of what all marketing research tries to achieve in measuring what people think and what they are likely to do.

The webinar (scheduled for January 26, 2012 with archived replay available afterwards) is being co-sponsored by the American Association of Public Opinion Research, and will be led by Claudia Deane, associate director for public opinion and survey research at the Kaiser Family Foundation.  It is designed for non-researchers, and will focus on:

• The science of how polling and survey research works
• How to determine the legitimacy of a poll and the rigor of survey research
• How to assess the quality and usefulness of survey questions

If you miss the webinar or hunger for more, Versta Research has (and can point you toward) additional resources to help PR professionals, marketing professionals, and market researchers understand, keep abreast of, and communicate the importance of research.  We are one of the few research firms in the industry with a background in university teaching and academic research striving to bring that expertise to the practical worlds of communications campaigns and marketing insight.  Please give us a call and we would be happy to help you further.

–Joe Hopper, Ph.D.

One powerful way to gain visibility and credibility in your marketplace is by sponsoring survey research that documents problems and solutions in areas where you have expertise.  To be successful, it requires (1) rigorous research carefully designed to uncover the right topics, and (2) savvy PR work that uses data to tell a credible and compelling story.

The Public Relations Society of America (PRSA) and the American Statistical Association have just published a handy guide for PR professionals that outlines best practices for using, interpreting and reporting statistics in press releases and other PR materials.  Some of those best practices include the following:

• Disclose who paid for the work, and who did the research
• Clearly describe how the research was conducted
• Describe the sample of the survey, and whether it was truly random
• Remember that almost any survey can suffer from bias
• Rely on descriptive statistics to report the data
• Remember that all statistical research includes some level of uncertainty
• Clearly describe trends and effects
• Avoid making claims about the future based on recent history
• Use causal statements cautiously, as they are very hard to prove
• Run your insights by the person who did the research to be sure the data support it

Turning data into stories is never easy, but as this partnership between PRSA and the ASA makes clear, it is becoming more and more important for PR professionals to have a working knowledge of statistics.  And if Shonali Burke’s report from the 2011 PRSA conference is any indication, PR professionals are embracing the challenge.

Versta Research can help you with this challenge.  We are experts in research.  We know how to design surveys and report statistics that get your message heard.  Give us a call at (312) 348-6089 when you need a compelling custom survey or omnibus survey to help you tell your story.

–Joe Hopper, Ph.D.

Is it true that the Associated Press refuses to carry stories from online surveys?  Yes, as odd as that seems nowadays.  But news media face a difficult problem given how easy it is to conduct biased public opinion polling, especially now with online panels and social networks.  So some news organizations like the Associated Press (AP), The New York Times, and ABC News have developed guidelines that specify for a survey or public opinion poll to be valid and reliable, it must be conducted by telephone.

It is worth understanding their rationale even if you do not pitch research stories to the media because the guidelines provide a fascinating glimpse into current methodological debates about probability sampling, inferential statistics, and the rapidly changing world of online data collection and analysis. If your organization does use research for public relations and marketing, then a deeper understanding will help you offer recommendations to your organization about conducting PR research.

In our Fall 2011 Newsletter, we outline media guidelines for reporting on survey research.  Whether your goal is getting research into the board room to influence top decision makers, or in front of the public to promote your brand, you need to know the standards of rigor against which the research will be judged.

Our commitment at Versta is to advise you on the best research mode for your campaign strategy, and to conduct rigorous research that can withstand the highest levels of scrutiny.

–Joe Hopper, Ph.D.

“How likely is it that you would recommend Acme Solutions to a friend or colleague?  Please answer on a scale from zero to ten, where zero means not at all likely, five is a neutral score, and ten means extremely likely.”

There are many good reasons to use numeric scales and many types of research for which numeric scales are optimal.  The NPS scale is good because it has eleven points with meaningful endpoints  and a meaningful midpoint.  Research shows that scales like this can be highly reliable and valid, with sufficient variability to allow for sophisticated statistical modeling.

But if your objective is to use survey data for marketing materials, public relations, news releases, or white papers, numeric scales make things difficult.  They are not easy to summarize in words, and if you want to use charts that tell quick, compelling stories, you will end up having to do something like this:

A Poor Fit: Pie Charts and Numeric Scales

The problem with this graphic is that the numbers inside the pie chart are confusing, and the words highly willing, not willing, and neutral were never actually used or selected by most respondents.  Somebody wrote the questionnaire and used a numeric scale without first considering how they were going to use and present the data.

Here is the question that was used: “When thinking of your financial investments, how willing are you to take risks? Please use a 10-point scale, where 1 means Not At All Willing, and 10 means Very Willing.” Had this organization been working with us, we would have advised using a scale like this:

• Not at all willing
• Not very willing
• Somewhat willing
• Very willing

Depending on their objectives and the story they wanted to tell, we may have advised including a “Neutral” category as well.  A scale based on words rather than numbers would have been much more useful in talking about how investors are willing or not willing to take risks.

There are no “magic scales” or response categories that should always be used.  If you find a research professional arguing otherwise, chances are they are not listening carefully to what you need, nor are they thinking much about how the data they collect will deliver on the core objectives of your research.  Telling a story with data requires thinking about the very last endpoint (presentation of data to the audiences you want to reach) from the very beginning (conceptualizing and designing the research).

—Joe Hopper, Ph.D.

Last week, Allstate found itself in this unhappy situation when it released findings from “research” showing accident rates based on drivers’ astrological signs. The findings were absurd, showing that Virgos were 700% more likely to get into car accidents than Scorpios.  The worst part was that some consumers believed it meant Allstate was analyzing astrological signs as part of its insurance underwriting process.  Allstate scored a big hit for media placement, but found itself retracting the research and apologizing soon afterwards.

Regrettably, there are a lot of silly surveys, presumably to support PR.  The PR folks and the research agencies proffering this kind of “research” clearly miss the whole point of doing research for PR, which is to document problems and solutions.

Here’s our suggestion for how to do better.  The key to great research for PR is to focus on your core message and then use data to support it.  Forget about grabbing media headlines with sexy and attention-getting statistics.  Honest and compelling information about issues that concern people will provide a story with authenticity you can leverage.  Here are four steps we suggest:

• Articulate a clear message that supports your business objectives
• Brainstorm relevant and important problems and solutions that are related to that message
• Write “dream headlines” around those problems and solutions using hypothetical facts and figures to support the message
• Conduct real research to uncover and document those problems and solutions

Allstate didn’t do this, admitting that their effort was designed to be “entertaining” rather than substantive.  Had they thought about their message first and how research can be used support it, we doubt they would have tried to grab attention in the way they did.

For additional tips, read our recent article in Public Relations Tactics entitled “How to Create Surveys for PR Stories.”  If you are contemplating research for PR and want to ensure avoiding the pitfalls of silly surveys, feel free to give us a call.  We can advise you and your team on an optimal approach for conducting rigorous, substantive, and media-worthy studies to support your PR efforts.

—Joe Hopper, Ph.D.

As of this writing, Versta is one of 67 prominent survey and polling organizations currently supporting the initiative. The initiative is designed to create protocols and recognition for regular disclosure of methods when survey organizations conduct public polls.

Trust in survey research and public polls is critical not only for the public good, but for the important work that we do for our clients.  Our clients rely on us to give them accurate data and truthful assessments. They can’t make good decisions without that. And they rely on us to ensure that when they share data with the public, it is backed by the most rigorous and defensible methods available in the social sciences. Transparency is one means of ensuring that all organizations are continually held to the highest standards of rigor and accuracy.

AAPOR began the initiative in 2009 “to encourage routine disclosure of methodological information from polls and surveys whose findings are released to the public.” The organization lists the following as key elements of the efforts now underway:

• The association’s public recognition of excellence in transparency;
• A system for collecting and archiving disclosure information;
• Education for organizations in how to make transparency a routine part of their work;
• Outreach to survey sponsors, users of survey data and the public to reinforce the value of transparency and to publicize the organizations that participate in the initiative;
• Collaboration with other associations and academic disciplines in these activities.

One benefit of working with an organization like Versta Research is that you can fully trust the outcomes of the research, and you can trust that we have the expertise to defend it.  You can also trust that we work diligently within the industry to protect the value of our collective investments in good research.

—Joe Hopper, Ph.D.

There are three tiers of survey research common in public relations, only two of which can truly optimize your PR:

Serious Surveys.  Beyond providing rich material for news releases, serious surveys are designed to establish true thought leadership.  They are leveraged for publication through white papers, conference presentations, or peer-reviewed journals.  The surveys we conducted and published about ulcerative colitis are excellent examples, and for the last three years have been used by the client to demonstrate a deep understanding of the issues faced by patients and physicians.

Solid Surveys. Solid surveys answer interesting questions or document issues that people care about, with solutions that people can use.  The primary goal is media placement, but a solid survey can be leveraged further.  For example, one client we worked for got significant story placements by surveying people about skin cancer myths and reasons for not taking simple precautions like wearing sunscreen.  Another gets ongoing coverage for a survey of IT professionals about future trends in the marketplace.

Silly Surveys. These are unfortunately not designed to answer authentic questions, but to grab attention in ways that may not even be relevant to your business.  We saw one survey not long ago that measured which song titles, from among five or six listed, captured people’s optimism or pessimism about the decade ahead.  The client?  An office products company.  Another recent survey announced the percentage of women who would give up TV, cell phones, computers, or sex in exchange for losing 10 pounds.  Who cares?  These are not meaningful comparisons; they’re just silly.

To be fair, silly surveys can attract good media, but there are costs.  First, they diminish your (and our) long-term credibility as journalists start to ask whether they’re reading yet another gimmick survey or whether your latest effort offers something of real value.  Second, there is an opportunity cost.  You could be investing in information that not only gets media attention, but is leveraged in multiple ways and for many years.  We have a client for whom we did a solid survey back in 2004, which they are still using on their website to engage consumers in 2010.

If you’re in the business of PR, we suggest leaving the silly surveys to Facebook polls.  Instead, focus on building a credible foundation with expert research and data analysis.  Versta Research can help you with this.  We can help you answer questions with expertise, turn data into stories, and ultimately help you communicate those stories to the audiences you need to reach.

—Joe Hopper, Ph.D.

Here are four tips for writing omnibus survey questions to ensure that your effort is successful:

1.  Tailor your questions to the interview mode.  Some omnibus surveys are conducted by phone, and some are done online.  Each mode requires a different style of asking questions.  For example, having an agree/disagree scale with seven points would work well for an online survey.  But on the phone, each of those points would have to be labeled with words and read out loud to the respondent.

2.  Ask the right number of questions.  If you ask too many questions, you may be defeating the cost advantage of an omnibus.  But if you ask too few questions, you will not get enough depth and leverage to tell a story.  Usually you need points of contrast or context, so you will need to ask more than just one or two direct questions.  Plan on asking five to eight questions for an omnibus survey.

3.  Keep a broad focus.  Most U.S. omnibus surveys include 1,000 respondents representing the full adult population.  Ask questions that will apply to all or most of them so that you are taking advantage of the full sample size.

4.  Ask questions that relate to your campaign. Your goal is to highlight the product or service you are selling.  Sensational, funny, or outrageous questions and survey findings will sometimes give you a quick flash of attention, but not the kind of substantive leverage that will truly carry a message into multiple forums over the course of several months.

At Versta, we will work with you on all phases of the omnibus process, including concept development, design, drafting and revising the questions, and then analyzing  and reporting the findings in a way that helps you tell the story.  Have more questions?  Give us a call — we are happy to help.

—Joe Hopper, Ph.D.

But new technologies have made data collection more efficient and less expensive, so omnibus surveys are not always the best option.  For many clients, a fully customized survey might be less expensive than an omnibus survey.

Here are the two most important factors that affect whether an omnibus survey will make sense for you:

1.  The number of questions you want to ask. If you have a small number of questions – typically no more than five or six – then an omnibus survey can be a good option.  Omnibus surveys are designed to aggregate just a few questions from many different clients, and are priced accordingly.  If you have more than five or six questions, then fielding a short custom survey can often be priced as competitively as an omnibus, and you gain some significant advantages.

2.  Who you want to survey. Omnibus surveys work only if many clients want to get survey responses from the same group of people.  As such, an omnibus survey can be a good option if the population you want to study is broadly defined.  In fact, most omnibus surveys are “general population” surveys, which means they rely on a sample of the entire U.S. adult population.  If you start narrowing your target group (such as moms with children under the age of 5), then a custom survey that recruits those specific people is likely to be better.

You need both conditions for an omnibus survey to make sense.  In other words, you need to have just a handful of questions, and you need to have a broadly defined group as your target.  Many surveys meet these conditions, and when they do, they save you money.

If you are not sure whether an omnibus survey makes sense for you, we would be happy to price both options for you, outlining the advantages and disadvantages of each.  We’ll advise you on the optimal approach given your objectives.  Feel free to give us a call (312-348-6089).

-Joe Hopper, Ph.D.

How does it save you money?  The biggest cost in conducting a survey is getting it set up and then finding a statistically representative sample of respondents to answer the questions.  Once you’ve got them on the phone or online answering survey questions, it is costs little to ask them “just one more question.”  You want to avoid asking questions for more than fifteen minutes, but suppose that you have fifteen questions you want to ask, and Acme Sports has fifteen questions, and Best Wireless has another fifteen questions.  Rather than each of you conducting your own short survey and duplicating all the work (and cost) of set-up, sampling, and communicating with respondents, you can pool your resources to do one survey.  Each of you pays proportionally to how much of the survey is devoted to your questions.

Here is an example how an omnibus survey works (shown second) versus traditional custom surveys (shown first) — the costs shown are for illustrative purposes only:

Since Versta Research and other research firms have clients asking for short and inexpensive surveys all the time, we do the work of finding the other companies to partner with you on an omnibus.  Versta will advise you on all phases of the work:  How many questions to ask, what to ask, how to ask, and we’ll provide you with several cost and survey-length options so that you can find the right approach for your budget and strategic needs.

Have more questions?  Give us a call at 312-348-6089.  We would be happy to answer your questions and to advise on an approach that works for you.

-Joe Hopper, Ph.D.