Last week, Allstate found itself in this unhappy situation when it released findings from “research” showing accident rates based on drivers’ astrological signs. The findings were absurd, showing that Virgos were 700% more likely to get into car accidents than Scorpios.  The worst part was that some consumers believed it meant Allstate was analyzing astrological signs as part of its insurance underwriting process.  Allstate scored a big hit for media placement, but found itself retracting the research and apologizing soon afterwards.

Regrettably, there are a lot of silly surveys, presumably to support PR.  The PR folks and the research agencies proffering this kind of “research” clearly miss the whole point of doing research for PR, which is to document problems and solutions.

Here’s our suggestion for how to do better.  The key to great research for PR is to focus on your core message and then use data to support it.  Forget about grabbing media headlines with sexy and attention-getting statistics.  Honest and compelling information about issues that concern people will provide a story with authenticity you can leverage.  Here are four steps we suggest:

• Articulate a clear message that supports your business objectives
• Brainstorm relevant and important problems and solutions that are related to that message
• Write “dream headlines” around those problems and solutions using hypothetical facts and figures to support the message
• Conduct real research to uncover and document those problems and solutions

Allstate didn’t do this, admitting that their effort was designed to be “entertaining” rather than substantive.  Had they thought about their message first and how research can be used support it, we doubt they would have tried to grab attention in the way they did.

For additional tips, read our recent article in Public Relations Tactics entitled “How to Create Surveys for PR Stories.”  If you are contemplating research for PR and want to ensure avoiding the pitfalls of silly surveys, feel free to give us a call.  We can advise you and your team on an optimal approach for conducting rigorous, substantive, and media-worthy studies to support your PR efforts.

—Joe Hopper, Ph.D.

As of this writing, Versta is one of 67 prominent survey and polling organizations currently supporting the initiative. The initiative is designed to create protocols and recognition for regular disclosure of methods when survey organizations conduct public polls.

Trust in survey research and public polls is critical not only for the public good, but for the important work that we do for our clients.  Our clients rely on us to give them accurate data and truthful assessments. They can’t make good decisions without that. And they rely on us to ensure that when they share data with the public, it is backed by the most rigorous and defensible methods available in the social sciences. Transparency is one means of ensuring that all organizations are continually held to the highest standards of rigor and accuracy.

AAPOR began the initiative in 2009 “to encourage routine disclosure of methodological information from polls and surveys whose findings are released to the public.” The organization lists the following as key elements of the efforts now underway:

• The association’s public recognition of excellence in transparency;
• A system for collecting and archiving disclosure information;
• Education for organizations in how to make transparency a routine part of their work;
• Outreach to survey sponsors, users of survey data and the public to reinforce the value of transparency and to publicize the organizations that participate in the initiative;
• Collaboration with other associations and academic disciplines in these activities.

One benefit of working with an organization like Versta Research is that you can fully trust the outcomes of the research, and you can trust that we have the expertise to defend it.  You can also trust that we work diligently within the industry to protect the value of our collective investments in good research.

—Joe Hopper, Ph.D.

There are three tiers of survey research common in public relations, only two of which can truly optimize your PR:

Serious Surveys.  Beyond providing rich material for news releases, serious surveys are designed to establish true thought leadership.  They are leveraged for publication through white papers, conference presentations, or peer-reviewed journals.  The surveys we conducted and published about ulcerative colitis are excellent examples, and for the last three years have been used by the client to demonstrate a deep understanding of the issues faced by patients and physicians.

Solid Surveys. Solid surveys answer interesting questions or document issues that people care about, with solutions that people can use.  The primary goal is media placement, but a solid survey can be leveraged further.  For example, one client we worked for got significant story placements by surveying people about skin cancer myths and reasons for not taking simple precautions like wearing sunscreen.  Another gets ongoing coverage for a survey of IT professionals about future trends in the marketplace.

Silly Surveys. These are unfortunately not designed to answer authentic questions, but to grab attention in ways that may not even be relevant to your business.  We saw one survey not long ago that measured which song titles, from among five or six listed, captured people’s optimism or pessimism about the decade ahead.  The client?  An office products company.  Another recent survey announced the percentage of women who would give up TV, cell phones, computers, or sex in exchange for losing 10 pounds.  Who cares?  These are not meaningful comparisons; they’re just silly.

To be fair, silly surveys can attract good media, but there are costs.  First, they diminish your (and our) long-term credibility as journalists start to ask whether they’re reading yet another gimmick survey or whether your latest effort offers something of real value.  Second, there is an opportunity cost.  You could be investing in information that not only gets media attention, but is leveraged in multiple ways and for many years.  We have a client for whom we did a solid survey back in 2004, which they are still using on their website to engage consumers in 2010.

If you’re in the business of PR, we suggest leaving the silly surveys to Facebook polls.  Instead, focus on building a credible foundation with expert research and data analysis.  Versta Research can help you with this.  We can help you answer questions with expertise, turn data into stories, and ultimately help you communicate those stories to the audiences you need to reach.

—Joe Hopper, Ph.D.

Keeping your survey efforts honest is critical.  Sincere efforts to document issues through rigorous surveys succeed, and when done correctly, people are influenced by numbers.  But trying to shock or manipulate with sensationalist, biased, and phony data gets you only so far.  For a great survey, keep it honest, rigorous, and valid, and carefully control for bias.  We, at Versta, can help you succeed with this.

-Joe Hopper, Ph.D.

So I’m always encouraged with yet more evidence that people do tell the truth on surveys.  Studies continue to show that surveys and polling are effective because (1) you need to ask only a small representative sample of people what they think to make statistical conclusions about the whole, and (2) the vast majority of people will tell you the truth about what they think.  It would seem that people are less concerned about privacy and keeping their opinions to themselves  than they are about being heard and making a difference.

The latest data demonstrating this was published in an industry magazine from the Marketing Research Association.  If a person volunteers, or is asked, to take a survey multiple times, what happens?  The vast majority will give you the same answers each time they take the survey, even on questions that ask for subjective ratings on 10 point scales.  If a person were lying or just racing through a survey with random answers to collect incentives, this would not happen.  It turns out that most people give thoughtful, truthful answers each time.

Of course you do not want the same person included in your survey more than once, and a good research vendor will do the careful work of ensuring it does not happen.  But when it does happen, it is fascinating that our efforts to get to the truth are so resilient. The study was done by Paul Johnson, a senior analyst at Western Wats.  If you would like a copy of the study, let me know and I will send you a copy (unfortunately the MRA keeps this content under lock and key, so I cannot include a web link here).

A corollary of this research will be a topic for another day, but here’s the gist:

If you want to know the truth about your product or your customers or your competition, just ask, because most people will tell you.

Versta Research can help you with what to ask, how to ask, whom to ask, and how many to ask, and chances are you will be delighted with how much insight you can gain through a systematic and rigorous effort of asking.

–Joe Hopper, Ph.D.

Two researchers recently came up with methods of testing whether polling data is legitimate in a case where a research firm is accused of falsifying publicly released data. The researchers examined the last digit of the numbers released by the polling firm, and calculated the probability of seeing these digits in their specific combinations and frequencies if the polls were legitimate.  It was an ingenious way of using sophisticated statistical methods to examine data where the accused firm refused to disclose details about its methods.  The New York Times calls it “forensic polling analysis” and notes that in this case, the odds suggest that the polling data were fraudulent.

There are a number of leading industry organizations that have outlined ethical guidelines and principles of disclosure for survey research, including:

• AAPOR, the American Association of Public Opinion Research
• NCPP, the National Council on Public Polls
• CASRO, the Council of American Survey Research Organizations

Versta Research believes these guidelines are important to the health and integrity of research.  We adhere to them and encourage our clients to do so as well.  If you are unsure about your ethical obligations when releasing polling data to the public, give us a call and we can help you make sense of the guidelines.

–Joe Hopper, Ph.D.

Surveys and polls can be powerful tools to understand what people are thinking and doing, and they can provide good data for public relations efforts and community outreach.  Unfortunately they can also be gimmicks, which erodes trust in polling and in the organizations sponsoring them.

Here is an example.  Walmart wants to build a new store in the city of Chicago, and so far the city council has said “no thank you.”  So Walmart conducted a “poll” of city residents to prove that the residents back them.  Eric Zorn at the Chicago Tribune has been following the story (so has Chicagoist).  Zorn describes several problems with the poll:  It was conducted via computerized telephone calls; there was only one question; the wording of the question was biased.

Here’s another issue to add to the list.  Walmart says they called every resident listed in the directory in the city of Chicago.  They made over ONE MILLION phone calls in one day.  Smart and effective pollsters do not do this.  A legitimate and accurate poll of Chicago city residents would have relied on a carefully selected and statistically adjusted sample of 800 residents.  Each phone call needs to be carefully handled, managed, and tracked.  You need to know who answered and who did not, where they live, how old they are, and so on.

Imagine the US Census Bureau conducting its annual American Community Survey by robo-calling every household or every telephone number in the nation (how long would it take — a week at most?) then declaring they have accurate data on the population.  Would you trust these results?

If you want to conduct an opinion poll for public relations efforts and community outreach, devote some time and thought to these key issues in designing your poll:

• Write neutral, well-worded questions
• Use an appropriate data collection method
• Design a credible sampling plan
• Carefully manage of the sample, outreach, and data collection

These will help ensure your findings can withstand media scrutiny.

–Joe Hopper, Ph.D.