Archive for the ‘Public Relations’ Category

Webinar on Polling for News and PR

Tuesday, January 24th, 2012

Election years are a perfect time to learn about (and brush up on) the fundamentals of survey research.  Not only are the airwaves inundated with public opinion polling, but methodological experts are called upon to talk about developments and current best practices as new technologies and methods become central to measuring consumer and public opinion and behavior.

This week the Poynter Institute is offering a webinar of particular interest for PR professionals, whether or not you care about political polls.  We also recommend it for any marketing professional because this type of polling is an exemplar of what all marketing research tries to achieve in measuring what people think and what they are likely to do. (more…)

Best Practices for Using Statistics in PR

Wednesday, November 9th, 2011

One powerful way to gain visibility and credibility in your marketplace is by sponsoring survey research that documents problems and solutions in areas where you have expertise.  To be successful, it requires (1) rigorous research carefully designed to uncover the right topics, and (2) savvy PR work that uses data to tell a credible and compelling story.

The Public Relations Society of America (PRSA) and the American Statistical Association have just published a handy guide for PR professionals that outlines best practices for using, interpreting and reporting statistics in press releases and other PR materials.  Some of those best practices include the following: (more…)

Internet Surveys and the Associated Press (AP)

Thursday, September 29th, 2011

Is it true that the Associated Press refuses to carry stories from online surveys?  Yes, as odd as that seems nowadays.  But news media face a difficult problem given how easy it is to conduct biased public opinion polling, especially now with online panels and social networks.  So some news organizations like the Associated Press (AP), The New York Times, and ABC News have developed guidelines that specify for a survey or public opinion poll to be valid and reliable, it must be conducted by telephone.

(more…)

An Interactive Graph for Choosing Sample Size

Thursday, June 9th, 2011

A good chart is the best way to understand the law of diminishing returns when it comes to sample size.  So for our June 2011 newsletter we built an interactive graph for choosing sample size.  It’s cool, educational, and useful.  Moreover, it will show you just how mind boggling the numbers behind sampling can be.  It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.

(more…)

Online Surveys Have Same Accuracy as Phone

Thursday, May 26th, 2011

A new study presented by two professors from Harvard University and the University of Massachusetts at Amherst was probably one of the liveliest and potentially disruptive presentations at least week’s annual meeting of the American Association of Public Opinion Research (AAPOR) in Phoenix.

Why?  Because their research challenges the beliefs of many AAPOR-ites who disregard most online research as being theoretically indefensible since it is not based on probability sampling.  The research presented was based on parallel surveys conducted last year, designed to allow careful comparison of three survey modes:  (more…)

Advice for PR Surveys: Avoid Numeric Scales

Thursday, April 28th, 2011

As much as we love numbers, we find ourselves often advising clients against using numeric scales in their surveys.  A numeric scale is any response format that asks people to give a number within a certain range to indicate the strength of their feeling or opinion.  The insanely popular survey question used to calculate Net Promoter Scores is a good example:

“How likely is it that you would recommend Acme Solutions to a friend or colleague?  Please answer on a scale from zero to ten, where zero means not at all likely, five is a neutral score, and ten means extremely likely.”

There are many good reasons to use numeric scales and many types of research for which numeric scales are optimal.  The NPS scale is good because it has eleven points with meaningful endpoints  and a meaningful midpoint.  Research shows that scales like this can be highly reliable and valid, with sufficient variability to allow for sophisticated statistical modeling.

But if your objective is to use survey data for marketing materials, public relations, news releases, or white papers, numeric scales make things difficult.  They are not easy to summarize in words, and if you want to use charts that tell quick, compelling stories, you will end up having to do something like this:

A Poor Fit: Pie Charts and Numeric Scales

(more…)

What People Think of Surveys

Wednesday, March 30th, 2011

One resource that we give far too little thought in market research are the people who offer their time and thoughts about the stuff we are researching.  They are truly the lifeblood of market research.

An article in the spring 2011 issue of Public Opinion Quarterly looks at trends over the last thirty years in Americans’ view of polling and market research surveys.   The news is mixed.  While the vast majority feel that public opinion polling is generally a good thing, fewer feel that market research surveys serve a useful purpose, and trust in the industry is not so great.  The most worrisome news (but not surprising, given the number of truly bad surveys flooding our lives nowadays) is the steady decline in people saying that participating in research is interesting and in their best interest:

Declining Satisfaction with Surveys

In addition to the chart above, key statistics outlined in the article include: (more…)

Allstate’s PR Misstep with a Silly Study

Friday, February 4th, 2011

Research can be powerful for PR efforts because new data and analysis often help anchor a story with facts and figures that have a deep interest and appeal to readers.  But if the research is not clearly designed to support key messages in your PR plan, you may find yourself apologizing to managers for a campaign that failed.  Worse yet, you may find yourself apologizing to an audience who got exactly the wrong message.

Last week, Allstate found itself in this unhappy situation when it released findings from “research” showing accident rates based on drivers’ astrological signs. (more…)

Sampling for Dummies

Friday, December 10th, 2010

The idea of sampling is not hard to grasp, but methodological discussions about sampling can quickly move into the higher reaches of mathematics and probability that confuse even researchers who are not statisticians.  Ever wonder what we are talking about when we refer to “probability samples?”  Or the rationale for not reporting margins of error?  Or why there is no such thing as a “statistically significant sample size?”

AAPOR and the ASA are offering a great way for non-statisticians to learn more about sampling.  AAPOR is the American Association of Public Opinion Research, and the ASA is the American Statistical Association.  In February they are hosting an introductory webinar on sampling for non-statisticians that that we recommend.  It is being taught by a senior statistician at NORC, and will cover such topics as: (more…)

Eliminate Your Margin of Error

Thursday, October 14th, 2010

Should you state a survey’s margin of error in your press release when pitching a story to the media?  In our view, the answer is no.  Why not?  Because margins of error refer to sampling error only, not about the overall accuracy or error of the survey itself.

But how many readers of your news story understand what sampling error is?  For that matter, how many researchers understand what sampling error is?  It was only after four college semesters of advanced mathematical statistics that I finally “got it.”  More importantly, how many readers understand that there are many other potential sources of survey error?  Most undoubtedly assume that all error is somehow accounted for when you confidently proclaim the margin of error being ±4%.  (Or, more absurdly, ±3.6% or even ±3.57% — examples of phony accuracy like this are all too easy to find.)

In short, margins of error are misleading because they deal with only one source of error.  They convey a false sense of accuracy.  And they should not be used.  This idea is not always popular among colleagues and clients.  But take comfort:  One of the giants of public polling, Harris Interactive, refuses to report margins of error in its work, for precisely the reasons outlined above.  Here we quote their methodological statement that accompanies every press release and report they issue: (more…)