After leaving academic research for the world of applied research, I found myself doing a lot of surveys for public relations. These surveys are designed to uncover surprising or newsworthy nuggets of data that companies use to focus attention on topics relevant to their concerns. My first boss despised such work, believing that a public relations agenda somehow dirtied the objectivity of rigorous research. In contrast, I love creating these surveys. Why? Let me count the ways:
If there is one super important lesson to be learned from this year’s round of election polling, it is that online surveys work. Google Consumer Surveys, which use non-probability online samples, predicted the election far better than Gallup did. And online surveys, overall, outperformed telephone surveys.
The New York Times’ Nate Silver compiled polling results from 23 organizations that conducted at least five surveys in the final three weeks of the campaign. He calculated how far their projections were from the actual outcome of the presidential race. Google (a fully automated, online solution) came in second place, predicting the actual outcome within 1.6 percentage points. Gallup (using “gold standard” telephone methods) came in last, predicting the outcome within 7.2 percentage points.
During a presidential election year there is no escaping the flurry of public opinion polling and the intense scrutiny that surveys get from the media. But love it or hate it, there are excellent reasons to pay close attention to this year’s political polling.
Last week there was a flash of news coverage for a PR-driven survey that documented these important facts:
I recently saw a press release about a study showing that only 19.5% of news release headlines are optimized for SEO. It brought to mind all kinds of issues about how best to report numbers in press releases. In particular it highlighted the important issue of whether specific numbers are meaningful and whether they communicate a misleading sense of precision.
For example, when a survey reports a margin of error to any decimal place, it suggests a level of precision that is misleading. Do a quick search, and you’ll find press releases reporting margins of sampling error such as +/- 4.8%, +/- 10.5%, or +/- 1.85%. These numbers are based on sample size formulas that assume perfect random sampling and one hundred percent response rates, which are almost never achieved. (more…)
Journalists and newsrooms are inundated with ever more data, information, and press releases that highlight survey findings in hopes of grabbing reporters’ and readers’ interest. While many surveys are poorly done and grossly self-serving, findings from other research surveys continue to generate newsworthy stories.
Consider this view recently expressed by a business columnist at the Chicago Tribune:
Election years are a perfect time to learn about (and brush up on) the fundamentals of survey research. Not only are the airwaves inundated with public opinion polling, but methodological experts are called upon to talk about developments and current best practices as new technologies and methods become central to measuring consumer and public opinion and behavior.
This week the Poynter Institute is offering a webinar of particular interest for PR professionals, whether or not you care about political polls. We also recommend it for any marketing professional because this type of polling is an exemplar of what all marketing research tries to achieve in measuring what people think and what they are likely to do. (more…)
One powerful way to gain visibility and credibility in your marketplace is by sponsoring survey research that documents problems and solutions in areas where you have expertise. To be successful, it requires (1) rigorous research carefully designed to uncover the right topics, and (2) savvy PR work that uses data to tell a credible and compelling story.
The Public Relations Society of America (PRSA) and the American Statistical Association have just published a handy guide for PR professionals that outlines best practices for using, interpreting and reporting statistics in press releases and other PR materials. Some of those best practices include the following: (more…)
Is it true that the Associated Press refuses to carry stories from online surveys? Yes, as odd as that seems nowadays. But news media face a difficult problem given how easy it is to conduct biased public opinion polling, especially now with online panels and social networks. So some news organizations like the Associated Press (AP), The New York Times, and ABC News have developed guidelines that specify for a survey or public opinion poll to be valid and reliable, it must be conducted by telephone.
A good chart is the best way to understand the law of diminishing returns when it comes to sample size. So for our June 2011 newsletter we built an interactive graph for choosing sample size. It’s cool, educational, and useful. Moreover, it will show you just how mind boggling the numbers behind sampling can be. It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.