Knowledge

1. Why am I or my friends never included in political polls conducted
for the media?

The reason is fairly simple. There are about 200 million adult or voting age Americans. But the average poll has a sample size of 1,000 adults. This
means that only one person in 200,000 will be included in any one national
or state poll. To put it another way, it would take 200,000 polls with samples
of 1,000 for pollsters to get around to all Americans - and this assumes no
one is called twice. Of course, national and local media organizations conduct several polls in one year. The number of national or local media political polls
you see in a single year is about 250 - but can vary depending on where you
live. But even two hundred fifty polls in a single year means your chance of
being interviewed at least once is still small.

2. Is a larger sample always better than a smaller sample?

Larger samples are generally more precise, but sometimes not. The important
rule in sampling is not how many poll respondents are selected but, instead,
how they are selected. A reliable sample selects poll respondents randomly
or in a manner which insures that everyone in the area being surveyed has a known chance of being selected.

3. How can a sample of only 800 or 1200 truly reflect the opinions of
200 million Americans within a few percentage points?

Sampling methods and measures of sample reliability or precision are derived
from a mathematical science called statistics. Statistics is a subject taught in colleges and some high schools. Text books on the subject are available in
most libraries. At the root of statistical reliability is probability; i.e., the odds of obtaining a particular outcome by chance alone. As an example, the chances
of having a coin come up heads in a single toss is 50%. Heads is one of only
two possible outcomes. The chance of getting two heads in two coin tosses is
less because two heads are now only one of four possible outcomes; i.e., a head/head, head/tail, tail/head and tail/tail. As the number of coin tosses increases, it becomes increasingly more likely to get outcomes which are
either very close to half heads or exactly half because, as with two coins,
there are more ways to get such outcomes. Sample survey reliability works the same way - but on a much larger scale. As in coin tosses, the most likely
sample outcome is the true percentage of whatever it is we are measuring
across the total population. Next most likely are outcomes very close to this
true percentage. A statement of potential margin of error or sample precision reflects this and often appears in poll stories. Using a sample of 1000 as an example, the statement could read: the chances are 95% of coming within
+/- 3% of a hypothetical survey conducted among all members of the
population. This means that 95% of all samples which could possibly be drawn
will yield an outcome within 3% of the true percentage among the population.
Keep in mind that estimates of potential sample error always assume random samples. But even in true random samples, precision can be compromised by other factors such as the wording of questions or the order in which questions were asked. There is no single ideal sample size. Samples of any size have
some degree of precision. The question is always whether there is sufficient precision to draw conclusions as determined by statistical formulae.

4. Why do polls often disagree with how I and my friends feel about things?

It is unlikely that you have a circle of friends as diverse as a randomly selected sample described before, a sample of the entire nation or of the state where
you live. This would mean you have friends from all neighborhoods, of all
ages, very wealthy friends and friends with no wealth, who are from all walks
of life and with educational levels ranging from grade school to post-graduate.
No poll has ever shown all people feeling exactly the same way on one issue.
So the next time you see a poll showing only 30% in agreement with your point
of view, remember, although you may not be in the majority, it still means that
60,000,000 Americans feel the way you do.

5. Why is job approval for elected officials asked so often in polls?

Job approval is a question asked in polls for over 50 years, asked about presidents from Harry Truman to Bill Clinton. It is a measure of potential
electoral success for an incumbent president (or state and local elected
officials) and is useful for trend purposes. Incumbents who fail to win approval
from a majority of the public for the job they are doing are very often in trouble
on election day. Experience shows presidential job approval has a lot do with
how well people think things are going in the country today. High job approval does not mean the office-holder is necessarily held in high, personal esteem.
Job approval is just one question which when analyzed in combination with response to other questions gives a good overall picture of how an incumbent
is perceived.

6. How accurate are polls?

The NCPP analyzed final presidential election polls conducted by the national media dating back over 50 years. When compared with actual election
outcomes, average poll error for presidential elections between 1956 and
1996 has been declining. Average poll error on each candidate during this
period was 1.9 percentage points. Important to this analysis of accuracy is
that most of these polls were conducted within days or even hours before
election day. Polls conducted 1-2 weeks before election day or even longer
by local newspapers and TV stations cannot usually be expected to closely
match election outcomes. Earlier polls are intended to monitor the success of campaigns and to identify the issues or events which will influence voter preferences on election day.

7. Can wording of questions bias poll results? How questions in a
poll are worded is as important as sampling procedure in obtaining
valid results.

Most professional polling organizations and their media clients review the
wording of questions as carefully as editors would examine a manuscript
before publication. This process usually calls for a review of several drafts
prior to fielding a poll. Questions are checked for balance, that is, are they
worded in a neutral fashion without taking sides on an issue? Does the
question represent both sides of an issue fairly? Answer choices read to
poll respondents must also be balanced; e.g., approve or disapprove, favor
or oppose. The order of questions must be in logical order. That is, general
questions are asked before specific questions. For example, overall job
approval of an incumbent must be asked before specific questions are asked
which may remind respondents about the incumbent’s successes or failures.
The same goes for questions asking respondents which side they take on
an issue which may influence a later question about opinion of a candidate
who takes the opposite side. Questions are written using clear, unambiguous,
concise language to insure all respondents regardless of educational level
understand them. And since most polls are conducted by phone, a writing
style suitable for the ear is often adopted as opposed to a style more
suitable for reading.

8. Why should I respond to a poll?

Each year, tens of thousands of Americans respond to poll interviews on
subjects of national interest. They do so because they want their opinions
heard. A poll or survey is an unbiased way for people to make their views
known to each other, to their government, to businesses, toeducators and
many other institutions. This is one way for average Americans to add their
voices to the debate over important issues of our day. It is our sense that
people want to be included in polls. Polling is only a part of a $3 billion
U.S. survey research industry that affects your life in many ways. Medicines
in your cabinet became available only after they had been tested for
effectiveness and safety, tested among samples of patients. Manufacturers
use sampling methods to guard against producing defective products.
Businesses conduct market research by sampling consumer opinions to help
develop products people want and avoid costly mistakes. Even the Bureau of
the Census only interviews everyone in the nation once every ten years. The
Census Bureau along with other government agencies also conduct sample
surveys to obtain social and economic data we see in the news. One thing
legitimate survey researchers will never do is try to sell you something.
That is called telemarketing. According to direct marketing industry
estimates, there are now 2 million telemarketers who call the general public
to sell products and services. The number of survey research interviewers who
conduct telephone interviews is probably only a small fraction of 2 million.