Polling Ignorance
by Mark Rhoads
Issue 115 - September 10, 2008

Many campaign managers in both parties agree that one of the nicest people in the polling business is Kellyanne Fitzpatrick Conway. She sticks to the facts without spin and she is very smart. She was elected to Phi Beta Kappa at Oxford. I interviewed Kellyanne recently for another article and I asked this question. If the approval ratings for Congress hover at only 19 percent--ten points below President Bush, and if Congress has been led by Democrats in both houses for two years, how come Democrats are preferred over Republicans by an average of 11 points on the generic ballot question?

She gave me some reasons and then I asked, do pollsters ever ask if the person being polled even knows which party controls Congress? Kellyanne blew me away. Yes she said that question is sometimes asked as a screen question and 38 percent of the respondents admit that they are not sure which party is in charge of Congress! With all we hear and read about Speaker Nancy Pelosi and Sen. Harry Reid, apparently well over a third and approaching almost forty percent of Americans are "not sure" which party these people belong to.

Polls measuring the horse race between Sen. Obama and Sen. McCain have been fairly stable for seven weeks and the margin is often inside the margin of error. The Aug. 15 Gallup Tracking poll has the race tied at 44 percent each. While Obama has had a slight lead of about three points on average, this race is very competitive and often within the margin of error. There is a reliable pattern. If you want to show the best lead for Obama, then take a small sample of registered voters. If you want to show the best lead for McCain, take a large sample of likely voters.

Should we trust polls? If you read many of them and watch their methodology, you can rely on the fact that they reflect general trends and movements over a period of several days. But an honest pollster will admit to the limitations of polls. It is getting harder all the time to get good samples and takes more time and effort to reach enough people for a good sample. Kellyanne Conway still thinks it is possible to get a good sample with people who own land phone lines in their homes but she admits that it now takes four nights instead of three to reach enough people for a good sample. When asked about the problem of nonpublished cell phone numbers, she admits that makes it very hard to get the best samples and might soon make it impossible to get good samples even with random digit dialing methods.

Polls cannot correct for people who lie, people who do not want to take the 20 minutes to answer questions when a favorite evening TV show is on, and they cannot correct for the kind of civic ignorance that is demonstrated by the fact that almost forty percent are not sure which party controls the Congress. In fact, it is amazing that pollsters do as good a job as they often do but when events move quickly and the calendar gets close to an actual decision.

If you follow politics, you follow polls. But don't fool yourself into thinking you are observing anything close to an exact science in measuring public opinion. At best, you can hope to spot early trends and wonder how much of the commercial messages are getting through.

Mark Rhoads

One post-script : Pollsters do not as a rule call only registered voters who are listed on county or city records of voters. They call up people who are land line subscribers and hope they are being honest when they say they are registered to vote. Also, they hope people are being honest when they say they are likely to vote. The best samples are the most expensive to get because they require so much time and many calls and so many screen questions. That is why pollsters do not go to that expense unless they really have to or have a client willing to pay for the most accurate results.