Gallup's Racially Biased Polls

By Ruy Teixeira, The Center for American Progress and The Century Foundation. Posted October 27, 2004.

From the Center for American Progress and The Century Foundation:

In this edition of Public Opinion Watch:
(covering polls and related articles from the week October 18-24, 2004)

• Bush's Battleground Blues


• Kerry Leads by Thirteen Points among College Students, Twenty-five Points among New Voters


• Gallup Poll Racially Biased


• Bush's Barriers to Re-election


• The Nader Nonfactor


• Tracking the Tracking Polls

Bush's Battleground Blues

Poll results in the battleground states have generally been good for Kerry lately, especially in the most important of these states (for example, Pennsylvania, Ohio, Florida).

In that light, it's interesting to note that five recently released national polls give Kerry solid leads of six to seven points in the battleground states overall. In 2000, these states broke evenly between Gore and Bush, so a six- to seven-point Kerry lead, if real, would be quite significant. Here are the polls and the numbers:

Democracy Corps (October 20-21): 52 percent to 45 percent
Marist (October 17-19): 50 percent to 43 percent
Pew (October 15-19): 49 percent to 43 percent
NBC/Wall Street Journal (October 16-18): 49 percent to 43 percent
Harris (October 14-17): 51 percent to 44 percent

Of course, not all polls show a Kerry lead of this magnitude in the battleground states. But Mark Blumenthal of Mystery Pollster has looked at a substantially wider range of recent polls and finds Kerry's battleground performance running ahead of his national performance in every single one. As Chris Bowers points out over at MyDD, Blumenthal's data show Kerry averaging a 49 percent to 45 percent advantage in the battleground.

And Bush's woes don't end there. The latest unemployment data from the battleground states show Wisconsin and Iowa with increased unemployment rates in the last month and Ohio's remaining stubbornly high at 6 percent.

It could be tough for Bush to get votes where he needs them the most.

Sources used for this section:
Marist College poll of 1,175 adults, released October 18, 2004 (conducted October 17-19, 2004)

Hart/McInturff poll of 1,004 registered voters for NBC News/Wall Street Journal, released October 19, 2004 (conducted October 16-18, 2004)

Princeton Survey Research poll of 1,568 adults for Pew Research Center, released October 20, 2004 (conducted October 15-19, 2004)

Harris poll of 1,016 adults, released October 20, 2004 (conducted October 14-17, 2004)

Greenberg Quinlan Rosner poll of 1,015 likely voters for Democracy Corps, released October 22, 2004 (conducted October 21-22, 2004)

*****

Kerry Leads by Thirteen Points among College Students, Twenty-five Points among New Voters

John Kerry Leads George Bush 52 percent to 39 percent among American college students, according to a Harvard University Institute of Politics Poll conducted October 7-13. The poll also found that Kerry leads among college student likely voters (LVs) in fourteen swing states by 55 percent to 38 percent.

In addition, a new Ipsos-AP analysis of their poll data shows new voters leaning very heavily toward Kerry.

Among LVs who are new voters, Kerry is favored over Bush by a smashing twenty-five points, 60 percent to 35 percent. Moreover, these new voters were twice as likely to say they'd been contacted by the Kerry campaign (38 percent) than by the Bush campaign (16 percent).

The Ipsos-AP analysis provides this sketch of new voters' demographics and political attitudes:

New voters tend to be young (64% are under 35), unmarried (54%), with some college experience (36%) and holding down a full-time job (63%), often in the service sector or skilled trades. They say the country is heading in the wrong direction (68%) and disapprove of Bush's performance as President (63%) and his handling of Iraq (65%), in particular.

Obviously, the more of these voters that show up at the polls on November 2, the better for John Kerry.

Sources used for this section:
SDS poll of 1,202 undergraduate college students for Harvard University Institute of Politics, released October 21, 2004 (conducted October 7-13)

Ipsos poll of 1, 540 adults for AP, released October 22, 2004 (conducted October 18-20, 2004)

*****

Gallup Poll Racially Biased

By this I don't mean that Gallup's pollsters are themselves racially biased. Rather I mean that their likely voter (LV) samples – whose results Gallup continues to promote above all others – tend to be racially biased because of the methodology Gallup employs to draw them.

Here's a basic sketch of how Gallup's methodology works:

Gallup asks each [RV] respondent seven LV screening questions, and gives each person an LV score of 0 to 7. [Assuming a turnout of 55 percent], the top 55% are classified as likely voters. In practice that typically means all of the "7"s – given full weight – plus some proportion of those with lower scores (usually the "6"s), who are weighted down so that the size of the likely voter sample matches the projected turnout for the year (apparently 55 percent this year). All other voters are discarded from the sample.

Note that the demographics of Gallup's LV sample are not adjusted in any way (as their overall samples are) and are simply allowed to fall where they may.

What this means is that if, say, minority voters are much less likely to answer the seven questions "right," they will be correspondingly under-represented in the LV sample – perhaps severely under-represented.

That is exactly what turns out to be the case. According to data obtained by Steve Soto over at the Left Coaster, Gallup's latest LV sample – the one that showed Bush with an eight-point lead – has only 14.5 percent minority representation and only 7.5 percent black representation.

How plausible is this as a representation of the election day electorate? Not remotely plausible. In 1996, minority representation among voters was 17 percent; in 2000, 19.4 percent. In 2004, the minority proportion of voters should be more than this, because minorities are growing, not declining, as a percentage of the U.S. population. So 14.5 percent for nonwhites as a prediction of the 2004 electorate is very, very unlikely. It would defy both recent history and powerful demographic trends.

As for 7.5 percent blacks? Come on. Blacks were 10.1 percent in 1996 and 9.7 percent in 2000. And they're 12 percent of the voting age population. There's just no way in the world blacks will only be 7.5 percent of voters in 2004.

So, in effect, Gallup's likely voter approach is disenfranchising minorities in assessing American voters' inclinations on the coming election. That's wrong and Gallup should stop doing it.

And speaking of disenfranchisement, how about America's young people? This group is also full of voters who are relatively unlikely to answer the seven LV questions right and thus qualify for admission into the exalted realm of the Gallup LV sample.

Sure enough, Gallup informs us that young voters (age eighteen to twenty-nine) only compose 11 percent of likely voters. Well, that would be quite a trick. In 1992, young voters were 21 percent of voters; in 1996, 17 percent of voters; and in 2000, 17 percent again. And we're supposed to believe that young voters are all of a sudden going to drop to 11 percent this year? Please, this doesn't pass the laugh test.

As it happens, minorities – no big surprise – lean very heavily toward Kerry this year. But young voters are also Kerry's best age group this year. Systematically under-representing these groups in Gallup's LV samples will therefore have an obvious, and fairly substantial, effect on their results, tilting them in the direction of Bush and the Republicans.

That's not right. Gallup should know better. And we should all know better than to trust results that are based on effective disenfranchisement of large numbers of minority and young voters.

Paul Krugman does know better. He had this to say in his October 22 column in the New York Times:

If the election were held today and the votes were counted fairly, Senator John Kerry would probably win. But the votes won't be counted fairly, and the disenfranchisement of minority voters may determine the outcome.

Recent national poll results range from a three-percentage-point Kerry lead in the A.P.-Ipsos poll released yesterday to an eight-point Bush lead in the Gallup poll. But if you line up the polls released this week from the most to the least favorable to President Bush, the polls in the middle show a tie at about 47 percent.

This is bad news for Mr. Bush because undecided voters usually break against the incumbent – not always, but we're talking about probabilities. Those middle-of-the-road polls also show Mr. Bush with job approval around 47 percent, putting him very much in the danger zone.

Electoral College projections based on state polls also show a dead heat. Projections assuming that undecided voters will break for the challenger in typical proportions give Mr. Kerry more than 300 electoral votes.

But he goes on to point out that this picture is not the one you would get from watching cable news, where polls of Gallup and Fox News set the frame for the state of the horse race. And he specifically cites the data reported by myself and others exposing the bias of Gallup's likely voter samples against minority and young voters.

As Krugman emphasizes, this distorted view of the state of the campaign not only misleads voters and the public, it potentially can be used to cover up actual disenfranchisement of minority and young voters on election day. Perhaps Krugman's column will finally help get "the Gallup problem" the attention it so richly deserves.

******

Bush's Barriers to Re-election

Two recent polls underscore just how high the barriers to Bush's reelection could be. The first is the latest CBS News/New York Times poll. Here are some of the key findings from that poll:

1. Kerry leads by seven points among independents.

2. Bush's approval rating is only 44 percent among RVs, but a truly abysmal 34 percent approval/50 percent disapproval among independents.

3. Democrats are up by six points in the generic house content (ten points among independents).

4. Right direction/wrong track is 39 percent/57 percent, and a stunning 34 percent/61 percent among independents.

5. Bush's approval ratings on Iraq, the economy and foreign policy are, respectively, 42 percent, 42 percent, and 41 percent (35 percent, 34 percent, and 33 percent among independents).

6. More voters think the economy is getting worse (34 percent) than think it is getting better (24 percent).

7. Among independents, the Democrats' favorable/unfavorable rating is 54 percent/35 percent, while Republicans' rating is 42 percent/49 percent.

8. More than four times as many voters believe Bush administration policies have increased the cost of prescription drugs (47 percent) than believe their policies have decreased the cost (11 percent).

9. By 60 percent to 8 percent, voters believe Bush administration policies have mostly benefitted the rich, rather than the middle class.

10. An amazing 81 percent of independents believe that Bush administration policies have either decreased the number of jobs in the country (55 percent) or had no effect (26 percent).

11. Perhaps even more amazing, 72 percent of independents believe that Bush administration policies have either increased their taxes (27 percent) or had no effect (55 percent).

12. About three-quarters of independents (74 percent) believe the Bush administration did a poor job of thinking through what would happen as a result of the Iraq war.

13. Independent voters are evenly split, 46 percent to 46 percent on whether Kerry has the same priorities for the country as they do, but, by 60 percent to 24 percent, they believe that Bush does not have the same priorities.

14. Independent voters believe, by eight points (50 percent to 42 percent) that Kerry understands the needs and problems of people like themselves but, by 59 percent to 37 percent, they believe Bush does not understand these needs and problems.

15. Independents believe, by 65 percent to 21 percent, that Kerry is more interested in protecting the interests of ordinary Americans than in protecting the interests of large corporations; by 69 percent to 22 percent, they believe Bush sides with large corporations rather than ordinary Americans.

16. Among independents, just 30 percent believe that Bush will make sure Social Security benefits are there for them, but 58 percent believe Kerry will make sure those benefits are there.

17. Also among independents: 65 percent believe that the Iraq war is only a minor part (12 percent) or not a part at all (53 percent) of the war on terrorism; 63 percent believe efforts to bring stability and order to Iraq are going badly; and 70 percent believe that Iraq was a threat that could have been contained without immediate military action or was not a threat at all.

18. Finally, 67 percent of voters believe that their family is not better off today than four years ago and 75 percent believe the number of jobs in their community has not increased in the last four years.

The approval ratings for Bush cited above are bad. But his approval ratings in the new Pew Research Center poll are so bad as to make those CBS/New York Times ratings look robust by comparison.

Bush's overall approval rating in the Pew poll is 44 percent, just one point above the worst ever recorded for Bush in this poll, and very inauspicious for an incumbent seeking reelection, of course.

But it is his ratings in specific areas such as the economy, Iraq, and foreign policy that are truly remarkable. They are all under 40 percent (!) and the worst ever recorded for Bush in this poll: 38 percent on the economy, 37 percent on Iraq, and 37 percent on foreign policy.

And let's not forget handling "terrorist threats," his best and perhaps only area of strength. He receives his worst rating ever in this area as well: 49 percent, putting him below the 50 percent level even in his strongest area.

If CBS and Pew are accurately capturing voters' assessment of the job Bush has been doing, it's going to be quite difficult for him to convince these same voters to stay the course and keep him in office.

*****

The Nader Nonfactor

The Washington Post had a good front-page article on October 22, "A Fading 'Nader Factor'?" The article points out, as I have repeatedly, that Nader's vote is likely to be a lot smaller than last time and, hence, less dangerous to the Democrats.

But the article also provides empirical backing for the idea that the Nader will not only be smaller, but also less likely to hurt the Democratic candidate than last time. Here's an excerpt from the article:

A survey conducted this month for the Democratic National Committee by pollster Stanley Greenberg showed Nader averaging 1.5 percent of the vote in a dozen battleground states where his name appears on the ballot, compared with about 3 percent in the summer. It also showed that most of the support Nader lost had shifted to Kerry and indicated that his remaining backers would be as likely to vote for Bush as for the Massachusetts Democrat, if Nader were not running.

And it's not just Greenberg who says this:

Frank Newport, editor in chief of the Gallup Poll, said his research has shown for months that when Nader is removed from poll questionnaires, the margin separating the two major candidates is unaltered.

All this suggests that Democrats may have a lot less to worry about from Nader's candidacy than many have assumed.

Source used for this section:

Manuel Roig-Franzia and Jonathan Finer, "A Fading 'Nader Factor'?" Washington Post, October 22, 2004

*****

Tracking the Tracking Polls

Here are last Sunday's tracking poll results (all data based on LVs, Rasmussen data include leaners):

Rasmussen: tie (49-49) from +3 Bush (50-47) 2 days previously
WP/ABC: +1 Bush (49-48) from +4 Bush (50-46) 2 days previously
Zogby: +2 Bush (48-46) from +2 Bush (47-45) 2 days previously
TIPP: +4 Bush (47-43) from tie (45-45) 2 days previously

So Bush's position at that point was either weakening, strengthening or staying about the same. That certainly helps clear things up.

Unfortunately, these disparate results are more common than not with the national tracking polls in this election. One is therefore tempted to say movement in these polls can't possibly be providing much meaningful information on the state of the race. Alan Abramowitz, professor of political science at Emory University, explains below why that might indeed be the case.

For many political junkies, including myself, following the presidential tracking polls has become a daily obsession. We wait with bated breath each morning for Zogby to release his latest results. At the stroke of noon, we log onto the Rasmussen website to get our second daily fix. Finally, at 5 P.M. we eagerly await the latest update on the Washington Post/ABC News tracking poll. Some of us have even discovered a fourth tracking poll, done by an organization called TIPP (the TechnoMetrica Institute of Policy and Politics). The TIPP tracking poll usually releases its daily update sometime in the afternoon.

Tracking polls are different from other political polls. Most polls are done over several days to allow time for multiple attempts to reach those who do not answer their phone the first time. In contrast, in the case of tracking polls, all interviews must be completed the same day so callback attempts are limited or nonexistent. These daily samples are combined over three or four days, with the most recent day's interviews added to the sample and one earlier day's interviews dropped from the sample. The result is a kind of "rolling sample" that, theoretically, tracks day-to-day trends in support for the candidates.

Political campaigns have long used tracking polls to gauge voter response to the campaign and formulate strategy. In recent years, however, a number of media outlets have also been conducting tracking polls and reporting their results to the general public. Four years ago, for example, the Gallup organization conducted a tracking poll during the final month of the campaign. However, the results were so controversial that Gallup dropped its tracking poll this year. The problem with the Gallup tracking poll was that its results gyrated wildly from week to week, and sometimes even from day to day.

In order to avoid the kinds of problems that affected the Gallup tracking poll, the four tracking polls being conducted this year all weight their nightly samples based on certain assumptions about the demographic and partisan composition of the electorate. The result is that this year's tracking polls have been much more stable than Gallup's 2000 tracking poll. And all of the tracking polls have produced similar results. On average, during the month of October, President Bush has had a lead of 1 percent in the Zogby tracking poll, 3 percent in the TIPP tracking poll, 2 percent in the Rasmussen tracking poll, and 3 percent in the Washington Post/ABC News tracking poll. Since October 12th, when TIPP joined the other three, the daily average of the four tracking polls has ranged from a 1 point lead for Bush to a 3 point lead for Bush with no evident trend.

So does it make sense to monitor the daily movements of these tracking polls? The answer is that if you're hoping to learn something about real trends in support for the presidential candidates, it probably doesn't make sense. That's because there is no correlation between the day-to-day movements of the four tracking polls. In other words, they don't move together – each poll's movements are unrelated to all of the other polls' movements. For example, the average correlation between the daily movements of the Zogby Poll and the daily movements of the other three polls is -.18. The average correlation for the Rasmussen Poll is -.07, the average correlation for the TIPP Poll is -.09, and the average correlation for the Washington Post/ABC News Poll is -.12. The combined average for all four tracking polls is -.11. These weak negative correlations mean that there is actually a slight tendency for the polls to move in opposite directions.

What these results indicate is that the day-to-day movements of the tracking polls are essentially random. Rather than reflecting real shifts in voter preferences, the day-to-day movements of the tracking polls are simply reflecting sampling error. This doesn't mean that the overall results of these polls are wrong. In fact, the average margin between George Bush and John Kerry in the tracking polls has been very close to the average margin in other recent national polls. It just means that the day-to-day shifts in the tracking polls are probably not real and that the real level of support for George Bush and John Kerry within the electorate has not changed over the past few weeks: the presidential race has been very close since the beginning of October and it is likely to remain that way until Election Day.

So relax political junkies. Stop obsessing over the daily movements of the tracking polls and get a life! Follow the World Series. Follow your favorite college or professional football team. Follow the weather report. Follow something that is more real than the day-to-day movements of the tracking polls.

Words of wisdom from the good Professor. Heed them well and you'll get through the rest of the campaign with a considerably lower stress level.

              

Ruy Teixeira is a senior fellow at the Center for American Progress and The Century Foundation.

Liked this story? Get top stories in your inbox each week from AlterNet! Sign up now »