How political polling can go catastrophically wrong
The question looms in nearly every U.S. presidential election, even in this year's race: Could the polls be wrong? If they are, they likely will err in unique fashion. The history of election polling says as much.
That history tells of no greater polling surprise than what happened in 1948, when President Harry Truman defied the polls, the pundits and the press to defeat Thomas E. Dewey, his heavily favored Republican foe.
Pollsters were certain Truman had no chance. One of them, Elmo Roper, was so confident of Dewey's victory that he announced two months before the election he would release no further survey data unless a political miracle intervened.
Rival pollsters George Gallup and Archibald Crossley largely completed their poll-taking by mid-October – and missed a decisive shift in support to Truman in the campaign's closing days.
As I point out in my latest book, “Lost in a Gallup: Polling Failure in U.S. Presidential Elections," the misfire of 1948 was exceptional. And that describes most polling failures in presidential elections: They tend to be exceptional, unlike previous polling errors.
When the polls go wrong, they almost always do so in some unanticipated way. Errors spring from no single template.
This variety helps explain why polling failure is so unpredictable and so jarring. The epic miscall of 1948 has never been duplicated in U.S. presidential elections – although the shock of Truman's victory may have been rivaled by the profound surprise that accompanied Donald Trump's win in 2016.
Trump's victory represented polling failure of another kind: Polls in 2016 were not so much in error nationally as they were in states such as Wisconsin, Pennsylvania and Michigan.
If Hillary Clinton had carried those states, as polls had indicated, she would have won the electoral votes to become president. But errors in state-level polls upset national expectations, in part because those polls tended to include too few white voters without college degrees, a key Trump constituency in 2016 and this year.
Recalling the 1936 debacle
Another type of polling failure is that of the venerable pollster who is singularly and astonishingly in error – as was the Literary Digest in its infamous mail-in survey of 1936.
The Digest was a weekly magazine whose massive mail-in polls had identified the winner in each of the three presidential elections since 1924. Some newspapers acclaimed the Digest's mass-polling technique for its “uncanny" accuracy.
In 1936, the Digest employed the same methodology that had served it so well. After sending 10 million postcard ballots and tabulating the 2.3 million returned from around the country, the Digest reported that Republican Alf Landon was bound for a comfortable victory over President Franklin D. Roosevelt.
Landon ended up carrying two states – Maine and Vermont – and lost the popular vote by 24 percentage points. Roosevelt's victory was one of the most lopsided in presidential election history.
That also was the year Gallup, Crossley and Elmo Roper initiated their election polls, which relied on smaller samples than the Digest. With varying degrees of accuracy, all three newcomers in 1936 signaled Roosevelt's victory.
The Digest's debacle offers an enduring reminder that the roots of polling failure run deep. The stunning miscall occurred at the dawn of modern survey research and introduced a nagging sense about polling's potential to mislead.
After all, if the great election oracle of its time could err so spectacularly, why would other polls be immune to failure?
The answer: They weren't, and aren't, immune.