Within Bad Samples
The Huge Poll That Got It Wrong
The 1936 Literary Digest poll shows why a huge number of responses can still mislead when the sample is badly selected.
On this page
- What the poll predicted
- Why the sample frame skewed the result
- Why size cannot rescue biased evidence
Page outline Jump by section
Introduction
The 1936 Literary Digest poll is one of the most famous examples of how a hasty generalisation can arise from a weak sample. The magazine collected an astonishing number of responses—more than two million—and yet reached a conclusion that was dramatically wrong. It predicted that Republican candidate Alf Landon would defeat President Franklin D. Roosevelt in the United States presidential election. Instead, Roosevelt won in a landslide. [History Matters]historymatters.gmu.eduHistory MattersLandon in a Landslide: The Poll That Changed PollingRepublican presidential candidate Alfred Landon would win 57 percent o…
The case remains important because it demonstrates a crucial lesson about evidence: a large quantity of data does not guarantee a reliable conclusion. If the people being measured are not representative of the wider population, increasing the sample size may simply produce a more precise version of the wrong answer. [Cambridge University Press & Assessment]cambridge.orgCambridge University Press & Assessment“President” Landon and the 1936 Literary Digest Pollby D Lusinchi · 2012 · Cited by 36 — The disas…
What the Poll Predicted
For several elections before 1936, The Literary Digest had successfully predicted the winner and had built a reputation for polling accuracy. In 1936 the magazine launched an enormous survey, mailing millions of ballot cards to potential voters and receiving more than two million responses. [Taylor & Francis Online]tandfonline.comTaylor & Francis OnlineTaking the Next Step in Exploring the Literary Digest 1936…by B Chance · 2024 — Their methods correctly predict…
Based on those returns, the magazine forecast that Landon would receive about 57% of the popular vote and comfortably defeat Roosevelt. The prediction appeared convincing because the number of responses was unprecedented. [History Matters]historymatters.gmu.eduHistory MattersLandon in a Landslide: The Poll That Changed PollingRepublican presidential candidate Alfred Landon would win 57 percent o…
The actual election result was almost the reverse. Roosevelt won more than 60% of the popular vote and carried all but two states. The polling error was so large that it became a landmark case in the history of survey research. [Wikipedia]WikipediaThe Literary DigestThe Literary Digest
For students of logical fallacies, the significance is clear. The magazine treated its evidence as representative of the electorate as a whole and generalised from it to a national conclusion. The problem was not a lack of data. The problem was the quality of the sample.
Why the Sample Frame Skewed the Result
The central flaw lay in how the magazine chose the people it contacted. Its mailing lists were drawn largely from telephone directories, automobile registration records, and its own subscriber lists. [Bit by Bit]bitbybitbook.comBit by Bit Asking questionsBit by BitAsking questions - 3.3.1 Representation… telephone directories and automobile registration records. Here's how they… Non-r…
Today, owning a telephone or a car may not seem unusual. In 1936, however, the United States was still experiencing the effects of the Great Depression. Telephone owners, car owners, and magazine subscribers tended to be wealthier than average citizens. Those wealthier voters were more likely to support Landon than Roosevelt. [Wikipedia]Wikipedia19361936 (MCMXXXVI) was a leap year starting on Wednesday of the Gregorian calendar, the 1936th year of the Common Era (CE) and Anno D…
As a result, the poll’s sampling frame—the pool from which respondents were selected—did not closely match the voting population it was supposed to represent. Even before anyone returned a ballot, the survey was already tilted towards a particular segment of society. [Bit by Bit]bitbybitbook.comBit by Bit Asking questionsBit by BitAsking questions - 3.3.1 Representation… telephone directories and automobile registration records. Here's how they… Non-r…
A second problem made matters worse. Only a minority of the people who received ballots actually replied. Research conducted after the election suggested that Roosevelt opponents were more likely to return the survey, creating an additional layer of bias. This is known as non-response bias. [Bit by Bit]bitbybitbook.comBit by Bit Asking questionsBit by BitAsking questions - 3.3.1 Representation… telephone directories and automobile registration records. Here's how they… Non-r…
The result was a sample that differed from the electorate in two ways:
- The people contacted were not representative of all voters.
- The people who chose to respond were not representative even of those contacted. [Bit by Bit]bitbybitbook.comBit by Bit Asking questionsBit by BitAsking questions - 3.3.1 Representation… telephone directories and automobile registration records. Here's how they… Non-r…
These distortions combined to produce a highly misleading picture of public opinion.
Why Size Cannot Rescue Biased Evidence
The most enduring lesson of the Literary Digest failure is that sample size and sample quality are different issues.
Many people assume that a survey becomes reliable once enough responses are collected. The 1936 poll appears to support that intuition: more than two million people participated. Yet the prediction was disastrously inaccurate. [History Matters]historymatters.gmu.eduHistory MattersLandon in a Landslide: The Poll That Changed PollingRepublican presidential candidate Alfred Landon would win 57 percent o…
The reason is that adding more observations from a biased source does not remove the bias. If a survey repeatedly measures the wrong group, collecting additional responses simply reinforces the same distortion. A huge sample can reduce random error while leaving systematic error untouched. [Bit by Bit]bitbybitbook.comBit by Bit Asking questionsBit by BitAsking questions - 3.3.1 Representation… telephone directories and automobile registration records. Here's how they… Non-r…
The contrast with George Gallup’s polling operation became famous. Gallup used a far smaller sample—tens of thousands rather than millions—but paid much closer attention to representativeness. His poll correctly identified Roosevelt as the likely winner. [Wikipedia]Wikipedia1936 Summer Olympics1936 Summer OlympicsOlympiade) and officially branded as Berlin 1936, were an international multi-sport event held from August 1st to…
This comparison highlights a key principle behind avoiding hasty generalisations:
- More data helps only when the data reasonably reflects the population being studied.
- Representative sampling is often more important than sheer quantity.
- Biased evidence remains biased, even when collected on a massive scale. [Bit by Bit]bitbybitbook.comBit by Bit Asking questionsBit by BitAsking questions - 3.3.1 Representation… telephone directories and automobile registration records. Here's how they… Non-r…
The Lasting Lesson for Hasty Generalisations
The Literary Digest poll endures as a cautionary example because it challenges an intuitive but mistaken belief: that a very large number of examples automatically creates strong evidence.
In reality, the poll’s millions of responses encouraged confidence in a conclusion that rested on a badly selected sample. The magazine effectively generalised from a subset of wealthier and disproportionately anti-Roosevelt voters to the entire electorate. That leap from an unrepresentative sample to a broad claim is exactly the kind of reasoning error that lies behind hasty generalisations and weak-sample arguments. [Wikipedia]WikipediaThe Literary DigestThe Literary Digest
The episode remains memorable because it shows that the crucial question is not merely “How many examples do we have?” but “Do these examples genuinely represent the wider group?” When the answer is no, even millions of observations can lead to the wrong conclusion. [Cambridge University Press & Assessment]cambridge.orgCambridge University Press & Assessment“President” Landon and the 1936 Literary Digest Pollby D Lusinchi · 2012 · Cited by 36 — The disas…
Amazon book picks
Further Reading
Books and field guides related to The Huge Poll That Got It Wrong. Use these as the next step if you want deeper reading beyond the article.
The Signal and the Noise
Discusses prediction, polling, and why large datasets can still mislead.
How to Lie with Statistics
Shows why impressive numbers do not guarantee valid conclusions.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: Wikipedia
Title: The Literary Digest
Link: https://en.wikipedia.org/wiki/The_Literary_Digest -
Source: cambridge.org
Link: https://www.cambridge.org/core/journals/social-science-history/article/president-landon-and-the-1936-literary-digest-poll/E360C38884D77AA8D71555E7AB6B822CSource snippet
Cambridge University Press & Assessment“President” Landon and the 1936 Literary Digest Pollby D Lusinchi · 2012 · Cited by 36 — The disas...
-
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/1936Source snippet
19361936 (MCMXXXVI) was a leap year starting on Wednesday of the Gregorian calendar, the 1936th year of the Common Era (CE) and Anno D...
-
Source: Wikipedia
Title: 1936 Summer Olympics
Link: https://en.wikipedia.org/wiki/1936_Summer_OlympicsSource snippet
1936 Summer OlympicsOlympiade) and officially branded as Berlin 1936, were an international multi-sport event held from August 1st to...
-
Source: historymatters.gmu.edu
Link: https://historymatters.gmu.edu/d/5168/Source snippet
History MattersLandon in a Landslide: The Poll That Changed PollingRepublican presidential candidate Alfred Landon would win 57 percent o...
-
Source: bitbybitbook.com
Title: Bit by Bit Asking questions
Link: https://www.bitbybitbook.com/en/asking-questions/total-survey-error/representation/Source snippet
Bit by BitAsking questions - 3.3.1 Representation... telephone directories and automobile registration records. Here's how they... Non-r...
-
Source: tandfonline.com
Link: https://www.tandfonline.com/doi/full/10.1080/26939169.2024.2395505Source snippet
Taylor & Francis OnlineTaking the Next Step in Exploring the Literary Digest 1936...by B Chance · 2024 — Their methods correctly predict...
Additional References
-
Source: merriam-webster.com
Link: https://www.merriam-webster.com/dictionary/literarySource snippet
LITERARY Definition & Meaning6 days ago — The meaning of LITERARY is of, relating to, or having the characteristics of humane learning or...
-
Source: randomservices.org
Link: https://www.randomservices.org/random/data/LiteraryDigest.htmlSource snippet
The Literary Digest PollThe candidates were Franklin Delano Roosevelt (the incumbent president, a democrat) and Alfred (Alf) Mossman Land...
-
Source: researchgate.net
Title: 315940924 Roosevelt Predicted to Win Revisiting the 1936 Literary Digest Poll
Link: https://www.researchgate.net/publication/315940924_Roosevelt_Predicted_to_Win_Revisiting_the_1936_Literary_Digest_PollSource snippet
Roosevelt Predicted to Win: Revisiting the 1936 Literary...The Literary Digest poll of 1936, which incorrectly predicted that Landon wou...
-
Source: theatlantic.com
Title: cellphones skew political polls did landlines do the same thing in 1936
Link: https://www.theatlantic.com/technology/archive/2012/09/cellphones-skew-political-polls-did-landlines-do-the-same-thing-in-1936/262640/Source snippet
Cellphones Skew Political Polls: Did Landlines Do...20 Sept 2012 — The infamous Literary Digest poll of 1936 predicted Alf Landon would...
-
Source: pivotalresearch.ca
Title: the poll that changed polling selection bias and the 1936 us election
Link: https://www.pivotalresearch.ca/2024/07/03/the-poll-that-changed-polling-selection-bias-and-the-1936-us-election/Source snippet
The Poll that Changed Polling (Selection bias and the 1936...3 Jul 2024 — In advance of the 1936 presidential election, the Literary Dig...
-
Source: nationalww2museum.org
Title: 1936 year worker labor action and reelection franklin d roosevelt
Link: https://www.nationalww2museum.org/war/articles/1936-year-worker-labor-action-and-reelection-franklin-d-rooseveltSource snippet
1936, a Year for the Worker: Labor Action and...Jul 13, 2023 — In 1936, strikes and protests achieved major gains for American workers a...
-
Source: medium.com
Link: https://medium.com/data-science/nonresponse-bias-how-the-silent-majority-decided-an-election-and-sunk-a-beloved-publisher-90a40e73d3a1Source snippet
Publisher. An intro to a statistical bias that makes its...Read more...
-
Source: blogs.library.duke.edu
Link: https://blogs.library.duke.edu/digital-collections/adaccess/timeline/1936-1940/Source snippet
duke.edu1936-1940 | Digital Collections Blog1936. Part of Roosevelt's New Deal policies, the Agricultural Adjustment Act (AAA), is deemed...
-
Source: mathcenter.oxford.emory.edu
Title: historical Blunders
Link: https://mathcenter.oxford.emory.edu/site/math117/historicalBlunders/Source snippet
Math CenterFamous Statistical Blunders in HistoryIn 1936, Literary Digest, a national magazine of the time, sent out 10 million "straw" b...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2107.14166
Topic Tree