Within Anecdotes
How to Use Stories Without Cherry Picking
Stories strengthen an argument when they illustrate supported patterns instead of replacing missing evidence.
On this page
- Illustration, signal, analogy, and proof
- What evidence should accompany a story
- Red flags for selective storytelling
Page outline Jump by section
Introduction
Stories are often the most memorable part of an argument. A personal account can make an abstract issue concrete, reveal a problem that statistics overlook, or help readers understand what a broader pattern looks like in practice. The danger arises when anecdotes are selected because they support a conclusion while contrary cases are ignored. That practice—often called cherry-picking—creates the appearance of evidence without providing a fair picture of reality.
Using anecdotes responsibly does not mean avoiding stories. It means matching the story to the claim. A well-used anecdote illustrates a pattern that is supported by broader evidence. A poorly used anecdote substitutes for evidence that is missing. The difference is central to avoiding logical fallacies based on anecdotal reasoning and hasty generalisation. Research in medicine, communication, and evidence evaluation consistently shows that anecdotes can be valuable as signals, examples, and hypothesis generators, but they become misleading when presented as proof of how things generally work. [PMC]pmc.ncbi.nlm.nih.govPMCby JK Aronson · 2003 · Cited by 132 — Anecdotal reports of adverse reactions should be published, for they have different functions to… [PMC]pmc.ncbi.nlm.nih.govPMCAnecdotes impact medical decisions even when presented…by EN Line · 2024 · Cited by 9 — We found that reading anecdotes for either…
Illustration, Signal, Analogy, and Proof: Know What Role the Story Plays
The safest way to use an anecdote is to be explicit about what it is doing in the argument.
An anecdote can serve as an illustration. For example, after presenting evidence that long waiting times affect access to healthcare, a patient’s story can help readers understand the human experience behind the statistics. The story does not establish the pattern; it helps readers visualise it.
An anecdote can also function as a signal. In medicine, individual reports of unexpected side effects have often alerted researchers to possible problems long before large studies were available. Such reports do not prove causation, but they can identify questions worth investigating. Aronson argues that anecdotal reports have functions different from randomised trials because they can draw attention to previously unrecognised effects and generate hypotheses. [PMC]pmc.ncbi.nlm.nih.govPMCAnecdotes that provide definitive evidencethat provide definitive evidence - PMCby JK Aronson · 2006 · Cited by 189 — Anecdotal reports of adverse drug reactions are generally reg…
A story may also work as an analogy, helping readers grasp a concept through a concrete example. In this role, the anecdote is explanatory rather than evidential.
What anecdotes rarely provide on their own is proof. A single success story cannot demonstrate that a treatment works for most people. A single failure cannot demonstrate that a policy is ineffective. Determining what is typical requires broader and more systematic evidence. [Wikipedia]WikipediaAnecdotal evidenceAnecdotal evidence
A useful test is to ask: if the anecdote were removed, would the underlying claim still stand on independent evidence? If the answer is yes, the story is probably illustrating a case. If the answer is no, the story may be carrying more evidential weight than it deserves.
What Evidence Should Accompany a Story?
Anecdotes become much less vulnerable to cherry-picking when they are paired with evidence that helps readers judge representativeness.
Several forms of supporting evidence are especially valuable:
- Statistics showing prevalence or frequency. If a story describes a problem, readers should know whether it affects one person in a thousand or one in ten.
- Comparative evidence. Showing outcomes among people who did not have the same experience helps separate coincidence from pattern.
- Multiple independent examples. One case may be unusual. Several cases from different sources can suggest a broader trend.
- Systematic research. Surveys, experiments, observational studies, or official records provide context that individual experiences cannot.
- Counterexamples. Acknowledging cases that point in the opposite direction demonstrates that evidence has been considered fairly rather than selected selectively.
In fields such as user research, practitioners routinely distinguish between qualitative insights from individual users and claims about how common a problem is. Observations from a few users can reveal important issues, but estimates of prevalence require broader measurement. [Nielsen Norman Group]nngroup.comresponding skepticism small usability testsNielsen Norman GroupHow to Respond to Skepticism of Testing Small Groups…Feb 24, 2019 — Here are some techniques you can use to help t… [Nielsen Norman Group]nngroup.comresponding skepticism small usability testsNielsen Norman GroupHow to Respond to Skepticism of Testing Small Groups…Feb 24, 2019 — Here are some techniques you can use to help t…
The strongest arguments therefore combine stories and data. The story explains what the pattern looks like; the evidence shows whether the pattern actually exists.
How Cherry-Picking Happens
Cherry-picking is not always deliberate. People naturally notice and remember vivid, emotional, or unusual experiences. This makes selective storytelling easy even when the storyteller is acting in good faith.
Several common mechanisms produce distorted anecdotal arguments:
Memorable cases crowd out ordinary ones. Dramatic successes and failures attract attention, while routine outcomes rarely become stories. As a result, the anecdotes available for discussion are often unrepresentative. [Wikipedia]WikipediaAnecdotal evidenceAnecdotal evidence
Self-selection shapes what gets reported. People with strong experiences are more likely to speak up. Those with ordinary or neutral experiences may remain silent. This can create a misleading impression of what is typical. [LinkedIn]linkedin.comLinked In Anecdotes are not EvidenceLinkedInAnecdotes are not EvidenceMay 22, 2021 — Anecdotal evidence is evidence where small numbers of anecdotes are presented. There is…
Confirmation bias influences collection. Individuals often notice examples that support their existing beliefs more readily than examples that challenge them. Once a conclusion is favoured, supporting stories may be gathered enthusiastically while contradictory cases are overlooked. [Wikipedia]WikipediaAnecdotal evidenceAnecdotal evidence
Availability affects judgement. A vivid anecdote is easier to recall than a statistical summary. People may therefore overestimate the importance or frequency of events that are easy to imagine. [PMC]pmc.ncbi.nlm.nih.govPMCby JK Aronson · 2003 · Cited by 132 — Anecdotal reports of adverse reactions should be published, for they have different functions to…
Recognising these tendencies is essential because they can distort both the creation and the evaluation of anecdotal evidence.
A Practical Method for Fair Use
Writers, speakers, and readers can evaluate anecdotal evidence through a simple sequence of questions.
First, identify the exact claim being made. Is the anecdote being used to show that something is possible, common, effective, harmful, or causal? Different claims require different levels of evidence.
Second, ask how the example was selected. Was it chosen because it is representative, or because it is especially dramatic?
Third, look for the missing cases. What experiences are not being shown? What would the argument look like if unsuccessful, ordinary, or contradictory stories were included?
Fourth, compare the anecdote with independent evidence. Does broader research point in the same direction, or is the story being asked to outweigh a larger body of evidence?
Finally, evaluate whether the conclusion exceeds what the story can support. A story can usually establish that an event occurred. It rarely establishes how often it occurs or why it occurs.
These questions do not eliminate the value of personal experience. Instead, they help place that experience in its proper evidential role.
Red Flags for Selective Storytelling
Certain warning signs frequently indicate that anecdotes are being used in a misleading way.
- A single story is presented as decisive proof of a broad claim.
- No information is provided about how common the experience is.
- Contradictory examples are ignored or dismissed without examination.
- Exceptional cases are treated as typical cases.
- Emotional impact is emphasised while supporting evidence is absent.
- The argument relies heavily on phrases such as “I know someone who…” or “It worked for me…” without broader corroboration.
- Stories are gathered only from communities already committed to a particular belief or outcome.
When these signs appear together, the risk of cherry-picking increases substantially. The problem is not that the stories are false. The problem is that readers are given no way to determine whether the stories represent the wider reality.
The Most Reliable Balance
The most reliable use of anecdotes treats them as human examples embedded within a larger evidential picture. Stories can reveal possibilities, illustrate consequences, and motivate investigation. They can even identify important problems before formal research catches up. Yet they become persuasive without being trustworthy when they are selected selectively and detached from broader evidence. PMC [Oxford University Research Archive]ora.ox.ac.ukAnecdotal reports, by which we mean either individual cases or small case series…Read more…
Avoiding cherry-picking therefore requires a simple discipline: let anecdotes show what happened in particular cases, and let systematic evidence determine how much those cases tell us about the world in general.
Amazon book picks
Further Reading
Books and field guides related to How to Use Stories Without Cherry Picking. Use these as the next step if you want deeper reading beyond the article.
The Skeptics' Guide to the Universe
Explains anecdotes, evidence quality, bias, and how to evaluate claims responsibly.
Thinking, Fast and Slow
Shows why memorable stories can distort reasoning and evidence assessment.
Calling Bullshit
Teaches readers to distinguish illustration from proof and spot cherry-picking.
How to Lie with Statistics
Highlights misleading presentation techniques and selective evidence use.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC1126236/Source snippet
PMCby JK Aronson · 2003 · Cited by 132 — Anecdotal reports of adverse reactions should be published, for they have different functions to...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11345347/Source snippet
PMCAnecdotes impact medical decisions even when presented...by EN Line · 2024 · Cited by 9 — We found that reading anecdotes for either...
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCAnecdotes that provide definitive evidence
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC1702478/Source snippet
that provide definitive evidence - PMCby JK Aronson · 2006 · Cited by 189 — Anecdotal reports of adverse drug reactions are generally reg...
-
Source: Wikipedia
Title: Anecdotal evidence
Link: https://en.wikipedia.org/wiki/Anecdotal_evidence -
Source: linkedin.com
Title: Linked In Anecdotes are not Evidence
Link: https://www.linkedin.com/pulse/anecdotes-evidence-glen-alleman-ms-systems-managementSource snippet
LinkedInAnecdotes are not EvidenceMay 22, 2021 — Anecdotal evidence is evidence where small numbers of anecdotes are presented. There is...
Published: May 22, 2021
-
Source: nngroup.com
Title: responding skepticism small usability tests
Link: https://www.nngroup.com/articles/responding-skepticism-small-usability-tests/Source snippet
Nielsen Norman GroupHow to Respond to Skepticism of Testing Small Groups...Feb 24, 2019 — Here are some techniques you can use to help t...
-
Source: nngroup.com
Link: https://www.nngroup.com/videos/maximize-user-research-insight/Source snippet
Nielsen Norman GroupHow to Maximize User Research Insight (Keynote address...Increase the reliability and validity of your research find...
-
Source: nngroup.com
Title: Nielsen Norman Group First Rule of Usability?
Link: https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/Source snippet
Don't Listen to UsersAug 4, 2001 — To design the best UX, pay attention to what users do, not what they say. Self-reported claims are unr...
-
Source: ora.ox.ac.uk
Link: https://ora.ox.ac.uk/objects/uuid%3Adde0582a-6622-4dfa-a437-1acd4287f83d/files/m1091e3e599b0de15efdd4d05fd7dde38Source snippet
Anecdotal reports, by which we mean either individual cases or small case series...Read more...
-
Source: philpharmblog.wordpress.com
Link: https://philpharmblog.wordpress.com/wp-content/uploads/2016/09/aronson.pdfSource snippet
of Adverse Drug ReactionsIn this paper we detailed different types of between-the-eyes (or definitive) adverse drug reactions (Aronson JK...
Additional References
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/10699067_Anecdotes_as_evidenceSource snippet
(PDF) Anecdotes as evidenceWe need guidelines for reporting anecdotes of suspected adverse drug reactions... adverse reaction, listed on...
-
Source: reddit.com
Link: https://www.reddit.com/r/UXDesign/comments/1bodmk9/we_should_ignore_nn/Source snippet
We should ignore NN?: r/UXDesignWould love to have a discussion about this recent Medium article. I’ve not been following the personalit...
-
Source: facebook.com
Link: https://www.facebook.com/unbiasedscipod/posts/anecdotal-evidence-is-data-collected-in-a-non-scientific-manner-to-assert-specif/403464464775223/ -
Source: reddit.com
Link: https://www.reddit.com/r/UXDesign/comments/1h06d6l/is_nngroup_trustable/ -
Source: nourishedbyscience.com
Title: Nourished by Science Is Anecdotal Evidence Valuable?
Link: https://nourishedbyscience.com/is-anecdotal-evidence-valuable-5-reasons-to-be-cautious/Source snippet
5 Reasons To be Cautious!10 Jun 2022 — So, the first big problem with anecdotes is that an anecdote doesn't provide any data on how gener...
-
Source: nngroup.com
Link: https://www.nngroup.com/videos/in-person-usability-testing/Source snippet
rs unique benefits, such as building rapport, observing non-verbal cues...
-
Source: reddit.com
Link: https://www.reddit.com/r/PhilosophyofScience/comments/1xys7o/why_is_anecdotal_evidence_considered_to_be/Source snippet
represent a statistical cross section accurately. matts2.Read more...
-
Source: ovid.com
Title: When I use a word … Medical definitions
Link: https://www.ovid.com/journals/bmjd/fulltext/10.1136/bmj.p1032~when-i-use-a-word-medical-definitions-pharmacovigilanceSource snippet
BMJby JK Aronson · 2023 · Cited by 4 — It is not therefore surprising that a lot of information about adverse events in people taking med...
-
Source: nngroup.com
Link: https://www.nngroup.com/videos/opinions-expert-design-review/Source snippet
identifying usability problems and strengths. Video Author...
-
Source: nngroup.com
Link: https://www.nngroup.com/videos/dont-listen-to-customers/Source snippet
You must watch what people actually do when using your...
Topic Tree