The Trials and Tribulations of Anecdata

“The plural of anecdote is not data” -Roger Brinner (probably)

As an editor at UX Booth, I frequently remind authors of one of our key guidelines – show, don’t tell. By this we mean that readers are more likely to understand a concept when it is illustrated with specific examples. Abstract theories are great, but we are more likely to remember stories or case studies than facts and figures, and if we’re up against an unfamiliar concept, we’re more likely to grasp its meaning when we can see it in action.

Occasionally, authors will respond to this request by attempting to “prove” a new concept with a single example. They will tell write about how useless usability testing is, based on the fact that it failed when their company attempted it, and they will illustrate this fact with a detailed story about their company’s experience. Or they will write about the importance of UX in enterprise software, based on a single personal experience. In both situations, the author is confusing “show, don’t tell” with “anecdotal evidence.” Their thesis is based on what I like to call anecdata.

What is Anecdata?

Anecdata an-ec-da-ta (noun): Personal or isolated stories being displayed as fact or statistical evidence.

Anecdata is dangerous for the same reason that a platypus is dangerous. Platypuses have poisonous venom, but they look so cute and cuddly that we don’t fear them! Equally, anecdata is fun, compelling, and illustrative, so readers don’t cultivate a healthy fear. Anecdata typically begins as a singular example, but writers will then use the single story to “prove” a point.

In the example above, where the writer’s thesis was “UX is vital to enterprise software,” I believe the thesis is correct. But rather than supporting this thesis with statistical evidence, multiple studies, or years of growing data, the writer is “proving” the point with a single story, turning their anecdote into their supposed evidence. In this case, even though their thesis is accurate, the article is weak because the thesis appears unproven.

It’s my experience that the worst culprits of anecdata are the misnamed “case studies.” True case studies state a thesis, and clarify that the case in question serves to do one of several things:

1. Further prove a thesis already proven by past studies.
2. Identify lessons learned in a very specific case (i.e. not meant to identify big picture best practices).
3. Illustrate details of a bigger picture story, such as describing the way a company works.

If a case study is not doing any of these, but is instead

Replace Your Anecdata

Anecdata has no place in our work. The key to creating examples that are not anecdata is that their purpose is to illustrate a concept, rather than prove it. Examples and stories and anecdotes are key to a well-written article, but not as proof of a thesis. They are supporting elements. Data, statistics, and research work alongside stories and examples in the same way sound and video work side by side in a film. Together, they make a movie, but neither is meant as a replacement of the other.

I’ll continue to urge authors to show examples of their concepts – but never as a replacement for well-researched data. Down with anecdata!