Every day, headlines announce that something doubles your cancer risk, that productivity improved by 40%, or that crime fell dramatically under the current government. Most of these claims are technically true in some narrow sense and misleading in their presentation. Not always through deliberate dishonesty — though that certainly occurs — but through the far more common problem of communicating statistics to a general audience without the context that makes them meaningful.
Developing even basic statistical literacy transforms your relationship with news, policy claims, and advertising. Here are the most important techniques to understand.
Absolute vs Relative Risk: The Most Common Misleading Technique
Health headlines are especially prone to this distinction. "Eating processed meat increases your risk of bowel cancer by 20%." That sounds alarming. But 20% relative to what baseline?
If your baseline lifetime risk of bowel cancer is 5%, a 20% relative increase raises it to 6% — a 1 percentage point absolute increase. Meaningful for population health planning across millions of people, but considerably less dramatic than "20% higher risk" suggests when applied to any individual's personal decision-making. The relative risk figure is technically accurate; the absolute risk is more informative for personal decisions.
Always try to find the absolute risk alongside any relative risk claim. Use our percentage calculator to convert between percentage increase figures and absolute values when you have both numbers. If a report gives only relative risk without mentioning the baseline rate, that omission is itself informative.
Cherry-Picked Timeframes and Baselines
Statistical claims about trends are only meaningful when the comparison period is appropriate and honestly chosen. A government claiming crime fell 30% "since we came to power" might be comparing from an unusually high baseline year that preceded a structural trend already underway. An investment fund claiming superior returns might showcase the last three years — their strongest period — rather than a ten-year horizon that reveals a more modest track record.
Our percentage change calculator lets you verify any cited percentage change when you have access to the raw start and end figures. Always ask why a particular start date was chosen — the selection of comparison period can make identical underlying data look dramatically better or worse depending on the narrative being served.
Sample Size and Statistical Significance
A study finding that a new drug reduces symptoms in 80% of participants sounds compelling. A study reaching that conclusion from 20 participants is a different matter entirely. Small samples produce unreliable results with wide confidence intervals — the uncertainty bands around any finding are so large that the result may be statistical noise rather than signal.
Statistical significance — usually reported as a p-value below 0.05 — indicates whether a result is unlikely to have occurred by chance. It does not indicate that the effect is large, practically important, or replicable. A statistically significant but tiny effect might be real and completely irrelevant to any practical decision. Effect size matters as much as significance, and large studies are generally more trustworthy than small ones even when both report p-values below the threshold.
Standard deviation is closely related here — a finding with a very large spread of outcomes (high standard deviation) tells a different story than one with tight, consistent results even if the means are identical. Our standard deviation calculator helps contextualise the spread in any dataset you encounter.
Correlation vs Causation: The Classic Problem
Two things moving together does not mean one causes the other. Ice cream sales and drowning rates both rise in summer — because both are driven by heat, not because ice cream causes drowning. More police officers are often deployed to high-crime areas, which can produce a correlation between officer numbers and crime rates that runs in the wrong causal direction entirely.
When a study demonstrates correlation, the meaningful questions are: Is there a plausible causal mechanism? Were confounding variables controlled for? Has the finding been replicated independently? Observational studies — where researchers observe what happens in the world rather than running controlled experiments — are inherently weaker evidence for causation than randomised controlled trials.
The Base Rate Problem in Probability
Understanding probability is critical for evaluating diagnostic tests, risk models, and security claims. Consider a disease test that is 99% accurate. That sounds excellent. But if the disease affects only 1 in 1,000 people and you test 100,000 individuals, you'd expect approximately 100 true positives but roughly 1,000 false positives (1% of the 99,900 disease-free people). A positive result in this scenario has only about a 9% chance of being genuine.
This is Bayes' theorem in action, and it explains why screening programmes require careful design. Use our probability calculator to explore conditional probability scenarios. The results are routinely counterintuitive — which is precisely why probabilistic claims in medical testing, security screening, and risk assessment are so frequently misunderstood by both the public and the professionals presenting the data.
Misleading Charts and Visual Manipulation
Axes that don't start at zero make small differences look enormous. A chart showing annual revenue growing from £98m to £102m with a y-axis starting at £95m looks like explosive growth. The same data on an axis starting at zero reveals a 4% increase. Dual-axis charts frequently imply correlations between unrelated variables. Pie charts are reliably misread for differences under about 10 percentage points.
When a chart is designed to communicate a conclusion rather than display data, the axis choices, colour selections, and scale usually reveal the intent. Looking at the numbers directly — rather than the visual impression — is always worth doing before accepting what the chart appears to show.
The Replication Crisis Context
A significant proportion of published findings, particularly in psychology and nutritional science, have failed to replicate when tested in larger, more rigorous studies. Many dietary recommendations from the 1990s and 2000s have since been substantially revised or reversed. Single exciting studies — especially small ones — should be treated as preliminary and hypothesis-generating rather than conclusive.
Systematic reviews and meta-analyses aggregate findings across multiple studies, averaging out noise and producing more reliable conclusions. When a major health or social science claim matters to you personally, checking whether a systematic review supports it is always a worthwhile step.
The UK's independent fact-checking organisation Full Fact at fullfact.org applies exactly these analytical tools to political and media claims in real time, and is an excellent resource for developing the habit of statistical scepticism.