Reading a statistical report critically

Evaluate reports for sampling bias uncertainty and unsupported causal claims

Problem types

Margin of error and what a sample supports
Studies, experiments, and samples
Testing a difference by randomization · Useful idea
A significant randomization test from random assignment supports a causal claim, which is what a critique of a report asks whether the design justifies.
A margin of error from simulated samples · Useful idea
Checking whether a reported margin of error fits the sample size is part of critiquing a statistical report.
Association in a two-way table · Useful idea
Reports that state percents from a table are checked by comparing the right conditional frequencies.
Association is not causation · Useful idea
Judging whether a report's causal claim is justified depends on its study design.
Asking a statistical question · Useful idea
Reading a report starts with its question: whether it expects variability and whether the data can answer it.

Comparing uncertain strategies · See it applied
A decision rests on the evidence reported, so a report is critiqued for flaws before it is used.

Checking a model against evidence · Related idea
Asking whether a report's evidence supports its claim is the same question as checking whether a model's predictions agree with the data.
Comparing center, spread, and shape · Related idea
A report that gives only a mean can hide skew or outliers, which a comparison of summaries reveals.

Open the interactive map to explore lessons and skill connections.