Guides / Regression to the mean
Why does the five-star pick let you down?
In brief
Because the five stars measure two things folded into one: how good the pick truly is, and how much luck rode along on the reviews that pushed it to the top. Luck rarely repeats itself in the same direction on your own turn, so your personal experience drifts toward the ordinary middle those five stars skipped past. An extreme score is a peak, and a peak is the least repeatable part of any range. Francis Galton clocked the same pull in the height of parents and children a century before anyone clocked it in star ratings, and gave it a plain, almost unkind name: regression toward mediocrity.
The instructor who screamed#
Picture a training strip, propeller engines still ticking in the heat, and a room of Israeli Air Force flight instructors listening to a young psychologist named Daniel Kahneman. He has just finished explaining why praise teaches a skill better than punishment. He feels good about the talk. Then one of the room’s most seasoned instructors raises a hand and, in a few sentences, ruins his afternoon.
The instructor told him plainly. On many occasions I have praised flight cadets for clean execution of some aerobatic maneuver, and in general when they try it again, they do worse.
Scream at a cadet after a clumsy landing, on the other hand, and the next attempt usually improves. His verdict landed hard: please don’t tell us that reinforcement works and punishment does not, because the opposite is the case.
Kahneman had the instructor’s pattern right and his story wrong. He called recognizing the mix-up one of the most satisfying moments of his career, because the pattern carried zero information about praise or punishment. Every cadet’s best runs and worst runs blend real flying skill with the ordinary noise of a single attempt, a gust of wind, a half-second of fatigue, one lucky or unlucky snap of judgment. Praise arrives after a peak, and a peak tends to get followed by something closer to that cadet’s own average. A scream arrives after a trough, and the same drift toward average follows it too, regardless of the scream.
He put the general shape of it into words years later, and it explains far more than flight school: because we tend to reward others when they do well and punish them when they do badly, and because there is regression to the mean, it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them.
Two thirds of the way back#
Decades before Kahneman’s flight instructors, Francis Galton ran the identical pattern through hard numbers. His 1886 paper measured the heights of 930 adult children against their 205 parent pairs, and found that tall parents raised children who were tall, yet on average a little shorter than themselves, drifting back toward the height of the wider population. Short parents raised children a little taller than themselves, drifting the same way, upward. Galton put the ratio in his own words: the Height-deviate of the offspring is, on the average, two-thirds of the height-deviate of its mid-parentage
. He titled the paper Regression towards Mediocrity in Hereditary Stature, and in a statistical sense, the word regression has meant this pull ever since.
Here is the mechanism underneath that fraction. Any single score blends two ingredients: a real underlying trait, and the luck specific to the one occasion the score got measured. A flight maneuver, a round of golf, a batch of five-star reviews, all sit at the mercy of both. Luck can push a score away from a person’s or a product’s true level in either direction, and, unlike the real trait, luck rarely lines up the same way twice. So the more extreme the first score, the larger the share of it that probably came from luck, and the more room the second measurement has to drift back toward the level that was true all along. Arithmetic, doing what arithmetic does. Never a curse, never a jinx, never a punishment for winning too well.
The second draw#
Here is the pull, plotted. Each dot is one item, measured twice, the first score fixed on the horizontal axis, the second on the vertical. Replay the draw and watch where the highest first scores land on the second try.
The dashed line is the world where the first score is the whole story, where whatever you scored on the first attempt you would score again exactly, zero drift. The real dots refuse that world. The solid line lies flatter, because an extreme first score carries an outsized share of one-time luck, and luck carries no memory from the first draw to the second. That is what a five-star product’s next customer is walking into. A second, independent draw, rarely a repeat of the review that convinced them in the first place.
Every peak gets photographed#
Once you know the shape, it turns up everywhere a peak gets celebrated in public. A rookie has an outstanding first season, and the following year gets branded with a sophomore slump, as though some curse took hold. Wikipedia’s own definition treats it as a statistics problem now, folklore stripped away: the sophomore slump is explained as an effect of regression toward the mean, in which an outlier case is followed by a case closer to the average
.
Sports Illustrated has carried a cousin of the same story for decades under a spookier name, the cover jinx: an athlete photographed at a career peak often has a quieter few months right after. Statisticians and science writers read the mechanism plainly. A cover subject gets chosen for a standout run, precisely the kind of run least likely to repeat, so the dip that follows needs no curse to explain it.
The same peak sits under every best of
list, including the ones that rank a product to the very top of a category. A list’s number one earns that slot from a blend of real quality and whatever pushed extra attention its way that season, a launch, a marketing budget, a wave of early adopters posting all at once. None of that blend survives the trip to your own device. The record already carries this exact claim, NordVPN is the best VPN
: 60 reports, 10 platforms. Read as a peak and a second draw, the pattern is textbook: plenty of steady, ordinary service, alongside the dropouts and slow speeds any heavy user runs into eventually. Nowhere near the flawless number one the list promised. Nowhere near a disaster either. Only the middle, which is where a second draw usually lands.
“NordVPN is the best VPN”
60 reports · 10 platforms · strong evidence · the second draw · signed d6f6 811f
The objection#
Galton, “Regression towards Mediocrity in Hereditary Stature,” Journal of the Anthropological Institute of Great Britain and Ireland 15 (1886), pp. 246-263: full text. Kahneman, autobiographical essay, The Nobel Prize (2002): full text, the same account recurring in Thinking, Fast and Slow (2011), ch. 17. Wikipedia, “Sophomore slump.” Wikipedia, “Sports Illustrated cover jinx.” All fetched 18 July 2026.