Doug ran that study in the midst of the 2004-05 playoffs, so it shouldn’t be surprising that Tom Brady (who was at the time 8-0 as a playoff starter and would run it to 10-0 before ever suffering a loss) came out on top, winning 3.5 more games than you’d expect from the particulars of the games he started. Fast-forward eight years, though, and you get this list of quarterbacks who debuted after 1977:
Flacco, whom Chase wrote about yesterday, predictably finishes near the top of the list. As usual, the table is fully sortable (the table includes 130 quarterbacks, and you can change the number of quarterbacks displayed or use the search function to find other quarterbacks), and you can click on the “Index” column to bring the “biggest chokers” to the top. A few other notes: I generated the expected wins for this Manning Index not from Doug’s formula, but from the pregame win probabilities we can derive from the Vegas lines. Also, since I don’t have a list of all QB starts since 1978, I’m considering a player to be the QB of record for his team if he led them in pass attempts in a game (with yards as the tiebreaker).1
By now, you've probably noticed that the Manning Index has reached its full potential, with a Manning brother at the top and at the bottom! But I also want to focus on the strange career arc of Brady, and how much his Manning Index has changed since the end of the 2004 season.
Here are Brady's career playoff stats as the Patriots' primary QB in a game, year-by-year:
The big thing that jumps out is the difference between his Manning Index early (through age 27 it’s +3.5; through age 29 it’s +3.8) and late in his career (starting in 2007, it’s -2.5). As an aside, two years ago, Jason Lisk wrote about Brady’s career in reverse (and Bill Barnwell did the same earlier this week), showing how easy it is to flip the narrative about him depending on which direction you view his playoff record. It’s almost as though perceived early-career clutchness has no predictive value for the latter half of a QB’s career… How shocking!
But it’s worth noting that Brady isn’t alone — here are the QBs who had the most playoff games before and after age 27:
Gm Thru 27
W Thru 27
eW Thru 27
Gm After 27
W After 27
eW After 27
There are some exceptions, of course, but the majority of QBs who had playoff success (as measured by the Manning Index) through age 27 saw their late-career performance fall off a cliff. In fact, if we run a linear regression attempting to predict a QB’s Manning Index from age 28 onward using his Manning Index through age 27, we get a statistically significant negative coefficient:
Post27ManningIndex ~ -0.3997 * Thru27ManningIndex
In other words, the better you do relative to expectations (i.e., the “clutcher” you are) early in your career, the more you can expect to choke later in your career. The same trend also holds up, albeit with less significance, if we split quarterbacks’ Manning Indices at age 29 (through which Brady was +3.8) instead of 27. It’s unclear what causes this “Brady Effect” — whether it be Vegas and/or the betting public becoming overconfident in a QB who had success early in his career, the effect of opponents having more tape on a QB as he ages, or just a matter of QBs peaking as playoff performers at an early age — but it’s a real historical phenomenon.
While you could argue that quarterbacks lose their clutchness over time, maybe this just means clutch quarterbacking doesn’t exist — or at least isn’t detectable to the point that it could be used to predict future outcomes. Here’s a final bit of evidence for the latter camp: the correlation between a quarterback’s Manning Index in even-numbered years and his Manning Index in odd-numbered ones is just 0.05, meaning there’s really no relationship there at all.
If a quarterback’s Manning Index/clutchness was a persistent skill, we would expect to be able to predict it in one random subset of a player’s career from another random subset. But we see here that it’s impossible to predict the “even” half of a player’s playoff career from the “odd” half… and if we try to predict future performance from a quarterback’s Manning Index at an early age, we end up with a negative correlation! In other words, this basically means we should avoid forming any opinion of a quarterback’s clutchness from his playoff W-L record. Period.
- [Chase Note: Scanning the old comments to Doug's post, Jason Lisk chimed in with this still-applicable comment: Steve Young is unfairly penalized for throwing more than 10 pass attempts in the 1987 loss to Minnesota at home (13-2 vs 8-7). Montana threw more passes that game, and was the starter, so Young got mop up duty in the loss. Take that game out when he was not the starter and did not throw the most attempts in the game, and Young moves up the list and is closer to zero. [↩]