Adam Steele is back for another guest post. You can view all of Adam’s posts here. As always, we thank him for contributing.

---

With the 2020 NFL playoffs having just concluded, now seems like a great time to tackle a question I’ve had for years: How strongly does quarterback performance correlate with team success in the postseason?

First, we need to measure quarterback performance. I graded each playoff start on a 0-4 scale, with zero being terrible and four being dominant. I chose this because it matches the academic GPA scale we’re all familiar with. The grade is based on performance while adjusting for era, opponent, weather, and other contextual factors.

Now you may be wondering why I don’t just use stats to grade games. Well, for the tiny sample of a single game stats can be very misleading, especially given the opponent and weather extremes we tend to see in the playoffs. I also prefer a rough instrument for this study so parsing all games into five buckets is ideal.

I included all quarterbacks with six or more playoff starts, giving us a sample of 69 players and 745 graded games. We’ll look at individual quarterbacks later, but for now let’s evaluate the big picture:

Full Sample
GPA: 2.05
Record: 421 – 324 (.565)
Point Diff / Game: +2.4

In Wins
GPA: 2.49
PD/G: +13.4

In Losses
GPA: 1.49
PD/G: -11.8

In this table I’ve separated win % and point differential by grade:

This data makes the relationship between quarterback play and team success look very strong. Each grade bucket yields significantly more wins and scoring margin than the one below it. But before the #QBWins crowd takes their victory lap, the relationship is weaker if we put all 745 games into one bucket. The correlation between grade and wins is .48 and for grade and point differential it’s .56. In the real world this means that quarterback play only determines 23% of the variance in playoff wins.

How does the data look if we compare the GPA of individual quarterbacks and their teams’ postseason results? Below is every quarterback in the study along with his number of playoff starts, GPA, win %, average point differential, and his rank in the latter three categories.

Well, the correlations get even weaker. At the individual level, a quarterback is only responsible for 17% of his postseason record! Put another way, 83% of a quarterback’s postseason record is determined by factors other than the quarterback himself. The inescapable conclusion is that judging a quarterback by his teams’ playoff results is the laziest possible way to evaluate his performance.

I’ll spare you my personal opinions about the quarterbacks in this study, but it’s worth noting that many of them have a huge disconnect between their performance and their team results. Compare Trent Dilfer and Ken Anderson or the four quarterbacks who have exactly seven starts and a 2.14 GPA. Check out the point differentials for the big four of the most recent era.

What do you take from this study? I’d love to hear your thoughts in the comments!

Editor’s Note: I created a bubble graph using the data above; the size of each bubble corresponds to the number of playoff starts (can you find Brady?). The X-Axis shows QB grade, while the Y-Axis shows winning percentage. Yes, that’s Dilfer up on the top left.