In 2006, I took a stab at ranking every quarterback in NFL history. Two years later, I acquired more data and made enough improvements to merit publishing an updated and more accurate list of the best quarterbacks the league has ever seen. In 2009, I tweaked the formula again, and published a set of career rankings, along with a set of strength of schedule, era and weather adjustments, and finally career rankings which include those adjustments and playoff performances.
If nothing else, that was three years ago, so the series was due for an update. I’ve also acquired more data, enabling me to tweak the formula to better reflect player performance. But let’s start today with an explanation of the methodology I’m using. To rank a group of players, you need to decide which metric you’re ordering the list by. I’ll get to all of the criteria I’m not using in a little bit, but the formula does use each of the following: pass attempts, passing touchdowns, passing yards, interceptions, sacks, sack yards lost, fumbles, fumbles recovered, rush attempts, rushing yards and rushing touchdowns. Most importantly, the formula is adjusted for era and league.So where do we begin? We start with plain old yards per attempt. I then incorporate sack data by removing sack yards from the numerator and adding sacks to the denominator1. To include touchdowns and pass attempts, I gave a quarterback 20 yards for each passing touchdown and subtracted 45 yards for each interception. This calculation — (Pass Yards + 20 * PTD – 45 * INT – Sack Yards Lost) / (Sacks + Pass Attempts) forms the basis for Adjusted Net Yards per Attempt, one of the key metrics I use to evaluate quarterbacks.
For purposes of this study, I did some further tweaking. I’m including rushing touchdowns, because our goal is to measure quarterbacks as players. There’s no reason to separate rushing and passing touchdowns from a value standpoint, so all passing and rushing touchdowns are worth 20 yards and are calculated in the numerator of Adjusted Net Yards per Attempt. To be consistent, I also include rushing touchdowns in the denominator of the equation. This won’t change anything for most quarterbacks, but feels right to me. A touchdown is a touchdown.
Fumbles are an overlooked aspect of quarterback play. The book “The Hidden Game of Football” values a fumble at -50 yards, making them slightly more costly than interceptions (THGOF also first derived the -45 yard penalty for interceptions). Now I don’t have ‘fumbles lost’ data but I do have information on “fumbles” and “fumbles recovered” going back to 1945; the difference between those two could be characterized as net fumbles. However, just because a quarterback fumbles and does not recover the ball does not mean his team loses possession (and remember, he is already being penalized for the sack and the sack yards lost).
For those of you who haven’t given a bunch of thought to fumbles data, here are some numbers from 2000 to 2011:
- 52% of all fumbles are by quarterbacks
- 25% of all quarterback fumbles are ultimately recovered by the quarterback, and an additional 3% bounce out of bounds, making them relatively harmless.
- Who generally recovers the other 72%? A defensive player recovers roughly 44% of all fumbles, leaving the remaining to be recovered by the offense (offensive lineman are responsible for recovering 15% of quarterback fumbles, while 13% are grabbed by an offensive skill position player). This means that roughly 60% of all quarterback fumbles not recovered by the quarterback are recovered by the defense.
Therefore the value of “net fumbles” — the number of fumbles by a quarterback minus the number of fumbles he recovered — is -30 yards, since, on average, a fumble lost is worth -50 yards. So we can update the formula to:
[(PYD + 20*(PTD + RTD) - 45*INT - SKYDLST - 30*(FUM-FumRec)) / (ATT + SK +RTD)]
That calculates each QB’s value per play; we then compare that number to the league average, and multiply the difference by his total number of plays (i.e., ATT + SK + RTD) to get each QB’s value added over average. The last step is to add a rushing component. I still haven’t figured out a very good way to handle quarterback rushing, but what I’ve done in the past is to add all QB rushing yards over 4.0 yards per carry. It still doesn’t feel very scientific, but the results have been noncontroversial, so I’ll continue to use it until I think of something better. I’m going to call final per-play measure of quarterback value added “converted yards per play” or CYP. Once you have each quarterback’s CYP, you then compare it to the league average. I take the CYP for each quarterback, subtract it by the average baseline for that season, and multiply the difference by the number of plays that quarterback had. This does a nice job of balancing the trade-off between compilers and guys that excelled for a short period of time. I have also added in adjustments for shortened seasons, for non-NFL leagues, and for the wartime era from 1943 to 1945.
But enough about methodology for now: let’s take a look at the 2011 results:
[table id=114 /]
Those who overrate counting stats were likely to put Drew Brees ahead of Aaron Rodgers last year, but there was no contest as to which quarterback produced the most dominant statistics. You might be surprised to see Tony Romo 4th on the list — I know I was — but he had better numbers than Eli Manning in both touchdowns, interceptions and fumbles, which was enough to beat out Eli Manning’s edge in yards per attempt. Romo is a polarizing figure, but how many know that he had 31 TDs and just 10 INTs last year? Few remember how efficient Matt Schaub was last season: on a per-play basis, he came in only behind the big three (he also ranked 4th in ANY/A in 2011).
Of course, the point of this system is to be able to compare players across eras. More on this tomorrow, but for now, here’s a look at the top 100 single-season performances:
[table id=115 /]
And now, the huge caveat: It’s important to remember that this is just a measure of each team’s passing game, assigned to the quarterback on the field for those plays. Obviously the quality of the offensive line, the ability of the receivers, the versatility of the tight ends and running backs, the philosophy of the coaches, the strength of the schedule, and good old randomness have a significant impact on the above numbers. The reason for these posts is to accurately measure quarterback statistics, and nothing else. Once we have strong measures of QB performance, we can then judge QBs based on how much of their success (or lack thereof) we want to assign to the QB and how much to other people/factors.
And, of course, even if supporting casts are the same, numbers don’t tell the full story. The point here is simply to get the most out of the numbers we have.
- I have individual sack data for every quarterback since 1969. For seasons before then, I have team sack data going back to 1949. For seasons before 1950, I ignored sacks; for seasons between 1950 and 1969, I gave each quarterback an approximate number of sacks, giving him the pro-rated portion of sacks allowed by the percentage of pass attempts he threw for the team. While imperfect, I thought this “fix” to be better than to ignore the data completely, especially for years where one quarterback was responsible for the vast majority of his team’s pass attempts. [↩]