The Greatest QB of All-Time IV, Part I (Methodology)

In 2006, I took a stab at ranking every quarterback in NFL history. Two years later, I acquired more data and made enough improvements to merit publishing an updated and more accurate list of the best quarterbacks the league has ever seen. In 2009, I tweaked the formula again, and published a set of career rankings, along with a set of strength of schedule, era and weather adjustments, and finally career rankings which include those adjustments and playoff performances.

If nothing else, that was three years ago, so the series was due for an update. I’ve also acquired more data, enabling me to tweak the formula to better reflect player performance. But let’s start today with an explanation of the methodology I’m using. To rank a group of players, you need to decide which metric you’re ordering the list by. I’ll get to all of the criteria I’m not using in a little bit, but the formula does use each of the following: pass attempts, passing touchdowns, passing yards, interceptions, sacks, sack yards lost, fumbles, fumbles recovered, rush attempts, rushing yards and rushing touchdowns. Most importantly, the formula is adjusted for era and league.

So where do we begin? We start with plain old yards per attempt. I then incorporate sack data by removing sack yards from the numerator and adding sacks to the denominator¹. To include touchdowns and pass attempts, I gave a quarterback 20 yards for each passing touchdown and subtracted 45 yards for each interception. This calculation — (Pass Yards + 20 * PTD – 45 * INT – Sack Yards Lost) / (Sacks + Pass Attempts) forms the basis for Adjusted Net Yards per Attempt, one of the key metrics I use to evaluate quarterbacks.

For purposes of this study, I did some further tweaking. I’m including rushing touchdowns, because our goal is to measure quarterbacks as players. There’s no reason to separate rushing and passing touchdowns from a value standpoint, so all passing and rushing touchdowns are worth 20 yards and are calculated in the numerator of Adjusted Net Yards per Attempt. To be consistent, I also include rushing touchdowns in the denominator of the equation. This won’t change anything for most quarterbacks, but feels right to me. A touchdown is a touchdown.

Fumbles are an overlooked aspect of quarterback play. The book “The Hidden Game of Football” values a fumble at -50 yards, making them slightly more costly than interceptions (THGOF also first derived the -45 yard penalty for interceptions). Now I don’t have ‘fumbles lost’ data but I do have information on “fumbles” and “fumbles recovered” going back to 1945; the difference between those two could be characterized as net fumbles. However, just because a quarterback fumbles and does not recover the ball does not mean his team loses possession (and remember, he is already being penalized for the sack and the sack yards lost).

For those of you who haven’t given a bunch of thought to fumbles data, here are some numbers from 2000 to 2011:

• 52% of all fumbles are by quarterbacks

• 25% of all quarterback fumbles are ultimately recovered by the quarterback, and an additional 3% bounce out of bounds, making them relatively harmless.

• Who generally recovers the other 72%? A defensive player recovers roughly 44% of all fumbles, leaving the remaining to be recovered by the offense (offensive lineman are responsible for recovering 15% of quarterback fumbles, while 13% are grabbed by an offensive skill position player). This means that roughly 60% of all quarterback fumbles not recovered by the quarterback are recovered by the defense.

Therefore the value of “net fumbles” — the number of fumbles by a quarterback minus the number of fumbles he recovered — is -30 yards, since, on average, a fumble lost is worth -50 yards. So we can update the formula to:

[(PYD + 20*(PTD + RTD) - 45*INT - SKYDLST - 30*(FUM-FumRec)) / (ATT + SK +RTD)]

That calculates each QB’s value per play; we then compare that number to the league average, and multiply the difference by his total number of plays (i.e., ATT + SK + RTD) to get each QB’s value added over average. The last step is to add a rushing component. I still haven’t figured out a very good way to handle quarterback rushing, but what I’ve done in the past is to add all QB rushing yards over 4.0 yards per carry. It still doesn’t feel very scientific, but the results have been noncontroversial, so I’ll continue to use it until I think of something better. I’m going to call final per-play measure of quarterback value added “converted yards per play” or CYP. Once you have each quarterback’s CYP, you then compare it to the league average. I take the CYP for each quarterback, subtract it by the average baseline for that season, and multiply the difference by the number of plays that quarterback had. This does a nice job of balancing the trade-off between compilers and guys that excelled for a short period of time. I have also added in adjustments for shortened seasons, for non-NFL leagues, and for the wartime era from 1943 to 1945.

But enough about methodology for now: let’s take a look at the 2011 results:

QB	Tm	Att	Yd	TD/INT	Sk/Yd	Rsh-Yd-TD	Fum/FR	CYP	VAL
Aaron Rodgers	GNB	502	4643	45/6	36/219	60-257-3	5-2	9.3	2063
Drew Brees	NOR	657	5476	46/14	24/158	21-86-1	1-0	8.2	1838
Tom Brady	NWE	611	5235	39/12	32/173	43-109-3	6-3	8.2	1704
Tony Romo	DAL	522	4184	31/10	36/227	22-46-1	7-6	7.4	1009
Eli Manning	NYG	589	4933	29/16	28/199	35-15-1	9-3	7.2	997
Matthew Stafford	DET	663	5038	41/16	36/257	22-78-0	5-1	6.8	869
Cam Newton	CAR	517	4051	21/17	35/260	126-706-14	5-3	6.5	705
Matt Ryan	ATL	566	4177	29/12	26/173	37-84-2	5-2	6.7	680
Matt Schaub	HOU	292	2479	15/6	16/98	15-9-2	3-1	7.7	661
Michael Vick	PHI	423	3303	18/14	23/126	76-589-1	10-4	6.1	528
Philip Rivers	SDG	582	4624	27/20	30/198	26-36-1	9-3	6.4	480
Ben Roethlisberger	PIT	513	4077	21/14	40/269	31-70-0	9-3	6.2	323
Carson Palmer	OAK	328	2753	13/16	17/119	16-20-1	2-1	6.3	225
Alex Smith	SFO	445	3144	17/5	44/263	52-179-2	7-4	6	194
Jay Cutler	CHI	314	2319	13/7	23/159	18-55-1	7-4	6	140
Matt Hasselbeck	TEN	518	3571	18/14	19/153	20-52-0	4-1	5.7	44
Kyle Orton	2tm	252	1758	9/9	10/54	11-13-0	4-0	5.2	-114
Andy Dalton	CIN	516	3398	20/13	24/160	37-152-1	5-0	5.4	-114
Joe Flacco	BAL	542	3610	20/12	31/203	39-88-1	11-4	5.4	-151
Matt Moore	MIA	347	2497	16/9	36/229	32-65-2	14-3	4.9	-274
Kevin Kolb	ARI	253	1955	9/8	30/219	17-65-0	8-0	4.7	-277
Dan Orlovsky	IND	193	1201	6/4	14/84	6-5-0	7-1	4.2	-288
Ryan Fitzpatrick	BUF	569	3832	24/23	22/148	56-215-0	7-4	5.1	-289
Tim Tebow	DEN	271	1729	12/6	33/225	122-660-6	14-3	4.1	-313
John Skelton	ARI	275	1913	11/14	23/162	28-128-0	4-2	4.3	-383
Curtis Painter	IND	243	1541	6/9	16/104	17-107-0	5-1	4	-389
Matt Cassel	KAN	269	1713	10/9	22/120	25-99-0	5-0	4.3	-402
Tarvaris Jackson	SEA	450	3091	14/13	42/293	40-108-1	9-4	4.8	-416
Rex Grossman	WAS	458	3151	16/20	25/201	20-11-1	8-2	4.6	-521
Christian Ponder	MIN	291	1853	13/13	30/164	28-219-0	6-0	3.7	-522
Mark Sanchez	NYJ	543	3474	26/18	39/243	37-103-6	10-1	4.7	-527
Colt McCoy	CLE	463	2733	14/11	32/173	61-212-0	11-5	4.4	-631
Josh Freeman	TAM	551	3592	16/22	29/164	55-238-4	9-2	4.5	-654
Sam Bradford	STL	357	2164	6/6	36/248	18-26-0	10-2	3.9	-695
Blaine Gabbert	JAX	413	2214	12/11	40/293	48-98-0	14-4	3	-1207

Those who overrate counting stats were likely to put Drew Brees ahead of Aaron Rodgers last year, but there was no contest as to which quarterback produced the most dominant statistics. You might be surprised to see Tony Romo 4th on the list — I know I was — but he had better numbers than Eli Manning in both touchdowns, interceptions and fumbles, which was enough to beat out Eli Manning’s edge in yards per attempt. Romo is a polarizing figure, but how many know that he had 31 TDs and just 10 INTs last year? Few remember how efficient Matt Schaub was last season: on a per-play basis, he came in only behind the big three (he also ranked 4th in ANY/A in 2011).

Of course, the point of this system is to be able to compare players across eras. More on this tomorrow, but for now, here’s a look at the top 100 single-season performances:

Year	Team	QB	Att	Pyd	TD	INT	Sk	Skyd	Rsh	Ryd	RTD	NetFum	CYP	VALUE
1984	MIA	Dan Marino	564	5084	48	17	13	120	28	-7	0	4	8.7	2436
2004	IND	Peyton Manning	497	4557	49	10	13	101	25	38	0	2	9.7	2314
2007	NWE	Tom Brady	578	4806	50	8	21	128	37	98	2	5	8.7	2192
2011	GNB	Aaron Rodgers	502	4643	45	6	36	219	60	257	3	3	9.3	2063
1953	CLE	Otto Graham	258	2722	11	9	26	209	43	143	6	3	8.1	1879
2011	NOR	Drew Brees	657	5476	46	14	24	158	21	86	1	1	8.2	1838
2006	IND	Peyton Manning	557	4397	31	9	14	86	23	36	4	2	7.9	1740
1943	CHI	Sid Luckman	202	2194	28	12			22	-40	1	0	11	1739
1992	SFO	Steve Young	402	3465	25	7	29	152	76	537	4	6	7.8	1727
1994	SFO	Steve Young	461	3969	35	10	31	163	58	293	7	3	8.2	1706
2011	NWE	Tom Brady	611	5235	39	12	32	173	43	109	3	3	8.2	1704
1982	SDG	Dan Fouts	330	2883	17	11	12	94	9	8	1	-1	7.8	1694
1976	BAL	Bert Jones	343	3104	24	9	29	284	38	214	2	2	7.7	1675
1999	STL	Kurt Warner	499	4353	41	13	29	201	23	92	1	9	7.8	1631
1998	MIN	Randall Cunningham	425	3704	34	10	20	132	32	132	1	2	8.4	1585
2000	SFO	Jeff Garcia	561	4278	31	10	24	155	72	414	4	3	7.3	1570
1970	SFO	John Brodie	378	2941	24	10	8	67	9	29	2	1	7.5	1548
2004	MIN	Daunte Culpepper	548	4717	39	11	46	238	88	406	2	9	7.6	1506
1984	SFO	Joe Montana	432	3630	28	10	22	138	39	118	2	2	7.9	1498
1981	CIN	Ken Anderson	479	3754	29	10	25	140	46	320	1	3	7.3	1489
1981	SDG	Dan Fouts	609	4802	33	17	19	134	22	56	0	7	6.9	1471
2003	IND	Peyton Manning	566	4267	29	10	18	107	28	26	0	2	7.2	1459
1967	WAS	Sonny Jurgensen	508	3747	31	16	18	151	15	46	2	5	6.4	1455
1964	BAL	Johnny Unitas	305	2824	19	6	34	241	37	162	2	1	7.9	1453
1942	GNB	Cecil Isbell	268	2021	24	14			36	83	1	0	7	1450
1986	MIA	Dan Marino	623	4746	44	23	17	119	12	-3	0	4	6.8	1450
2010	NWE	Tom Brady	492	3900	36	4	25	175	31	30	1	2	8.2	1434
2009	SDG	Philip Rivers	486	4254	28	9	25	167	26	50	1	5	8	1413
2009	NOR	Drew Brees	514	4388	34	11	20	135	22	33	2	8	7.9	1412
1993	SFO	Steve Young	462	4023	29	16	31	160	69	407	2	6	7.2	1400
1953	RAM	Norm Van Brocklin	286	2393	19	14	11	94	8	11	0	2	6.7	1395
1975	CIN	Ken Anderson	377	3169	21	11	32	247	49	188	2	3	6.8	1385
1991	WAS	Mark Rypien	421	3564	28	11	7	59	15	6	1	6	7.9	1385
1983	WAS	Joe Theismann	459	3714	29	11	34	242	37	234	1	1	7.2	1378
1960	CLE	Milt Plum	250	2297	21	5	35	283	17	-24	2	2	7.6	1373
2005	IND	Peyton Manning	453	3747	28	10	17	81	33	45	0	5	7.7	1366
1998	SFO	Steve Young	517	4170	36	12	48	234	70	454	6	8	7	1364
1963	NYG	Y.A. Tittle	367	3145	36	14	30	268	18	99	2	1	7.5	1363
1988	CIN	Boomer Esiason	388	3572	28	14	30	245	43	248	1	1	7.7	1356
2008	NOR	Drew Brees	635	5069	34	17	13	92	22	-1	0	4	7.4	1341
1989	SFO	Joe Montana	386	3521	26	8	33	198	49	227	3	6	8	1338
1967	NYJ	Joe Namath	491	4007	26	28	27	270	6	14	0	0	5.8	1335
2006	NOR	Drew Brees	554	4418	26	11	18	105	42	32	0	7	7.2	1323
2001	STL	Kurt Warner	546	4830	36	22	38	233	28	60	0	8	7	1312
2000	IND	Peyton Manning	571	4413	33	15	20	131	37	116	1	4	7	1309
1949	CHI	Johnny Lujack	312	2658	23	22	13	101	8	64	2	1	6.2	1290
1959	BAL	Johnny Unitas	367	2899	32	14	23	181	29	145	2	5	6.7	1286
2000	MIN	Daunte Culpepper	474	3937	33	16	34	181	89	470	7	6	7.1	1280
1968	OAK	Daryle Lamonica	416	3245	25	15	26	216	19	98	1	3	6.3	1275
2009	GNB	Aaron Rodgers	541	4434	30	7	50	306	58	316	5	6	7.3	1267
1989	RAM	Jim Everett	518	4310	29	17	29	214	25	31	1	0	7.2	1260
2009	MIN	Brett Favre	531	4202	33	7	34	247	9	7	0	2	7.5	1256
2009	HOU	Matt Schaub	583	4770	29	15	25	149	48	57	0	2	7.3	1255
1967	SDG	John Hadl	427	3365	24	22	10	99	37	107	3	5	6.1	1253
1961	PHI	Sonny Jurgensen	416	3723	32	24	26	212	20	27	0	4	6.7	1251
1980	CLE	Brian Sipe	554	4132	30	14	23	217	20	55	1	2	6.7	1241
1947	WAS	Sammy Baugh	354	2938	25	15			25	47	2	8	7.2	1241
1979	DAL	Roger Staubach	461	3586	27	11	36	240	37	172	0	2	6.7	1237
2009	IND	Peyton Manning	571	4500	33	16	10	74	19	-13	0	2	7.4	1237
2004	PHI	Donovan McNabb	469	3875	31	8	32	192	41	220	3	7	7.5	1235
2010	SDG	Philip Rivers	541	4710	30	13	38	227	29	52	0	5	7.5	1221
1975	MIN	Fran Tarkenton	425	2994	25	13	27	245	16	108	2	-2	6.1	1220
2009	DAL	Tony Romo	550	4483	26	9	34	196	35	105	1	4	7.4	1211
1968	NYJ	Joe Namath	380	3147	15	17	16	118	5	11	2	1	6.5	1198
1973	PHI	Roman Gabriel	460	3219	23	12	31	219	12	10	1	4	5.7	1196
2008	SDG	Philip Rivers	478	4009	34	11	25	151	31	84	0	6	7.7	1195
1974	CIN	Ken Anderson	328	2667	18	10	36	292	43	314	2	2	6.2	1192
1969	OAK	Daryle Lamonica	426	3302	34	25	11	100	13	36	1	1	6.3	1191
1974	OAK	Ken Stabler	310	2469	26	12	18	141	12	-2	1	2	6.9	1188
1967	BAL	Johnny Unitas	436	3428	20	16	24	189	22	89	0	1	6.3	1188
1991	SFO	Steve Young	279	2517	17	8	13	79	66	415	4	2	8.2	1185
2005	CIN	Carson Palmer	509	3836	32	12	19	105	34	41	1	4	7.1	1182
1995	GNB	Brett Favre	570	4413	38	13	33	217	39	181	3	8	6.9	1175
1966	DAL	Don Meredith	344	2805	24	12	35	257	38	242	5	4	6.4	1170
1966	KAN	Len Dawson	284	2527	26	10	23	213	24	167	0	5	7.3	1169
1999	IND	Peyton Manning	533	4135	26	15	14	116	35	73	2	4	6.9	1169
2002	OAK	Rich Gannon	618	4689	26	10	36	214	50	156	3	6	6.7	1154
2003	KAN	Trent Green	523	4039	24	12	20	130	26	83	2	5	6.9	1144
2007	IND	Peyton Manning	515	4040	31	14	21	124	20	-5	3	3	7.2	1140
1983	SFO	Joe Montana	515	3910	26	12	33	224	61	284	2	3	6.6	1138
1962	NYG	Y.A. Tittle	375	3224	33	20	16	127	17	108	2	4	7.1	1132
1976	OAK	Ken Stabler	291	2737	27	17	19	203	7	-2	1	4	7.1	1129
1976	MIN	Fran Tarkenton	412	2961	17	8	25	221	27	45	1	2	6.1	1129
2002	KAN	Trent Green	470	3690	26	13	26	141	31	225	1	0	7.1	1127
2010	GNB	Aaron Rodgers	475	3922	28	11	31	193	64	356	4	3	7.4	1125
1971	DAL	Roger Staubach	211	1882	15	4	23	175	41	343	2	5	7.3	1124
1961	HOU	George Blanda	362	3330	36	22	10	128	7	12	0	3	7.6	1122
1967	OAK	Daryle Lamonica	425	3228	30	20	37	323	22	110	4	2	5.6	1121
1998	NYJ	Vinny Testaverde	421	3256	29	7	19	140	24	104	1	4	7.4	1120
1990	HOU	Warren Moon	584	4689	33	13	36	252	55	215	2	14	6.6	1115
1980	PHI	Ron Jaworski	451	3529	27	12	27	213	27	95	1	2	6.8	1114
1948	PHI	Tommy Thompson	246	1965	25	11			12	46	1	0	8.1	1113
1950	RAM	Norm Van Brocklin	233	2061	18	14	12	93	15	22	1	4	6.5	1112
2009	NWE	Tom Brady	565	4398	28	13	16	86	29	44	1	4	7.2	1109
1984	STL	Neil Lomax	560	4614	28	16	49	377	35	184	3	9	6.3	1096
1990	KAN	Steve DeBerg	444	3444	23	4	22	191	21	-5	0	6	7.2	1092
1978	DAL	Roger Staubach	413	3190	25	16	32	219	42	182	1	3	6	1085
1977	BAL	Bert Jones	393	2686	17	11	26	221	28	146	2	1	5.5	1084
1966	GNB	Bart Starr	251	2257	14	3	24	181	21	104	2	7	7.4	1084
1977	DAL	Roger Staubach	361	2620	18	9	30	219	51	171	3	5	5.8	1083

And now, the huge caveat: It’s important to remember that this is just a measure of each team’s passing game, assigned to the quarterback on the field for those plays. Obviously the quality of the offensive line, the ability of the receivers, the versatility of the tight ends and running backs, the philosophy of the coaches, the strength of the schedule, and good old randomness have a significant impact on the above numbers. The reason for these posts is to accurately measure quarterback statistics, and nothing else. Once we have strong measures of QB performance, we can then judge QBs based on how much of their success (or lack thereof) we want to assign to the QB and how much to other people/factors.

And, of course, even if supporting casts are the same, numbers don’t tell the full story. The point here is simply to get the most out of the numbers we have.

1. I have individual sack data for every quarterback since 1969. For seasons before then, I have team sack data going back to 1949. For seasons before 1950, I ignored sacks; for seasons between 1950 and 1969, I gave each quarterback an approximate number of sacks, giving him the pro-rated portion of sacks allowed by the percentage of pass attempts he threw for the team. While imperfect, I thought this “fix” to be better than to ignore the data completely, especially for years where one quarterback was responsible for the vast majority of his team’s pass attempts. [↩]

Tagged as: Aaron Rodgers, Dan Marino, Eli Manning, GQBOAT, Joe Montana, Johnny Unitas, Peyton Manning, quarterbacks, Steve Young, Tom Brady