## Guest Post: Brad Oremland’s Stat-Based Quarterback Ranking System

Brad Oremland is a longtime commenter and a fellow football historian. Brad is also a senior NFL writer at Sports Central. There are few who have given as much thought to the history of quarterbacks and quarterback ranking systems as Brad has over the years. What follows are Brad’s thoughts on a stat-based quarterback ranking system.

I recently concluded an eight-part series on the greatest quarterbacks in the history of professional football. Those rankings were subjective, based on everything I know about the players: stats, awards and honors, coaching and teammates, team success and postseason performance, reputation, the eye test, and so forth.

But I also have a method for classifying quarterbacks statistically. I actually published the results of this formula three months ago, but without revealing the process that produced those results. A number of readers were curious about my methodology, and in this post, I’ll finally explain how the sausage gets made. The math is not complicated — you don’t need a stats background to understand this — but there’s a lot of it: you could calculate most of this with a pencil and paper, but by the end, you’re going to want a spreadsheet.

I’m a fan of baseball analytics — I’ve even done some writing on sabermetrics — and I’m convinced that this kind of project must be based on linear weights. The all-time ranking process includes some additional steps, but here’s the basic formula, which I call QB-TSP: Quarterback Total Statistical Production (I call all my stat-based rating systems TSP).

Passing Yards – Sack Yards – (constant * (Attempts + Sacks) ) + Completions + (20 * Passing Touchdowns) – (40 * Interceptions) + (0.5 * Rushing Yards) + (20 * Rush Touchdowns) – (20 * Fumbles)

The constant approximates replacement level and varies depending on the era: it moves to reflect quality of competition, changes in the game, and the level of passing efficiency. Here are the values I use presently:

AAFC, 1946-49: 4.0
NFL, 1946-49: 3.5
NFL, 1950-69: 3.0
AFL, 1960: 4.0
AFL, 1961: 3.9
AFL, 1962: 3.8
AFL, 1963: 3.7
AFL, 1964: 3.6
AFL, 1965: 3.5
AFL, 1966: 3.4
AFL, 1967: 3.3
AFL, 1968: 3.2
AFL, 1969: 3.1
NFL, 1970-77: 3.0
NFL, 1978: 3.25
NFL, 1979-94: 3.5
NFL, 1995-2003: 4.0
NFL, 2004-08: 4.5
NFL, 2009-present: 5.0

Obviously, this is not a precise valuation of replacement level. But it’s a functional approximation, and there are additional tweaks (see below) that smooth it out across eras. The value varies according to league: NFL, AFL, AAFC. But even within the post-merger NFL, the expected value of a pass attempt rose in the late ’70s, due to the Mel Blount rule and eased restrictions on offensive linemen. The value increases again with the 1995 expansion (and widespread adoption of the West Coast Offense), the 2004 illegal contact policy, and the 2009 defenseless receiver rules. All of these adjustments correspond to noticeable differences in QB production and efficiency; TSP scores remain relatively stable even when the constant changes. In a normal year, 90-95% of qualified passers will have a positive score.

Let’s use Aaron Rodgers’ 2014 season as an example of the system in action.

Passing Yards – Sack Yards – (constant * (Attempts + Sacks) ) + Completions + (20 * Pass TDs) – (40 * Interceptions) + (0.5 * Rushing Yards) + (20 * Rush TDs) – (20 * Fumbles)

Rodgers passed for 4,381 yards, with only 174 sack yards lost. That’s +4207. Rodgers threw 520 passes and took 28 sacks. That’s 548 * -5 = -2740, updating his TSP to +1467. Rodgers completed 341 passes (+341), with 38 pass TDs (+760) and only 5 INTs (-200), bringing his score to +2368. Rodgers rushed for 269 yards (+134.5) with 2 TDs (+40), and he fumbled 10 times (-200). That’s -25.5, so his total TSP for the year was +2342.5. That’s an excellent score, by far the highest of any quarterback in 2014.

This system rewards both production and efficiency. In 2014, Cam Newton had an 82.1 passer rating and 7.8% sack rate, on 486 dropbacks. His backup, Derek Anderson, posted a 105.2 rating and 4.0% sack rate on 101 dropbacks. Newton also rushed for 539 yards and 5 TDs, while rushing was negligible. Newton was a slightly below-average passer, and he scores a 729 TSP, 20th in the NFL. Anderson’s passing stats were far above average, but in about 1/5 the attempts, and without any rushing value. He scores 316, the 30th-ranked QB of the season (between Charlie Whitehurst and Geno Smith). Newton’s massively higher production (3,366 net yards, 23 TDs) earns him a higher score than Anderson’s greater efficiency in very limited action (708 net yds, 5 TD).

The opposite can apply, as well: Tony Romo (465 dropbacks, 3474 net passing yards) had a higher TSP than Drew Brees (688 dropbacks, 4766 net passing yards) or Matt Ryan (659 dropbacks, 4489 net passing yards). TSP balances production and efficiency: a player won’t have a great score unless he was efficient and a driving factor in his team’s offense.

Comparing QB-TSP Across Eras

Even with the differing constant in the TSP formula, era adjustments are necessary. Shorter seasons are pro-rated to 16 games. Furthermore, each season is assigned a value, based on the top 10 in QB-TSP, or top-5 during the 1950s. Instead of using league average, this focuses on the best players, and it’s not distorted by bottom-outliers. I calculate the average of the top 10, and average that with the median score among the top 10.

Continuing to use 2014 as an example, the year’s score is derived from an average of Aaron Rodgers (2343), Ben Roethlisberger (2097), Peyton Manning (2002), Andrew Luck (1862), Tony Romo (1684), Drew Brees (1676), Matt Ryan (1582), Tom Brady (1542), Russell Wilson (1493), and Eli Manning (1393). That average (1767.0) provides 50% of the value assigned to 2014. The remaing 50% comes from the median within that group — Romo and Brees, averaged — 1680. So the value for 2014 is (1767 + 1680) / 2 = 1723.5 (actually, 1723.3, because Romo had 1683.5). Incorporating the median helps to reduce distortion from positive outliers.

1723.3 is an unusually high value for a single season, but I use a rolling five-year average to calculate the adjustment for a given season. So for the 1975 season, I use (5 * 1975) + (3 * (1974 + 1976)) + (1973 + 1977), divided by 13. When I don’t have five years of data, I use as much adjustment as possible: 2014’s value is [(5 * 2014) + (3 * 2013) + 2012] / 9. That comes to 1642.1. The five-year average insures that players aren’t punished for having a good season in a year when other players also had a good season, unless it’s a trend across multiple years, indicating something about the league-wide passing environment.

To obtain a player’s era-neutralized TSP, I multiply his raw TSP by 1750, then divide it by the value for that season. So in 2014, that’s: (TSP * 1750 / 1642.1). Using Rodgers as an example, his QB-TSP was 2342.5. His era-neutralized TSP is 2496.4. The 1750 figure is generous, so it’s normal for the era-neutralized TSP to be higher than the raw figure.

Rather than using era-neutralized TSP, I use 1/4 of the raw TSP and 3/4 of the era-neutralized TSP. Retaining some of the raw TSP rewards players who actually did more. If you played in an era when quarterbacks played a larger role in the team’s offense, the system reflects that. This final figure is era-adjusted TSP. For Rodgers, we use (.25 * 2342.5) + (.75 * 2496.4) = 2457.9. That ranks 50th all-time. The table below lists the top 200 seasons since 1946:

Dan Marino1984495748108.93725
Sammy Baugh1947283227923432
Dan Fouts198227971893.33197
Peyton Manning2004449449121.13196
Otto Graham194926942297.53099
Peyton Manning2013532656115.13091
Steve Young1994409942112.83073
Otto Graham1947263826109.23039
Bert Jones1976303426102.52951
Aaron Rodgers2011468148122.52914
Johnny Unitas1959285934922908
Y.A. Tittle1963303938104.82900
John Brodie197029032693.82888
Dan Fouts198147243390.62877
Drew Brees2011540447110.62872
Otto Graham195326201799.72866
Sonny Jurgensen196135393288.12851
Tommy Thompson194819372698.42782
Y.A. Tittle196232403589.52781
Daunte Culpepper2004488541110.92761
Ken Anderson197531102393.92760
Kurt Warner1999424442109.22756
Charlie Conerly1948227327842753
Peyton Manning20064347351012752
Dan Marino198646244492.52718
Otto Graham1946158818112.12678
Steve Young1993427031101.52671
Steve Young19923850291072664
Steve Young1998439042101.12657
Milt Plum1960199823110.42630
Jeff Garcia200045373597.62627
Sonny Jurgensen196736363387.32627
Randall Cunningham19983704351062626
Ken Anderson198139343098.42609
Ken Anderson198224261695.32605
Warren Moon199046523596.82578
Johnny Lujack1949269125762577
John Elway198733642383.42571
John Brodie196530713195.32553
Joe Montana1984361030102.92540
Brian Sipe198039703191.42533
Brett Favre199543774199.52514
Johnny Unitas196427322196.42503
Johnny Unitas196334072089.72491
Neil Lomax198444213192.52477
Rich Gannon200246312997.32473
George Blanda196132143691.32466
Aaron Rodgers2014447640112.22458
Roman Gabriel1973301024862453
Drew Brees200849763496.22451
Bernie Kosar198729262395.42443
Joe Theismann1983370630972435
Joe Montana1982256518882432
Sid Luckman194727302567.72427
Fran Tarkenton197528572791.82426
Peyton Manning200043983494.72426
Dan Marino198731632789.22423
Kurt Warner2001465736101.42420
Joe Montana198339702894.62416
Dan Fouts198045203284.72406
Peyton Manning2003418629992406
Dan Marino199443343189.22390
Tobin Rote195624242970.62381
Johnny Unitas196031042573.72360
Norm Van Brocklin195323431984.12356
Ken Anderson197426892095.72350
Jim Everett198941273090.62349
Scott Mitchell199542973692.32346
Fran Tarkenton197627851889.32344
Sammy Baugh194825342378.32329
Joe Montana1989355029112.42329
Drew Brees2013497042104.72318
Joe Montana1987303732102.12312
Mark Rypien199135112997.92311
Drew Brees200643452696.22308
Daunte Culpepper2000422640982303
Bobby Layne195825461777.62279
Norm Van Brocklin195925581879.52274
Roger Staubach197935182792.32270
Daryle Lamonica196932383579.82269
Johnny Unitas1957250925882255
Peyton Manning2012452837105.82250
Ken Stabler197423262794.92244
John Brodie196125371684.72239
Norm Van Brocklin195019801985.12228
Norm Van Brocklin196023622486.52226
Jim Hart1976282118822223
Ken Stabler1976253228103.42218
Peyton Manning199940922890.72202
Ben Roethlisberger2014480732103.32200
John Elway199338902592.82198
Norm Van Brocklin195425471371.92198
Roger Staubach197831532684.92193
Boomer Esiason198835752997.42176
Donovan McNabb2004390334104.72175
Dan Marino199240092485.12175
Fran Tarkenton197425761982.12170
Trent Green200339922692.62165
Peyton Manning2005371128104.12160
Steve Beuerlein199942803894.62158
Johnny Unitas196733192083.62142
Jim Kelly199136623497.62142
Ron Jaworski1980341128912138
Johnny Unitas1958202622902135
Erik Kramer199537823093.52129
Philip Rivers2013440032105.52125
Roman Gabriel196925932986.82124
Brett Favre199438963590.72122
Sonny Jurgensen196426892785.42117
Joe Namath197227231972.52115
Philip Rivers2008394234105.52113
Joe Montana1990395327892109
Lynn Dickey198341633587.32107
Warren Moon199145842581.72105
Peyton Manning2014458539101.52101
Brett Favre199738783692.62101
Drew Brees2009428636109.62094
Frank Ryan196629072988.22092
Roger Staubach1977257221872089
Dan Marino198843862880.82089
Otto Graham194826273185.62082
Boomer Esiason198639112587.72077
Joe Montana198535603091.32073
Y.A. Tittle194824782090.32072
Dan Marino199138202685.82069
Fran Tarkenton197027642182.22067
Trent Green200237742792.62063
Bert Jones197726111980.82062
Roger Staubach1971205017104.82061
Fran Tarkenton197226281880.22061
Fran Tarkenton196928012387.22056
Bert Jones197524792189.12053
Daryle Lamonica196831192680.92043
Vinny Testaverde1998322030101.62041
Charlie Conerly1959167915102.72035
Drew Brees201249924496.32028
Philip Rivers2009413729104.42023
Fran Tarkenton196731113185.92022
Jim Everett198838713189.22021
Steve DeBerg199032482396.32014
Peyton Manning2007391134982011
Sonny Jurgensen196230932474.32009
Carson Palmer2005377233101.12007
Elvis Grbac200040662989.92006
Roger Staubach197626841779.92006
Don Meredith196627862987.71995
Randall Cunningham199039773591.61995
Bill Kenney198341232780.81993
Frankie Albert1948215537102.91991
Philip Rivers2010453530101.81988
Dan Marino198539563084.11988
Brett Favre199637944195.81986
Roger Staubach197324092694.61984
Joe Montana198134672188.41983
Otto Graham195227042466.61980
Roman Gabriel196727763185.21980
Neil Lomax198731442488.51975
Matt Schaub200946782998.61974
Rich Gannon200038353292.41962
Aaron Rodgers20124261411081961
Greg Landry197125351980.91960
Jim Zorn197937392277.71955
Tony Romo200741643897.41954
Andrew Luck201448734396.51953
Joe Namath196737602673.81953
Aaron Rodgers2009444435103.21949
Archie Manning197833171881.71947
Earl Morrall196827472793.21945
Fran Tarkenton196527152083.81942
Troy Aikman199234382489.51939
Ken O'Brien198535472596.21938
Vinny Testaverde199640953588.71936
Sonny Jurgensen196929362385.41935
Peyton Manning200944133399.91935
Bob Waterfield195115781681.81933
Y.A. Tittle195420881378.71930
Joe Namath196830461772.11929
Doug Williams198136372376.81926
Greg Landry197224412771.81925
Steve Young1991285321101.81920
Jeff Garcia200136783794.81917
Jay Cutler2008465727861916
Sonny Jurgensen196630132884.51916
Mark Rypien198937162388.11912
Jim Hart197422982279.51911
Steve Young1997300822104.71909
Rudy Bukich196524602393.71905
Dan Fouts197939362682.61900
Bobby Thomason195323562275.81899
Matt Ryan201246503399.11898
Norm Van Brocklin195116671580.81897
Rich Gannon200139042995.51895

For those keeping track, the top 200 includes 11 seasons of Peyton Manning; eight each of Dan Marino, Joe Montana, and Fran Tarkenton; seven from Johnny Unitas; and six each by Drew Brees, Otto Graham, Sonny Jurgensen, Roger Staubach, Norm Van Brocklin, and Steve Young. Those 11 players account for 78 of the top 200.

The top 100 includes 7 Manning, 5 Marino, 5 Montana, 5 Unitas, 5 Van Brocklin, 4 Ken Anderson, 4 Brees, 4 Graham, 4 Young; 3 Tom Brady, 3 John Brodie, 3 Dan Fouts, and 3 Tarkenton. Those 13 players account for 55 of the top 100.

The top 50 features: 4 Graham, 4 Young, 3 Anderson, 3 Manning, 3 Unitas, and two each from Brady, Brodie, Jurgensen, Marino, Aaron Rodgers, and Y.A. Tittle. Those 11 players account for 29 of the top 50.

As a general guide:

* Anything under 500 TSP is an inconsequential season. The quarterback had very limited playing time, played poorly, or both. 2014 examples: Derek Anderson, Derek Carr.

* 500 era-adjusted TSP is a bad starter or a good backup. 2014 examples: Nick Foles, Drew Stanton.

* 1000 era-adjusted TSP is an average season. The player had some value to his team, but he wasn’t a Pro Bowl-quality performer. 2014 examples: Alex Smith, Andy Dalton.

* 1500 era-adjusted TSP is a good season, a top-10 season, a borderline Pro Bowl season. This is a clear positive contribution to any player’s résumé. 2014 example: Russell Wilson.

* 2000 era-adjusted TSP is a great season. There are 149 Modern Era seasons that meet that standard, a little more than two per year. The player will almost always make the Pro Bowl, and he’ll usually generate some all-pro support. 2014 example: Andrew Luck.

* 2500 era-adjusted TSP is an exceptional season. There are only 45 such seasons in the 69-year Modern Era, so these seasons only occur about twice every three years. About half of them were named league MVP, and most were first-team all-pro. 2014 example: Aaron Rodgers.

* 3000 era-adjusted TSP is a legendary season. The player always wins MVP1, and these are seasons that educated fans know about: Marino in ’84, Young in ’94, Peyton in ’04.

Using QB-TSP For Multiple Years

Simply adding up a player’s era-adjusted TSP is not an effective method to determine his career value: it rewards compilers and does not reflect what we think of as greatness. For anything over four or five years, I use an additional step.

I started doing this years ago, for RB-TSP. It works better for running backs than quarterbacks, but it can apply to any “value over replacement” rating system. When we talk about the best players ever, part of what we mean is, who was the best at his best? Someone who started for 20 seasons but was never the best in the league accrued a lot of value, but he wouldn’t be in the discussion for best of all time. Using TSP as a frame of reference, one 3000-point season is worth much more than three 1000-point seasons. A player who performs at that level gives his team an excellent chance of reaching the championship game, and for the length of a season, he was as good as anyone we’ve ever seen. Compared to three average seasons, well there is no comparison.

To acknowledge that, I employ exponents. Take the player’s era-adjusted TSP, divide it by 1000, and then raise it to the 1.5 power: (EA_QB_TSP / 1000) ^ 1.5. For Aaron Rodgers in 2014, this is 2.458 ^ 1.5 = 3.9. Rodgers’ career value by year, starting in his rookie season of 2005: 0.0, 0.0, 0.0, 2.2, 2.7, 2.6, 5.0, 2.7, 1.3, 3.9. Summing those yields 20.4, Rodgers’ career value in my rating system. This is the 20th-ranked score of the Modern Era, between Roger Staubach (22.4) and Boomer Esiason (19.9). I published the top 125 career ratings in April.

Let’s revisit the guide above:

* 500 era-adjusted TSP is a bad starter or a good backup. This translates to roughly 0.35 in the exponent formula.

* 1000 era-adjusted TSP is an average season. This is worth 1.0 toward career value.

* 1500 era-adjusted TSP is a good season. This is worth about 1.8 in the career ratings.

* 2000 era-adjusted TSP is a great season. This will earn 2.8 in the exponent formula, almost three times as much as a 1000-TSP season.

* 2500 era-adjusted TSP is an exceptional season. This is worth 4.0 toward career value.

* 3000 era-adjusted TSP is a legendary season. This is worth 5.2. Dan Marino’s 1984 season, the best of all time, scores 7.2.

The exponent method prevents compilers from dominating the career rankings. Average and slightly-below-average seasons have some value, but not very much, whereas great seasons raise a player’s score very quickly. The ^ 1.5 adjustment seems a little conservative to me, and I will likely increase it the next time I revise the formula.

To review, we began with the basic QB-TSP formula: Pass Yds – Sack Yds – (constant * (Att + Sacks) ) + Comp + (20 * (Pass TD + Rush TD – Fumble) ) – (40 * INT) + (0.5 * Rush Yds). We obtain an era-neutral rating: TSP * 1750 / (5-Year Rolling Value), then use 25% of QB-TSP and 75% of era-neutral TSP to produce the era-adjusted TSP score. This value is divided by 1000, then raised to the 1.5 power. Sum the value for each season, and this is the career rating I use.

This is not the same ranking I used for the Top 101 QBs series. That was a subjective ranking, incorporating stats but not relying upon them exclusively. In fact, of the 39 ranked players in that project, only three have the same placement in the article series as in the stat-based method outlined above. This system is limited to regular-season stats, and is not sufficient to evaluate a player’s career. But it does help me to put players’ regular-season stats into context.

Most stat-based systems lean too heavily toward either efficiency or production, and I believe this method balances the two. I like that this system recognizes some value in seasons that are below average but above replacement level, which is essential in evaluating quarterbacks who don’t rank among the very best of all time. Players like Drew Bledsoe, Dave Krieg, and Kerry Collins, who were starters for a long time, do reasonably well in this formula, without dominating the list based on their gross production. The exponent layer insures that the top of the list is dominated by players who actually had exceptional seasons.

I don’t claim this system is perfect, because it clearly is not. But I know some readers have been curious about the way I evaluate quarterback stats. This is the way.

1. Unless a kicker does. []
• I don’t feel like I have all that much to say so I’m sorry this is a rather inane comment but I really, really like this system. It seems to find a good balance between production and efficiency and be very inclusive. Great work, Brad!

Every time I try to type your name it comes out Bard. Apparently we should play DnD.

• Andropov

I think you’re right that the exponential modifier you use for career rankings is a low. On the career list, longevity seems to have a disproportionate impact on total value. Aside from that, though, I really like this system. It’s fairly simple, and likely isn’t the best ranking system for directly comparing two QBs across eras, but I think it’s probably very good at demonstrating in what neighbourhood a QB’s performance is. That’s really all I would ask for with a system made to analyze the whole of NFL history.

• sacramento gold miners

Advanced stats are providing a new way to understand the game, which is a positive, as long as we don’t use them to the exclusion of everything else. In other words, all categories must be used, career totals mean something, the eye test is still important, and winning will always be the primary objective. There are also factors which simply can’t be measured about the QB position, yet are important for success. I’m a little cautious about making assumptions about players I didn’t see live, and QB numbers involve both the construct of the offense, along with the context of a specific game.

For the greatest QB season ever, I cannot put a player who played poorly in his team’s most crucial game that season. Dan Marino did have a legendary 1984 season, but not getting his team into the end zone after the first quarter of said game is a definite negative. Some folks also have Tom Brady’s 2007 campaign as the best ever, and it’s the same issue. SB 42 was the most important game for the 2007 season(and Brady’s entire career), and he came up short.

• You should probably just multiply Brad’s numbers by weighted winning percentage.

• Richie

LOL

• Ralph Skinner

*zing*

• Tom

Freaking hilarious.

• Tom

I agree with you to some degree, but I think this ranking specifically leaves out the playoffs. I could be wrong, but I’m assuming that Brad would make some tweaks to the “formula” if the playoffs were included. Kind of like Chase’s Super Bowl leverage-adjusted ANYA post he did a while back (the one where Kurt Warner, I believe, came out on top as having the greatest post-season performance).

• How did you derive the constants?

• I used a thorough scientific process to determine what “seemed right” to me. The constant varies to keep year-to-year scores fairly stable, and to keep 90-95% of qualified passers in positive scores.

• Wolverine

Good stuff. I enjoy Football Outsiders advanced stats, but one of the limitations is that there’s no way to combine DVOA (efficiency compared to average) and DYAR (total value above replacement level). They try to get around this in their DVOA rankings by having a minimum # of plays to qualify for the rankings. I think your method of combining efficiency and volume is brilliant, as it doesn’t require setting some arbitrary minimum.

Speaking of which, anyway we could see season rankings from 2012,2013,and 2014? I think it would give some great context as far as where the current 32 starters rank.

• Tom

The other limitation with DVOA is that you have no idea what the heck you’re even looking at. OK, that’s not entirely true, but what I love about the stuff on this site – ANY/A, SRS, Brad’s formula, etc. – is that for the most part, you can crank up Excel and try some of this stuff out yourself…although there is a little bit of “black-boxness” to Brad’s constants, he’s at least giving us the whole formula so that we can experiment with it, etc.

• QB-TSP, Top 32 of 2012

P.Manning 2165
D.Brees 1952
A.Rodgers 1888
M.Ryan 1827
R.Griffin III 1634
C.Newton 1531
T.Romo 1370
R.Wilson 1362
E.Manning 1193
M.Schaub 1163
M.Stafford 1153
B.Ben 1098
J.Freeman 1020
J.Flacco 1008
C.Palmer 987
A.Luck 895
C.Kaepernick 876
A.Dalton 859
R.Fitzpatrick 669
A.Smith 609
J.Cutler 455
R.Tannehill 433
P.Rivers 393
C.Ponder 383
J.Locker 353
M.Vick 324
B.Weeden 307
C.Henne 153
M.Cassel -52
M.Sanchez -241

QB-TSP, Top 32 of 2013

P.Manning 2927
D.Brees 2195
P.Rivers 2012
N.Foles 1747
R.Wilson 1337
A.Dalton 1314
C.Kaepernick 1256
M.Stafford 1223
T.Romo 1207
A.Luck 1177
A.Rodgers 1137
B.Ben 1114
C.Newton 1068
J.McCown 981
A.Smith 952
M.Ryan 949
C.Palmer 764
J.Cutler 702
R.Griffin III 598
M.Vick 506
R.Fitzpatrick 500
R.Tannehill 398
M.Cassel 374
J.Locker 325
J.Campbell 320
M.McGloin 301
C.Henne 297
C.Keenum 212
E.Manuel 196
M.Glennon 160
C.Ponder 147

QB-TSP, Top 32 of 2014

A.Rodgers 2343
B.Ben 2097
P.Manning 2002
A.Luck 1862
T.Romo 1684
D.Brees 1676
M.Ryan 1582
R.Wilson 1493
E.Manning 1393
J.Flacco 1328
P.Rivers 1237
M.Stafford 1018
R.Fitzpatrick 984
A.Smith 981
R.Tannehill 956
A.Dalton 873
C.Kaepernick 824
B.Hoyer 758
C.Newton 729
J.Cutler 700
K.Orton 634
C.Palmer 596
T.Bridgewater 589
M.Sanchez 569
K.Cousins 516
N.Foles 489
D.Stanton 474
C.Whitehurst 358
D.Anderson 316
G.Smith 313
M.Glennon 293

Responding to Tom’s comment, I completely agree: I really like formulas you can test yourself. The basics of TSP, you can do in your head if you’re good at basic math.

• Wolverine

Awesome. Much appreciated!

• Great stuff, Brad. Actually, perhaps too much for me to chew at once, but here’s one question that leaps to mind.

What do you think of Starr’s 1966 season? Was he 4th in your formula? I note that he was a pretty clear choice as MVP and QB1 by the voters of that day, while I had him neck and neck with Meredith but a bit ahead of Jurgensen/Ryan: http://www.footballperspective.com/the-greatest-qb-of-all-time-v-part-iii-adjusted-dropbacks/

Perhaps the ’66 season is a good jumping off point to discuss efficiency vs. volume. Also, why isn’t there a penalty for rushing attempts? Do you think you just don’t need it if we only give half credit for each yard? Was this a trial-and-error result, or a theoretical point?

• Starr’s ’66 ranks 5th in QB-TSP (Ryan, Meredith, Jurgensen, Dawson), though he’s really close to Jurgensen and Dawson — Sonny has 1799, Dawson 1753, Starr 1727. Joseph and I discussed this very season in the comments of my earlier post. The short version is: would you rather have 462 pass plays by Jurgensen, or 277 by Starr? Jurgensen was an efficient passer that season, so even though Starr was more efficient, TSP rates Jurgensen slightly ahead. I have no problem with Starr’s MVP, but I’m comfortable that the stat-based system sees those two seasons as close to equal.

Adam was right about why I calculate rushing the way I do. For most QBs, a large percentage of their rush attempts are kneel-downs or sneaks, so applying a deduction per rushing attempt is unfair and misleading. I know you and I have commiserated in the past that giving credit to rushing is the hardest part of evaluating this position statistically, but the results with this method seem pretty reasonable to me. I don’t think good rushing QBs are significantly overrated or underrated in this formula, and for players who don’t run much, the contribution to their score is negligible, as it should be.

• Gotcha. That all makes sense. Thanks, Brad.

I see you reference a TSP for running backs. Can you please show us that formula as well? Wide receivers have one?

• RB-TSP is the best statistical system I have ever created. I’m going to save that for later. I’m still tinkering with WR-TSP — it’s gotten a lot less attention than QB and RB — but the formula I used in 2014 was Receptions + Receiving Yards + (5 * Receiving First Downs) + (15 * Receiving TDs). The top 20 last season were:

Antonio Brown 2447
Demaryius Thomas 2240
Julio Jones 2167
Jordy Nelson 2167
Emmanuel Sanders 1985
Dez Bryant 1948
Randall Cobb 1913
Odell Beckham Jr. 1861
T.Y. Hilton 1847
Jeremy Maclin 1833
Golden Tate 1780
Alshon Jeffery 1668
DeAndre Hopkins 1661
Calvin Johnson 1563
Mike Evans 1529
Anquan Boldin 1500
DeSean Jackson 1490
Kelvin Benjamin 1471
Steve Smith Sr. 1459
A.J. Green 1430

Thanks for sharing this, Brad. Overall I think this system does a great job, but I have some questions.

So to clarify, you’re evaluating passing by both efficiency and volume, but rushing is strictly a volume bonus? Given the distortion of kneels in QB rushing data, I think this makes sense.

By using a yearly constant AND a five year rolling average, it seems as though you’re adjusting for era twice. Perhaps I’m missing something, but I don’t really understand your reasoning here. I’m also confused as to why each year in the 70’s gets its own specific constant, yet other long periods of time are lumped together with one constant. For example, ’09 and ’14 were significantly different passing environments (0.5 ANY/A apart) yet they have the same constant. That doesn’t make sense to me, especially when you treated pre-1978 with much more specificity.

How did you choose the 40 yard penalty for interceptions and 20 yard penalty for fumbles? Not that I even disagree, but I’ve seen different values assigned to turnovers and I’m still not really sure which ones are right.

I really like the exponent method for career rankings, but I agree with you and others that the exponent should be higher. Perhaps 1.7 would be better.

How many iterations of this system did you try before finally settling on this version? If you’re anything like me, the urge to tweak is quite enticing.

• It seems to me that the constant era adjusts for volume, while the moving average era adjusts for efficiency. It would be similar(ish) to finding RANY and then multiplying dropbacks by (2014avg per tm/yearNavg per tm) [or the average of that and 1, if prorating that much gives you the willies].

The only thing I don’t like about normalizing for era is the implicit assumption that average in 1950 is the same as it is in 1975 or 2015. I personally believe the average QB from today is probably much more talented than the average QB from 1969.

That makes sense. However I’d still like to know how Brad derived the constants (hint hint Brad).

As to your second point, I’ve grappled with the same issue when creating baselines. The starting QB’s of today are almost surely better, on average, than the QB’s of yesteryear. But how do we measure that? How do we separate the effects of rule changes, schedule length, and strategy innovations from gains in actual QB talent?

What baseline would you use in place of a league average?

• I have gone back and forth regarding baselines. RANY uses league average as a baseline, but that means you get 0 credit for being average. I think there is value to being a league average starter, so I try to come up with a replacement level. I usually use 80% (I believe Chase uses 75%), but I have toyed around with using either 1 standard deviation or 1.333 standard deviations below average (ANY/A+ score of 85 or 80). Of course, the lower you go the more you allow compilers to climb the rankings.

I have been talking with Chase about some multipliers I use for AFL, AAFC, earlier NFL (segregated/wartime/low-paid/baseball more attractive years). However, I think we both agree that the multipliers I am currently using are a bit too harsh on older players. For instance, my 1961 AFL modifier drops Blanda from 1359 to 700 era-modified marginal adjusted yards. That seems pretty harsh, but it seems fair to me too, given how terrible the 1961 AFL was.

TLDR – I don’t really know, but I’m trying to find out.

• Exactly right on rushing. Bryan is right about the constant (adjusts for volume), while the 5-year rolling average is the true era adjustment. By way of example, if I used a 3.0 constant in 2014, Tony Romo would fall from 5th to 9th, while Jay Cutler would rise 5 spots and Derek Carr would rise 15. No amount of era adjustment would correct where they fall within the year. The constant keeps volume passers from being overrated or underrated.

As far as 2014, let’s wait a few years to see what history shows us about this era. I remarked in the article that 2014 had an unusually high era adjustment (1723), but the season before was unusually low (1482), so it’s misleading to imply that last season is representative of a fundamental change in the league. The 5-year rolling average adjusts for what players did at that time, but if 2014’s high TSP was indicative of a trend, that will become apparent in the next few seasons. Perhaps it will become clear that an adjustment to the formula is required; more likely, I suspect the 5-year rolling average will be sufficient.

Chase and I have both read The Hidden Game of Football, which is why our base formulas are pretty similar. He and I also — independently — concluded that TDs were undervalued in that book. I like -40 because [1] -45 seems too harsh, especially for downfield passing eras and eras when most pass attempts were on 3rd and long, [2] 40 is a round number, and the math with TDs and INTs is super easy, [3] 40 yards is approximately what a team might expect from a punt, and [4] probably some other stuff I’m not recalling off the top of my head. I’ve used that figure for a long time, and the results jive with my own perceptions — and often with conventional wisdom, as well. Similar thinking WRT fumbles. Approximately 42.5% of QB fumbles are recovered by the opponent.

Yeah, the exponent is too conservative. I’ve run the numbers with 1.6, and it doesn’t make much difference. I’ll probably go 1.75 the next time I revise the formula.

I’ve been tweaking this system for over a decade. I began calculating a RB-TSP in the 1990s, and QB-TSP in the early ’00s.

Yes I was referring to the 60’s, not sure what compelled me to keep typing 70’s. Now that I understand the purpose of the constants, the gradual increase through the AFL years makes perfect sense.

I agree with using the five year average to smooth out statistical bumps, but I think the method breaks down when there are rule changes in a given year that clearly affect the difficulty of passing. I don’t think it’s right to include ’77 and ’78 in each other’s baselines when the rules were significantly different. Same with ’03 and ’04. The reason I brought up ’14 is because of the re-emphasis of illegal contact, which resulted in a ton of extra flags compared to previous years, and thus an easier passing environment. I don’t see the jump between ’13 and ’14 as random variation; I believe it was caused by the officiating.

Regarding the value of interceptions, I’ve toyed with the idea of adjusting the penalty based on that player’s Y/C. In theory, a higher Y/C indicates deeper passing, which means a lower cost for INT’s. It feels wrong to penalize Joe Namath’s interceptions the same as Alex Smith’s, when Smith’s shorter picks are clearly more damaging.

• John

Elway really has no all time great seasons. Tbh I don’t think there was a year where you can definitely say he was a top 2 QB

I think this is a product of Elway’s prime years coinciding with his worst supporting casts. If he had Davis, Sharpe, Smith, and McCaffery from age 26-30, I’ll bet his peak years look far more exceptional.

• And Shanahan. His recent troubles in Washington have somehow made a lot of fans forget how exceptional his track record with QBs is.

True. Jake Plummer’s splits with and without Shanahan are jarring. Brian Griese even looked like one of the best QB’s in the league for a season under Shanny.

• Who do you think were the top two QBs in 1987 and 1993? Elway’s ’87 ranks 40th in the Modern Era by my methodology, and he was NFL MVP that season. For ’93, Young (2475) and Elway (2037) have TSP miles ahead of third-place Troy Aikman (1738). Elway was 2nd in passing yards and TDs, with a passer rating of 92.8, and he was second-team all-pro.

• Ralph Skinner

In line with what some of the other commenters have said, I personally would raise the exponent incrementally. However, it’s neat that this variable allows you to add more or less weight to longevity depending on your individual preferences for a great quarterback. Seeing Brett Favre outrank Steve Young, Drew Brees, and demolish Roger Staubach and Aaron Rodgers was an indicator for me that 1.6 or 1.7 (as Adam suggested) might be my preference. Young’s four single seasons above Favre’s best shows to me an undervaluation of a quarterback’s peak 3-5 years.

• It seems to me that Favre is really underrated among readers at this site. He was really good for a long time. I don’t know if people only remember the washed-up narcissist of the late ’00s, but Brett Favre was a really special player from 1994-2004.

By TSP, he had nine seasons worth at least 1775 (2.4 after the exponent adjustment). Young had six, Brees has six, Graham had seven, Staubach had seven, Rodgers has five. And Favre had more average and above-average seasons than any of them. He was the clear best QB of the late ’90s, he was very good (and better than his numbers) in the early ’00s, and he had surprising good seasons in ’07 and ’09.

I agree about raising the exponent, but I don’t think you’ve given Favre the credit he deserves. Young’s four best seasons score 18.4, while Favre’s four best add up to 12.9 — Young’s score is 43% higher. I would be okay with elevating that even further, but I don’t think the current system dramatically misrepresents their peaks: Young is way ahead, as he should be. But Favre had a lot of good seasons, which Young did not; Favre was a starter for 19 seasons, and Young for 8 seasons. That’s why the system rates Favre slightly ahead.