Brad Oremland: Quarterback Total Statistical Production — 2017 Update

Brad Oremland is a longtime commenter and a fellow football historian. Brad is also a senior NFL writer at Sports Central. There are few who have given as much thought to the history of quarterbacks and quarterback ranking systems as Brad has over the years. What follows is Brad’s latest work on quarterback statistical production.

Author’s Note: This is a very long post, but I discourage you from skimming it. Wait to read it until you can go over it without feeling distracted.

Two years ago, I wrote an exhaustive series on the greatest quarterbacks of all time. That was a subjective ranking, but I also discussed the formula for Quarterback Total Statistical Production, QB-TSP. This post concerns that stat, QB-TSP, so you may want to read that link if you haven’t already.

I’ve made three minor adjustments to the formula since that writing:

1. Eliminated the +1 for completions and decreased all the constants by 0.5. This has minimal effect on scoring but de-emphasizes completion percentage.

2. Made the AFL’s constants more charitable by a factor of 1/3. I think TSP was underrating AFL QBs.

3. Increased the exponent to 2. That’s probably a shade too high, but it’s easy to work with.

I’ll quickly review seasonal values, then the blind spots of TSP, and then we’ll do a list: the top 100 Modern-Era QBs, as ranked by TSP.

Let’s start by distinguishing [1] raw TSP, [2] era-adjusted TSP, and [3] yearly value points.

Raw TSP is the basic formula: Passing Yards – Sack Yards – [ constant * (Attempts + Sacks) ] + (20 * Passing Touchdowns) – (40 * Interceptions) + (0.5 * Rushing Yards) + (20 * Rush Touchdowns) – (20 * Fumbles). You can calculate this with pencil and paper, or even in your head, which I like.

Era-adjusted TSP is modified by a 5-year rolling average. This is the appropriate figure to use when comparing across different years.

Yearly value points indicates the exponent-modified stat. This is the one you should use when adding up career values, so you don’t overrate compilers.

So what do all these numbers mean? From this point on, all references to TSP signify Era-adjusted TSP.

* Zero TSP (0 pts) indicates replacement-level performance. 2016 example: Brock Osweiler. I have data going back to 1946, and I believe Osweiler is the only player with 100+ attempts to score exactly zero. Jared Goff scored lowest this season, -422.

* 500 TSP (0.25 pts) is an inconsequential season, a bad starter or a good part-time player. 2016 examples: Brian Hoyer, Matt Moore.

* 1000 TSP (1 pt) is an average season. The player had some value to his team, but he wasn’t a Pro Bowl-quality performer. 2016 examples: Alex Smith, Sam Bradford.

* 1500 TSP (2 pts) is a good season, a top-10 season, a borderline Pro Bowl season. This is a clear positive contribution to any player’s résumé. 2016 examples: Derek Carr, Matthew Stafford.

* 2000 TSP (4 pts) is a great season. It’s a top-5 performance, the player almost always makes the Pro Bowl, and he’ll usually generate some all-pro support. If you have more than two or three seasons like this, you’re probably going to the Hall of Fame. 2016 examples: Drew Brees, Aaron Rodgers.

* 2500 TSP (6 pts) is an exceptional season. These only occur about twice every three years. Most of them were first-team all-pro, and about half were named league MVP. 2016 example: Matt Ryan.

* 3000 TSP (9 pts) is a legendary season. The player always wins MVP, and these are seasons that educated fans know about: Graham in ’47, Fouts in ’82, Marino in ’84, Young in ’94, Peyton in ’04 and ’13, and Brady in ’07.

I will refer to these numbers a lot, so I recommend re-reading them if you’ve been skimming.

Last item before the career rankings: what do these get wrong? First of all, this list is purely statistical. It doesn’t know the offensive scheme, how good the linemen and receivers were, or anything else. It’s just stats, and it differs from my opinion, sometimes significantly.

Second, these are Modern-Era (1946-present) rankings only. I did a work-up on pre-Modern QBs whose careers stretched into the late ’40s, like Sammy Baugh, but only looking at seasons from 1946 onward; they’re not technically ranked. Actually, Baugh and Tommy Thompson are the only ones who made the top 100 anyway.

Third, I only used data from the NFL, AFL, and AAFC, so players who had good seasons in other leagues are underrated. The USFL was a major league, and its players are underrated, most notably Jim Kelly. The CFL is not a major league, but readers who remember my rankings of Warren Moon and Doug Flutie will recall that I believe it’s inaccurate to exclude CFL seasons from a comprehensive analysis of their careers. Nonetheless, they don’t count here. Moon is underrated by TSP, and Flutie’s not in the top 100.

Fourth, these are regular-season rankings, so outstanding postseason quarterbacks are underrated. Fifth — I know this is heresy to some folks — I think compilers are underrated here. Exceptional seasons score a little too high, while above-average and good seasons score a little too low. Anyone who was great in a partial season is probably underrated (e.g. — Tom Brady in 2016). I think rushing is probably a little underrated, so great scramblers may rank one or two spots too low.

TSP aims to balance efficiency with production, so that neither one alone will yield a good ranking. If a QB passed 600 times but kind of sucked, he’s not going to get a good score. If he was really awesome in both games he played, that’s not going to do much for him, either. That aligns with my philosophy, it’s what I want from this stat. But there are some cases where it goes astray: good QBs on run-oriented offenses,1 good players who got hurt or took over partway through the season, etc. Good QBs stuck on bad teams sometimes generate massive volume with poor efficiency, because [1] they’re the best player on the team, so the coaches call pass plays, [2] they’re usually playing from behind, so the coaches call pass plays, but [3] the defense knows they’re going to throw, so it sends in extra DBs, dials up exotic blitzes, and won’t bite on play-action, and [4] the blockers and receivers aren’t very good. Those seasons — “Archie Manning seasons”, let’s call them — are probably underrated.

Player Career TSP Career Value

1. PEYTON MANNING 34,057 76.8

Oddly, I think Peyton Manning playing so poorly in 2015 and still winning a Super Bowl ring helps his best-ever argument more than a great season and Super Bowl would have. If Manning can play like a bum all year and still win the big one, maybe Super Bowls are a better indicator of team quality than of individual skill?

2. DAN MARINO 31,474 66.9

In his first 10 seasons, Marino scored 52.0 TSP, the highest for any QB in any 10-year stretch. That includes nine seasons in a row (1984-92) with at least 1750 TSP. Peyton Manning had eight in a row (2002-09), and no one else has more than five. It seems like Marino’s name comes up almost every Sunday, because — despite the new rules that help QBs and receivers, and the explosion of passing stats — Marino still holds a bunch of records. He was really incredible.

3. TOM BRADY 26,529 52.7

Two years ago, Brady ranked 9th in career TSP. Since then, he’s had two more great seasons (4.2 and 3.5). I stand by my subjective ranking of Brady two years ago — I think a lot of people were projecting his career forward, rather than ranking what he’d already accomplished. The most consistent feedback I got was that people felt active players were too low: I heard similar, albeit less accusatory, complaints about Drew Brees, Aaron Rodgers, Ben Roethlisberger, etc. I think a lot of folks got about two years ahead of things on all those guys. I prefer to rate active players conservatively; I’d rather adjust their rankings upward when we’ve had some time to establish a context for their accomplishments, rathing than make a premature proclamation of greatness.

4. JOHN UNITAS 25,981 52.3

Most 2000-TSP seasons: Peyton Manning (10); Dan Marino, Joe Montana, Fran Tarkenton, Unitas (7); Drew Brees (6); Tom Brady, Otto Graham, Roger Staubach, Norm Van Brocklin (5).

5. DREW BREES 26,083 51.4

Like Brady, he’s still moving up the list. Subjectively, today I’d rank him 10th all-time. I think his numbers are somewhat misleading, several of the players below him have excellent postseason résumés, and he’s the pretty clear third-best QB of his generation. I’m skeptical that in the 70-year Modern Era, three of the top five QBs were continuously active from 2001-15.

6. FRAN TARKENTON 28,550 51.2

Tarkenton ranks 6th in career value points, but 3rd in total TSP (28,550). He never had the one really big season like Manning in ’04, or Marino in ’84, or Brady in ’07, or Unitas in ’59, or Brees in ’11 — his best statistical season, in 1975, earned 5.6 points and an MVP award — but for consistently valuable play, there’s no one in history who’s obviously ahead of him. Tarkenton led in QB-TSP from 1964-68, 1965-69, 1966-70, 1967-71, 1968-72, 1969-73, 1970-74, 1971-75, and 1972-76 — nine five-year blocks in a row. Most 5-year leads: Manning, Tarkenton, and Unitas (9 each); Otto Graham and Dan Marino (6); Steve Young (5).

Tarkenton is probably the most underrated player in the top 10, seldom named among the very best of all time.

7. OTTO GRAHAM 21,285 49.7

The constant in the TSP formula approximates replacement-level production per attempt. It prevents volume passers from dominating the yearly rankings. The constant is 2.5 throughout the 1950s, compared to 3.0 for the NFL from 1946-49, and 3.5 for the AAFC. That is not a generous approach to AAFC seasons, and Graham is the only AAFC quarterback to rate in the top three for any year from 1946-49. Most value points from the AAFC:

1] Otto Graham, 28.9
2] Frankie Albert, 5.9
3] Y.A. Tittle, 5.2

The AAFC doesn’t score particularly high; it’s just Otto Graham.

8. JOE MONTANA 24,331 48.6

I thought squaring the yearly values might push Dan Marino ahead of Montana as the top-rated QB of the 1980s, because of Marino’s monster 1984 (3688 TSP, 13.6 points), the highest-scoring season of all time. But Montana held on. He’s an underrated regular-season QB. Highest-rated QB by decade:

1946-55: Otto Graham, 49.7
1950-59: Norm Van Brocklin, 30.3
1955-64: Johnny Unitas, 40.4
1960-69: Sonny Jurgensen, 32.4
1965-74: Fran Tarkenton, 34.0
1970-79: Roger Staubach, 31.7
1975-84: Dan Fouts, 37.3
1980-89: Joe Montana, 40.3
1985-94: Dan Marino, 42.5
1990-99: Steve Young, 42.6
1995-04: Peyton Manning, 33.1
2000-09: Peyton Manning, 50.4
2005-14: Peyton Manning, 43.7

9. BRETT FAVRE 26,386 45.3

I think a lot of people in the analytics community underrate Brett Favre. He broke the career records for yardage and TDs, and he wasn’t a compiler. He had four 2000-TSP seasons, as many as Steve Young, and he led the league in TSP three seasons in a row (1995-97). Young had Jerry Rice; Favre had Antonio Freeman.

10. STEVE YOUNG 20,573 44.2

Otto Graham and Steve Young are the only quarterbacks with four exceptional seasons (2500 TSP). From 1992-94, Young scored 2652 TSP, 2650 TSP, and 3063 TSP. Those are all 7-point seasons. Young accumulated more value points in those three years than Jim Kelly did in his entire NFL career. The most similar three-year stretch is Dan Fouts from 1980-82, with at least 2390 TSP each year.

11. DAN FOUTS 21,885 43.0

Led all QBs in TSP during the 1975-84 decade. He played with exceptional receivers, for a gifted offensive coach, and he doesn’t have an outstanding postseason résumé, but he was the dominant QB of his generation.

12. KEN ANDERSON 20,133 38.2

Anderson was essentially an elite game manager: high completion rate, low interception percentage, not really a downfield bomber. When those QBs play for great teams, like Bart Starr’s Packers or Troy Aikman’s Cowboys, their greatness gets recognized. When they play for second- and third-place teams — the Bengals won only two division titles in Anderson’s 13 years as the primary QB, and went 2-4 in the postseason — there’s a sense that the quarterback needs to do more. If you had to run a two-minute drill, you didn’t want Kenny Anderson, you wanted Staubach, Stabler, Bradshaw.

In my subjective rankings two years ago, I rated Otto Graham as the second-greatest quarterback of the Modern Era. I am a strong proponent of methodological pluralism, and that’s what sets Graham apart. Whatever your preferred approach, he ranks among the very best of all time. There is probably no other QB in history who excelled so dramatically by every measure we use to evaluate quarterbacks.

Anderson represents the other end of spectrum: statistical analysis is the only method of evaluating greatness that suggests he was a Hall of Fame-caliber QB. Anderson made four Pro Bowls and won an MVP, which isn’t too shabby. But so did his successor, Boomer Esiason. Rich Gannon made four Pro Bowls and won an MVP. Roman Gabriel made four Pro Bowls and won an MVP. Daryle Lamonica made five All-Star Games and won two MVPs. Jack Kemp made seven All-Star Games and won an MVP. Matt Ryan has four Pro Bowls and an MVP, and I doubt many of us are ready to put him in the Hall of Fame yet. Anderson’s awards and honors are consistent with a lot of guys you’d probably rank around 30-45. By other measures, Anderson fares even worse, and doesn’t look like a special player.

Ken Anderson is underrated outside the analytics community, but if you rank him among the top 20 QBs of all time, you’re essentially rejecting methodological pluralism, saying stats tell the whole story. I love statistics, and I find them immeasurably useful, but I would really discourage you from such a credulous approach.

13. SONNY JURGENSEN 20,228 36.8

Jurgensen (1957-74) and Unitas (1956-73) were contemporaries, but Jurgensen’s TD/INT differential (+66) is substantially better than Johnny U’s (+37). If you rate Unitas in the top 10 — which you should — you need to rate Jurgensen in the top 20, at least.

14. JOHN ELWAY 22,258 36.7

In 1987, Elway led the NFL in QB-TSP, 2582. With Vance Johnson, Ricky Nattiel, and Clarence Kay as his leading receivers.

15. NORM VAN BROCKLIN 19,204 35.2

Van Brocklin is TSP’s highest-rated QB of the 1950s, 30.3. That’s the lowest high-score for any decade. Furthermore, if you add the top five for the ’50s, it comes to only 103, which is also the lowest of any decade, followed by 1946-55 (111) and 1955-64 (112). The top 10 for the ’50s sums to only 142, which is the lowest of any decade, again followed by 1946-55 (154) and 1955-64 (170). The ’50s featured two Hall of Famers in their primes (Van Brocklin and Bobby Layne), as well as prime years from three others (Graham, Unitas, and Tittle), and Charlie Conerly is sometimes mentioned among the greatest quarterbacks of all time.

I know some people refuse to consider older players in their all-time rankings. I think that’s misguided, and — by definition — not an all-time ranking. But it’s true that passing is more important now than it was in the past. QB-TSP reflects that: players score higher today than they did in previous decades.

16. Y.A. TITTLE 19,087 35.0

By this method, still the greatest quarterback in New York Giants history. He scored 18.7 in four seasons with the Giants, including back-to-back years with over 2700 TSP (joining Dan Fouts as the only players to do so). But tenure is so critical to team identity, I prefer accumulated TSP to value points when determining franchise leaders. Here are the Giants’ top five, through 2016:

1. Phil Simms — 12,333
2. Eli Manning — 11,831
3. Charlie Conerly — 11,431
4. Fran Tarkenton — 8,995
5. Y.A. Tittle — 7,263

I suspect that more closely approximates how Giants fans feel: Simms and Eli pretty close at the top, with Conerly also in the mix for older fans. I don’t think Giants fans want to hear that their best QBs were a Viking and a 49er — and I’m inclined to agree with them.

17. AARON RODGERS 16,976 34.7

One of the nice things about this system is that it’s easy to calculate a player’s climb up the leaderboard. One more great season (2000 TSP, 4 pts) would vault Rodgers to 12th all-time.

18. ROGER STAUBACH 15,770 31.7

Most years leading the league in QB-TSP:

1] Otto Graham, 6
2] Peyton Manning, 5
t3] Roger Staubach and Steve Young, 4

Staubach was a brilliant QB in a short career, essentially just eight seasons. Why does he rank below similar players like Graham (10 years), Young (8 years), and Rodgers (9 years)? Staubach didn’t have exceptional seasons. His highest TSP was 2294, and no major organization ever named him first-team all-pro. Paul Zimmerman, an admirer of the ’70s Cowboys, lamented in The New Thinking Man’s Guide to Pro Football, “Every time Roger had a good year, someone else picked that time to have a slightly better one.” TSP sees it the other way around, but it’s true that Staubach never posted the statistical production that normally suggests a first-team all-pro.

Staubach was great, and his fine postseason record doesn’t count toward TSP, but he wasn’t as outstanding as his four years leading the league would imply.

19. WARREN MOON 19,322 31.0
20. JOHN BRODIE 17,839 30.7

I would take Warren Moon’s consistency over Brodie’s erratic excellence — the 1961, 1965, and 1970 seasons account for over 60% of his 31.2 value points — with zero hesitation. Moon had more 1000-TSP seasons, more 1200-TSP seasons, more 1400-TSP seasons, more 1600-TSP seasons… Moon was good every year, and he did have a 2500-TSP season in 1990. But as it stands right now, the value formula overrates seasons like Brodie in ’65 and ’70, and underrates seasons like Moon in ’87 and ’97. There’s a comparison to be drawn between Ken Anderson and Brodie.

21. PHILIP RIVERS 15,819 26.2

I think there’s a growing sense, in both mainstream and analytic circles, that Philip Rivers has had or is rapidly approaching a Hall of Fame-caliber career. I believe his HOF chances are directly linked to Eli Manning’s. If Rivers and Roethlisberger are elected, the voters aren’t going to leave Eli out, or at least not forever. Eli’s induction would probably boost Rivers’ chances as well, if it comes to that, but this was the first year I got the sense that Rivers might have a better chance of enshrinement than Eli.

22. BOOMER ESIASON 15,881 25.8
23. ROMAN GABRIEL 15,982 24.0
24. JOHN HADL 15,238 23.24
25. JIM KELLY 15,425 23.18
26. BOBBY LAYNE 15,510 23.17
27. JIM EVERETT 13,937 23.1
28. KURT WARNER 12,202 22.8

Beginning in 1998, Kurt Warner’s TSP by year: 1, 2730, 1770, 2381, -62, -57, 381, 751, 312, 1176, 1856, 963. That’s two exceptional seasons, two very good seasons, two essentially average seasons, and six years that don’t really add anything to his legacy. Is that a Hall of Famer? Maybe, but I understand why people see it both ways.

29. JIM HART 16,076 22.5
30. BEN ROETHLISBERGER 16,024 21.5
31. JOE NAMATH 13,253 21.3
32. MATT RYAN 12,753 20.9

As I mentioned at the beginning, the career values are derived using a ^2 exponent, when the ideal exponent is probably something like 1.85. That means great seasons are overrated, while average and good seasons are underrated. Normally this isn’t a big deal, but there are a few players, and types of player, for whom it distorts their rankings.

Matt Ryan’s 2016 season scores higher (6.8) than Joe Flacco’s career (6.1). That seems excessive to me. Flacco scores 500-1050 TSP in seven of his nine seasons; such seasons are somewhat underrated, I think. I alluded earlier to Archie Manning seasons and the Bob Griese problem. Let’s add a couple more:

Joe Flacco seasons — 500-1050 TSP, classic compiler seasons. All-time leaders: Vinny Testaverde (10); Dave Krieg, Carson Palmer (8 each); Joe Flacco (7).

Ben Roethlisberger seasons — 1100-1600 TSP, good years that score a fraction of the value of great years. All-time leaders: Ben Roethlisberger (10); John Elway, Bob Griese (8 each); Jim Kelly, Donovan McNabb, Phil Simms (7 each).

If I used a ^1.85 exponent, Krieg would rise four spots, Simms three, everyone else two or fewer. So the difference isn’t enormous, but it’s there. Flacco would pass Kirk Cousins (6.3), which I think I agree with for now. My sense is that the ^2 adjustment works well in the top 25 or so, ^1.85 is better for 26-75, and something like ^1.75 the best below that. Around the border of the top 100, one-year wonders are competitive with compilers, which seems wrong to me. Someone who held a starting job for a decade was better than someone who fluked into one great season. With the ^2 modifier, Scott Mitchell (2376 TSP in 1995) ranks ahead of Michael Vick, Steve Beuerlein, Jim McMahon, and Neil O’Donnell. At ^1.85, he doesn’t.

Archie Manning seasons — Above-average QB on a bad team, with high volume and low efficiency. Examples: Archie Manning, Drew Bledsoe, Tobin Rote, Norm Snead.

Bob Griese seasons — Good QB on a run-oriented offense, with low volume and high efficiency. Examples: Bob Griese, Troy Aikman, Ben Roethlisberger, Bart Starr.

The issues regarding Joe Flacco/Ben Roethlisberger seasons are purely mathematical; I could “fix” those if I wanted. I didn’t, because the top of the list is more important to me than the middle.2 The Archie Manning/Bob Griese problem is a little dicier, because it’s inside the TSP formula; it involves the balance of efficiency vs. production. Furthermore, helping one group would hurt the other. And a mathematical formula can’t tell the difference between, say, Norm Snead in 1961 and Joey Harrington in 2003, or Bob Griese in 1973 and Alex Smith in 2011. You can’t differentiate those seasons with math, or at least not in any way I’m aware of. You have three choices: conclude that players like Griese were wildly overrated, conclude that players like Smith are wildly underrated, or look beyond statistics and apply subjective adjustments when evaluating these players.

This is an area where I differ from many statistical analysts. Football is such a complicated sport, and quarterback such a complicated position, that I don’t believe stats can ever tell the full story. So when I consult a statistic, I assume it to be imperfect. It gives me a frame of reference, a starting point, from which to ask, “What’s missing? What is distorted or misrepresented?” This is why I find a methodologically pluralistic approach to be of such importance.

I don’t want to misrepresent my position on QB-TSP. I have been refining this formula for over a decade, and I think it’s the best stat out there for evaluating QBs. I would caution readers, though, to be judicious in using stats — any stat — to make sweeping judgments about pro football. I’ve done a subjective ranking that incorporates not only stats, but also a lifetime of research on pro football history, to create a more cohesive evaluation of great quarterbacks. If that’s what you’re looking for, please read that series, not this article. This isn’t a list of the greatest QBs, per se.

33. BERT JONES 10,551 20.6
34. RANDALL CUNNINGHAM 12,557 20.4
35. TONY ROMO 13,052 20.0
36. DARYLE LAMONICA 10,940 19.4
37. RICH GANNON 12,258 19.21
38. DONOVAN MCNABB 13,717 19.19
39. TROY AIKMAN 13,375 18.9
40. JEFF GARCIA 11,605 18.68
41. TRENT GREEN 11,087 18.67
42. LEN DAWSON 14,225 18.4

Does TSP underrate AFL seasons? Certainly Dawson ranks lower than one would expect. In 1962, when The Sporting News named Dawson AFL MVP, he scored only 1646 TSP. That year, the constant that penalizes attempts and sacks was -2.5 for the NFL, but -3.033 for the AFL. The highest-scoring QBs in AFL history were Dawson, Namath, Hadl, and Lamonica, all bunched together between 13.3 – 14.6. Hadl, with significant production in the ’70s as well, has the highest career mark, 23.2.

TSP rates Hadl (24th), Namath (31st), and Lamonica (36th) about where knowledgeable fans would expect. Rather than a systemic error in TSP’s treatment of AFL quarterbacks, this suggests a problem particular to Dawson. He had a long career and impressive counting stats, but he didn’t have big seasons. His passer rating was fantastic, but his sack and fumble rates are among the worst of all time. Of the top 100 QBs in TSP, Dawson ranks 88th in sack percentage (9.3%) and 92nd in fumble rate (1.92%). Don Meredith is the only player worse in both categories. Everything that doesn’t show up in the passer rating formula, Dawson was below average.3 I don’t believe Dawson is substantially overrated; he’s in my top 30. But I do believe that a casual look at his statistics makes him look much more accomplished than a careful look does. At this point in my career as an analyst and historian, I have little doubt that Joe Namath was better than Dawson.

43. TERRY BRADSHAW 13,342 18.1

Bradshaw, like Bart Starr, is a Hall of Famer largely because of his fantastic postseason performances, which don’t count here. Any regular-season ranking inevitably underrates him.

44. DAUNTE CULPEPPER 8,772 17.6

Culpepper only had five seasons in which he scored at least 175 TSP.4 You can view him as a fluke who had a few good seasons, playing mostly with Randy Moss, or as a Bert Jones-type whose career never fully recovered following an injury. I thought Daunte was a good QB, but not as outstanding as his stats would imply, so I lean more towards the first theory.

45. CARSON PALMER 12,917 17.5
46. JOE THEISMANN 11,069 17.33
47. NORM SNEAD 12,784 17.32
48. BART STARR 13,829 17.30

How on earth does Norm Snead have a higher rating than Bart Starr?

Starr had a lot of seasons that the system reads as compiling, and Snead only had one or two. I’m not sure fans realize how pedestrian Starr’s regular-season stats usually were. Consider 1960, when the Packers made their first NFL Championship appearance under Vince Lombardi. Starr passed for just 1,358 yards and 4 touchdowns. Yeah: four.5 He threw 8 interceptions. TSP scores that season at 875, a little below average, which doesn’t seem especially unfair to me.

Well, that’s one season. But it’s not atypical for Starr. He passed for at least 2,000 yards only five times, never for 2,500. His passed for double-digit TDs just seven times, and his career high was 16. He had a very short prime, basically just 1961-66. Look, Bart Starr was a great quarterback. He doesn’t have great stats.

Norm Snead passed for 2,000 yards eight times, including four years over 2,500. His passed for double-digit TDs 10 times, and his career high was 29. He had 1700-TSP seasons a decade apart, 1962 and ’72, and was a productive starter in at least eight seasons.

Snead threw way too many interceptions (5.9%), and Starr was the best of his generation at avoiding INTs (4.4%). Snead’s INT% is about one-third higher than Starr’s, a massive difference. But because of the way we use statistics, QBs who throw interceptions are underrated, and QBs who hold the ball, leading to sacks and fumbles, are overrated. Snead’s sack rate was about 7.5%, with a 1.2% fumble percentage. Starr’s sack rate was about 10%, with a 1.7% fumble percentage. His sack% is about one-third higher than Snead’s, and his fumble rate is almost 50% higher.

Even considering sacks and fumbles, Starr was a more efficient quarterback than Snead. But Snead was far more productive than Starr. Other than 1966, Starr was probably never the best player on the Green Bay offense. Jim Taylor and Paul Hornung were the motor, not Starr. Snead often was the best player on his offense, and TSP values his productivity, as I believe it should. Starr (1956-71) and Snead (1961-76) played at more or less the same time, and they made an equal number of Pro Bowls (4 each); I’m actually not proposing anything too radical here.

I don’t want to give anyone the wrong idea. Bart Starr was a better quarterback than Norm Snead, and it’s not close. But depending on how you balance volume and efficiency, their stats are very close.

49. VINNY TESTAVERDE 14,297 17.1

His 14,297 TSP ranks 29th all-time, but he only had four average (1000 TSP) or better seasons.

50. STEVE MCNAIR 12,716 17.0

He’s underrated by this method. McNair was an excellent runner, he struggled with injuries in some of his best seasons, and he had good years that don’t score a lot of value points. The Titans went 13-3 back-to-back seasons in 1999 and 2000, and McNair scores just 2.5 for those seasons combined. That strikes me as conservative. To some extent, it’s a Bob Griese problem, since Tennessee’s offense ran first and foremost through Eddie George.

51. MILT PLUM 9,869 16.9
52. KEN STABLER 11,234 16.4
53. CHARLIE CONERLY 11,431 16.1
54. MARK BRUNELL 12,138 15.9
55. DREW BLEDSOE 12,406 15.4
56. CHARLEY JOHNSON 11,910 15.33
57. NEIL LOMAX 9,279 15.27
58. BERNIE KOSAR 9,914 15.21
59. BRIAN SIPE 9,456 15.19
60. BILLY WADE 10,535 15.1
* SAMMY BAUGH 7,027 14.8

This is where Baugh would rank based on 1946-52. That time frame excludes most of his best seasons.

61. PHIL SIMMS 12,333 14.7
62. ELI MANNING 11,831 14.6

At this rate, he may never reach the top 50.

63. TOBIN ROTE 9,153 14.4
64. CRAIG MORTON 12,033 13.4
65. BILLY KILMER 11,407 13.3
66. STEVE DEBERG 11,424 13.2
67. EARL MORRALL 10,848 12.92
68. BOB GRIESE 12,311 12.88

It’s incongruous to find him below his backup with the Dolphins, Earl Morrall. I don’t believe this ranking reflects Griese’s quality or accomplishments as a quarterback, but he didn’t have any statistically outstanding seasons. Slow-but-steady is underrated by this methodology, which rewards big years more than consistency.

69. JOE FERGUSON 10,288 12.72
70. STEVE GROGAN 11,946 12.68
71. MATT HASSELBECK 9,710 12.52
72. RON JAWORSKI 10,309 12.49
73. DAVE KRIEG 12,590 12.48
74. MARK RYPIEN 8,023 12.3
75. JEFF GEORGE 9,067 12.2
76. DON MEREDITH 8,336 12.0
77. GEORGE BLANDA 9,433 11.9

A Hall of Fame kicker; a backup QB who had one exceptional season in a marginal major league.

78. RUSSELL WILSON 7,602 11.8
79. FRANK RYAN 8,929 11.743
80. CAM NEWTON 7,942 11.738

I named Cam Newton MVP in 2015. These are the top four in QB-TSP that season:

Carson Palmer, 2235
Russell Wilson, 1962
Cam Newton, 1953

Did I get suckered into Newton’s narrative, subverting what TSP told me about Palmer and Newton? I don’t think so. It was a very close call — I wouldn’t argue with any of the four — but I thought Newton accomplished the most relative to his support from teammates: “Working with an undistinguished group of receivers, Cam made plays you have no business making in the NFL, like the ghost of Randall Cunningham.”

Chase put it well in preseason: “After Greg Olsen, the Panthers top targets are second round draft pick Devin Funchess, Corey Brown, Jerricho Cotchery, Ted Ginn, Jr., and Ed Dickson. So when the Panthers go 6-10, it’s because Cam Newton isn’t a true leader.” They didn’t go 6-10, of course; they went 15-1 and led the NFL in scoring. That wasn’t because Funchess turned into A.J. Green; it was because Newton played like Aaron Rodgers. Newton passed or ran for 45 of the team’s 54 offensive touchdowns. In context, I thought Newton was more impressive than Palmer, and created the most on his own.

81. BRAD JOHNSON 9,493 11.6
82. GREG LANDRY 8,041 11.44
83. MATTHEW STAFFORD 8,033 11.39
84. TOMMY KRAMER 9,152 11.1
85. DOUG WILLIAMS 8,231 11.0
86. JIM ZORN 8,229 10.79
87. KEN O’BRIEN 8,342 10.76
88. STEVE BARTKOWSKI 8,020 10.72
89. JOHNNY LUJACK 6,022 10.69
90. DANNY WHITE 8,631 10.4
91. JAKE PLUMMER 7,968 10.3
92. ARCHIE MANNING 7,139 10.1
93. MATT SCHAUB 7,549 10.0
94. ED BROWN 7,826 9.9
* TOMMY THOMPSON 5,365 9.7

Like Sammy Baugh, he’s rated only on what he did from 1946 onwards. Unlike Baugh, that mostly tells the story of his career anyway.

95. BOBBY HEBERT 8,163 9.5
96. KERRY COLLINS 9,029 9.2
97. CHRIS CHANDLER 8,533 9.1
98. LYNN DICKEY 6,713 9.0
99. CHAD PENNINGTON 6,267 8.8
100. ANDREW LUCK 5,909 8.5

101-110: Jim Plunkett, Jay Cutler, Bill Kenney, Marc Bulger, Jim Harbaugh, Scott Mitchell, Michael Vick, Steve Beuerlein, Andy Dalton, Bill Nelsen.

It surprised me to see Andrew Luck (8.53) and Andy Dalton (8.09) separated by less than half a point. But I think that’s a function of popular narrative. Luck has been hyped and beloved since Day One, with Dalton the red-headed stepchild.6 Dalton has more passing yards than Luck, and more net yards per attempt. He has more passing TDs and a higher passer rating. He has more rushing TDs and fewer fumbles. Luck had a great season in 2014, and he rates ahead because of that, but the gap is not large.

1. In my head, I call this the “Bob Griese problem.” []
2. And because I can square values in my head, but ^1.85 requires a calculator. Simplicity and accessibility are more important to me than the illusion of precision. []
3. Dawson was a reasonably productive scrambler — he ran for 250 yards his first two seasons in the AFL — but not really a rushing threat, and he only scored 9 rush TDs, which is very low. Dan Marino scored more rush TDs than Dawson. Peyton Manning scored twice as many rush TDs as Dawson. []
4. That’s Ryan Fitzpatrick’s score for 2016. []
5. Fifteen players in the 13-team NFL threw more TDs than Starr. Ten of them threw at least twice as many. Johnny Unitas led the NFL with 3,099 yards and 25 TDs. []
6. I’m not proud. []
• WR

I don’t understand why Marino looks so good by this method. His rate stats and career totals just aren’t that impressive.

• Le Petit Cochon

Which career totals don’t impress you? He held the record for passing yards and TDs for 12 years each. He held the record for game winning drives for 14 years. He’s tied for fifth in first team all pros, and he’s tied for sixth in Pro Bowls. He attempted the fourth most passes of any quarterback and ranks sixth in career ANY/A+. Those seem like pretty impressive career totals to me, but maybe I’m not quite as wise and wonderful as you.

• Here’s a quick query that suggest otherwise: http://pfref.com/tiny/TDbE9

• Richie

Are you trying to bait the Brady fans? Your 500 attempt threshold cuts off 3 good Brady years.

Dropping to 400 puts Brady and Marino at #2: http://pfref.com/tiny/ZKORu

• Ha, no. Just thought I’d try to quantify both high attempts and high rates.

• Over on FO, Marino was top 6 in DYAR in 9 seasons, and that excludes 83-86 which were his best years. Six of the 9 were top TWO. When FO does get back to 83-86, Marino could well have 13 top-six seasons and nine or 10 top-two years out of 16 played (excluding ’93 when he got hurt). He has a 119 ANY/A+, which Le Petit Cochon alludes to as 6th all time. Even his passer rating very strong for his era (113 Rate+) despite giving him zero credit for avoiding sacks.

Personally, my appreciation for Marino has probably increased in the last 10 years more than any other QB, even as it may be diminishing for others as his career yard & TD records continue to get passed in the 21st century passing explosion.

• Richie

Marino is, not surprisingly, my favorite player of all time. I am one who has had a diminishing appreciation of his career. The more I learn about statistical analysis, the more he slips in my opinion. Sucks.

• The more I learn about statistical analysis, the more he slips in my opinion.

You must be analyzing the wrong statistics.

When Marino retired, he held the all-time career records for passing yards and TDs by about 20% each, an enormous margin. He held the single-season records as well. His passer rating was excellent (though not a record), and his relative sack% is still the best in history.

During Marino’s first decade in the NFL, 1983-92, he had the most completions by 32%, the most net passing yards by 37%, the most pass TDs by 44%, the best TD/INT +/- by 34%, and he’s a close 2nd in ANY/A, to a player with less than 2/3 as many attempts. Even if you look only at the Top 10 in passing yards, Marino is two standard deviations above second place in basically every counting stat, plus he’s top-three in every rate stat except raw Y/A, which by omitting sacks misses that Marino was the single greatest QB in the history of football at avoiding sacks.

He led the league in passing yards more times than Montana, Elway, Moon, and Kelly combined. In 1984, he led the league in touchdowns by 50%. His +31 TD/INT led the league by 72% and broke the record (+22) by 41%. He passed for 48 and 44 TDs in a season when the record was 36. His statistics were unthinkable; he completely redefined what it was possible for quarterbacks to accomplish statistically. It took over a decade, and significant change to rules that affect the passing game, for anyone to catch up.

Marino’s efficiency stats were merely great, rather than historic, in part because defenses knew he was going to pass. One of the principles guiding this statistic (TSP) is that the more one passes, the more difficult it is to do so successfully. This stat is unusual in its emphasis on both production and efficiency. Marino wasn’t a Griese or Flacco who only passed on 3rd-and-long and who were almost incidental to the team’s success; he was the Dolphins’ offense. That’s significantly to his credit, I believe.

Marino had terrific arm strength but uncanny touch. He had the quickest release of his generation, and maybe all time. He was an awe-inspiring downfield passer when the league was trending towards horizontal WCO passing, and he ran a brilliant two-minute drill. I know you remember these things; they’re all true, and they’re reflected in the numbers.

• Richie

I think 2 of the things where Marino suffers, that I didn’t realize while he was playing, are his mediocre playoff performances (both in wins and stats); and that most of his really great seasons came before 1987.

• Have you read Sean Lahman’s Pro Football Historical Abstract? You should check out his summary of Marino, especially as regards the postseason.

It is true that Marino’s two or three best seasons all came before 1987. Yet he leads in QB-TSP from 1987-91 and 1988-92, and he would have led beyond that if not for the injury in ’93.

From 1988-92, Marino led the NFL in passing yards. He’s the only player with at least 3,500 yards each season, and he and Warren Moon are the only ones with multiple 4,000-yard seasons. Marino led in pass TDs, and he’s the only player with at least 20 TDs each season. Among players with at least half as many pass attempts, he ranks 4th in passer rating and 1st in lowest sack%. His rushing is pretty worthless but his fumble rate is low.

QB-TSP for 1988-92:
Marino — 9,856 — 19.6
Moon — 9,024 — 17.4
Everett — 8,524 — 15.7
Kelly — 8,179 — 13.7
Rypien — 7,185 — 12.0
Montana — 6,034 — 12.0
Cunningham — 5,968 — 9.4
Young — 5,785 — 11.3

I don’t know that Marino was the best QB in any given year after 1986, but he was consistently near the top, which no one else can really say.

Marino’s TSP rank by year, beginning in 1983:
8, 1, 2, 1, 3, 2, 7, 5, 4, 2, injured, 2, 10, 8, 7, 12, 24

Most Top-10 seasons: Peyton [15], Marino [14], Favre and Tark
[12], Elway, Montana, and Unitas [11], Brady, Brees, and Van Brocklin
[10].

At least one major outlet named him second-team all-pro in ’88, ’92, ’94, and ’95. I’m especially partial to ’92. Compared to AP all-pro Jim Kelly, Marino had more yards, more TDs, fewer INTs, and a higher passer rating. Plus, I don’t think there’s any question that Kelly had better teammates. The Dolphins won the AFC East that season.

• WR

So Brady, am I correct in surmising that part of the reason why your system rates Marino so highly, is because relative to his contemporaries, he was so much better than them? Because I don’t disagree with that. It’s the guys that have come along since then, and matched or surpassed Marino’s output, that appear underrated to me. I’d take Brady over Marino right now, based on career production. But your rankings suggest the Patriots QB would need 2 more elite level seasons just to pull even with Marino.

• Four Touchdowns

I think the only fair way to compare these guys is to match them with their contemporaries — Brady and Marino played under different rules with very different football strategy and coaching philosophies guiding their careers.

• garymrosen

Absolutely. Johnny Unitas’ career QBR is 78.2 and his int % is 4.9. However I would still be reluctant to rate him below Alex Smith (85.3/2/2).

• garymrosen

That was 2.2 for Smith’s career int%. Also below Joe Montana’s.

• garymrosen

“It’s the guys that have come along since then, and matched or surpassed Marino’s output, that appear underrated to me.”

Most basic rule of stats is that when comparing across eras you have to take into account the average of the era. E. g. in the entire decade of the 1980s there were 4 QBs who surpassed 100 QBR for a season – Marino and three guys named Montana. In recent years there have sometimes been that many QBs in a single season over 100. That one stat is not the be-all and end-all but it is indicative of changes in the game that affect QB stats.

• In 06-07 or so the ESPN boards were a pretty interesting and sometimes wild place. One of the characters there was a guy who loved Marino and hated Favre. His explanations/excuses about Marino in the playoffs were infamous. The main one was Miami’s run defense. There were a lot of games they allowed 200+ yards (most egregiously, 329 to the ’95 Bills). This poster played up the 1994 game vs. Kansas City a lot, and would add that in that year’s loss to SD, Marino got into range for a GWFG that missed. I remember he once said the only playoff loss he truly blamed Marino for was ’97. It was an interesting and, yeah, pretty biased outlook.

Certainly, however, Miami’s teams were very lacking, missing the playoffs in a number of years Marino dominated. When they did make it, his playoff stats (rating of 77.1) actually aren’t *that* bad considering when he played and how often he had to throw. They were worse than his regular season numbers, but so are almost everyone else’s. And he’ll always have the ’84 AFCCG ; p

• Tom

I’m in agreement about how great Marino really was…putting the stats aside, I’ve watched some clips of him on YouTube, and couldn’t get over how fast he gets the ball out…and he could move, too. Here’s a great play, 3rd-and-18:
https://youtu.be/Pcn0uMJkYEw?t=639

• Richie

It’s so weird trying to watch games without the score/quarter/time graphic showing. How did we do it back then?

Is there a bigger “what if” matchup in history than Super Bowl XX? If the Dolphins had done their job and gotten to the Super Bowl, could they have given the Bears a run? I’m guessing the Bears probably would have still won handily. But it would have been nice to see what happens.

• Tom

Ha! Yeah, it’s frustrating watching those old games because you’re like “What quarter are we in? What’s the down and distance? What the hell is going on?”

Hard to say what would have happened, but I’m thinking the Bears would have the edge…they were insanely dominant that year…then again, if Miami could beat them once, who knows?

• sacramento gold miners

I miss the yellow first down line we have today, but enjoyed watching the NFL more during that era. Fewer moronic promos and breaks, and I liked how the play usually started faster back then. Today, it’s tedious watching everyone standing at the line, while the QB gestures and scans the defense. I also liked the fact defenses could hit more liberally without getting penalized.

• sacramento gold miners

Yes, I think the 1985 Dolphins could have beaten Chicago for a second time that season. They had the blueprint for neutralizing the Bear’s defense, and this time, would be playing indoors. Marino’s release and those receivers were lightning quick, and I’m not sure what Chicago would try schematically the second time around to fool Miami. In the regular season game, I believe Payton did rush for over a hundred yards, but some of that came after the game had been decided, Chicago wanted to feed him the ball for a record of some sort.

My guess is for the hypothetical rematch, Chicago would try to execute the running game better early. Mix in the pass, and use ball control, to avoid a huge deficit. Also, hitting Marino more often(even if it drew a penalty), would be high on the Bears list.

Unlike Tony Eason, Dan Marino wouldn’t have been intimidated the second time around. Too bad we didn’t get the opportunity to see this game.

• Corey

FO going back farther in time has also shown just how bad the defenses Marino got saddled with were. Miami ranked dead last in defensive DVOA in 1987, 1988, and 1989.

• sacramento gold miners

Phillip Rivers does have a postseason issue, leading the Chargers to only one conference title game in his career. Glad to see Dan Fouts rated ahead of him, always thought Fouts was the more lethal player.

Ken Anderson’s last two postseason performances were underwhelming, and he was a poor comeback QB historically.

Always felt Manning’s 2015 season was underrated, a great player getting the job done even in the downside of a career is impressive.

• Four Touchdowns

It’s underrated in the sense that he was physically limited (beyond just age, we’re talking significant nerve damage in his throwing arm) AND playing in a brand new run-heavy scheme with a poor run-blocking offensive line.

But the numbers and the eye-test doesn’t lie, he was mostly terrible that year, getting by solely on football intelligence.

• sacramento gold miners

But that football intelligence was on full display in the postseason, when Manning made some of the key plays which had eluded him in some past defeats. The numbers don’t tell the full story, whether it was switching to better plays, or converting third downs, even a diminished Manning was good enough to deliver a title. We also can’t underestimate how important it is to grab early leads in conference title games and super bowls, and that’s exactly what Manning helped to do. Put Trevor Simeon on the 2015 Broncos, and they’re likely going down against the Steelers in the opening playoff round.

Manning also was far more composed in his final game, as opposed to the super bowl disaster versus Seattle. A great player can still pass the eye test when it counts the most. I definitely think Manning’s final season added to his career value, even though the stats weren’t impressive.

• Tom

I’m only kind of with you on this…perhaps in the playoff games against the Steelers and the Patriots, but in the Super Bowl, I don’t know…there are a couple of huge plays that could have turned the game around had the Denver defense not been there to save the day: the 2nd quarter interception was big – being on the CAR 24, it took some points off the board when it was still a one-score game – and then there was the 4th quarter sack fumble (and of course, we can talk about who’s “fault” that was). I’ve spent a decent amount of time looking at the plays for this game (EP and WP numbers, etc.), and there’s a lot of 3rd down incompletions in there, or short gains that don’t get the first down (and again, we can always debate how much of this is Manning’s fault, I’m OK with that). On the flip side, yes he made some key plays: the 22-yard and 25-yard passes to Sanders in the 3Q and the 16-yarder to Sanders in the 4Q. These are important key plays, but in my mind they just even out the key plays that he missed on.

Anyway, I love Manning and I’m pleased as punch that he got to win another Super Bowl, but as someone and myself said in another post, I think he played just well enough in that game to not lose it. Frankly, I think it’s the worst performance by a winning QB in the Super Bowl, and I hate saying that, believe me.

• Tom

Brad – nice article! I’m working on something like this myself…using WP numbers, Chase’s “value” stat, etc…not anywhere near as systematic as what you could do, but fun anyway. I’m not done so I’m missing some guys, here’s what I’ve got currently for the bottom 5:

5. Bob Griese, SB 8
4. Joe Theismann, SB 17 (I watched this game the other day, and really, he was giving this game away)
3. Johnny Unitas, SB 5 (Johnny is kept from the bottom of this list by one freakish play – the 75-yard TD to John Mackey which may or may not have touched another Colts player making it an illegal catch)
2. Ben Roethlisberger, SB 40
1. Peyton Manning, SB 50

So, matching your list fairly closely…Dilfer may end up in the bottom 5 once I get around to processing that game.

Brad – quick question, and we’re way off topic here: so far, I’ve got Montana’s 1988 game as the best SB performance in history. I know a lot of folks put the 1989 above that one (or have other QB performances above that), and that makes sense, but I have my reasons. Your quick thoughts (I know this discussion needs to be held elsewhere)?

Finally – I agree with your comments in that article about Manning in SB 50…*twice* he had to bring up that beer stuff, and it was stupid…kind of ruined the moment.

• Best Performances By Super Bowl-Winning Quarterbacks

I rate Montana’s XXIV better than XXIII.

• Tom

Thanks Brad. I’ve got the Bengals game as “better” because Montana played almost perfectly in a game that was tight throughout. The 27-yard pass to Rice on 2nd-and-20 with 1:17 left that put the 49ers on the Bengals 18 might be one of the biggest passes in SB history. Anyway, thanks again!

• You’re welcome! I enjoy these discussions. Obviously — you bring up topics I’ve already chosen to write about 🙂

Regarding Montana’s best Super Bowl, I hesitate to agree for several reasons.

Montana played in four Super Bowls and was named MVP of three; you’re saying the only time he didn’t win MVP was not only his best personally, but the best of all time. That’s a bold position.

The Niners scored 26 points in Super Bowl XVI, 38 points in Super Bowl XIX, and 55 points in Super Bowl XXIV. They scored only 20 points in Super Bowl XXIII, against a pretty average defense. If this was the single greatest performance in history, the scoreboard doesn’t reflect it.

Montana went 23-of-36 in that game; “almost perfectly” makes me think more of Phil Simms in Super Bowl XXI.

It’s an interesting argument, and maybe there’s something I’m forgetting or misunderstanding, but I have trouble seeing it.

• Tom

Brad – you’re right on all counts…not as many points, not the MVP, and yes, I overstated things by saying he was “almost perfect”. So there’s nothing you’re forgetting or misunderstanding, it has to do with how I’m doing this…without getting into the numbers (hopefully I’ll do a post), here’s my general position…

1. For starters, and obviously this can be debated, I agree with Chase’s methodology in the post I mentioned, and that puts Montana’s 1988 game as 10th overall. True, 1989 and 1984 are ranked higher, but I’m starting from the position that the 1988 game was statistically a very good game for Montana (Top 10).

2. OK, the 49ers only scored 20 points, but their kicker Mike Cofer missed two field goals. A 19-yarder and a 49-yarder. We can say that Montana “should have” scored TD’s on those drives, and the 49-yarder wasn’t a gimme, but in my mind it’s fair to say that they could have reasonably put up at least 23, which is only 3 shy of the 26 scored in SB 16. On top of this, Roger Craig fumbled, which killed another drive…yes, that happens in games, but just trying to point out that if you look at a drive chart, it makes a little more sense how they only scored 20 points.

3. Here’s where we diverge,drastically – I’m looking at “clutch” performance and using Win Probability to evaluate that. Yep, there’s A LOT wrong with what I just said, I totally get it – “clutch” ability probably doesn’t exist and WP has all kinds of problems when used on its own (Malcolm Butler would have the greatest Super Bowl performance of all time, for starters). And you’re right, there’s something to be said for just putting the game away and not letting the other team hang around, that’s true.

But that being said, I can’t help myself. I love the drama of this game, in my mind the “great” plays of all time are the ones that matter the most, and the “great” games are the ones where the ending is in doubt…definitely not the *best* way to evaluate a player, and that’s why I’m trying to use both approaches – just plain stats plus “clutch”. Who played great (or at least very good) throughout the game *and* played great when it mattered most?

I don’t want to get into the WP numbers…I use them, but I’d rather just describe the situation:

Montana is not playing great in the first three quarters in 1988, but he’s not bad either. So here’s where he gets a gets a huge bump for me; these three drives in the 4Q:

1. Started at the end of the 3Q, on the 49ers 15-yard line, score was 6-13 Bengals, Montana goes down the field 85 yards in 4 passes including a TD and now it’s tied at 13-13
2. Starts at 11:14 on their own 18-yard line, goes down the field 50 yards, Montana throws incomplete on a clutch 3rd down, Cofer misses 49-yarder.
3. Starts at 3:04 on their own 8-yard line, score is 13-16 Bengals, goes 92 yards, including a 27-yard pass on 2nd-and-20 which lands them on the 18-yard line, two plays later, pass to Taylor, TD, wins the game.

The 49ers were behind or tied on every drive, and Montana was lights out: 12 for 15, 226 yards, 2 TD’s, no sacks. Every single one of those drives, there is pressure to score…time is running out, you need a score, and they start deep in their own territory on each one. And on each one he puts his team in a position to score (even though the second drive was a “failure”, per se).

(end of story)

There are other games where QB’s lead amazing comeback drives and play great in the clutch – Brady’s done it a bunch of times – and I view those games as “great”. And there are games where QB’s just put up monster numbers and don’t need to play great “in the clutch” – Montana, Young, Williams etc. – and those games are also “great”. By my “system” Montana did both very well in this game, and that’s why I stick by it being the best SB game ever.

When I get around to it, I’ll do a post, and we can all pick it apart. Thanks again Brad, sorry for the long post guys!

• Heh — interesting stuff. I don’t necessarily agree, but I admire the passion and dedication. I suspect that if you were to widen your sample, a lot more bad QBs have games like this than games like Montana vs. DEN.

• garymrosen

“a lot more bad QBs have games like this”

Sure – maybe once in their career. I believe backup QB Frank Reich still has the biggest 2d-half comeback ever, in the playoffs no less. And Alex Smith had a terrific finish to that 49er playoff game against NO. But Montana has a long string of miracle comebacks going back to his career at Notre Dame and stretching through his final years at KC.

• sacramento gold miners

Montana really exploded in the fourth quarter of Super Bowl 23, with 195 passing yards, and the 14 points to pull out the win. However, he could only produce a pair of field goals in the first three quarters of that game, against a defense which lost Pro Bowl NT Tim Krumrie early in the first quarter. Part of the reason was the zone blitz scheme, the Dick LaBeau version, which was new at the time.

• Tom

You’re right, he wasn’t too hot for the first 3 quarters..maybe I’m out to lunch on this “Montana ’88” thing. His drives in the first three quarters ended like this:

Punt
FG
Missed FG
Roger Craig fumble
Punt
Punt
Punt
FG

I guess that really doesn’t scream “greatest game of all time”. It’s his WP numbers in that explosive 4th quarter that put him at the top of my list. He leads two drives where his team takes the lead or ties…the first one gives him 31%, the second one gives him 91%. Maybe back to the drawing board…

• garymrosen

“he doesn’t have any huge WP-draining mistakes early on”

That is one side of Montana’s greatness. In an internet discussion on QBs a few years ago someone made the astute observation that Montana had the “biggest ratio of good plays to mistakes”. Some QBs play a conservative game, have good completion and low interception ratios but never seem to make a lot of crucial plays. Montana’s play was as flawless as any other QB ever, it seemed like a “percentage” game out of Walsh’s short-pass WCO though later on with Rice he threw deep more often. But he is associated with as many if not more dramatic game-breaking drives and plays than anyone.

• sacramento gold miners

I can’t have Bob Griese in my bottom five, because he did exactly what the Dolphins needed. 6-7, on the day, as Paul Warfield was playing hurt with a leg injury. Miami’s offensive line was brutalizing Minnesota’s defense, and Larry Csonka was dominant.

John Unitas was knocked out of SB 5 early with damaged ribs, not sure I would have him in the bottom five.

Agree with Roethlisberger as the worst game, it was surprising, he he was outstanding in the the road playoff wins.

• Tom

You make a good point about Griese…it’s unfair to call his performance in SB 8 “one of the worst” when he barely threw the ball. He’s number 5 for me because he just didn’t do much, and I don’t mean that in a bad way. The Dolphins won the Super Bowl and he played a part, but compared to other winning QB’s, that part was small. But anyway, I agree, he did what was need to do, and a guy certainly can’t be blamed for not slinging it around when it’s not needed, etc.

• Richie

Even worse than Roethlisberger in the 2005 Super Bowl?

• Tom

Yeah, I’ve got Big Ben as the second worst. I’ll probably do a post about this; honestly, it’s kind of goofy, but I like it. I’m looking at how well each guy played according to this Chase post:

http://www.footballperspective.com/best-and-worst-super-bowl-passing-performances/

And combining that with a WP analysis. Any “real” football stat-analysis guy would shoot me for how I’m doing it, but whatever.

I probably have Ben ahead of Manning because of this play: 37-yard pass to Hines Ward on a 3rd-and-28 in the 2nd quarter when they were down by 3. That’s a big (not massive) clutch play. And then he’s got a slew of other minor 3rd-down conversions, including his rushing TD. He’s not way better than Manning, but I have him ahead.

• Richie

That 37-yard pass was a lame duck, that Ben is lucky wasn’t intercepted. And, I guess Seattle didn’t bother rushing the passer, just trying to keep any gains short?

It’s funny how these things go. A few plays earlier, Ben put one right in Ward’s hands that he dropped.

https://youtu.be/-AkPMPqnjuA?t=6m31s

• Tom

Hahaha…you’re right, very, very close to being picked off! But then again, he scrambled out of the pocket, kept the play alive, yada, yada, yada.

• sacramento gold miners

I’ll always believe Peyton’s SB 50 was stronger than his SB 48 performance, the Broncos were a different offense with a better defense in 2015. In the context of the game, Manning was far more composed, and gave Denver the early lead, which is so important in Super Bowls. He also delivered the final TD as well, and didn’t make the costly turnover the SB 48 Manning committed which sealed Denver’s fate versus Seattle. I agree Manning definitely looked his age, missing some throws, but made just enough plays, while Cam Newton appeared overwhelmed from the get go.

Manning’s SB 48 stat line is one of the most deceiving in SB history. 24 of his 34 completions, and 239 of his 280 passing yards came after his poor play had given Seattle a 22-0 lead. The Seahawks didn’t mind some of those short completions, as it kept the clock moving. And some of those stats were complied with Seattle playing softer coverages. It’s like Sam Bradford’s 2016 season, the stats are lying in the context of what performance really was.

The other bad aspect of Manning’s SB 48 showing was how frustrated he became early, you wouldn’t have known this was a future HOF QB with plenty of experience in noisy environments. The sulking on the bench while Denver’s defense was on the field for most of the first half was damaging. Technically, Denver was still alive with roughly four minutes left in the second quarter, trailing 15-0, but it must have felt like 30-0 on that Broncos sideline. Manning had a spectacular 2013 season, but played like a rookie in SB 48.

• Tom

I still think SB 50 is “worse”, but not by much, and you make a great point (as you did in another comment) about Manning’s poise and demeanor in SB 48. He needed thicker skin in that game…perhaps the shock of their offense not playing well after possibly the greatest season by an offense ever was too much, who knows.

I agree that Manning’s SB 48 stat line is deceiving, this is why I’m including game situation in my rankings of best SB QB performance. Those numbers he put up in the second half just arent that meaningful *in the context of how this game was won*, and I feel the same way about Steve Young’s 5th and 6th TD’s in 1994. I don’t throw the numbers out, but I like trying to make a distinction between plays that matter and plays that don’t.

• Tom

Brad, this is awesome. As you and others have noted many times, it’s a hell of a thing to try and rank these guys – different eras, offensive schemes, strength of opponent, surrounding talent, etc., etc. – but your systematic approach, always well-explained and transparent, once again gets us further down that road. Thanks for the great read.

• I’m glad, Tom, thanks much for the compliment.

• No doubt. Always appreciate this stuff.

• On Rivers, I like this chart:

Rivers is about a year and a half behind Manning in playing time, but he’s nearly caught him in TDs. He obviously is way ahead in INTs.

Stats aren’t everything, of course, but Rivers has a massive advantage in ANY/A. And he now even has a higher postseason ANY/A. And a better winning percentage.

http://www.pro-football-reference.com/play-index/pcm_finder.cgi?request=1&sum=1&player_id1_hint=Eli+Manning&player_id1_select=Eli+Manning&player_id1=MannEl00&player_id2_hint=Philip+Rivers&player_id2_select=Philip+Rivers&player_id2=RivePh00

I disagree with the idea that Rivers is picking up HOF steam; to me, he’s been a HOF caliber player for a long time. If anything, Eli is the one who is picking up HOF steam as he keeps rising on the cumulative leaderboards. Go back to 2010, and Rivers was #2 (and just a hair behind #1 Rodgers, on significantly fewer attempts) in career AY/A: http://www.pro-football-reference.com/leaders/pass_adj_yds_per_att_career_2010.htm

• You don’t believe there’s a growing sense that Philip Rivers has had or is rapidly approaching a Hall of Fame-caliber career?

I think Eli will probably get in at some point, but I believe his chances are dropping at this point; he’s losing the aura he had after those two Super Bowls. I think the mainstream media is less adoring of Eli than they were a few years ago; the “clutch” myth has gotten pretty hard to maintain. Maybe I’m wrong, that’s just been my perception.

• Perhaps I just have internalized natural progression. Four years ago, I would have said Rivers had a better chance than Eli, because (1) Eli wouldn’t win another Super Bowl and (2) Rivers would keep having the edge in rate stats. Maybe if both retired 4 years ago, I would agree with you, but I think I had internalized the idea that by the time Manning was up for induction he’d be about 15 years removed from his last Super Bowl. I think over the last 4-5 years, Rivers has done worse and Manning has done better, but I understand your point, too. I think I just had projected that into my mind already, if that makes any sense.

• Tom

Agree about the “aura” of Eli…regardless of his regular season stats, the fact that he and and the Giants beat arguably the greatest dynasty in the history of the league *twice* has created a mythology of greatness about him that is hard to argue against. But a lot of time has passed, and that perception of him is starting to wane…What he needs, and what I myself pray for every year, is a THIRD Giants/Patriots Super Bowl…it’s a trilogy that has to happen like Frazier/Ali and Johnson/Bird, etc.

• sacramento gold miners

I’m in the opposite camp, I think Phillip Rivers is losing steam, although the 300 plus TDs will eventually get him elected. Hasn’t been to the playoffs since 2013, the franchise is in flux, and the final years could be tough. Had that excellent playoff game against the Colts, but that’s not as memorable as the Fouts’ Miracle in Miami classic. I think voters who saw Fouts won’t be overwhelmed by Rivers numbers amassed in an easier era to throw the ball.

Eli was in after beating the Patriots for a second time in a SB, and the Giants are a more stable, storied franchise. I also think 10-20 years from now, the SB 42 upset will gain traction in the history books. Manning returned the Giants to the playoffs again last season. I feel Manning’s signature moments on the bigger stage with a more storied franchise will have him in Canton at least five years ahead of Rivers.

• I may be in the minority, but I wouldn’t put anyone from the 2004 QB class in the HOF. From the realignment era, it’d just be Peyton, Brady, Brees, and Rodgers for me. (So no Warner or Romo either).

• I agree with modifications one and two, although my decision on two can’t be final till I see what Blanda’s 1961 looks like. I think I agree with three, too, but it seems to go against the philosophy of rewarding greater value over replacement.

• Blanda is the only AFL QB among the top eight in 1961:
TSP / Era / Pts
S.Jurgensen 2262 / 2748 / 7.6
G.Blanda 2096 / 2547 / 6.5
J.Brodie 1783 / 2166 / 4.7
B.Wade 1521 / 1848 / 3.4
Y.Tittle 1498 / 1819 / 3.3
M.Plum 1442 / 1752 / 3.1
B.Starr 1220 / 1482 / 2.2
J.Unitas 1128 / 1370 / 1.9
J.Lee 815 / 990 / 1.0
J.Kemp 731 / 888 / 0.8

I’m not sure what you mean about going against the philosophy of rewarding greater value over replacement.

Forgive me if I missed it in the article, but what is the difference in constants between the NFL and AFL in 1961? I’m of the opinion that the early AFL was significantly weaker than it was from `65-`69, to the point of it barely qualifying as a major league. How do we know that the 1960 AFL was a more legitimate league than Warren Moon’s CFL? I’m honestly not so sure.

• The NFL’s constant is 2.5 from 1950-77. The AFL’s constant is +0.667 higher for each year removed from the merger. Thus, 2.567 in 1969, 2.633 in 1968, etc. It’s 3.1 in 1961. I agree that the AFL in ’61 was, as I wrote in the Blanda section, “a marginal major league”, but one player in the Top 8 seems about right to me.

Good point about Dawson. I tend to think Dawson’s `62 must be better than Blanda’s `61, but objectively there’s not really evidence for that claim.

• I should’ve elaborated on three. In conversations with both me and others, you’ve been much higher in VOR than VOA. I tend to agree that average play is important and should be rewarded. Using the higher exponent seems to go against that ideal. Would it be fair to look at ERA as a VOR and Career Value Points as VOA? Or, to rephrase, would it be fair to view one as a measure of value and the other as a measure of dominance?

• I get what you’re saying, but the answer is “not really”, esp WRT the first question. What I like about this system is that it still begins with VOR. The constant always approximates replacement level. The exponent is a way of rewarding big seasons rather than compilers, but the baseline is always replacement level. The high exponent probably places a little too much emphasis on big seasons, but the relative value of production vs efficiency is unchanged. Hopefully I explained that in a way that makes sense to people besides me.

I agree with using replacement level for the constant, but I struggle to understand how Brock Osweiler’s 2016 rates at precisely that level. By any other measure (ANY/A, DVOA, PR, etc.) he rates significantly below replacement, which I believe accurately reflects how poorly he played. Does TSP boost him up to zero simply because he attempted a lot of passes?

• No, volume is neutral here. Each attempt or sack incurs a -4.5 penalty. Doesn’t the “A” in the stats you cited represent Average? Replacement level is different. I think Osweiler is a pretty great representative of replacement level. Do you really think you could sign someone off the street, or lure Kyle Orton or somebody out of retirement, and get significantly better play than Osweiler’s?

I should have mentioned DYAR instead of DVOA, but ANY/A and passer rating don’t have a set baseline.

Yes, I do believe there are a number of backup QB’s who would play better than Osweiler if given the chance. Hell, his own backup Tom Savage looked noticeably superior in limited action. To me, replacement level is equal to the 32nd best starter in the league. Perhaps you’re using a different definition?

• Yeah, you’re using a MUCH different definition of replacement level than I am. Tom Savage wasn’t a street free agent; that’s what I think replacement level implies.

If Osweiler was significantly below replacement level, what on earth was Jared Goff? Division-III? High school?

Any QB who isn’t good enough to start is, by definition, replaceable. At least that’s the way I see it.

Your quip about Goff leaves out some important context. The NFL’s talent margin is razor thin, so even the worst starter is still 95% as good as the best QB in an absolute sense. Let’s say Osweiler is a 95 and Goff is a 90; Goff is twice as far from elite as Osweiler, but still far better than a D-III or high school QB (who might be a 60 and 30, respectively). By the definition of replacement level you’re using, shouldn’t we give bonus TSP to every QB who makes an NFL roster? If street free agent is the baseline, then any QB who’s not on the street should theoretically have positive value.

• That’s fine, except that it’s not what replacement level means. This is a term with an established definition, and it’s not the one you’re using.

I apologize if my Goff question came off as snippy; it wasn’t my intention, though it does seem to be yours in the reply. No, TSP is based on production and efficiency, so a player who doesn’t actually produce anything will never score based on his potential.

If that’s the correct definition, then most stat analysts seem to disagree. Chase and Bryan both use 75% of league average for their replacement level baseline, and FO uses something like 87%. Sounds like this is more a semantics issue than anything; perhaps the baseline I’m using should be calling it “Starter Level” or something of that nature.

Haha this is where the internet fails us. I thought you were being a smartass on purpose, so I returned the favor. Apology accepted and reciprocated.

• I’m looking at the AFL modifier again, and it still seems pretty high for at least the first few years. The AAFC modifier has the league at about 85.7% strength versus the NFL, which I think is fair. However, it currently has the 1960 AFL at about 78.9% strength, which I think is pretty high for a league full of expansion teams. I might go with .085 in order to make 1960 AFL about 75% strength, although I think using .10 and making it around 71% strength is probably better. The problem with that is that I think it underrates the later AFL. A 0.1 puts the 1969 AFL around 96% strength, while the .0667 gives us 97.4%, and a 0.085 leaves us with 96.7%.

• The proper weight for AFL seasons, and especially early AFL seasons, is tricky. Like you, I think there’s no question the very early AFL was weaker than the AAFC. Beyond that, however, anything I write below that sounds confident is just tone being difficult to interpret in this medium.

Do you think the career values for players like Dawson, Namath, Hadl, and Lamonica ring true? Other than Hadl, I think most AFL QBs rate lower than people might expect. Jack Kemp ranks 135th, I think, below Brian Griese and Tony Eason.

Highest-scoring AFL QB each season, score is Era-Adj, brackets indicate # of AFL QBs in the Top 10:

1960: Jack Kemp, 1100 [5]
1961: George Blanda, 2547 [3]
1962: Len Dawson, 1646 [2]
1963: Tobin Rote, 1625 [2]
1964: Babe Parilli, 1653 [3]
1965: John Hadl, 1221 [1]
1966: Len Dawson, 1891 [4]
1967: Joe Namath, 1997 [4]
1968: Daryle Lamonica, 2086 [4]
1969: Daryle Lamonica, 2278 [2]

That seems tenable to me, especially since in 1960 Kemp ranks 5th; a bunch of AFL guys snuck into the bottom half of the top 10. Bart Starr (1358 yds, 4 TD, 8 INT, 70.8 rating) ranks ahead of George Blanda (2413 yds, 24 TD, 22 INT, 65.4 rating) that season.

There’s an element of guesswork, obviously, and I’m interested in your feedback, because this is something I struggled with.

• It’s probably impossible to know how to approach this, and I wouldn’t trust anyone who claimed to be certain about his or her results. Keep that in mind so I don’t sound (as much) like an overconfident jerk. It seems to me that the AFL probably wouldn’t progress in a linear pattern, and I think Jim Glass does a fair enough job explaining the reasoning for that here. I don’t agree with everything he said, but he makes some good points.

It seems fair to me to say the best players and teams in the AFL would do well in the NFL too. I think the Gillman (and Alvin Roy) Chargers, the Stram Chiefs, and the Saban Bills would have competitive games with the Bears and Browns, but I don’t think they’d touch the Packers in the early part of the decade. I think Lance Alworth and Cookie Gilchrist would embarrass NFL defenses just like they embarrassed AFL defenses.

To me, the distinction is the median team and median player, who I think wouldn’t have been anywhere close to the NFL till at least 1966-67. Even if AFL teams could lure great talents, to have solid depth in a league full of expansion teams is unrealistic. It takes time to build a decent roster, and I don’t think it’s unfair to compare early AFL teams to the free-agency-loving-Dan-Snyders with big names on the roster but flotsam and jetsam filling out the bottom half. I think that is a problem that didn’t exist with nearly the same magnitude in the AAFC.

Putting aside philosophical aspects, I also performed research similar to Jason’s in the linked article. Instead of coming up with SRS scores, I came up with estimated Pythagorean scores and used them to come up with what I guess you could call an average AFL team rate. In other words, if the NFL is always set to 1.000, the AFL is set to 0.???. I’m sure you will disagree with my results, which I think is a good thing. If we always agree, we’d make very little progress. Here’s what I have:

Year …..VsNFL
1960 …..0.529
1961 …..0.606
1962 …..0.667
1963 …..0.683
1964 …..0.694
1965 …..0.717
1966 …..0.722
1967 …..0.761
1968 …..0.833
1969 …..0.861

• WR

To clarify my earlier comment on Marino, I’m not saying that he’s overrated, or anything like that. I meant that relative to the other guys near the top of the list, his numbers aren’t all that impressive to me. This ranking has Marino ahead of everyone except Manning, and by a huge margin. This ranking gives Marino 18% more career TSP than Tom Brady, and 20% more than Drew Brees. By career yearly value, he’s ahead of Brady by 27%, and ahead of Brees by 30%. When you consider that Brady and Brees have already surpassed Marino’s career totals in yards and TDs, and have era-adjusted rate stats very comparable to Marino, I don’t think that’s an accurate reflection of the relative value of those three players.

I’m not saying Brad is wrong. I’m just saying that I don’t understand how he got results with margins that large.

Nice to see Tarkenton ranked so high; I believe he’s top 5 all-time, but very few seem to agree. Despite a lack of great teammates, Fran consistently produced throughout his career. I also fully agree that Brees and Anderson are overrated by this methodology, but it’s damn near impossible to devise a statistical ranking that doesn’t overrate them. Bizarre seeing Everett at #27; he’s underrated but I think that’s at least ten positions too high.

I would prefer you used the 1.85 exponent, but understand going with 2 for simplicity’s sake. IMO Daunte Culpepper is vastly overrated here, as Cunningham and Jeff George (!) both had career years with the same supporting cast. Johnny Lujack is cartoonishly overrated for his one season peak; I wonder if you could modify the formula a bit to require x number of halfway decent seasons in order to sniff the top 100. Even with a 1.85 exponent, the Lujack types would still come out much higher than they should.

I don’t have a problem with Bradshaw at #43. Maybe it’s a little low, but subjectively I don’t think he’s better than 35th all-time. IMO his postseason play isn’t enough to overcome an underwhelming (for his reputation) regular season career, accentuated by him getting chance after chance in his early years despite poor play. Also happy that Joe Flacco doesn’t crack the top 100. Sure there is value in stringing together average seasons, but he has often struggled to clear even the bar of mediocrity, especially since winning the SB. Frankly I consider Flacco among the most overrated quarterbacks in history.

Overall a great list and fantastic read. I appreciate the untold hours of effort you’ve put into these studies over the years.

• sacramento gold miners

Tarkenton’s greatness was on display with the Giants, when he was key in turning them around. In fact, Tarkenton beat the Vikings in 1969. In his second tour of duty with Minnesota, Tarkenton did have a strong supporting cast, but they were underdogs in those three SB losses.

• Thanks, Adam. I tend to agree with your critiques, but I’d suggest you look more closely at Johnny Lujack. He probably is overrated here (1949 scores 6.3), but it was hardly his only good season. He was a consensus all-pro in 1950 (granted that today it’s hard to understand why), and he was very good in ’51. Don’t overlook his rushing; from 1950-51 he scored 18 rushing TDs, and this was an era when you could lead the league if you threw 20 pass TDs.

Where you rate Bradshaw depends heavily upon how much you weight postseason performance, but I think you’re underselling his peak somewhat. TSP shows Bradshaw as the 2nd-best QB in football from 1977-81 and 1978-82, behind Fouts. I don’t agree with punishing him for “getting chance after chance in his early years despite poor play”. It’s true that many other QBs haven’t gotten the second chances Terry did, but he made the most of them; the unusual opportunity shouldn’t diminish what he accomplished in the second half of his career. No better than 35th is a very conservative ranking for a regular-season MVP who also was arguably the greatest Super Bowl quarterback of all time. Again, I wonder if you’ve given enough weight to his rushing accomplishments.

That said, Griese, not Bradshaw, is the ranking I’d really like to hide. Miami’s offense was so successful on the ground, it didn’t make sense to have Griese throw much, including many situations in which he might have caught the defense off-guard, and it really limited his productivity.

I can’t say I’ve ever noticed Lujack’s rushing numbers, but they do paint his short career in a better light. Since pass attempts were so sparse in his day, rushing value for QB’s was probably more important than it is in the modern game.

Our differences on Bradshaw are mostly philosophical. You’re evaluating careers based on what actually happened, while I’m injecting a bit of hypothetical into my judgments. The circumstances of Bradshaw’s career were extremely fortunate for him (due to his draft position and being backed by the Steel Curtain), and I believe a number of QB’s with lesser careers would’ve done just as well in his shoes.

• I’m content to differ on Bradshaw, but:

1] I’m much more comfortable evaluating careers based on what actually happened, and

2] The circumstances of Bradshaw’s career were extremely fortunate for him from 1972 or 1974 or thereabouts onward; he was drafted onto maybe the worst team in professional football, and in the years before free agency, that took a while to turn around.

3] It’s possible that a number of QB’s with lesser careers would’ve done just as well in the regular season, but Bradshaw’s postseason career was spectacular and distinguished. It seems like you’re underselling that.

• A quick note on Jeff George. By TSP, the year you mention is actually George’s fourth-best season, behind 1995, 1997, and 1994. That mostly due to volume; George threw 329 passes for Minnesota, compared to over 500 the other three seasons. His INT% with the Vikings was pretty high, too.

• eag97a

Minor nitpick. I would characterize this as quarterback total regular season statistical production. Some of the greats have a lot of playoff production

• Four Touchdowns

I am curious why post-season production was excluded, especially with a study this exhaustive.

• eag97a

You’ll have to ask the poster. I would think issues regarding the one and done format of the playoffs makes quantifying them a bit of a challenge. I think that qb’s should also get some credit for successfully leading scoring drives regardless if they just hand it off to running backs and results to a FG or rush TD by a running back. I subscribe to Bill Parcells’ school of thought regarding qbs (QBs are evaluated by how successful they are in leading the team to the end zone. Commandment # 9 (https://billyliggett.wordpress.com/2007/10/09/bill-parcells-11-quarterback-commandments/).

• It’s not a direct comparison. TSP scores player seasons, so postseason games would require a substantially different statistical approach. Postseason opportunities are not equally available to all players, which is another problem — this system already underrates QBs stuck on bad teams — and there’s the question of how to weigh successive rounds of the postseason. How much more is a Super Bowl worth than a wild-card game? Is a wild-card game worth more than a regular-season game? Is playing well in a wild-card better than getting a first-round bye? Is playing poorly in a wild-card worse than missing the playoffs altogether? Those are moral judgments more than football questions; they have no place in a math project.

I think quantifying those things fairly is impossible; I am much more comfortable using TSP as a starting point and making subjective adjustments for the Starrs and Montanas, etc.

That’s essential: stats are a starting point. TSP is a starting point. People like things to be simple, so some will want a stat that tells the entire story. That stat is a fairytale. The person who tells you he has that stat is the same one 500 years ago who knew how to find the Fountain of Youth.

TSP doesn’t overreach; it doesn’t try to be anything it’s not, or anything it can’t be. It measures regular-season production, and I believe it does so better than any other system. That’s a tremendous starting point from which to begin a rigorous analysis. But it’s still just a starting point. Critical thinking is a powerful tool, and if we don’t exercise it we’ll reach bad conclusions. TSP is a basis from which to begin critical thinking: what has this stat failed to capture? Obviously postseason performance is part of that, but relying on a single statistic to capture such a complicated sport is inevitably misleading, not to mention intellectually lazy.

I have a strong grasp of pro football history, so I’m really comfortable making subjective adjustments based on non-statistical information, which I think helps me develop formulas that don’t reach beyond their limitations.

• Tom

Right on. I’m with you as far as starting off with some baseline…TSP is great, Bryan Frye’s TAY/P is great, and then the discussion starts…we can make those subjecive adjustments for postseason, sheer athleticism, etc. I almost feel like the postseason is a separate beast altogether, but I also understand the need to include it when we’re doing the GOAT thing. Tough problem, as you noted.

• Ryan

Great work as always, do you have/can you post a list with the 1.85 exponent?

• 1 PEYTON MANNING — 67.7
2 DAN MARINO — 59.3
3 TOM BRADY — 47.2
4 JOHN UNITAS — 46.9
5 FRAN TARKENTON — 46.7
6 DREW BREES — 46.2
7 JOE MONTANA — 43.6
8 OTTO GRAHAM — 43.5
9 BRETT FAVRE — 41.5
10 STEVE YOUNG — 39.0
11 DAN FOUTS — 38.4
12 KEN ANDERSON — 34.3
13 JOHN ELWAY — 33.8
14 SONNY JURGENSEN — 33.3
15 NORM VAN BROCKLIN — 31.9
16 Y.A. TITTLE — 31.5
17 AARON RODGERS — 31.1
18 WARREN MOON — 28.6
19 ROGER STAUBACH — 28.5
20 JOHN BRODIE — 27.9
21 PHILIP RIVERS — 24.2
22 BOOMER ESIASON — 23.9
23 ROMAN GABRIEL — 22.3
24 JIM KELLY — 21.731
25 JOHN HADL — 21.730
26 BOBBY LAYNE — 21.6
27 JIM EVERETT — 21.21
28 JIM HART — 21.16
29 KURT WARNER — 20.55
30 BEN ROETHLISBERGER — 20.47
31 JOE NAMATH — 19.6
32 MATT RYAN — 19.2
33 RANDALL CUNNINGHAM — 18.8
34 TONY ROMO — 18.7
35 BERT JONES — 18.4
36 DONOVAN MCNABB — 18.2
37 TROY AIKMAN — 17.8
38 DARYLE LAMONICA — 17.73
39 RICH GANNON — 17.68
40 LEN DAWSON — 17.6
41 JEFF GARCIA — 17.20
42 TERRY BRADSHAW — 17.19
43 TRENT GREEN — 17.17
44 BART STARR — 16.6
45 CARSON PALMER — 16.5
46 NORM SNEAD — 16.42
47 VINNY TESTAVERDE — 16.39
48 STEVE MCNAIR — 16.2
49 JOE THEISMANN — 16.0
50 DAUNTE CULPEPPER — 15.7
51 MILT PLUM — 15.5
52 KEN STABLER — 15.3
53 MARK BRUNELL — 15.2
54 CHARLIE CONERLY — 15.1
55 DREW BLEDSOE — 14.8
56 CHARLEY JOHNSON — 14.7
57 BILLY WADE — 14.3
58 PHIL SIMMS — 14.2
59 ELI MANNING — 14.1
60 BERNIE KOSAR — 14.03
61 BRIAN SIPE — 14.02
62 NEIL LOMAX — 14.00
63 TOBIN ROTE — 13.4
* SAMMY BAUGH — 13.14
64 CRAIG MORTON — 13.05
65 BILLY KILMER — 12.82
66 STEVE DEBERG — 12.80
67 BOB GRIESE — 12.7
68 STEVE GROGAN — 12.5
69 DAVE KRIEG — 12.4
70 EARL MORRALL — 12.3
71 JOE FERGUSON — 12.2
72 RON JAWORSKI — 12.0
73 MATT HASSELBECK — 11.9
74 JEFF GEORGE — 11.5
75 MARK RYPIEN — 11.4
76 DON MEREDITH — 11.3
77 GEORGE BLANDA — 11.2
78 FRANK RYAN — 11.13
79 BRAD JOHNSON — 11.11
80 RUSSELL WILSON — 11.06
81 CAM NEWTON — 11.0
82 MATTHEW STAFFORD — 10.8
83 GREG LANDRY — 10.71
84 TOMMY KRAMER — 10.67
85 DOUG WILLIAMS — 10.4
86 KEN O’BRIEN — 10.30
87 JIM ZORN — 10.25
88 STEVE BARTKOWSKI — 10.2
89 DANNY WHITE — 10.0
90 JAKE PLUMMER — 9.81
91 JOHNNY LUJACK — 9.71
92 ARCHIE MANNING — 9.54
93 MATT SCHAUB — 9.52
94 ED BROWN — 9.37
95 BOBBY HEBERT — 9.23
96 KERRY COLLINS — 9.12
97 CHRIS CHANDLER — 8.90
* TOMMY THOMPSON — 8.82
98 JIM PLUNKETT — 8.50
99 LYNN DICKEY — 8.49
100 CHAD PENNINGTON — 8.24
101 ANDREW LUCK — 8.02

• Ryan

Many thanks!

• Anders

Maybe it shouldnt be as surprised, but I thought Romo would rank much higher than a guy like McNabb.

For Cunningham, I suppose he ranks higher than Simms and Aikmann because of his 98 season?

• Five best seasons by TSP:

Cunningham: 6.9, 4.0, 2.5, 2.2, 1.5 (6.9 is ’98)
Aikman: 3.7, 3.4, 3.2, 1.9, 1.8
Simms: 2.6, 2.1, 1.8, 1.7, 1.7

’98 is a big deal, but Cunningham had seven seasons that score at least 1.4.

• Corey

Nice work. I’m curious about using a five-year rolling average for era adjustments. I agree this is generally the right approach, but does it make sense when we know there are clear turning points with rule changes in 1978 and 2004? Won’t this kind of adjustment inflate the values for the years immediately after the rule changes, as they’re being compared to a pre-rule change baseline?

• This is a great question, but I believe it works. For 1978 especially, it took coaches and QBs a couple years to adjust to the new rules and take full advantage. The game changed, but not overnight. The changing constants smooth things out as well.

• I did what I believe to be TSP, down to the letter, per your instructions in this and the previous post. However, I am getting different numbers for most people, and I am wondering where I went wrong. Using Tom Brady as an example in the attached picture:

The last 6 columns are raw TSP, yearly replacement level multiplier, raw TSP rank within year, straight up era adjustment vs the 5 year average, the modifier to include 75% of era adjusted and 25% of actual value, and the (/1000)^1.85 exponent. I know that part is different, but I’m more concerned about the results prior to that last step. Anything jump out at you?

https://pbs.twimg.com/media/C7DOkj5U0AAXGTn.jpg

• I also went back further than 1946 and immediately realized it was a bad idea when Luckman’s 1943 outscored Aaron Rodgers’s career.

• I’ll investigate this further, but I will say that raw TSP looks good every year except 2016. I believe Brady had 5 fumbles, not 3.

• Looking at Brady’s fumbles, it appears I didn’t include the two he had on plays in which he picked the ball up immediately and threw a pass.

These are my yearly values since 2002:

2002 – 1593

2003 – 1579

2004 – 1558

2005 – 1469

2006 – 1427

2007 – 1491

2008 – 1588

2009 – 1623

2010 – 1596

2011 – 1616

2012 – 1593

2013 – 1558

2014 – 1599

2015 – 1667

2016 – 1730

• I’m with you through 2010. After that the values I used were:

2011 – 1618
2012 – 1601
2013 – 1573
2014 – 1613
2015 – 1676
2016 – 1734

I’m thinking there’s a discrepancy in ’13 and ’14. Below are the Top 10 in Raw TSP each season:

P.Manning 2815
D.Brees 2093
P.Rivers 1921
N.Foles 1716
R.Wilson 1305
A.Dalton 1258
C.Kaepernick 1241
M.Stafford 1180
T.Romo 1150
A.Luck 1135

A.Rodgers 2276
B.Roethlisberger 2009
P.Manning 1914
A.Luck 1803
T.Romo 1612
D.Brees 1564
M.Ryan 1496
R.Wilson 1455
E.Manning 1328

• I think removing kneels and spikes may have something to do with this.

• That’ll do it. Obviously that data is better in a vacuum, but I try to treat each season equally, and it’s not available for most years.

• I have the data completely back to 2002, and I am still working on finishing it up through 1983. Prior to that, I don’t think it makes much difference anyway, given the rules in place that made those less favorable plays to call than they are now.

I still hate the way the NFL’s stats are tracked, with kneels and spikes counting as individual QB stats rather than team stats. Or with fumbles completely missing for the AAFC. Or no sacks data before 1963 or sack yards data before 1947. Or fumble data before 1945. Or individual first downs before 1991, or YAC/air yards before 1992. Football stats are still the wild west compared to baseball.

• Preach.

Fortunately, the difference is minor. For Brady, it comes to 0.3, which is negligible.

“I have the data completely back to 2002, and I am still working on finishing it up through 1983.”

I am very excited for this! We would have complete box score data for 25 seasons (back to `92).

• One of the hardest parts of finding spikes and kneels in game books is that they’re often not labeled as such. Back to 2002, it’s pretty clear, but prior to that it’s rough. Many spikes are just listed as incomplete passes, and I have to try to find game tape to confirm whether they are spikes or actual attempts. If I don’t know for sure, I don’t call it a spike. I’m sure I’ve made some mistakes.

Many kneels are listed like this: J. Montana up middle for -2 yards. You have to pay close attention to context in order to parse these.

Are you doing this manually prior to 2002?

• I have been and will continue to.

That takes commitment. I’m impressed.

• Four Touchdowns

Another great article from Brad! Thanks for sharing, I love stuff like this!

• Four Touchdowns

Sorry, I’m having trouble understanding what the Career Value is calculating and what it means — I assume it isn’t the total TSP broken down into an average per season, is it?

• There’s a more detailed explanation in the link to the article explaining the QB-TSP formula, but it’s calculated by dividing TSP/1000 and squaring the result. This rewards big seasons and peak performance rather to avoid overrating compilers.

And thanks for your other comment, I’m glad you enjoyed the piece.

• garymrosen

“He[Montana]’s an underrated regular-season QB.”

His regular season play was not “underrated” by those who watched him consistently. His reputation was built not just on a handful of memorable postseason plays but by his level of play year in and year out. Bad games and bad plays were not nonexistent but extremely rare. Yes Marino had that great 1984. However Montana had the only other three seasons in the 1980s with a QBR over 100.

• It seems to be a thing among younger fans who didn’t actually watch him play or take the time to do their research before saying dumb words. I have actually heard people my own age (I’m 32) say that Montana got into the HOF based on his postseason performance, as if he wasn’t one of the greatest regular season QBs in history. I first got into football in 1991, because most of my family loved Washington, so I didn’t get to see him play live for quite a while after I began watching football. My uncles and grandfather would tell me how incredible he was, but I didn’t get to see him till he played for a mediocre offense, at an old age, after suffering numerous injuries that still plague him today. My little 9 year old mind was in awe of him, and he became my favorite QB.

My grandfather became a bit of a hoarder/collector, and he eventually amassed an impressive collection of old VHS tapes of NFL games, and I was finally able to see many of Montana’s games well after he retired (YouTube has also facilitated this). He didn’t just match the hype; he exceeded it. He wasn’t this screen-throwing, easy play making game manager so many modern critics seem to label him as. He made some incredible throws with beautiful touch, and his timing easily offset his lack of elite arm strength. And, man, before his 1986 injuries, he was a very good athlete.

• Richie

“I have actually heard people my own age (I’m 32) say that Montana got
into the HOF based on his postseason performance, as if he wasn’t one of
the greatest regular season QBs in history.”

I have never heard such claims. But that would have to be from people who didn’t see him play – particularly with SF. Of course, while with the Chiefs he had a regular season comeback against Denver on a Monday Night that was a classic. (and pissed me off)

I think the problem is that Montana now ranks way down the list of career passing stats, and he never had a 4,000+ yard season – which looks bad compared to players of the past decade.

According to PFR, Montana only started 16 games one time, and only played in 16 games one other time. So there was a bit of a reputation of a guy who missed games. I think that could play in to the perception that he wasn’t a great regular season player.

Also, Steve Young replacing him and playing so well may help obscure Montana’s excellence.

But when Montana retired after the 1994 season, he ranked #1 all time with a 92.3 passer rating. (Marino was second at 88.2) Now Montana is down to 11th in passer rating.

Montana retired #4 all-time in passing yards; 4th in TD, 1st in INT %, and 2nd an ANY/A.

• garymrosen

Yes Montana was somewhat frail, though his eventual comeback from the clocking he got in the 1990 NFCCG is a tribute to his fierce competitive determination. I remember him having to wear a “flak jacket” as early as the 1981 season. And obviously his 1986 back surgery was in the background of the Montana-Young rivalry for the 49ers QB position the next two years.

• Another piece to remember is that every QB missed more games back then. This was before all the rules changes that were designed to help QBs stay healthy.

• Tom

Saw an interview where Montana alluded to that…he’s too cool to complain, but I got the sense that he would have liked to have said “In my day, they could kick the crap out of you”, etc.

• FWIW, Montana was 8th in era-adjusted passer rating: http://www.footballperspective.com/adjusting-passer-rating-for-era-part-v-the-results/

And the only guys who came after him were Young and Rodgers. Luckman and Baugh were from a very different era, and Graham, Dawson (AFL-adjustment needed, and he was a passer rating superstar), and Staubach were the other 3.

• I’m seeing that Montana played 16 games four times and started 16 twice. He didn’t miss a start from 1981-1983, which unfortunately includes a 9-game season. Not that the difference really matters all that much.

• Richie

Yeah, I meant to type 16 starts “twice” . But I still think he only played 16 games 3 times. PFR lists 4, but one of those is 1979 when he only attempted 23 passes. They must be counting games where he was a holder or something.

• Some of PFR’s oddities are confounding at times, but I’d be lost without them.

• sacramento gold miners

Montana had three other memorable games with the Chiefs, and I think he’s the only KC QB outside of Dawson to lead them to a conference title game. In 1993, Montana came back to beat the Steelers in the playoffs. In 1994, he beat the eventual World Champion Niners in the regular season, and later upset the Oilers at Houston. Even in a diminished capacity without the offensive weapons in San Francisco, Montana could still be great.

• garymrosen

Thanks, Bryan. I have been a strident advocate for Montana here to the point where I may have inadvertently belittled Bill Walsh in a recent thread. Of course Walsh helped to make Montana but the reverse is also true as it has been for great coach/QB combinations from Brown/Graham to Belichick/Brady.

“He didn’t just match the hype; he exceeded it.” That was my observation a few years ago when the NFLN showed the entire original broadcast of the Dallas game. I saw it at the time and while I remembered many details I had forgotten that Montana was not getting great protection in the game and made a number of crucial drive-preserving throws on the run and/or under pressure. In fact in the 3d quarter he was getting hit so badly they were warming up reserve QB Guy Benjamin, not a Steve Young-level backup.

• sacramento gold miners

Sometimes we can see potential greatness in college, especially on the bigger stage. Think Montana in the Cotton Bowl, Brady in the Orange, or Marino in the Sugar Bowl. Conversely, I couldn’t believe how poorly Vinny Testaverde played in the Fiesta Bowl, and that was a red flag for me entering the NFL.