≡ Menu

Today’s guest post comes from James “Four Touchdowns” Hanson, a relative new reader to the site. As always, we thank our guest posters for contributing.


Elite Quarterbacks: Measuring Overall Team Support

It’s easy for football fans to buy into the mainstream logic that if you have an elite quarterback, your team will have a winning record, enjoy trips to the post-season and even win a few championships. The better the quarterback, the more wins and titles you can expect… right?

But that logic doesn’t always hold up.  Dan Marino, Fran Tarkenton, Warren Moon, Dan Fouts, Jim Kelly, Sonny Jurgensen, Philip Rivers, and so on, provide examples to the contrary. And while his talents have been unfairly portrayed at times, the fact that Terry Bradshaw has four Super Bowl rings while superior passers have none presents a disconnect if you think great quarterback talent is measured by titles.

If we go by an average time of possession of 30 minutes per team, that means that half the time, a team’s quarterback isn’t even on the field. And if 35% to 55% of your team’s offensive plays are running plays… doesn’t that mean the quarterback really only affects 22% to 33% of the total game time? And once you get into other factors that affect a passer’s game, like play design and coaching, offensive line talent, receiving talent, quality of opposition, etc., attributing credit and blame gets pretty murky.

So while we have a general feeling that some quarterbacks receive more support than others, so how do we go about measuring it through metrics? Unfortunately, due to the nature of passing stats, I don’t know of a way to separate a quarterback from his receivers, pass blocking, scheme and play-calling. Whether it’s passer rating, ANY/A, or passing EPA, none of them can tell you which percentage of the credit (or blame) should be shared with those external factors.

That said, we can measure a quarterback’s support by looking at the numbers produced by his running game, defense and special teams. While I’d love to run these numbers for all quarterbacks, my ability to collect them is fairly limited (basically, cutting and pasting from Pro Football Reference – I don’t know enough about Python or R to run a spider to scrape them all in one go), so I will be focusing on four quarterbacks that are perceived to be “elite” by general mainstream consensus – Peyton Manning, Tom Brady, Drew Brees and Aaron Rodgers.

Due to length, I will be looking at these numbers in three separate articles. This one will focus on what support they received in these areas overall and the second will take a look at team support relative to the quarterback’s performance – how often their teams win when they have below-average performances and how often they lose when they perform at a high level (as measured by stats, natch).

Overall ANY/A, Passer Rating, & Expected Points Metrics

So let’s dig into the numbers. Here are the quarterbacks’ average stats per game for their overall careers including playoff games (which is why you may notice a difference between these and career averages that only include the regular season). The cells shaded blue indicate that quarterback is the leader among the four in that metric. While all of these metrics should be familiar to readers of the site, I have also included a metric I’m calling Relative Passer Rating (rPR) and it’s essentially the same concept as Relative ANY/A – it measures passer rating compared to the average for that season. For example, if the average is 80.0 and the passer earns a 90.0, his rPR would be +10.0.

As Peyton Manning is the only one of the four to suffer a physical decline in a career ending season, I have included his pre-2015 averages for the efficiency metrics in parenthesis next to his actual averages, though this will not affect any of the analysis from this point on – it’s more of a “nice to know”.

Great numbers across the board, as to be expected by elite players but it’s amazing how much higher Brady’s win percentage is than the other players despite not producing any clear statistical advantage in the efficiency or traditional metrics – and how much lower Brees’ win percentage is than the others despite his numbers being on par with the other three QBs.

Additionally, as their success in the playoffs makes us the biggest difference in the way people perceive these players, I will break out their playoff numbers in a separate table –

The numbers here probably come as a surprise – I know I didn’t expect Brees to sweep nearly every passing metric for the playoffs. While his smaller sample size comes into play, it doesn’t seem to have affected his win percentage, which is about on par with everyone except Tom Brady.

So there’s our starting point – we can see how each player has performed relative to the others, and while there are some clear leaders in the playoffs, they’ve produced at a similar level overall throughout their careers. Now let’s see how their teams have supported them.

Team Support Measured by Expected Points

As they’re the metrics that seem mostly closely tied to the margin of victory and defeat, I figured we’d start with Expected Points. Mike from Sports Reference defines Expected Points as a way to “break down the contributions each team’s various squads made to the margin of victory.” Those last few words are key; EP are applicable to the margin of victory – or defeat.

With that in mind, I went about measuring the EP of each quarterback’s passing offense against the rest of the team – running game, defense, and special teams. The sum of those three squads make up the Total Support EP number while the point differential is the average margin of wins and losses across their careers.

The EP Gained / Lost metric represents the average EP each quarterback’s running game, defense, and special teams have added or subtracted per game from the passing EP they generated. I also felt that per game average didn’t fully illustrate what effect those added or subtracted points had over the course of each player’s career, so I included the sum or difference in Total EP Gained / Lost.

Finally, since EP is tied to the margin of victory or defeat, I included the Point Differential to give the numbers some context. I also showed what each QB’s passing EP as a percentage of the average point differential in Pass EP % of Outcome so we could see how much credit or blame each quarterback should take for his team’s outcomes on average.

Red indicates negative EP while green indicates positive EP; dark green and dark red indicate who finished first and last in each category respectively.

First off, it’s clear that these players’ passing offenses have been the driving force behind their teams’ success. Compare their passing EP to the average margin of victory and you see that it’s each quarterback’s passing game that has created most or all of that point differential – it looks like most of the time, these guys carry their teams.

That said, we can see the EP support (AKA negative plays) has been very different for one player compared to the other three – Brady is the only one to have positive EP support in any category and amazingly has positive EP in ALL support categories. It’s said that the quarterback affects all other players on the field and while that may be true, we’re not seeing it in these results – every other QB has a higher EP average per game than Brady but gets worse support than him.

If we were to round the numbers, it looks like Manning’s and Rodgers’ team support have cost their teams around a field goal in points, on average, while Brees loses roughly 5 points. Meanwhile, Brady’s support has been worth an extra point to his Patriots. We shouldn’t misinterpret these numbers to suggest that somehow Brady is being carried by great support – it’s clear that Brady’s passing game is the engine that drives the Patriots’ success as a team but he’s the only quarterback of the four whose supporting elements haven’t cost him points.

This has led to his teams actually adding positive EP to his games overall, giving Brady nearly 264 expected points to his overall point differential over the course of his career. Manning and Brees, on the other hand, has had their teams cost them over 1,000 expected points and at the rate Rodgers’ offenses have been producing, Rodgers will likely join them by the end of his career.

Of course, there’s only so much we can see from these per game averages – let’s see what that EP looks like spread out over the course of their careers. After all, a team that provides -5 EP in five games and a team that provides +5 EP in two games and -20 EP in three games will both average out to -5 EP per game. But the first team had five straight bad games while the second team had three good games and two catastrophically bad games. Clearly, not the same thing.

So let’s look at how many games each passer and team have provided positive and negative support – and then let’s see how often that support has created a two-possession lead or deficit (9 points), three-possession lead or deficit (17 points), and while I know 24 points is technically three possessions, I think the odds of any team getting three TDs with three two-point conversions is very low – so I set 3 TDs (21 points) as my threshold for three-possessions. Additionally, we should see how often our QBs do the same, so those passing EP numbers are also included.

The leaders in passing EP are marked blue, the leaders in support EP are marked green, and ones with the least EP in each category are highlighted in red –

The numbers reflect what we’ve seen before – Aaron Rodgers leads in almost all categories with passing EP, having the largest number of games with positive EP and the fewest with negative EP. Meanwhile, Brady has the most games with positive support and the fewest games with negative support across all categories by a substantial margin, though he’s tied for the lead with Rodgers for the fewest games with negative EP over 9 points.

That all said, when you look at the actual number of games with negative passing EP, it’s really only a handful of difference between these players – they’re all incredible, posting positive EP in the vast majority of their games and two possessions worth of positive EP in about half of their games. These guys are considered the elites of the sport for a reason and these numbers bear that out.

As before, I included their playoff numbers since that’s the core difference in how fans perceive them. One difference here from the previous chart, though – since none of them enjoyed totally positive playoff support, the QB with the smallest deficit of supporting cast EP is highlighted in yellow–

If you think about it, it’s kind of amazing that despite Manning, Brady and Brees playing over 240 games (Rodgers at 151), a small fraction of them determine how the general public sees these players — the playoff games. Manning has played 292 games and the outcome of 27 will determine how he’s remembered. Brady has played 269 and 34 are what makes him the GOAT to most of the public. Brees has played 243 and the outcome of 11 is what keeps him out of most people’s conversation for GOAT. Kinda nuts – but at least Rodgers still has a lot of his career ahead of him. Hopefully, Green Bay will get the man some help!

With that in mind, I decided to add three rows to this column since we’re dealing with such a small sample size for the playoffs so we can understand the narrative behind the averages a little better – how many playoff appearances they had with positive support EP, their record with that support, and then their record with negative support. While they all had negative support in the playoffs, Brady’s did the least amount of damage – despite both Brees and Rodgers having higher passing EP, their playoff point differential is less than half of Brady’s. Meanwhile, it seems Manning has to assume some responsibility for his lack of playoff success as his passing EP drops dramatically in the playoffs.

That said, in terms of win percentage, they all do better with positive support – Manning seems to do worst but still has an incredible 75% win percentage, way up from his 52% overall.

And while the results so far of this study have shown that Brady has received the most support while not always posting the best efficiency metrics or raw stat totals, we have to give him his due – he is the only QB of the four with a winning playoff record in games with negative EP support.

So how does this all translate to Super Bowls? Two of Manning’s three playoff runs with positive support netted him a Super Bowl title (his other was in 2014 when he was hindered by injuries that would ultimately end his career), while Brees’ and Rodgers’ only playoff run with positive support led to them winning a Super Bowl (Brees’ Special Teams EP in his Super Bowl was nearly 10 points alone!). On the other side, Tom Brady has won three Super Bowls with positive support and two Super Bowls with negative support (though all Super Bowl runs featured at least one game with positive support that contributed more to the margin of victory than passing EP), which certainly adds some credibility to the idea that he’s got a bit more “clutch” to him than the other three quarterbacks.

All that said, we can see how much more the other three quarterbacks lost due to poor playoff support – despite having the lowest passing EP by far, Manning also contributed the most to his team’s point differential, suggesting that overall, they would have done much worse without him.

Below, we see the numbers that tell the story of Manning’s playoff struggles – he’s generated the fewest games with positive EP and the most games with negative EP as a percentage and his poor games have been more catastrophic than the others, while his great games have been more dominant than the others. His passing offenses seem to be “feast or famine” in the playoffs. Beyond that, the story stays the same for the other QBs – Rodgers and Brees generate the best playoff passing EP but suffer the worst support EP of the four QBs, while Brady’s great EP performances seem to be hindered the least by his supporting squads.

So, what is the deal with Manning in the playoffs? We’ve established his support was poor in general, but poor support hasn’t hindered the other QBs as much as it has Peyton (though both Brees and Rodgers playoff sample size is much smaller than Manning’s). It’s a question many have asked and for my money, probably most satisfyingly answered in Scott Kacsmar’s two-part article on Football Outsiders, which you can read here and here.

His critics seem to feel that the pressure and anxiety caused by the high-stakes “lose and you’re out” playoffs format causes Manning to play worse. Perhaps this is true – but I find it hard to believe that he can, for example, run the same “levels” play over and over in the regular season but once the playoffs start, he suddenly can’t make the reads and throws he’s made literally thousands of times before. I’d need to see film study done to believe this theory – if someone can show me that Manning consistently made more bad reads and missed open receivers in the playoffs, while under the same amount of pressure as usual, I can believe it was psychological.

But there are other possible explanations. The first is that the small number of playoff games can skew numbers due to a high degree of variability found in small sample sizes. Perhaps Manning was just unfortunate that he had a higher percentage of his poor games in the playoffs than the other QBs. After all, his career efficiency averages, even when adjusted for opponent like DVOA, suggest he’s had the same percentage of bad games overall as the other three QBs. I find this plausible but perhaps a bit unsatisfying.

Ultimately, it’s something we won’t be able to figure out here, so let’s move on to looking at support measured by traditional metrics.  That will come in Part II.

  • eag97a

    I question the assumption that a qb affects only 22 to 35% of game time. Handing off to rbs and the running game is still within the purview of he qb as the coach on the field. His decisions to audible succesfully to running games to extend drives and score points directly impacta the game. As I have mentioned numerous times before, passing is just one part of being a qb.

    • The way I look at it, it can’t be higher than 25% … two teams each contributing 50%, and each team’s offense providing half of that (even when ST is included). Add in the RB/OL contribution to the running game and after-catch passing yardage being largely on the receiver, and it’s not close to 25% either (or close to 50% in the typical 1-team level of analysis).

      • Four Touchdowns

        I look at it as the a team’s offense and the opposing defense are sharing the same game time. The QB is never throwing the ball without an opposing defense making their game contributions simultaneously.

      • eag97a

        Reducing the game as contributions totalling 100% just misses the team nature of the game. That reductive logic ignores the synergistic effects of good team play and ignores a lot of the mental and non-passing contributions of the qb who directs the offense. Reducing the game to 100% and dividing the pie to each contributor according to arbitrary criteria is IMO not really capturing the spirit of the game.

        • Four Touchdowns

          In fairness, I didn’t break it down by contributions for the reasons you stated — I did it by total time on the field. It’s a two-way street; if your defenses hold opposing teams to less points, yards, time of possession and force punts that benefit your average starting field position, a QB will probably have more success.

          • eag97a

            Same way that a qb directing the offense to long drives and then scoring indirectly affects their defensive team mates specially the d-line which needs to be rested adequately to have an excellent pass rush. That example shows the qb contribution on a teams’ defense. FWIW it really is hard to assign credit and/or blame in football.

            • Four Touchdowns

              Absolutely, I agree — which is why I’ve always had a hard time accepting that one player’s contributions are worth so much more than the 52 other men on the roster and the coaching staff.

              • eag97a

                I also note that disentangling the contribution of the different units and players are very hard to tease out and that is why I’m leery if any metric assigning specific weights to specific player contributions in this team game.

            • Paul

              Teams lead by these 4 always rank high in drive stats, so there might not be a lot of difference in what your saying compared between them.

        • It’s no more arbitrary than how you’re doing it. Just less metaphysical.

          • eag97a

            I’m not doing anything. You might want to check with the one posting. Arbitrary is arbitrary. If we are going meta here I think being metaphysical won’t score you many points. Just my 2 cents.

            • I may have misread you. But if you were to devise a frame of analysis to make sense of the game, I assume you’d try to account for “the synergistic effects of good team play.” That’s the “metaphysics” I was referring to. Sounds like it would get you a number besides 100%, even though when you’re talking about any whole with which to make parts, that’s kind of the number you’re stuck with.

              • eag97a

                I’m not a mathematician but with a game (process???) like football with multiple changing variables, you can design a partial differential equation to try to solve for the qbs’ “contribution”. If you did that you probably will find whole families of solutions instead of exact solutions since many of the variables will be unknowable or not quantifiable. With mathematical rigor we will remove it from the metaphysical plane down to ordinary analysis. Worth a shout out to any mathematicians here IMO. 🙂

    • Four Touchdowns

      That’s a fair perspective — perhaps it should read “quarterback’s passing only affect 22% to 35% of game time”.

    • sacramento gold miners

      Agreed, there are a number of other factors which go into QB play which affect the outcomes of games. Audibles, leadership, overcoming adversity, are just a few. The QB position will not only continue to be the most important position in football, but all of sports as well. As history has shown us, elite QBs win more often than good QBs. And while postseason accomplishment isn’t the only metric for measuring the position, it should always be part of the resume.

      100 years from now, we still won’t be able to quantify everything in sports, it’s just the way it is. Tom Brady isn’t Tom Brady just because he’s an accurate passer, Patriots fans know exactly what I’m talking about. Intangibles will always affect outcomes in sports, does anyone think James Harden’s indifference helped the Houston Rockets the other night?

      Now let’s unpack the inevitable list of elite QBs who never won a Super Bowl, when teams draft, not every situation is the same. As I’ve said, a QB can still be in the HOF without winning a SB. Championships aren’t determined by whether or not a player deserves one, and there will always be exceptions. Trent Dilfer wasn’t even a good QB, but lucked out with the 2000 Ravens.Let’s start by eliminating Sonny Jurgensen and Warren Moon from the list. Jurgensen played most of his career before the expanded playoff format in 1970, and free agency. He also had the bad luck of George Allen preferring the style of Billy Kilmer. Moon lost roughly five years of his NFL career because of how poorly black QB
      were evaluated, but still managed to win multiple Grey Cups.

      Dan Marino was the last Dolphin QB to lead his team to the SB. But the lack of defense and running game were just enough to prevent another SB appearance. Fran Tarkenton had a similar situation to Jurgensen, but age, and superior opposition were a problem in the big game, and the Vikings still haven’t returned to the SB. Buffalo has made one postseason appearance since Jim Kelly played, the Bills defense was usually poor on the biggest stage.

      Phillip Rivers isn’t finished yet, and reached one more conference title game than Tony Romo. Rivers did have the bad luck of playing that game at NE on a damaged knee. Dan Fouts led San Diego to the postseason despite a porous defense. Was great in the 1980 AFC TG, tough to criticize him for the frigid conditions in 1982.

      Wrapping up, data won’t provide the answers in this topic, and SB titles aren’t predetermined. The fact that a few elite QBs didn’t win it all in the modern era, doesn’t mean the larger number is an accident. And it’s ok if Griese, Bradshaw, and Aikman aren’t inner circle QBs. Those guys walked into bad situations, and developed into stars while leading legendary teams. Their individual numbers aren’t more prolific simply because those teams usually had a formula of running the ball more for success.

      • The best player on an NBA team is more important, but QB is a clear second

        • Four Touchdowns

          He is — but the question is how *much more* important than the other players? How much more important than the coaches?

          Is one quarterback more important than a team’s entire defensive line? Is he more important than a team’s front seven? What about the entire starting defense?

          Within his own passing offense — how much more important is he than his offensive line and wide receivers? The play callers?

          Imagine yourself as the quarterback — the play is called, you make your pre-snap read, and then you get the ball. Assuming you do everything perfect — is your first read open? That’s likely not up to you — it’s up to the play design, receiver, and opposing defense. What about your second read? Again, you’re relying on other people. Uh oh, the pass rush is getting close… is the third read open? Again, not up to you.

          A quarterback will always be limited by external factors — his options are predetermined for him by a play-caller and play-design. Yes, many have the ability to audible, but it’s not as wide-open as you think, even for guys like Manning and Brady. And even if the perfect play is called, if your receivers just can’t beat coverage or pass blocking can’t give you the time needed for plays to develop, it doesn’t matter — your odds of success become lower.

          • sacramento gold miners

            A QB affects the entire team, because he’s the leader on the field, with the ball in his hands on every offensive snap. The team’s best defensive player must react to the opposing formation, that’s a huge difference in this conversation. The good to great QBs can overcome mistakes by their teammates on individual plays more often than the poor to mediocre QBs.

            And when we look at postseason play, the elite QBs overcome adversity, make the right audible, to make just enough plays for victory.

            • Four Touchdowns

              I guess my problem with taking what you’re saying as fact is the evidence — is there anything you can point to that proves any of that?

              I know analytics are imperfect and only tell part of the story, but they provide concrete evidence.

              • sacramento gold miners

                The proof would be breakdown on each individual play, but I’m not going to trust Pro Football Focus on that one. The great ones just keep producing, I don’t believe luck is always on their side.

                • Four Touchdowns

                  I agree that all-22 analysis is the best… but why don’t you trust PFF? It seems like their methodology is pretty rigorous — three separate grades by three separate analysts and then reviewed by a coach? That sounds good, no?

                  • Paul

                    As a Tom Brady fan, and great qb fan in general, it pretty ridiculous they graded him at 99.5 or whatever, they’re saying Brady was nearly perfect, and as great as he was, that cant be nearly right. Also PFF doesn’t take into account defensive performance really, the best graded PFF seasons always come in seasons those qbs played average or worse schedules (like Brady this year). so the tape is limited. They graded the patriots and cowboys as two of the best secondary’s despite advanced stats from fivethirtyeight and football outsiders saying they were average or worse. grading only goes so far, when a player faces bad units such as Brady facing the terrible jets, browns, 49ers, ect, of course he’ll grade higher then someone facing great defenses.

                    • Four Touchdowns

                      Interesting stuff, thanks for pointing that out.

                      That said, there is value in looking at the all-22 since metrics are imperfect — not every interception or sack is the QB’s fault and sometimes a receiver bails him out of a bad play.

                      I guess they could do some defense-adjusted grade but it doesn’t seem like they’re in that business. They just look at the plays available and base grades on them; weighting things by defense is more of an advanced analytics thing.

                    • Paul

                      that’s all true, I was just pointing out some flaws in PFF grades. I still read them and believe they have value but everyone grades differently, its subjective. I just purchased pre snap reads by Cian Fahney and his conclusions and adjusted stats (for WR drops, inaccurate throws, interceptable passes) and personal conclusions vary widely from PFF. Not saying either is wrong, but its obvious that they both determine and value things differently. So who is right? its interesting in a big picture kind of way, telling more then just stats do, but if different people cant even determine what constitutes a dropped pass (or the NFL unable to even determine what a reception is), or what is “pressure on a qb” or YAC, than I don’t know how valuable they are. For instance did a receiver get YAC from his great play or because the qb throw him open and put the ball in a position that benefited the WR the most, whos to tell? or was he under pressure from his own poor (or great) pocket presence. Fahney believes sacks are almost entirely the lines fault whereas chase’s research and stats say its mostly the QB, PFF has the Packers as one of the best offensive lines in football yet Aaron Rodgers still has a above average sack %, so whos at fault, Cian and PFF believe two completely different methodologies.

                      Anyway, thanks for all the hard work on this article, I had been wanting to do something similar but you did a excellent job at it.

                    • Four Touchdowns

                      Very true and I purchased Fahey’s new almanac as well. He does great work and I’m inclined to trust his subjective opinion over PFF just because he goes into far more detail about how he arrives at his conclusions.

          • I’m with you all the way. I’ve said around here before that it’s really annoying to hear “most important” conflated with “more important than everything else combined.”

            • Four Touchdowns

              Amen to that! I’ll be the first to agree that a QB is the most important player and if good enough, can often make a big difference to his teams’ fortunes — but that doesn’t always mean winning records and titles.

              For example, the Saints were 7-9 last season — but without Brees, I bet they’d have two or fewer wins for the entire season. But he’s not going to take that crappy team and turn them into a Super Bowl champion all by himself and neither could Joe Montana or Tom Brady.

              • Paul

                I think they would have won even less, the saints’ had a terrible defense, 2-5 win difference might be probable.

        • sacramento gold miners

          When your best player is off, and doesn’t seem to care, that’s a huge problem in the postseason.

      • Four Touchdowns

        Good post and thanks for reading. I agree that data can’t provide any true answers and in the end, people will believe what they want to believe about any given player.

        That said, stats are helpful in providing context to results and are one of our best, most non-subjective ways of analyzing the game of football.

        While I feel honest, unbiased all-22 review is the best way to judge players, it is hindered by the fact that they are inherently subjective in a way that passing yards or ANY/A isn’t — the numbers stand alone, as imperfect as they may be in allowing us a total analysis.

  • Four Touchdowns

    Thanks to Chase for posting this — it represents many hours of data collection and research that I can only chalk up to compulsion, LOL.

    • eag97a

      Looking forward to part 2. I must say its not just compulsion it is a labor of love.

  • Paul

    Interesting article. Very well done. But peyton mannings stats look alittle wonky. Profootballreference has his career ANY/A (1998-2015) as 7.17. His playoff is 6.29. So im not sure where 7.32 comes from? Is it 1999-2014? Thats more in line with the stat from the table. It would also be interesting if. This included strength of schedule, such as chase’s post on greatest qbs, after adjusting for that qbs perfromance against that defense. Doing that shows that peyton manning had some very underestimated great years before his record setting 2004, as the colts faced far tougher defenses in the afc east then in the afc south. I have all the tables on these qb with adjuated defenses performance and any/a over average if youre interested. Anyway good work.

    • Four Touchdowns

      It comes from an average of his ANY/A across every game he’s played — I couldn’t tell you why PFR’s numbers differ as I pulled his game-by-game data from their own gamelogs and just ran an average in Excel.

    • Four Touchdowns

      This is really bothering me (I put WAY too much time into this and want everything to be 100% accurate). I just checked my numbers, checked my ANY/A formula, checked my averages — I don’t get why their total would be different than mine.

      Chase has my spreadsheets, maybe he can figure out why.

      • WR

        Manning’s ANYPA figure, including playoffs and excluding his rookie season, is 7.22. Maybe you typed 7.32 by mistake?

        • Four Touchdowns
          • Paul

            Its certainly strange. Ill look also, see if i will verify your numbers.

            • Four Touchdowns

              Look up “Peyton Manning” on PFR and then click into the gamelogs. You need to get the playoff gamelogs separately.

              The Excel formula to calculate ANY/A (they don’t provide it in the gamelogs) is —

              (pass yards + 20*(pass TD) – 45*(interceptions thrown) – sack yards)/(passing attempts + sacks)

              • When I did the formulas from the gamelog info, I got 9683 dropbacks for 69480 ANY, or 7.18 ANY/A in the regular season. I got 1067 dropbacks for 6974 ANY, or 6.54 ANY/A in the playoffs. Overall, I got 10759 dropbacks for 76454 ANY, or 7.11 ANY/A.

                • Four Touchdowns

                  Argh, I don’t understand why this is happening? Can I email someone my spreadsheet so they can see where I went wrong?

                  I also see that my numbers for Brees and Rodgers don’t sync up as well to PFR’s data… only Brady looks similar.

                  • I’d love to look at your data. LaverneusDinglefoot AT gmail dotcom

                    • Four Touchdowns

                      Just sent.

                    • Four Touchdowns

                      So this is weird — when I calculate Manning’s average career ANY/A by averaging all the ANY/A over his whole career, I get 7.33 (I had his final game twice somehow, leading the 7.32).

                      But when I calculate his ANY/A by using his career averages of the stats that make up ANY/A (TDs, yards, INTs, etc.), I get 7.09.

                      So it looks like his total career ANY/A and his average ANY/A per game are two different numbers (though I’m unsure why it doesn’t match TheGridFe’s 7.11 ANY/A).

                    • In your workbook, you found his average career ANY/A by taking the average of all of his games. When I did it, I used a weighted average, so a game like 11/16/14 would count more heavily than a game like 11/15/15.

                    • Four Touchdowns

                      Phew, great to have an answer!

                      I’m actually comfortable leaving it as his average per game rather than his career total.

                      Thank you so much for looking at it!

                    • I think the error is pretty simple.

                      You took an average of the averages to calculate his career ANY/A This, by definition, weights each game equally.

                      But when calculating a player’s career average, we don’t usually weight each game equally. If a player averaged 10 ANY/A and 0 ANY/A over 2 games, an average of the average is 5.00. If he had 10 attempts in the first game and 40 attempts in the second game, his average ANY/A over those two games would be 2.00, since he had 100 ANY over 50 attempts.

                      In other words, the 7.33 number represents an average of the averages, but since Manning’s best games had fewer attempts, his average ANY/A is 7.09 (which I get in your cell is you copy the formula in V293 into V295) .

                    • Four Touchdowns

                      What do you think, Chase? As said, since I’m looking at these as individual games — how many games someone got positive support, how many games someone got negative support, etc. — I like the idea of it as the average ANY/A per game rather than the career total.

                      It doesn’t weight things but I don’t think this is a situation where weighting the numbers really serves the actual purpose of the study.

                    • FWIW, the correlation coefficient for Manning’s ANY/A and his dropbacks is -0.29. I didn’t calculate that for everyone, but I was a bit surprised to see that. It means, of course, that his best ANY/A games came with fewer attempts, and his worst games came with more attempts. That makes some sense, but it still feels like a high number. Manning also has 2-attempt and a 3-attempt game where he had a bad ANY/A; remove that, and the CC jumps to -0.35.

                      I’m not sure if Brady/Rodgers/Brees have this same issue or not.

                    • Four Touchdowns

                      Well, my “per game” averages for both Brees and Rodgers ANY/A and their actual total ANY/A have about the same amount of difference as the ones for Manning when I look at PFR (their total doesn’t include playoffs, though) — Rodgers is 7.64 to PFR’s 7.48 and Brees is 7.10 to PFR’s 6.91.

                      Meanwhile, Brady’s per game average and total career average ANY/A are almost the same — 7.08 to 7.09.

                    • Paul

                      that’s pretty crazy, maybe Brady has a relatively consistent amount of attempts for his career?

                    • Paul

                      it sounds right, when a team or quarterback struggles the qb usually throws more attempts, when you look at the total seasonal attempts by ranges of attempts I think it becomes obvious. more attempts less efficiency. Kerry Byrne of coldhardfoootballfacts.com was very big on Tom Brady’s record when “carrying the team” since his stats were so much better than others in high attempt games (for instance he had very high attempts in most all the Patriots superbowls).

                      It would be interesting to see a study on the correlation between attempts and player performance.

                    • Four Touchdowns

                      Chase did show that there was a slightly negative correlation between high attempts and wins.

                      http://www.footballperspective.com/correlating-passing-stats-with-wins/

                    • Four Touchdowns

                      It’s not a correlation but my stat sheets on these guys show that they all average between 34-38 attempts per game. So if we use the gameplay finder for games with, say, 45 attempts or more, we see —

                      Brady has a 87.5 rating with a 60% win percentage in 44 games.

                      Manning has a 80.9 rating with a 28% win percentage in 46 games.

                      Rodgers has a 87.5 rating with a 32% win percentage in 14 games.

                  • Paul

                    are youre numbers including attempts? and average of yearly totals would be skewed since they don’t through the same passes every time? many a season Attempts are off.

  • WR

    Nice work, James. I do think a couple of factors that explain Manning’s postseason struggles are that in the regular season, his numbers are weaker outside of a dome, and often struggles against elite defenses. Take him out of a dome, put him up against a great defense, and he’s not the same guy, though all QBs struggle against great competition. I also think it may be schematic. The Air Coryell style of offense struggles against bump-and-run, and fast, physical defenses that can keep up with the receivers.

    I think what this work really shows is that all 4 guys in the study are very close overall, and the different things they’ve accomplished are largely the product of the situations around them.

    • Four Touchdowns

      I also think scheme may have something to do with it — after all, Dan Fouts and Dan Marino also ran the Coryell offense. People talk about Manning’s superior receivers but it’s clear that those offenses don’t work without true #1 receivers who can stretch the field vertically. I think that’s why despite being more physically limited, Manning’s Denver numbers are better — they used some of the Edhardt Perkins system and more of the Patriots’ style schemes.

      That said, I think one of the issues with analyzing Manning’s outdoor games and comparing them to Brady or Rodgers outdoor games is that the vast majority of Manning’s games outdoors were away games while Brady and Rodgers played outdoors at home as well. Running the numbers, Manning’s rate from ’98 to ’14 in outdoor away games does drop down to 89.6 — but if we eliminate home games, Brady’s drops down to 92.5 as well. It is three points better than Manning’s, but I’m not sure how big a difference in rating 3 points is — I’m fairly new at the analytics game, LOL.

    • Four Touchdowns

      Also, as far as defensive quality goes — in a discussion about my last article, I did end up looking up Manning and Brady’s career DVOA / DYAR and they are fairly similar, so I’m not sure what you’re basing his performances against great defenses on.

      Here’s that post again:

      “I finally got a moment to look at their DVOA and DYAR… below are their career averages. Manning holds a small advantage over Brady in every category except average league ranking for DYAR looking at the stats straight up.

      If you go by Brady’s numbers post-2004, he does gain a small lead over Manning — but if we get rid of Manning’s 2015, which I think we can agree was not the same player as seasons prior, Manning’s numbers jump up quite a bit and overtake Brady’s post-2004 career averages.

      However, if you subtract Brady’s three worst seasons, I think it’s fair to look at Manning’s numbers without his three worst seasons and he beats Brady’s post-2004 numbers by quite a margin.

      And if you look at their single season best numbers in each category (Brady from 2007, Manning from 2004 and 2013), Manning wins across the board again in every category except DYAR.

      So I’m not sure why you thought it “wasn’t close” — even if you take Brady post-2004 and put them up against Manning’s, including his horrible 2015, the numbers are actually very close.”

      https://uploads.disquscdn.com/images/5c110e3fe140265d7f7d161132e648ed97946691acd2ca6819637090c48d8611.png

      • Four Touchdowns

        Additionally, here is a 2014 article about post-season performance by DVOA/DYAR (and my comments):

        “I did randomly find this one that shows that as of 2014, Peyton Manning had the 4th toughest strength of schedule by DVOA at 20.9% while Brady had the 24th hardest at 12.0%.

        It also shows that Manning had the better passing DYAR per game at 114.8 while Brady averaged a 82.6 DYAR in playoff games. It also showed Manning with three of the top ten single game DYAR scores while Brady’s best came in at #11.

        Manning also beats Brady’s playoff DVOA with 31.3% (ranked 4th) to 21.9% (ranked 11th).

        It also shows that Manning has the better playoff 3rd down conversion rate, 42.86% to 40.98%.

        Finally, it shows that Manning also has the better Playoff Win Probability Added and Expected Points numbers, with 4.80 / 145.8 to Brady’s 4.59 / 134.6.

        So going by Football Outsiders, it looks like through 2014, Manning has been better in the playoffs by a whole host of metrics despite not having the rings to show for it.”

        http://www.footballoutsiders.com/stat-analysis/2014/nfls-best-playoff-quarterbacks-dvoa-and-dyar

        • Tom

          And for me, the point of these studies isn’t that one guy is “better”…hell, a few wrong data entries here and there and one guy is better than the other, whatever. The point is that these guys are *both* legendary, and it’s plausible, or at least possible, that one guy may have been on a better team, may have had better luck, etc, and this resulted in more wins.

          Now…since that study was done in 2014, the numbers have most likely gotten better for Brady and worse for Manning, and it’s clearer, if it wasn’t before, and this is only my opinion, that Brady is the GOAT. But he’s not as far out in front as most of his fans might think, and that’s what these studies show.

      • WR

        Between 2004-2015, the YAR to DYAR adjustment helps Brady and hurts Manning. This is because over that time period, Brady faced significantly tougher defenses, which is why Brady beats Manning in DYAR in years like 2009 and 2012.

        • Four Touchdowns

          I’m not sure I understand your first statement… can you elaborate? Sorry, I think it might be the way it’s phrased.

          Are you saying Brady’s DYAR average from ’04 to ’15 is better than Manning?

          • WR

            So this is something I should have explained a long time ago. What I mean is that if you compare the unadjusted YAR figures to the adjusted DYAR figures for each season, you can see how strong the slate of opposing defenses was that year. So for example, in the 2009 season, Brady had a YAR figure of 1649, and a DYAR figure of 2021. So this reflects the fact that Brady played a particularly tough group of defenses that year, according to Football Outsiders. So we score this as a + 372 for Brady. Manning went from 1849 YAR to 1771 DYAR, so we score this as -78 for Manning.

            If you add up all the totals season to season for both guys from 2004-2015, what you get is a +1444 score for Brady, who faced tough defenses every single season, and a score of -497 for Manning. Now, Brady had a -77 score in 2016, so that brings him back to Earth a little bit. But it’s clear that Brady has faced much tougher defenses overall between 2004-2015. So when we see that Manning outpaced Brady in that period in YPA, passer rating, ANY/A, and TD%, we should remember that on the whole, Brady was facing a tougher slate of opponents.

            • Four Touchdowns

              Ah, that’s a very keen observation. Between that and Chase’s SOS numbers, I would have to agree Brady played tougher defenses over that time.

              I also have another theory — having better team support meant that Brady didn’t need to resort to ridiculous heroics as often as the other three to keep the Pats in games as often. I think it’s very possible he could have the same volume stats as the other three if he played for a weaker franchise.

        • Four Touchdowns

          I just ran the numbers for 2004-2014 (for obvious reasons, I’m not including 2015) and the averages per season are —

          Manning has an average DYAR of 1,853 and a 37.2% DVOA while Brady has an average DYAR of 1,623.7 and a 30.7% DVOA. Even when we add up all the DYAR and DVOA from that time period, we get 18,525 DYAR and 371.9% DVOA for Manning and 17,861 DYAR and 337.4% DVOA for Brady.

      • Paul

        according to Chase’s greatest qb article, http://www.footballperspective.com/the-greatest-qb-of-all-time-v-part-ii-career-rankings/, Tom Brady has faced significantly stronger defenses, with the 3rd highest relative SOS of any qb in NFL history that can qualify for NFL records (1500 career attempts), as of 2013. Peyton Manning was about exactly league average and Aaron Rodgers and Drew Brees faced slightly below average defenses. While I think Brady has played either average or below average schedules the past 3 seasons (I don’t know how to created a weighted Attempt formula for ANY/A) its certainly not different enough to change that.

        So Brady’s stats are likely underrepresented in any statistical debate that doesn’t include SOS. additionally, Peyton’s numbers are likely also, since defenses generally allowed lower ANY/As during his early career, especially compared to Aaron Rodgers, who has faced by far the easiest schedule when you factor in eras. with a career defensive ANY/A of 5.85 compared to 5.55 for Peyton Manning and 5.46 for Tom Brady. These are non weighted cumulative defensive totals, so they don’t include the amount of attempts each player had against each team. If they were it would likely throw the numbers even more in Brady’s favor, as the AFC East has been consistently above average in defensive quality (maybe a result of having 3 of the worst cold weather arenas, and a humid, hot outdoor stadium in Miami).

        However, I’m not aware if his numbers adjusted for the SOS those defenses faced, so the numbers may be skewed by that. I don’t know how to run SOS adjustments or create a weighted list, so once I learn how it might change these totals, also can someone let me know how to upload exel files here, do you have to turn it into a image?

        • Four Touchdowns

          Thanks for the link and all your thoughtful comments.

          I totally agree that SOS should be included in an article comparing stats and while this articles series does that to some extent, the primary topic here is team support. The only reason I included the players’ stats was to try and determine how much of any blame the QBs should take in their support being lower — if you’re constantly failing as an offense, it will hurt the amount defense and special teams can help.

        • Four Touchdowns

          FWIW, I just ran the averages with the new seasons (and without 2000 and 2008) and Brady’s career SOS is now 0.09 (as an average).

          I agree about stats being underrepresented due to era — that’s why I included both RANY/A and Relative Passer Ratings.

          • Paul

            that’s reasonable, the player performance isn’t as important to this study, but since you are comparing their performance in these situations it might likely change a few of the fields. Having unweighted averages is misleading tho, since the AFC East has consistenly been better than the other divisions each player played in, and as they play each of these teams twice, it will greatly skew the numbers if you use an average of just those teams. Anyway, I understand why you didn’t include them, its hard to calculate and needs adjustments in the first place for the teams those defenses played, like profootballreference does with team SOS. I don’t know how to do those.

            If anyone would like to help me learn how id appreciate it.

            • Four Touchdowns

              Honestly, doing this series was a ridiculous amount of work for me and I don’t know that I’d ever try it again, despite a real interest in doing others. Collecting this info without a data scraper is just way too tough… I’d be curious to see whatever you come up with!

              • Paul

                whats a data scraper? I can send you my spreadsheets if you like when I finish them (its a hobby, I’m taking my time).

                Also, how do you upload spreadsheet here, does it have to be in picture form, how do you do that with exel documents?

                • Four Touchdowns

                  I have no idea how to upload a spreadsheet in the comments section. I just take a screenshot and upload the JPG.

                  A data scraper is computer code developers use to “scrape” data from website — it goes through a website’s pages and targets data you program it to collect. For this article, I wish I could have run a scrape for all Expected Points from all box scores.

                  Then I could have included Philip Rivers, Kurt Warner, Ben Rothlisberger, and a few other elites. I ended up going through every boxscore and copying and pasting all the EP data into an Excel sheet.

                  Yes, it was tedious.

    • Paul

      in his career Manning has been tremendous against great defenses, if you adjusted a defense for its performance against Manning, you come up for a more valid representation of that defense, for instance, in 2002 the Colts played the Eagles at Philadelphia. The Eagles that season went 12-4, and after removing Manning’s performance against them, they allowed an ANY/A of 4.03. Manning threw 23 passes and averaged 16.48 that day. With a difference of 12.45 ANY/A, that’s a total adjusted yardage of 288.4. This is likely his finest regular season game statistically. Is it fair to judge Manning by the Eagles cumulative ANY/A that season of 4.5. I don’t believe so, so I went through and adjusted the numbers for every game of Manning’s career (regular season only) and came up with his adjusted ANY/A.

      Now for his performance against good defenses (ill go with anything under 5.00 ANY/A:

      Cmp Att Cmp% Yds TD Int Rate Sk Yds Y/A AY/A ANY/A
      2.00-2.99 : 3 games 58 102 56.9% 655 6 1 91.7 3 23 6.4 7.16 6.73 Total Value 392.2
      3.00-3.99: 9 games 191 296 64.5% 2089 15 7 92.3 9 41 7.1 7.01 6.67 Total Value 891.3
      4.00-4.99 76 games 1689 2661 63.5% 19876 127 91 87.8 95 670 7.5 6.89 6.40 Total Value 4839.3

      That’s hardly struggling against good defenses. I would like to upload the excel spreadsheet but don’t know how, can anyone help me do that?

  • Tom

    James, this is great work, thanks for posting this…very enjoyable read, love all the tables. I’m not surprised to see that Brady has had a lot of support, compared to the other guys, from special teams and defense – he’s on a team that consistently values those units (I’d love to see a study showing Brady’s average starting field position compared to other QB’s), and it’s something we can see with our eyeballs if we watch the games.

    So yes, that other stuff that happens when your favorite QB isn’t touching the ball “on every offensive play” matters and it matters a lot. If you don’t think Brees or Rodgers, or yes, even Manning, would have higher winning percentages (not saying higher than Brady’s), playing with better defenses and better special teams, than I’m certain you’re not watching the same game I am. Think about what a single defensive stop does…imagine the Saints having at least two or three more defensive stops per year in the last few years, how many more wins that means for Brees?

    • Four Touchdowns

      After all the talk about the QB stats and whether I should be weighting career efficiency metrics on pass attempts — it’s a welcome change to see someone actually talking about the whole point of the article, the team support!

      Thank you!

    • Paul

      profootball reference lists each teams average starting position for each season.

      • Tom

        You’re right…maybe the Drive Finder can gather all that stuff together (as opposed to having to grab each season, etc.). Thanks.

        • Paul

          your welcome, I hadn’t though of using the drive finder, profootball reference is just, so great.

        • Four Touchdowns

          Looks like from 2001-2016, the Pats average starting position was their own 32.

          From ’99 to 2010, the Colts average starting position was their own 30 (no 1998 and earlier). From ’12 through ’15, the Broncos average start was their own 28.

          From 2005-2016, the Saints average starting position was their own 28.

          And from 2008-2016, the Packers average starting position was their own 30.

          • Tom

            Man, that was fast…I was in the process of gathering up all that data, ha!

            Well, there you have it. Not a humongous advantage, but if we go by Brian Burke’s (and others) simple conversion that a yard is 0.06 points (I think it was that…might be 0.07), we’re looking at an advantage of around a 0.12 points per drive over a team that normally starts on their own 30. If Brady gets 9 drives in a game, he’s got about a 1 point advantage right out of the gate. Again, not huge, but still nice to have.

            • eag97a

              Not a very big advantage and opponent strength can heavily impact this. Ofc averaging across careers it will still be significant but that won’t explain TBs’ overwhelming lead in wins.

              • Tom

                Of course, the opponent strength, weather, a million things can impact this. And yeah, 1 point per game is not a huge advantage and certainly doesn’t explain the huge lead in wins. But let’s say we look at every team since 2001 and find that the Pats have the best average starting field position by a few yards. I’d say that’s worth noting, and is part of the bigger picture of why they are so successful. In the same way that historically, the Pats don’t fumble the ball…it’s like off the charts. Another piece of the puzzle…none of which, incidentally, is taking anything away from Brady.

                • eag97a

                  Correct. Another data point in our never ending quest to perfect qb eval! 🙂

                  • Tom

                    Hahaha, right! And it *is* never ending…not sure why we (I) find this so alluring…

                    • Four Touchdowns

                      Who the hell know, LOL.

                      BTW, who are your top five favorite QBs?

                    • Tom

                      I’m embarrassed to say that I really haven’t tought about that much… I’m kind of a weird bird with this football thing – I don’t even really have a favorite team (OK, the Rams ‘causd I live in LA) – I just love the game, the history, the STATS, rankings, narratives, etc.

                      After giving it some thought, here are the guys that I like (not the guys that I think are the best, I leave that to Bryan or Brad or Chase, etc):

                      1. Peyton Manning
                      2. Brett Favre
                      3. Aaron Rodgers
                      4. Fran Tarkenton
                      5. Terry Bradshaw

                      This could change tomorrow of course!

                      How about you? What are your top 5 guys?

                    • Four Touchdowns

                      If Chase is reading this, this concept could be a good complement to Wisdom of the Crowds — asking people to list their favorite players and seeing who’s popular among FP readers. You could even do a “fan favorite” all-star team.

                      Anyway, my favorite QBs in no particular order after Peyton —

                      1. Peyton Manning
                      2. Aaron Rodgers
                      3. Dan Marino
                      4. Joe Montana
                      5. Doug Williams (purely sentimental here — his backstory is amazing, going from the death of his wife to becoming the first Black QB to win it all)
                      6. Jim Kelly
                      7. Andrew Luck
                      8. Steve Young
                      9. Drew Brees
                      10. Dan Fouts

                    • Tom

                      Damn, forgot about Fouts…kind of nostalgic for me. Another guy with a great backstory like Williams is Kurt Warner, of course – went from bagging groceries to SB champ. I might have put him on my list as well.

                      The dudes on your list are obviously all studs…I love watching an Andrew Luck game – throw some picks early on to get your team in a hole, then be the hero late in the game and win (yes, I’m being very general here). Apart from the 4 Super Bowls, haven’t watched a lot of Jim Kelly…thought he was great on that last drive in 1990 (“Wide Right”).

                    • Four Touchdowns

                      For a lot of the older players, my favor tends to come from me liking their stories. I’ll admit to be being biased to “hard luck” players who have been held back by weak organizations — something about an underdog, I guess, LOL.

                      By all rights, I should really like Tom Brady and while it’s not his fault, the Patriots trolls bashing Manning have made me resent him a little (though I don’t let this affect any of my studies, I believe in fair and honest fact-gathering).

                    • Tom

                      I’m with you on the underdog thing…that’s probably why I’m not a huge fan of Brady. I mean, he wins all the time, what’s the fun in rooting for someone like that? It’s like rooting for the Yankees in the 1920’s or something…so yeah, he’s the GOAT in my opinion, but he’s not one of my favorite QB’s…maybe if he’d just lose a little bit more, I might like him.

                    • Four Touchdowns

                      If you consider titles a team achievement, what puts him over the top as the GOAT to you?

                      Not saying you’re wrong, just curious of your criteria since you’re not a “RINGZ” guy.

                    • Tom

                      For starters, don’t get me wrong, I’m not saying he’s running away with it…not saying he’s just way out in front of the pack, etc.

                      I haven’t really gone through this methodically yet (the way Brad, Chase and others have) but I guess it comes down to this: at the end of the last Super Bowl, my brother called me and said “He’s just done this too many times…how many more times do we need to see this guy play great at the end of the Super Bowl before we say he’s the best?”

                      So, yes, my “analysis” for now is based on playoff performances, especially the Super Bowl, but I feel comfortable doing that because we know he’s got solid regular season numbers as well.

                      But you are correct, I’m not a “RINGZ” guy…I don’t start with the rings and then say, “See? He’s good!”. No…for me it’s the fact that in every Super Bowl he’s been in, he plays well enough – either throughout or at the end – to put his team in a position to win. He hasn’t had that out-of-this-world performance like Montana, Young, Williams, etc., but he hasn’t had a “bad” performance either…and more importantly, he hasn’t played “bad” at the end when the game is in doubt.

                      I like using Win Probability; I’ve got the numbers for every Super Bowl game and in Brady’s worst game (per my numbers and we know WP can be wonky and there’s a lot to fool with there) – against the Giants in 2007 – he still has a WPA of +0.22. Remember – he led his team on what would have been the game winning drive had Eli not created his own bit of magic.

                      Again, I haven’t gone through everything, but to sum it up: I would most likely still call him the GOAT if Atlanta wins the toss and beats the Pats in overtime last year and if Beast Mode runs that ball in for the winning score in 2014. He’d have only three rings, and he’d be the GOAT.

                      All that being said, we both agree that he’s had a lot of help…so as we uncover more stuff (and as I more methodically nail this down) my opinion may change.

                    • Tom

                      To further explain what I’m talking about: per the numbers I’ve got and from watching the game, Manning’s best Super Bowl is in 2009 against the Saints. Yes, the pick-six ended the game, but without his great play earlier in the game, that pick-six would have been meaningless. If Peyton had played well in all four of his Super Bowls, even if he lost every last one, I might consider him the GOAT. It’s not the actual acquiring of the ring for me at all…it’s how the QB played in that game (and the playoffs, etc.) I’ve got a lot more analysis to do on this, but that’s where I’m at now…

                    • Four Touchdowns

                      That’s fair enough. While I’m not trying to dissuade you, I’ll just explain why I have a hard time feeling comfortable calling him the very best.

                      First off, I’m a tiers guy — too much noise from the different rosters, schemes, coaches, eras, etc. to say anyone is the best. The don’t start on level playing fields and the sample size of games per season is just too small — and with playoff games, the sample gets smaller.

                      But if I were to mainly use Super Bowl performance, it’s hard to pick Brady over Montana. Brady won his games in more dramatic fashion since they were all by one possession but Montana just dominated in Super Bowls with a 127.8 career SB rating and no interceptions, while Brady isn’t in the top ten for rating among multiple SB winners.

                      That said, and this is just me with no judgement of you or anyone else, I’d never use such a small sample of games as a way of judging players or even as a tiebreaker — there’s just too much noise when the samples are that small.

                      For example, Terry Bradshaw has a 112.8 career rating in Super Bowls and while he’s a better player than the stats suggest, we both know that doesn’t represent him as an overall player (especially since the average rating during those years was probably in the early 70s!).

                      Troy Aikman is right behind him at 111.9 — meanwhile players like Peyton Manning, Fran Tarkenton, Dan Marino, etc. have SB ratings far behind him.

                      And the SB record for passer rating? Phil Simms at 150.9.

                      Now these are extreme examples, as Brady’s SB stats mirror his regular season numbers, but my point is that players don’t get to choose when they have their best games or worst games, they just happen when they happen.

                      And beyond all that, consider that all of Brady’s SB wins and losses were by one possession (and I feel he’s good enough to deserve them all, I’m bringing this up to illustrate randomness) — and then consider that only 64% of MLB World Series game one winners win the title. In the NFL, game one winners win 100% of the titles. If these results carried over to football, 36% of Super Bowl winners would be different in a series of games.

                      I put him in the top tier of football QBs, but if we don’t give SB games an unusually large weighting in our ratings, there just doesn’t seem to be a case that he’s definitively better than the other elites in that tier.

                      But again, this all is just my own thinking and using my own criteria — I don’t expect it to apply to anyone else. 🙂

                    • Tom

                      OK, I agree with everything you said…the small sample sizes, the “one-and-done” aspect of NFL playoffs/SB, the perhaps unjustified weighting of the Super Bowl, the varying eras, the “tiers” idea, and finally, yes – even when we look at just Super Bowls, Brady has not had that dominating, mistake-free Super Bowl that Montana, Young and Simms have had (especially when we consider that Montana has had perhaps more than one Super Bowl like that).

                      And to further enhance your argument, Brady *himself* said that he doesn’t consider himself the GOAT for some of the reasons you mention:

                      “I don’t agree with that, and I’ll tell you why. I know myself as a player. I’m really a product of what I’ve been around, who I was coached by, what I played against, in the era I played in. I really believe if a lot of people were in my shoes they could accomplish the same kinds of things. So I’ve been very fortunate. … I don’t ever want to be the weak link.” (http://www.businessinsider.com/tom-brady-says-hes-not-greatest-quarterback-2017-5)

                      Perhaps he is being humble (I certainly don’t think “a lot of people” – he means QB’s of course – could accomplish what he did), but this is still very revealing and for now, we can take his word at face value.

                      OK, so why should I say he’s the GOAT, even after agreeing with your points, and seeing that Brady also doesn’t think he’s the GOAT (which we’ll just take at face value)?

                      Well, I’ve got to look at this more, in light of your insightful comments, and so what I’ve written below is incomplete and doesn’t address everything you’ve said.

                      For starters, it appears that, overall, I am placing a HUGE premium on Clutch Performance in the Super Bowl. What I am considering “great” is playing well at that moment when you absolutely HAVE to. There are problems with this way of thinking (so, Steve Young is not great because he didn’t get a clutch moment in the SB? That’s lame.), and that’s probably where we differ most. That being said, here’s my take:

                      Brady has played very well, if not great, *in the clutch* in SEVEN Super Bowls. If it was four or five, or maybe even six, me might not be the GOAT, but seven is just too much to ignore. So, yes – Montana, Simms, Young, Bradshaw, all of these guys have had better or more dominating SB games; in fact, Brady doesn’t really have a “dominating” SB game – the 2003 game against the Panthers I’d just call “very good”. But none of those guys have as many clutch performances in the biggest game ever as Brady. As I said when you first asked, it’s just been too many times with this guy.

                      Finally, here’s something else, and this distinction is important: I don’t think, necessarily, that Brady is “better” than the other elite (or whatever word we want to use) QB’s like Manning, Bradshaw, Rodgers, Montana, Marino, etc. I believe that any of those guys could easily have had seven games like Brady’s, because you’re right – these guys don’t get to choose what games they play in, etc. But, since he is the only that’s even *been in* seven Super Bowls, and he played well, in the clutch, in each one, I’m calling him the GOAT. If his performances were EXACTLY the same in each of those games, and he lost each game, I might still consider him the GOAT.

                      I must reiterate – I haven’t thought this all the way through, and your comments have given me a lot to chew on, but that’s where I’m at right now!

                    • Four Touchdowns

                      Holy shit, my respect for Brady just skyrocketed by reading that quote!

                      “I don’t agree with that, and I’ll tell you why. I know myself as a player. I’m really a product of what I’ve been around, who I was coached by, what I played against, in the era I played in. I really believe if a lot of people were in my shoes they could accomplish the same kinds of things. So I’ve been very fortunate. … I don’t ever want to be the weak link.”

                      Okay, back to the rest of your post…

                    • Tom

                      Yeah, met too. He’s a legend, has 5 rings, and when asked if he’s the GOAT says “Well, actually I’m on this really good team, and so you know, I’m pretty lucky”. Basically just said what all of us have been saying…you can be great, but being on a great team certainly helps you become “more great”!

                    • Four Touchdowns

                      I’m honestly not trying to convince you that he’s not the GOAT. As said, I don’t really feel anyone is actually the GOAT so if people want to decide for themselves who they think is the best based on the criteria they value the most, and they truly feel it in their heart, I will not tell them they are wrong.

                      If you feel “clutch” performances in the Super Bowl are the key deciding factor, I am fine with that. After all, the whole point of playing the game is becoming that season’s champion.

                      My post was only explaining why I couldn’t name him, or any other QB, the actual best quarterback ever and why the “clutch in the Super Bowl” argument doesn’t work for me — at the end of the day, they are just seven games out of hundreds and I don’t believe quarterbacks have special “clutch” powers they can activate at special times like the Super Bowl.

                      If their lines hold up and guys get open on time, they make the reads and throw the ball accurately. Sometimes QBs have amazing improvisation like with Rodgers or Elway, but Brady isn’t that kind of QB — like Manning and other pocket passers, he’s a guy that relies on his system and teammates to get open and create opportunities.

                      IMO, I think people place a premium on “in the clutch” because it’s incredibly dramatic and creates a lot of tension, making the moments more emotional and memorable. But if you really think about truly being clutch, wouldn’t that mean playing perfect football from the start and putting your team up by so many TDs that the chance of the other team winning is as low as possible? Isn’t posting a 147 rating, 5 TDs, and no interceptions more clutch — all with John Elway on the other side of the field — than having three bad quarters of play and relying on the outcome of a few drives and some lucky plays by teammates? I mean, that Edelman miracle catch really should have been an interception by the linebacker, but the Falcons defense was horrible last year, so a “clutch” play happened.

                      Again — all my own logic and feelings, I don’t expect anyone to change their minds or feel differently based on my thoughts. I just like to share, LOL.

                    • Tom

                      OK…regarding your comments:

                      – Right, I didn’t get the impression that you were trying to convince me of anything, we’re just talking about stuff and throwing around ideas, etc. And yeah, there’s a million ways to look at this subject.

                      – Agreed, seven games out of hundreds seems somewhat unfair, and I’m not convinced either. To stick with my point, I guess I might say that I’m including the fact that it took having a great season to get there and to win in the playoffs. But yeah, you have a good point.

                      – I agree that QB’s don’t have some special clutch ability…perhaps I’m splitting hairs here, but I’m congratulating Brady for being clutch in those moments, not that he is, in fact, “clutch”. One time, I made a great catch in little league to win the game…that was a clutch moment, but I am not “clutch”. Brady is the GOAT (for me, at the moment) because he has A LOT of clutch moments, not because I think he *is* clutch. I can tell you this – I was hoping to God he wouldn’t be clutch in the last Super Bowl, that’s for sure.

                      – You are 100% correct about why people, myself included, place a premium on clutch. It’s why I enjoy the game…the drama, the tension, the memorable moments. The 1989 and 1994 Super Bowls are an absolute bore, even though they are probably the most incredible QB Super Bowl performances ever. So I don’t really have an answer to this, except to say that you’re right, but I’m still really into the clutch aspect of the game (or the “drama” aspect, if we want to call it that).

                      – Yes, “being clutch” should also mean playing great the whole game, not just in the 4th quarter when you’re down. I do not have an answer to this either. I just know that there are plays that are made when the result of the game is doubt – when the score is tight and there isn’t much time left, and I place a premium on these plays. For one, it may not always be that the QB “didn’t play well” early on. Brady was doing alright in 2003 when Jake Delhomme decided he wanted to be the GOAT and led 3 TD-scoring drives in the 4th quarter when his team needed it. The Pats needed to put some points up and Brady helped them do it. Again, to be clear – if Delhomme scored last in that game, my analysis of Brady’s performance doesn’t change.

                      – I also agree that we run into problems when we give a guy credit for something he’s only half a part of: Edelman’s catch, a lot of Swann’s catches, the Julio Jones catch, Kurt Warner throwing the ball 20 yards (or whatever) and Fitzgerald running the rest of the way, etc. On this point, I have to repeat what I’ve been saying about Brady: it’s not just one time, or a few times…it’s A LOT of times. It’s seven Super Bowls, countless playoff games. Edelman shouldn’t have caught that ball, but Welker maybe should have made a catch in 2011, etc. He’s just come through too many times, for me anyway, to ignore. Remember how Montana got his reputation for being a “winner” and being “clutch” for those playoff and Super Bowl wins? That’s what Brady is doing, *even in the games where his team lost*. Look at the 2015 AFC Championship game against the historically great Denver Bronco’s D! He was getting mauled by Von Miller and the rest, and the Pats *still* almost won that game. Yes, we can point to great catches by Gronk, etc., but the fact is, Brady made the plays that needed to be made. And he’s been doing it for years and years. (OK, I need to stop right now…I’m defending the Pats for Christ’s sake and I hate them).

                      That’s kind of where I’m at right now…among your good points, I would say that I struggle most with “clutch” play (4th quarter, etc.) versus great play throughout the game. In this ranking by Chase http://www.footballperspective.com/best-and-worst-super-bowl-passing-performances/ Brady doesn’t even crack the Top 15 of Great Super Bowl Performances. I’m putting together a ranking right now which tries to blend “great play throughout” with “clutch play”, and I’ve got Montana as the greatest *SB* QB of all time. So, I guess when I flesh everything out, I might move off of Brady being the GOAT, who knows.

                    • Richie

                      I was rooting for Brady and the Patriots in 2001. They were basically a bad franchise up until then, that somehow managed to sneak into 2 Super Bowls. Brady was a good story. I was happy the tuck rule was called in their favor. I was happy they beat the Rams.

                      Somewhere between there and winning their second Super Bowl, I turned on them. Neither Brady nor the Patriots have had an underdog story since at least winning their 3rd Super Bowl.

                      Add in the Celtics, Bruins and Red Sox all winning championships in the past 15 years (and now the Celtics win the lottery?!?!)…and it’s been tough to be a Boston hater!

                    • Now I want to play too.

                      My favorites, among those I have actually seen while active, are:

                      1. Joe Montana
                      2. Steve Young
                      3. Steve McNair
                      4. Drew Brees
                      5. Aaron Rodgers

                      If we open it up to included guys I’m too young to have seen live but have watched tape on and read about, I’d include:

                      – Sammy Baugh
                      – Sonny Jurgensen
                      – Fran Tarkenton
                      – Roger Staubach
                      – Pat Trammell

                    • Four Touchdowns

                      That’s a good way of doing it. I’ll amend mine to do the same — seen while active:

                      1. Peyton Manning
                      2. Aaron Rodgers
                      3. Andrew Luck
                      4. Drew Brees
                      5. Russell Wilson

                      (Yes, that includes two guys who have beaten Peyton in the Super Bowl — but they’re just too cool and talented not to like!)

                      And from the history books:

                      1. Dan Marino
                      2. Joe Montana
                      3. Doug Williams
                      4. Jim Kelly
                      5. Steve Young

                      And honorable mentions to Dan Fouts, Otto Graham, and Sammy Baugh.

                    • My favorite thing about Dan Fouts:

                      https://www.youtube.com/watch?v=iPUcsLRX4pA

                    • Richie

                      One of the things I liked about Fouts was the way he dropped back without side-stepping. Has anybody else really done that?

                      At the 1:30 part of that video is great when Winslow tells Fouts he needs to find a new tight end. “tip-toeing like a mother f’er.”

                    • Tom

                      Dude…that was cool.

                    • Richie

                      Hmmmm…..Top 5 favorites I’ve seen. This can be tough, because my impression of many of them has changed. Either because they switched from underdog to hero or vice-versa. Plus, I can get annoyed by the way some guys get treated for winning or not winning Super Bowls. Sprinkle in me having them on a fantasy team or not, and things shift.

                      1. Dan Marino (there hasn’t been any shift here. He’s been my guy since 1984 when I was 12.)
                      2. Dan Fouts (My first memories of watching football are around 1980-81. Those Chargers teams were fun.)
                      3. Aaron Rodgers (I was indifferent to him until I added him to my dynasty fantasy team in 2011)
                      4. Warren Moon
                      5. Peyton Manning (I think I hated him his first few years. But somewhere in the mid-2000’s when the Brady-Manning stuff took off, I hated how Brady was considered better mainly because he had Super Bowl wins, and Manning didn’t.)

    • Four Touchdowns

      As I did this study, Expected Points kinda became my new favorite stat because it goes beyond traditional volume metrics and adds context to the plays in the same way some of the other advanced analytics sites do.

      For example, traditional stats will reward a 12 yard completion on a 3rd and 15 while Expected Points will see that as a negative play. Looking at things that way, we can see the EP gained and lost by all the various squads as a breakdown of how many negative and positive plays they had — if they have negative EP, they had mostly negative plays, even if they somehow got good yardage totals or whatever.

      • Tom

        Yep. Upon first stumbling on to EP, my initial reaction was that it was something that will allow us to compare all the different units’ contributions with the same metric – points. I’ve got EP numbers for every Super Bowl…the data I’m using is from Brian Burke from years back, and so yes, the numbers don’t quite work for Super Bowls in the 1970’s, maybe even through the 1990’s, Burke’s data is from 2002-2012 I believe.

        But regardless, these numbers give us picture of a game. Denver’s 2015 SB win looks like this:

        Off: -14.0
        Def: +17.9
        SpT: +8.5
        (+1.6 gained on end of half play, etc.)
        Margin of Victory: +14.0

        In an instant, we can see what happened in this game: the Denver defense (or Carolina’s bad offense, however you look at it) and special teams carried the load, those units provided the points necessary to win the game. Sure, we can look at Win Probability if we want to get into the “timeliness” of these points, but, knowing that game was fairly close throughout, the above picture tells us what happened.

        • Four Touchdowns

          Exactly, you see which units contributed to the win — it was a defense-driven Super Bowl win all the way.

          And as bad as Manning’s passing game was in Super Bowl 48, we see that his total support EP (all other units combined) was even worse, showing a total team collapse — (-13.24 passing EP, -4.52 rushing, -7.18 defense and -9.66 special teams EP).

          • Tom

            Agreed…my (Burke’s) numbers are a little different, but basically say the same story. The Denver offense was *real* bad (I’ve got -18.6 EPA), but the entire team had a bad day! And thus the blowout…with a little support – and I’m not just talking about not allowing the kick return TD – the game might have been 24-8, etc.

            • Four Touchdowns

              Wow, that’s a HUGE difference i defensive EP! I wonder why their models are so different?

              No wonder EP hasn’t caught on with that kind of variance between different models.

              • Tom

                James – what difference are you talking about? Between my EPA of -18.6 and PFR’s -16.3 for the Denver offense in SB48?

                Regarding the models, obviously I can’t speak with any kind of authority on either one, or any other model. But yep, the reason this probably hasn’t caught on, like Fantasy Points, etc., is that there’s no “official” model and the models differ. Here are the reasons I don’t use PFR’s EP numbers:

                1. I can’t make sense of their box score table…the numbers don’t add up. For SB48, Denver has -16.3 on offense. So I’m thinking the pass and rush should add up to that, right? They don’t. -13.2 pass, -4.5 rush. But then there’s this other column for turnovers, -18.0. How does that all add up? Have no idea, and so it makes me a little uneasy with it…I don’t know what they’re doing.

                2. A first-and-goal from the 1-yard line has an EP of 6.97. This, to me, just seems too high. Burke has that state at 5.96, Keith Goldner’s model has it at around 6.3.

                To be clear, I don’t think PFR’s numbers are “wrong” or “bad”, just for the reasons above I don’t use them…once I got Burke’s numbers and created my own play-by-play templates in Excel, I just stuck with that.

                • Four Touchdowns

                  Sorry, I thought your -18.6 was the for the Denver defense!

            • Four Touchdowns

              Just to put things in perspective, Manning played worse in SB50 in terms of EP and ended up winning that game!

  • bluestatehostage

    Jim Kelly and Fran Tarkenton each won 4 conference championships. The rest of those guys you mentioned in that same sentence won a combined total of 1. A conference championship is not nothing.

    • Four Touchdowns

      I totally agree but team support matters in winning those as well. Jim Kelly and Fran Tarkenton had better overall teams than Sonny Jurgensen and Philip Rivers.

      Also — go Skins!

    • Four Touchdowns

      To me, the Jim Kelly Bills are the poster children for bad luck — your worst games at the worst times.

      From 1990-1993, they had a dominant winning record against NFC teams during this time period (I want to say 17-2 but I can’t recall what the 30 for 30 special said specifically) and in between two Super Bowl losses to the Cowboys, they went down to Dallas in the regular season and beat them at home!

      At the end of the day, it’s just four games and unfortunately for them, they happened to be their four worst games.

  • Tom

    James – need to get into the weeds a little bit; I know I’m just missing something. In your first EP table, under the Brady column, you’ve +1.37 rushing, +1.36 defense and +0.42 special teams; and the total is +0.99. Shouldn’t the rushing, defense and special teams numbers all add up to the total? What am I missing?

    • Four Touchdowns

      I’m not sure either — for each individual game, I add the columns with the rushing EP, total defensive EP, and the ST EP together. Then at the bottom for the career averages per game, I do an average of all of those games to get the team support EP average.

      Logically, it would be 3.5 EP as you say — but I’m not sure why the total support average wouldn’t reflect that.

      I can send you my spreadsheet if you shoot me your email address so you can check it out on your own.

      • Tom

        I have a hunch as to why this discrepancy is there: as I noted below, for whatever reasons, there’s some funkiness in PFR’s numbers, they don’t seem to match up. I understand how this can happen…I was so excited about EP when I found out about it in 2013 that I compiled the numbers for every game that year, and it’s freaking tedious. There are errors in the play-by-play, penalties that your parsing might mistake for actual plays, etc., etc.

    • Four Touchdowns

      Tom, I see what’s going on — the best rushing, defensive, and ST games aren’t all happening at once.

      For example, let’s take an imaginary set of three games:

      Rush EP: 5.0
      Def EP: -3.0
      ST EP: -1.0
      Support EP: 2.0

      Rush EP: -3.0
      Def EP: 7.0
      ST EP: 1.0
      Support EP: 5.0

      Rush EP: 1.0
      Def EP: 2.0
      ST EP: -9.0
      Support EP: -6.0

      Average Rush EP: 1.0
      Average Def EP: 2.0
      Average ST EP: -3.0
      Average Support EP: 0.3

      See what happened? If you added the support EP averages of rushing, defense and special teams, you would get an average of 0.0 EP support.

      However, if you average the final support numbers of 2.0, 5.0, and -6.0, you get an average of 0.3 EP.

      The average support per game is different than the averages of each category individually.

      • Tom

        Right, I think you’ve got it. Well, that solves that mystery at least!

  • Tom

    Well, apart from the usual Manning/Brady discussion, another topic worth bringing up is the pitiful Support Drew Brees has been getting. None of us doubt how good this guy is, and it’s a damn shame he doesn’t get more help. If I’m reading your chart correctly, in 70% of Brees’ games, his Support has put up negative EP numbers. It’s unfortunate that we don’t have league averages for all of this stuff (and I understand why we don’t, what with you not having a state of the art Data Scraper!), because I suspect that this would be near the bottom. If anything, your charts show that if you’ve got an elite you QB, *and* your supporting cast is consistently better than average, you’ll win A LOT of games.

    • Four Touchdowns

      For me, the real story of this article isn’t Brady and Manning, though that was the inspiration for it — it’s Drew Brees and to a lesser extent, Aaron Rodgers.

      We can see that Brees has basically gotten shafted by having such horrible team support throughout most of his career. I honestly think that with better teams, he’d be as admired as Manning and Brady and might even have another ring or two to show for it. He should probably be in the “top five of the Super Bowl era” conversation but usually isn’t.

      And while I suspected Aaron Rodgers was the best, looking at his relative stats has basically confirmed it for me (and as a Peyton fan, I don’t want to admit that, LOL). He’s so far ahead of the other three and since the stats are higher than ever now, you’d think his relative stats would suffer but he’s just unbelievable.

      His only flaw is his high sack rate but for everything else he delivers, I think it’s worth it. He’s basically got better efficiency metrics than the greatest pocket passers but also adds the scrambling ability of guys like Young, Tarkenton, Archie Manning, etc. If I could have anyone on my team, it’d be Peyton Manning — but if I take away my sentimental attachment to the sheriff and just went about it logically, it’d be Aaron Rodgers.

      • Tom

        Agreed on all counts. I’m not a QB expert at all, but on top of Aaron’s numbers, he makes plays that I just don’t the think any of the other three could come close to making. Someone else said this, but he’s like a Jedi in the pocket, shifting and sliding around, having that sense of where everyone is (but yeah, he takes some sacks as well).

        Of any QB that I’ve seen play (on TV of course), I would take Rodgers first, as you said. Of course it’s a close call with Brady, Montana, Manning, etc., all the other greats. But I’m willing to put him in any system and take my chances.

        • Four Touchdowns

          I agree about him making plays I don’t ever see any other QB making — his arm strength and accuracy are just off the charts.