Author’s note regarding the numbers: The Win Probability (WP) numbers shown below were, for the most part, generated using a formula presented by Wayne Winston in his book Matheletics, and subsequently improved upon by Pro Football Reference for their WP model. ^[1]While in the process of messing around with PFR’s WP Calculator, I have noticed differences between the results from the Calculator and the results from using the formula as shown on their web … Continue reading

The heart of the formula is the Excel NORMDIST function, which returns the normal distribution (the probability) for a given mean and standard deviation. I have made some minor adjustments to this formula, but it is basically the same. The formula requires the use of Expected Points data; the EP dataset I use comes from Brian Burke’s Advanced Football Analytics site (when it was active), and I have adjusted those numbers for era. Since the formula falls apart in certain areas – most importantly, the 4^th quarter when the game is close – I abandon it and use other data to generate a WP number. Field goal success rates, 4^th down success rates, drive results, PFR’s Play Index and the recently provided play-by-play data from Ron Yurko on GitHub, are some of my alternate sources. I realize there’s a “black box” aspect to Win Probability analysis (if you can find two models that completely agree, let me know), since it’s not something that can be easily checked, and perhaps especially suspect when the author openly states that he adjusts the numbers “manually”. To that I can only say that my intent is to provide as accurate a picture as possible of the games that I analyze, and I’m open to any suggestions, comments or questions. Thanks.

Below is a table showing the performances of Tom Brady and Nick Foles in last year’s Super Bowl (“Rank” refers to how that stat ranks amongst all Super Bowl QB performances, and Value is explained in this footnote ^[2]“Value” is another term for Adjusted Net Yards Above Expectation, a metric frequently used by Chase Stuart on footballperspective.com and other sites. The formula is: (ANY/A – opponent ANY/A … Continue reading):

The numbers say Brady had the better day, and it’s true that his performance could be viewed as the best in Super Bowl history. But after the game, in the glow of the Eagles victory, I think a lot of fans agreed with what HOF QB Steve Young said on NFL Primetime:

“MVP Nick Foles outdueled (my italics) Tom Brady in the biggest offensive show in the history of the Super Bowl, and he made the plays that scored the touchdown that turned the game in the last part of the fourth quarter, more than anyone else that had done it, against [unintelligible] the Brady era…”

We can forgive Young’s jumbled praise toward the end of his statement because he’s pretty pumped up, and perhaps we can forgive him for crowning Foles without, possibly, looking at both QB’s numbers. But on that second item, I’m not sure Young would change his mind, even if he had the numbers in front of him. It appears to me that for Young, Foles “outdueling” Brady isn’t about who has the most yards or the most touchdowns or the best completion percentage. It’s about which QB made those key plays in crucial moments that significantly altered the outcome of the game…you know, the plays that scored the touchdown that turned the game. Young is saying that in this game, Foles was more “clutch” than Brady. Was he right?

I don’t think it’s a question that really can be answered, even putting aside all the baggage with the term “clutch”. But, if we wanted to take a shot at it, the Win Probability Added (WPA) stat would be the best place to start. As I’m sure most readers of this website know, WPA is simply the difference between a team’s chances of winning the game (its win probability, WP) at the beginning of a play and its WP at the end of that play. That difference tells us how much a play increased or decreased a team’s chances of winning, which is what we want to know if we’re trying to measure “clutch”. To find out the QB’s impact on the game, we can just add up all the WPA for the plays in which he passed, ran or caught the ball.

I’ve processed WPA numbers for Brady and Foles in this last Super Bowl, as well as every other quarterback in every other Super Bowl. I’m going to present my “findings” in a series of posts over the next few weeks; the first one is today, covering Super Bowls 36-52 (2001-2017).

Before diving into the table, there are some ground rules cover:

1. The QB gets credit for every play where he ran, passed, was sacked, or caught the ball. If he’s responsible for a delay of game penalty, he takes the WPA hit for that.
2. Other than delay of game penalties (if it’s deemed that the QB is responsible), plays that are deemed “no play” do not affect the QB’s WPA (e.g., the QB does not receive credit for pass interference calls).
3. If a QB throws a completion and then the receiver fumbles away the ball, it is a neutral play for him (he neither gains nor loses WPA).
4. The QB gets full credit for the entire play in which he makes a completion – if the receiver catches a 1-yard dump off and then runs 75 yards for a TD, the QB gets credit for all of it. There is one situation where I split WPA: penalty plays that add yards at the spot of the penalty (unnecessary roughness, etc.). The QB gets credit to where the play would have ended without the penalty; if the play results in a first down and the play would have ended short of the first down marker, he gets credit to at least the first down marker.

We’re almost there. I’ve tried to cram as much relevant information in here as possible to provide a full picture of not only the quarterback’s performance, but the context of the game he was in. Below are some explanations of some of the terms:

Vegas Spread: Negative number means the QB’s team was favored by that much at closing (per PFR box score in all cases)

WP at Start: The QB’s team’s probability of winning at the beginning of the game (calculated from the Vegas spread)

WPA: The QB’s total WPA in the game. This number includes the Vegas spread in the calculations and is generally the number that I prefer in my evaluations.

WPA NoSprd: The QB’s WPA assuming both teams have a 50% chance to win the game at the start. This is useful if you think the Vegas spread is unfairly skewing the numbers in a certain direction.

WPA Sup: The total WPA for the QB’s teams’ defense and special teams. Note that this does not include the team’s own rushing WPA. This stat includes the Vegas Spread.

Value: This is a measure of statistical production used frequently by Chase Stuart in various articles; it’s ANY/A above expectation multiplied by the number of dropbacks (see here for a post ranking Super Bowl performances by Value).

And finally, the table. As you can see, Foles did indeed “outduel” Brady: his WPA is +1.00 and Brady’s is +0.61.

There’s so much here I want to discuss, but since I’ve gone on long enough, I’ll limit myself to four comments:

1. Because WPA is only looking at the impact of plays in the context of winning and losing, it can be inherently unfair. Russell Wilson’s interception at the end of SB 49 was over 4 times as costly as Tom Brady’s two interceptions – combined – in that same game. You may not agree with that assessment, but keep in mind that this stat is supposed to work that way. Brady’s interceptions came earlier, when the outcome of the game was somewhat in doubt, Wilson’s came at a moment when it appeared that the Seahawks had the game all but wrapped up. I’ve already noted this in the “ground rules”, but it’s worth repeating: the QB, Wilson, is taking the full brunt of that WPA loss, there is no parsing to account for receiver error, play calling, great defensive play, etc. But what’s cool is that since we have the effect of the play as a whole, we can divvy that up any way we see fit.

2. Because of this “unfairness”, WPA is not very useful in evaluating quarterback talent. This quote by Brian Burke will put us in the right mindset: WPA is what I call a narrative stat. Its purpose is not to be predictive of future play or to measure the true ability of a player or team. It simply measures the impact of each play toward winning and losing. Wilson’s interception might be the most impactful play in Super Bowl history (I think it is), but the idea that that tells us anything about Wilson’s ability or how he performs in clutch situations is ridiculous.

3. Since everything must always, at some point, be about Brady/Manning: Tom Brady earns his reputation as playing well in the clutch, as he has four appearances in the Top 10 (of course, this will change once we add more games), and two appearances in the Top 3! His best Super Bowl game, to me, is the last one: he is statistically off the chart (Value), and he has a high WPA. Unfortunately for fans of “The Sheriff”, Peyton Manning does not fare so well in this ranking, as he has two performances in the Bottom 5. What’s interesting to me about his Super Bowls is that his best game is probably the one most detractors point to as evidence that he is a choker: SB 44 against the Saints, in which he threw the pick-six. His worst game, against the Panthers in 2015, results in him getting a ring. I never liked QB wins as a stat, at least for small sample sizes like the Super Bowl, but after looking at this data, I now really detest it.

4. Where I think things really get interesting is when we look at WPA Support, and I have to go Brady/Manning here again. It’s been shown by a few analyses – this one by Adam Steele covers regular season numbers, this one, also by Adam, covers the Brady/Manning playoffs, and this one by James Hanson covers support for Brady, Brees, Manning and Rodgers – that generally speaking, Tom Brady has had some solid support (good defenses and special teams) surrounding him for most of his career, and Peyton Manning not as much. But there’s an interesting paradox here: Brady’s support in the Super Bowls, apart from his first one, has generally been disappointing, from a clutch perspective (and from an overall perspective on the last game). In crucial moments, it gives way, setting the stage for Tom Terrific to save the day. On the flip side, Manning has had two Super Bowls where he really benefits from great defensive play; in 2006 against the Bears and 2015 against the Panthers, a game in which it’s safe to say that the defense “won the game”. I think this would be an interesting study just using traditional stats.

Thanks for reading, hope you found this as interesting as I did.

If you can get five people in a room to agree on what a sports dynasty is, you’ll probably have achieved the most miraculous agreement in history since the Congress of Vienna. We know a sports dynasty when we see one (the current Patriots, the New York Yankees, 1990s Bulls, etc.), but it becomes less clear once we attempt to actually define it:,When does the dynasty start? How long must it last? What are the requirements?

In this article on NFL dynasties, FiveThirtyEight does a nice job of negotiating the quagmire by just listing the “best team over any number of years”. ^[1]Their definition of “best” being their ELO ratings. I’m going to do the same thing here, but focusing solely on NFL defenses since the merger (regular season only). The metric is points allowed by the defense (meaning: fumble, interception, kick and punt return touchdowns, and safeties aren’t included), adjusted for era and strength of schedule (basically, SRS ratings). Regular readers may recall that I published these results back in August 2015. To differentiate this stat from Pro-Football-Reference’s DSRS, I’ll call it “DfSRS”.

Below is a table of defensive dynasties, ranging from 1 to 15 years:

I’m sure none of you reading this are surprised by the teams that show up here; all the heavy hitters are there, including the recent Seattle defenses. As noted, dynasties can be hard to define, but we can be sure that being consistent is one of the key ingredients, and the Seahawks defense has certainly been that. Below is a table showing how they’ve ranked in various metrics in the past 5 years:

Pretty impressive, and no doubt, that’s a dynasty. How does their 5-year run compare to the two legendary teams that are ahead of them in that category?

While the Seahawks don’t have those absolutely “elite” regular seasons — the Bears and Steelers both have two years of +8.0, and the Seahawks have none — they have the highest “worst” year out of the three. To underscore how remarkable their run has been, it’s worth taking a look at their contemporaries. Below is the Seahawks’ 5-year run compared to the current 5-year runs of six other teams that have had (or have) good defenses:

Carolina had a fantastic 2013, only to drop to average the following few years; the 49ers were great in the Harbaugh years, dipping in 2014 and then mediocre in 2015; the Bengals, Chiefs and Cardinals have been consistently good but not great, and Denver has two average and below average years sandwiched in between three good/great years. It’s not easy to be consistently great, year after year.

I’ll be watching the Seahawks closely this year. If they can hold teams to around 15 or 16 points per game (no easy task), they’ll replace the Bears as having the best 6-year dynasty, which, to me, is historically significant. Over the last five years, Richard Sherman, Bobby Wagner, and Earl Thomas have been three of the top defensive players in the NFL: Sherman and Thomas are two of just seven defenders with three first-team All-Pro nods since 2012, and Wagner ranks fifth in AV among all defensive players over the last five years.

A few other comments:

• This analysis is doing a big disservice to the Minnesota Vikings by making 1970 the cut-off year. In 1969 the Vikings DSRS was 10.6, and I’m sure if we remove “non-defense allowed” scores that number wouldn’t change much. Including that 1969 season in the survey puts Minnesota at the top of the “Elite” ranking, and would easily give them the best three and four year runs. For future posts, I’ll take a stab at incorporating pre-merger years.

• You might be surprised to see the Ravens showing up to the party so late – they don’t break into the table until we hit the 13-year run. There’s no doubt they are a defensive dynasty, but they had a lot of ups and downs, with great years interspersed with some down years. For example, their 5-year run surrounding their Super Bowl win in 2000 looks like this:

As noted, it’s hard to be continually be great, year after year, for any length of time.

• The 1970s Dolphins are another team that shows up late; as good as they were, they just weren’t as dominant as the other dynasties. If I included the 4th and 5th best dynasties, they’d most likely start showing up in the 4-6 year ranges.

• Other dynasties that aren’t listed but would appear if I showed more than just the Top 3:
  • Jacksonville Jaguars, 2004-2007
  • Los Angeles Rams, 1973-1978
  • New Orleans Saints, 1986-1992
  • New England Patriots, 2003-2010
  • New York Giants, 1981-1990
  • Pittsburgh Steelers, 1990-1996
  • San Francisco 49ers, 1981-1992

• I didn’t run the numbers, but my hunch is that if we keep stretching the length of the dynasty out, the Pittsburgh Steelers would probably continually be at the top once we pass 25 years or so. They really have three distinct defensive dynasties, and when these connect, I don’t see any team topping them:
  • The Steel Curtain years, 1972-1979
  • The Greg Lloyd/Kevin Greene/Rod Woodson years, 1990-1996
  • The Troy Polamalu/Dick LeBeau years, 2004-2012

• If you’re thinking playoff success should be considered when ranking dynasties, you’re 100% correct. I’m working on a way of including that; maybe some kind of point system or perhaps just add in points allowed above expectation for each playoff game, etc. Definitely worth doing for something like this.

Thanks for reading, would love to hear your thoughts.