## The 2013 Playoffs Were Not Very Random

In September 2012, Neil wrote that the NFL playoffs had become more random. And that was three months before Joe Flacco turned into Joe Montana. This year, however, feels like one of the least random playoffs in recent memory. And there’s a good reason for that: it is.

If you know the points spread for a game, you can derive the team’s probability of winning in Excel by using the following formula and typing the spread (with a negative number for the favorite) in cell C2:

=(1-NORMDIST(0.5,-(C2),13.86,TRUE)) + 0.5*(NORMDIST(0.5,-(C2),13.86,TRUE)-NORMDIST(-0.5,-(C2),13.86,TRUE))

Using that formula, the table below shows the winner of each game in the 2013 postseason, sorted in order of ascending pregame win probability.1 There was only one big upset this year, the Chargers victory in Cincinnati. Conversely, the least surprising outcome was San Diego’s loss in Denver the following week.

WinnerH/RLoserRdLinePregame WP
SDG@CINW730.7%
NOR@PHIW2.542.8%
INDKANW050%
SFO@CARD-1.554.3%
SFO@GNBW-2.557.2%
SEASFOC-3.560%
DENNWEC-5.565.4%
NWEINDD-769.3%
SEANORD-7.570.6%
DENSDGD-871.8%

Assuming independence, the above odds would imply a 0.28% chance of the 10 games going exactly the way they did. That number doesn’t mean much in the abstract — the chance of any 10 games going in one specific direction is really low — but as it turns out, that is a pretty large number, relatively speaking.

I used this same method to measure how likely or unlikely each set of 11 playoff games has been in each year since 1990. To make an apples-to-apples comparison, I have assumed that Denver wins the Super Bowl and that the Broncos are a 2-point favorite.2 If that happens, then there would have been a 0.157% chance of the playoffs finishing the way they did (if Seattle wins, that would make the playoffs a bit more random and drop the percentage to 0.124%). The graph below shows the likelihood of the results of each postseason since 1990. The odds are located on the Y-Axis, the year on the X-Axis.

This jives with what Neil discovered: the results from 2005 to 2011, with one exception, were pretty unlikely. And 2012 followed that same pattern. On the other hand, the 1991 playoffs bears no resemblance to modern playoff football.

That year, Washington was a 7-point favorite in the Super Bowl and won over Buffalo. Two weeks earlier, Washington was a 14-point home favorite against the Lions and Buffalo was a 12-point home favorite against Denver. In the division round, Washington and Buffalo were both double-digit favorites, too. In addition, Denver won as a 3.5-point favorite, while the Detroit/Dallas game was a pick’em. The Wild Card round featured a little suspense, as two road dogs won (Dallas (+3) over Chicago and Atlanta (+6) in New Orleans), while Kansas City (-5) and Houston (-9) won as big favorites. But that was as non-random as the NFL playoffs can get.

What about 2006? Why does that year appear to be not very random? Indianapolis was as a 6.5-point favorite in the Super Bowl over Chicago, and the Bears and Colts both won as three-point favorites two weeks earlier. In the division round, Chicago (-8.5) and New Orleans (-4.5) won as favorites, while Indianapolis (+4) and New England (+5) won as dogs. Yes, even Tom Brady and Peyton Manning can be underdogs. The Wild Card round that year was all chalk: Seattle (-2), Indianapolis (-6.5), Philadelphia (-6) and New England (-9).

Astute readers have probably picked up on something: the spreads have been very low during the 2013 playoffs. In fact, the average favorite has been giving only 4.3 points (including an expected 2 points in the Super Bowl). That’s the lowest since 1992 (4.1). From 1990 to 2012, the average spread was favorite -6.1, so 2013 does represent a pretty significant departure.

One could reasonably argue that the lack of big favorites means the method I’m using understates how predictable this year’s playoffs have been. My formula considers a win by a 3-point favorite to be more random than a win by a 7-point favorite (a defensible position, but a matter of taste). So consider this: only two underdogs have won in the entire postseason. At no point since 1988 has there been a postseason with fewer than two upsets. And there was three or more upsets in every postseason but from 1999 to 2012. So in that regard, 2013 does stand out as a significant outlier; of course, Seattle could still make it three upsets. 3

Here’s another way in which this year’s playoffs have been unsurprising: for the first time since 1993, there has been only one dog of more than 3 points to pull off a playoff upset. The table below shows, for each postseason since 1990, the probability of the playoffs turning out the way they did (this is the same data from the graph above but in table form), the number of upsets in the playoffs, the number of upsets by teams that were more than 3-point underdogs, the spread of the biggest underdog in the playoffs to win that season, and the average points spread of each game.4

YearProbUpsets> 3BiggestAvg
20130.157%2174.3
20120.09%3396
20110.071%5386.4
20100.028%52104.7
20090.048%5394.9
20080.04%53104.6
20070.019%5412.58.4
20060.278%2255.3
20050.057%528.54.8
20040.163%336.56.1
20030.107%3274.5
20020.199%336.55.8
20010.035%42146.5
20000.04%6364.6
19990.23%3376.8
19980.275%22118.3
19970.114%32115.7
19960.152%3312.58.6
19950.122%439.58.4
19940.352%2267.3
19930.415%2176.3
19920.099%4344.1
19910.572%2167.4
19900.125%3386

The average point spread has been very low in 2013, but the favorites are still winning. In that regard, this year has been very similar to the 1992 postseason. Low points spreads indicate uncertainty and are a reflection of parity, so one would think randomness would increase in that environment. But there has only been two upsets in the playoffs, and one of them came by a team with a better record than its opponent.

1. All points spread data from Pro-Football-Reference.com. []
2. Note to Seattle fans: I have assumed that Denver wins not because they’re the favorite and it would minimize randomness, but because of a personal vandetta I have against the Seahawks the city of Seattle Pacific Northwest. []
3. I will note that some outlets had the Chiefs as favorites by gametime, which would mean 2013 already has had three upsets. []
4. This is from the perspective of the underdog, not the losing team. In other words, it doesn’t matter who won or lost the game when deciding the average points spread. []
• Kibbles

On the other hand, we were expecting them to be random based on recent history, so the most random thing they could have possibly done was not be random at all!

Well played, playoffs. Well played.

• Nate

> … My formula considers a win by a 3-point favorite to be more random than a win by a 7-point favorite (a defensible position, but a matter of
> taste). …

I’m not sure what “more random” means. Certainly, 3 point favorites are less likely to win than 7-point ones.
The relationship to the money line suggests that the win percentage is around 1/(1+exp(-spread / 7) , though football scoring makes things a bit more complex.

> … Low points spreads indicate uncertainty and are a reflection of parity, so one would think randomness would increase in that environment. …

If we were to plot a distribution of expected scores, then the spread should be at the median – that is to say where the cumulative distribution hits 0.5 . Even if we assume that the distribution is roughly normal, we’d have to get a second measurement (like the money line) to get a good idea of what the uncertainty is.

It seems like the formula is tracking disparity in team strength more than it’s tracking upsets: Naively speaking, with 2048 scenarios, it seems like anything more than a 0.05% to occur would be considered reasonably likely.

• Chase Stuart

I’d probably double that and say anything more than 0.1% would be considered reasonably likely. It’s not as though each game is a coin flip, and a few games seem reasonably simple to predict. But prior to 2013, seven of the last 8 years fell below the 0.10% threshold.

The formula tracks both the disparity in team strength and upsets, I think.

• Nate

> I’d probably double that and say anything more than 0.1% would be considered reasonably likely. It’s not as though each game is a coin flip…

Let’s assume a somewhat more realistic scenario where every favorite is 2:1 to win. With 11 games we’d expect 11/3=3.3 upsets on average – so I’d call 4 upsets ‘reasonably likely’. (At least 4 upsets happens a bit less than 1/3 of the time.) The chance of any particular 4-upset scenario is 2^7/3^11 = 0.072%. If ‘reasonably likely’ is loosened to about 1 in 8, then 5-upset scenarios with individual likelihoods of 0.036% would qualify.

• Richie

> the average favorite has been giving only 4.3 points (including an expected 2 points in the Super Bowl). That’s the lowest since 1992 (4.1)

Would “median” point spread in the playoffs be more informative than “average”? All it takes is one 14-point game to throw off the average. Or maybe there aren’t enough huge spreads to make much difference.

• Another thought. Would there be a way to weigh the seedings in this analysis (or even playoff round)? Would that tell us anything more? I know the seedings are implicitly included in the point spreads. But, San Diego upsetting Cincinnati is the biggest upset, and that upset doesn’t really resonate much because Cincinnati was a 3 seed, and I think most people were not terribly surprised to see the Bengals lose.

It’s when the 1st and 2nd seeds lose games that really make the playoffs seem “wild” (like Denver losing last year or the Patriots getting pounded by the Jets a couple years ago). Although this year’s #2 seed losing to San Francisco was mitigated by the fact that San Francisco was actually favored in the game.

• George

Just a number of quick things on this;

Re: Cincinatti (and I will digress), I remember the post on predictions before the playoffs and I remember commenting afterwards – I think only three of about 20 of us picked the Chargers to win that one. I’m shocked that this was the only real upset of the play-offs. As a general rule, I’ve avoided the play-offs for the last couple of years as random things happen (e.g. Denver vs Baltimore last year). I’ve kept track of the ratings (or saved the numbers at least) since after week 8 (when they start coming together) and Denver and Seattle have usually been 1st or 2nd since then (with a little bit of Carolina mixed in). I think it’s no surprise to any of us that they are heading to the Superbowl but what js a surprise is that something random hasn’t happened to either of them on the way.

Re: working out the percentage chance of an upset by a given margin in a particular game (and I know this is slightly different to the above and is assuming that the ratings are accurate and other things), Neil explained this in a comment to a post that I’d commented on earlier this year (where from memory I was quite troubled by either an incident involving either the Jets or the Vikings and was before the Patriots vs the Browns game which also troubled me a lot). I know there have been quite a few games where the ratings would have suggested an 80-90%+ chance of covering that haven’t come in this year (which has just felt odd to me and has made me question the model in principle).

Re: the spread; for the Superbowl whatever way you work it (weighted or unweighted) I have Seattle as the better team by roughly 2 (which would be roughly right on several firms opening lines – I know Cantor properties in Vegas reportedly opened it as a Pick). I still can’t figure why everyone is favouring Denver so heavily (especially given the possible weather issues which I think will affect them more). In terms of the parity point – can we really consider the data from the early 90’s as this was pre the Salary Cap (which I think will have probably brought teams closer together I’m guessing in terms of talent)? I’ve just found the lines this year in the play-offs (and over the last couple of years in the play-offs) to be fairly accurate in terms of what you would expect if you were setting them based on a ratings system (e.g. SRS, Winston/Stern etc.). On that basis the ratings seemed to predict the right team would win (and it would be interesting to know how that approach would have faired over the whole season) I think the shock is how much the team won by in certain cases. I don’t know if the line making has got sharper over the last two decades (and the line when working out the percentage chance is relevant) but I definitely believe gamblers are sharper now and I expect that bookmakers have adjusted their line making approach to compensate for this (but I’m not sure how that would affect the figures).

• Kibbles

I wonder how much of the action on Denver owes to Russell Wilson’s weak play in recent weeks.

Also, I don’t think its as easy as saying that the potential weather favors Seattle. From what I’m reading, there’s a decent chance of snow. Snow on the ground doesn’t affect the flight of the ball, but it does make footing more treacherous. Treacherous footing favors the offensive players, because they know where they’re going ahead of time and can anticipate the cuts while the defender is backpedaling and reacting. For examples, think of Tom Brady throwing 6 TDs in the first half against the Titans, or think of the weekend this year where that freak blizzard hit the Eastern seaboard and the league set new single-weekend scoring marks.

In theory, something that benefits the offense benefits both teams, since they both have offenses. In practice, if I told you that there was an increased chance that the superbowl would be a shootout, that would probably cause you to adjust your estimation of Denver’s chances marginally upward.

• George

I think that you are right re: Russell Wilson’s play in recent weeks, but I don’t know why I’m still going to go with Seattle. I’m just amazed at the amount of money that must have been thrown at it to shift the line by up to 4 points in some books. I just don’t think that the oddsmakers would have been that wrong. I’d love to know what they are seeing and what the public is seeing as it is clearly different.

Re: my numbers from last night – I was slightly off, unweighted I have Seattle by give or take 2, weighted by about 0.5 a point. I just think that they are the better team. I haven’t looked at it but I would be keen to take a quick look at who’s defences in terms of rankings the Broncos have come up against during the season and offences (with respects to scoring on their defence) as my gut feeling is that it won’t quite match up to what they get against Seattle.

Re: the snow, I don’t think it comes. I think you have some rain in patches, cold (to the point where it may affect some players – I can’t remember what the statistic is when Manning is wearing a glove in cold weather) and the possibility of gusts of wind (to the point where it may affect deep ball plays). I’ve never been to Denver or Seattle I just feel that the conditions may suit Seattle better than Denver.

• Richie

I’d love to know what they are seeing and what the public is seeing as it is clearly different.

I think Peyton Manning is just a much more famous QB than Russell Wilson, plus good offense is more tangible than good defense. So the general public is going to be more inclined to bet on Denver. We’ll see if the line holds in Denver’s favor as the fortnight progresses. It wouldn’t surprise me if the big money professional gamblers push the line back towards Seattle as the game nears. Although, for a professional gambler I assume picking a side in the Super Bowl is “just another game to bet”, so maybe there is not enough professional action on the game to counteract the massive amounts of general public money that will be bet.

• Nate

Ostensibly, the sharps want to be where the suckers are, so when there’s lots of general public action, there should be additional sharp action as well.

• Rob Harrison

“Note to Seattle fans: I have assumed that Denver wins not because they’re the favorite and it would minimize randomness, but because of a personal vandetta I have against the Seahawks the city of Seattle Pacific Northwest.”

LOL . . .