≡ Menu

We don’t know anything and we never will

Five weeks in, you start to hear NFL experts trade their preseason overconfidence for regular season overconfidence. It’s tempting to fall into the trap thinking that with over a quarter of the season in the books, now we have an idea of how the rest of the regular season will unfold.

It’s tempting, but it’s not really true. The best way to measure whether someone knows what they’re talking about is to see if their predictions come true. Fortunately for us historians, each game a group of experts predict what will happen every week — it’s called the point spread.

Assuming we actually learn something each week, then the point spreads should reflect the actual results as the leaves change colors. But do they?

I looked at the point spread for all regular season games from 1988 to 2011. Now the question becomes how do we measure if a point spread was “right”? If the Texans are favored by 7 to beat the Bengals, and they win by 10, is that a “good” projection?

The simplest way to test this is to see the difference between the actual result and the projected result. The line that was most “off” in the database came two years ago. Likely due to the genius of Josh McDaniels, the Raiders were 7-point underdogs in Denver in 2010, but won the game 59-14. So in that game, the line was off by 52 points. Last year, the 49ers were 3-point favorites at home against Tampa Bay, but won by 45 points; that line was “off” by 42 points.

One interesting sidenote: you might think that big lines are more likely to be off by bigger margins than small lines. But that’s not really the case. The standard deviation of “how much a line is off by” is roughly 8 points regardless of the spread. It’s not exact, of course, but for our purposes, we can work under the assumption that lines are generally equally likely to be off by the same amount regardless of the spread.

Anyway, back to the point of the post. How accurate are lines early in the year? In week 1, the spreads are generally a little tighter than they are the rest of the way; perhaps the oddsmakers are just as unsure as the rest of us. But no matter what week it is, the lines are always off by about 10 points per week. Take a look. The table below shows how much the lines were off by, on average, in weeks 1 through 17 from 1988 to 2011. In the last column, I’ve shown the percentage of games where the line was within 10 points of the actual result.

wk#gmsSpreadLineOffw/i 10pts
13624.710.760%
23615.210.558%
33405.610.456%
43225.510.956%
53195.21064%
63155.41060%
73155.610.957%
83155.510.659%
93265.310.559%
103395.610.157%
113585.69.461%
123625.71062%
133635.410.159%
14360610.859%
15363610.957%
163635.710.957%
173355.710.461%

It’s tempting to think we know more once we see more, but that’s unfortunately not the case. Of course, if it was, it would be easy to make money gambling on football.

For those curious, week 12 of the 2003 season was as close as Vegas has ever come to perfection. Look at how close these lines were to the actual results (as always, the boxscores are clickable):

GameYrFavUnderlineFavPtsUnPtsLineOff
2003 rav -3 vs. sea2003ravsea344410
2003 tam -6 vs. nyg2003tamnyg619130
2003 clt -3 vs. buf2003cltbuf317140
2003 min -10.5 vs. det2003mindet10.524140.5
2003 oti -6.5 vs. atl2003otiatl6.538310.5
2003 nyj -4 vs. jax2003nyjjax413101
2003 dal -3 vs. car2003dalcar324201
2003 nwe -5 vs. htx2003nwehtx523202
2003 cin -3 vs. sdg2003cinsdg334274
2003 ram -7 vs. crd2003ramcrd730274
2003 mia -7 vs. was2003miawas724236
2003 gnb -3.5 vs. sfo2003gnbsfo3.520106.5
2003 phi -5.5 vs. nor2003phinor5.533207.5
2003 kan -11 vs. rai2003kanrai1127248
2003 cle -3 vs. pit2003clepit361310
2003 den -10.5 vs. chi2003denchi10.5101919.5

What about the other side of the coin? Vegas was really, really, really off in week 17 of the 1993 season. Just so you know, 1993 was the year the NFL tried the double-bye week approach resulting in an 18-game season, so this was really like a week 16 most years:

GameYrFavUnderlineFaPtsUnPtsLineOff
1993 nwe -6 vs. clt1993nweclt638032
1993 cle -2 vs. ram1993cleram2421426
1993 gnb -3 vs. rai1993gnbrai328025
1993 sdg -1 vs. mia1993sdgmia1452024
1993 kan -3 vs. min1993kanmin3103023
1993 den -13.5 vs. tam1993dentam13.5101720.5
1993 dal -17 vs. was1993dalwas1738318
1993 nyg -3 vs. crd1993nygcrd361714
1993 pit -3 vs. sea1993pitsea361613
1993 sfo -7.5 vs. oti1993sfooti7.571010.5
1993 chi -3 vs. det1993chidet314209
1993 phi -3 vs. nor1993phinor337268
1993 atl -3.5 vs. cin1993atlcin3.517217.5
1993 buf -7 vs. nyj1993bufnyj716145
{ 8 comments }