≡ Menu

Betting Bad: Thinking About Uncertainty in Prediction

Barack Obama was not the only winner in the 2012 presidential election. Nate Silver, now founder and editor in chief of Five Thirty Eight, and other stats-y election forecasters basked in the praise that came when the returns matched their predictions.

But part of the praise was overstated. At the very end, Silver’s models essentially called Florida a toss-up, with the probability of an Obama win going just a few tenths of a percentage point above 50%. But because his model gave Obama the slightest of edges in Florida, his forecast in most of the media essentially became a predicted Obama win there. In addition to accurately forecasting the national popular vote, Silver then received credit for predicting all fifty states correctly.

I am all in favor of stats winning, but the flip side of this is the problem. If Obama had not won Florida, Silver’s prediction―which, like that of other forecasters such as Sam Wang of the Princeton Election Consortium, was excellent―would have been no less good. [1]This is a column about football, but you might want to check out some of the stuff through that link on the differences between Silver and Wang on the upcoming midterm elections. They both know way … Continue reading And if stats folks bask too much in the glow when everything comes up on the side where the probabilities leaned, what happens the next time when people see a 25% event happening and say that it invalidates the model? [2]Of course, maybe Football Outsiders has already run into that with the 2007 Super Bowl prediction. Perhaps sports people are ahead of politics on this stuff.

Lots of people have made this point before — heck, Silver wrote about this in his launch post at the new 538 — but it is really useful to think carefully about the uncertainty in our predictions. Neil has done that with his graphs depicting the distribution of team win totals at 538, and Chase did so in this post last Saturday. Football Outsiders does this in its Almanac every year, with probabilities on different ranges of win totals.

But there are really two kinds of uncertainty that we want to think about: uncertainty in the outcome and uncertainty in the model. Silver’s model had an uncertain outcome for Florida. It was almost equally likely that Obama or Romney would win the state. I don’t think he had much uncertainty about the model, though. There was so much good polling data over a long period of time, that Silver likely felt confident that his 50% estimate was close to Obama’s true probability of winning.

Football is very different. Obviously, there is even more uncertainty about who is going to win the Super Bowl. Football Outsiders currently gives Denver the best chance at 19.2%, so that shows how there is a ton of uncertainty about who will win it all this year. According to their numbers, you should take the field over their top four teams. Together, those four teams have just a 47.2% chance of winning the Super Bowl.

Even beyond that uncertainty in the outcome, there is a great deal of uncertainty about whether 19.2% is the right number for Denver. For an election, it is pretty easy to know how strong a candidate is at any point in time. For a football season, it is pretty hard and particularly hard at the beginning of a season. [3]On the other hand, 538’s new Elo Ratings may provide some guidance early in the year. Because of this idea, you’re probably even more advised to take the field over the top 4, since the uncertainty in the model means that some of those lower teams are probably underrated.

The neat thing is that all the smart people in football analytics get that predicting football is so hard as almost to be foolish to try. In the past, just understanding how little we knew was enough to find profitable football bets. For example, the easiest way to win on season win totals was to take the over on the worst teams from the previous season. Most people thought they knew how bad those teams would be, but the smarter folks understood that regression happens and so your model of those bad teams better have a healthy dose of uncertainty built in. [4]Note that I am not talking about me since I did not bet on football then.

For the first five years of data that I have, here are the 21 teams that had an over/under win total of 6 or less along with the result for that season. [5]This is 2002-2007 with 2005 missing because thelogicalapproach.com did not have the data posted for that year.

YearTeamWin TotalOver LineUnder LineResult
2002CAR5.5-145115Over
2002DET6115-145Under
2002HOU4.5-100-130Under
2003ARI5.5150-180Under
2003CIN5.5-130-100Over
2003DET6-145115Under
2003HOU5-110-120Push
2004ARI5-170140Over
2004HOU6-175145Over
2004SDG4.5-115-115Over
2004SFO5.5110-140Under
2006GB6-160140Over
2006HOU5.5-150130Over
2006NYJ6105-125Over
2006SFO5-110-110Over
2006TEN5.5115-135Over
2007GB6-160140Over
2007HOU5.5-150130Over
2007NYJ6105-125Over
2007SFO5-110-110Over
2007TEN5.5115-135Over

You would have gone 15-5-1 betting the over on these teams. I made the decision up front to just look at the first five years of the data here, but you would have gone 4-1 in the next year (2008), too. [6]Chase note: I have wins total data for the ’05 season, courtesy of Wager Minds. The Browns (4.5 projected wins, 6 actual), 49ers (4.5, 4), and Dolphins (5.5, 9) from that season would have … Continue reading

Of course, those easy opportunities are now mostly gone. But it is pretty fantastic that Vegas rewarded humility so well. Understanding how random the NFL is was recently a moneymaker.

Randomness is endlessly fascinating. It’s sort of easy to assume that randomness means that teams ought to end up towards 8-8. That’s true in terms of regression, but randomness also means weird 2-14 seasons like Houston’s last year and shocking 14-2 seasons by teams such as the 1998 Falcons. And NFL games are not independent random events. Injuries happen and can torpedo seasons. Some team is going to lose repeatedly because of a bad draw from the injury deck, such as the 2011 Colts.

I think we are still underselling the chance that some team wins more than 14 games and some other team wins 2 or less. Yes, it looks like my choices for those teams just might be the wrong ones, but I think we should still be looking for randomness pushing some potential candidates towards the extremes for records.

The Stone-Cold (I Think There May Be a 60% Chance This Bet Will Win) Mega-Lock of the Week

Yes, I am a complete moron. I picked the New England Patriots and Green Bay Packers to lose a combined three games all year. They have already lost two. I picked the Arizona Cardinals to win two games. They have already won one. [7]This game left me thinking for a while. Most of all, I can’t understand how San Diego could be completely unprepared for the double A-gap blitzes that Arizona brought down after down. And yes, I … Continue reading My first mega-lock was a two-team teaser that was saved on one end by a garbage-time touchdown by the Raiders, only to lose when the Bills finished running over the Bears (5.8 YPC on 33 carries).

As I said on Sunday, that was a queasy-knees bet. I felt queasy specifically about Jay Cutler and he did not let me down, throwing a terrible pick that helped cost the Bears the game. This week, though, I have a bet that induces no quavering in the lower extremities. In fact, this is probably going to be one of the bets I feel most confident in this season.

Two-team teaser: Pittsburgh Steelers up to getting 8.5, Green Bay Packers down to giving 2.5

My only reservation on this bet is how shaky Pittsburgh’s front seven might be. Baltimore is among the teams least likely to kill the Steelers on the ground, however. Since Joe Flacco became the Ravens’ quarterback, the Steelers and Ravens have played 14 times and the margin of victory has exceeded 8.5 points only twice. One of those was the Steelers’ 9-point win in the AFC championship game in Flacco’s rookie year. An amazing ten of the 14 games have been decided by a field goal or less, including the last five. Since 2009, the line has been three or lower for every game except the one Charlie Batch started for the Steelers in 2012. I think the Steelers side of this bet is a little less good than it would have been in past years when these were two great defenses, but it still seems pretty sweet.

And I promise not to continue betting against the Jets every week, but getting Green Bay across the 3 and 7 mileposts is too enticing to pass up.

Season Mega-Lock Record: 0-1

References

References
1 This is a column about football, but you might want to check out some of the stuff through that link on the differences between Silver and Wang on the upcoming midterm elections. They both know way more than I do, but for the small amount that it is worth, I lean more towards Wang on this one.
2 Of course, maybe Football Outsiders has already run into that with the 2007 Super Bowl prediction. Perhaps sports people are ahead of politics on this stuff.
3 On the other hand, 538’s new Elo Ratings may provide some guidance early in the year.
4 Note that I am not talking about me since I did not bet on football then.
5 This is 2002-2007 with 2005 missing because thelogicalapproach.com did not have the data posted for that year.
6 Chase note: I have wins total data for the ’05 season, courtesy of Wager Minds. The Browns (4.5 projected wins, 6 actual), 49ers (4.5, 4), and Dolphins (5.5, 9) from that season would have also met Andrew’s criteria. Therefore, from ’02 to ’08, betting the over on teams with over/unders of 6 or fewer wins would have given you a 21-7-1 record. That’s pretty darn good.
7 This game left me thinking for a while. Most of all, I can’t understand how San Diego could be completely unprepared for the double A-gap blitzes that Arizona brought down after down. And yes, I think I’m done on my Arizona for worst record bet, but there was this team that overcame a double-digit fourth quarter deficit to the Chargers in Week 1 last year.
{ 12 comments }