Previous post:

Next post:

Checkdowns: How Much Does Last Year’s Record Matter When Predicting This Year’s Games?

by Neil Paine on November 1, 2012

in Checkdowns, Power Rankings, Statgeekery

In Tuesday’s post, I outlined a method of regressing a team’s record to the mean to estimate its “true winning percentage talent” (the trick is to add eleven games of .500 ball to their record, at any point in the season). In the comments, FP reader Dave wondered if we could incorporate last year’s true WPct talent into our talent assessment for this season, so I thought I’d run a quick regression to look at that.

My dataset was simply every game from 2003-2012 (including Monday night’s game). For each game, I recorded:

  • Whether the game was a win, loss, or tie for the team in question. Wins got you a “1″, ties a “0.5″, losses a “0″.
  • The team’s WPct talent estimate going into the game. So in the first game of the season, that’s (0+5.5)/(0+11)=0.500 for everybody; meanwhile, for an 11-4 team going into the final game of the season, it’s (11+5.5)/(15+11)=0.635.
  • The team’s WPct talent estimate from the previous season.

I then set up a logistic regression to predict whether the game was a win or a loss based on the two WPct talent variables, this year and last year:

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.7686  -1.1489   0.1616   1.1429   1.7072  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept)    -2.6936     0.1982 -13.589  < 2e-16 ***
currenttalent   4.0297     0.3509  11.485  < 2e-16 ***
prevtalent      1.3571     0.2666   5.091 3.57e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 6712.4  on 4843  degrees of freedom
Residual deviance: 6508.0  on 4841  degrees of freedom
AIC: 6516.1

Number of Fisher Scoring iterations: 4

That means to predict your likelihood of winning any given game, you plug your WPct talent numbers from this season and last season into this formula:

WPct ~ 1 / (1 + EXP(2.693606 - 4.029688*(Current_Talent) - 1.357123*(Prev_Talent)))

It's important to note the size of the coefficients here -- the current WPct talent coefficient is three times as big as that of last season's WPct talent, so it has much more bearing on the prediction.

At any rate, here are the probabilities of winning any given game that this formula implies for this year's teams:

Year
Team
Games
Wins
Current_Talent
Prev_Talent
p(W_any_gm)
2012atl770.6940.57470.8%
2012sfo860.6050.68566.3%
2012htx760.6390.57465.9%
2012gnb850.5530.75963.7%
2012chi760.6390.50063.6%
2012rav750.5830.64863.1%
2012nyg860.6050.53761.6%
2012nwe850.5530.68561.4%
2012pit740.5280.64857.8%
2012den740.5280.50052.8%
2012mia740.5280.42650.3%
2012crd840.5000.50050.0%
2012det730.4720.57449.7%
2012min850.5530.31549.0%
2012sea840.5000.46348.7%
2012cin730.4720.53748.5%
2012nor720.4170.68547.9%
2012dal730.4720.50047.2%
2012phi730.4720.50047.2%
2012rai730.4720.50047.2%
2012sdg730.4720.50047.2%
2012oti830.4470.53746.0%
2012clt740.5280.27845.3%
2012nyj830.4470.50044.7%
2012buf730.4720.42644.7%
2012tam730.4720.35242.2%
2012was830.4470.38941.0%
2012ram830.4470.27837.4%
2012kan710.3610.46335.2%
2012cle820.3950.35234.9%
2012car710.3610.42634.1%
2012jax710.3610.38932.9%

{ 11 comments… read them below or add one }

Dave November 1, 2012 at 11:33 am

I’m not sure I see how the weight given to LY’s ratings decline as the season progresses. It should do that shouldn’t it?

Reply

Neil November 1, 2012 at 11:40 am

It essentially happens automatically. The variation in current WPct talent will increase as the sample increases and we have more certainty about how “true” the talent we observe is. In week 1, all teams’ current WPct talent is .500, so any variation in expected win probability comes from last year’s WPct talent. The more current games you add, the more the probability rating is a product of the variation in this year’s performance, rather than last year’s.

Reply

Dave November 1, 2012 at 12:17 pm

Would it be possible to calculate just how the effective weighting of the two changes from week to week?

Reply

Dave November 1, 2012 at 11:54 am

I see what you mean. I’m just having a hard time wrapping my head around the specific numbers. The current season doesn’t reach that 3x ratio until the season has gone on long enough for this season’s variation to match LY’s. It just seems like LY is weighted too heavy for too long.

I’ve attacked this problem in a different manner and my work usually shows LY becoming a factor that affects current rating on a minor basis by about game 7 of the current season.

Reply

Richie November 1, 2012 at 2:02 pm

Neal,
I was going to use your formula to try and compare how the formula predicted games so far this year to how the games played out. (I assumed I would do this by taking each team’s p(W_any_gm) before each game, then recalculate after the actual W/L, and add these together for the 7/8 games so far.

But I tried using your formula in Excel, and my numbers aren’t matching. I don’t know if my order of operations is wrong or if I’m not understanding your formula.

Your formula: WPct ~ 1 / (1 + EXP(2.693606 – 4.029688*(Current_Talent) – 1.357123*(Prev_Talent))
my Excel formula: =1/(1+(EXP(2.693606-(4.029688*B1)))-1.357123*C1) where cell B1 = Current_Talent, C1 = Prev_Talent

Reply

Neil November 1, 2012 at 2:16 pm

For, say, the current Falcons, are you plugging in (5.5+7)/(11+7)=0.694 as Current_Talent and (5.5+10)/(16+11)=0.574 as Prev_Talent, like so?

1 / (1 + EXP(2.693606 – 4.029688*(0.694) – 1.357123*(0.574)))=0.707

(It doesn’t match the 70.8% from the table exactly because of rounding.)

Reply

Richie November 1, 2012 at 2:20 pm

Thanks. Order of operations error. I didn’t realize everything to the right of EXP was part of the exponent.

Reply

Neil November 1, 2012 at 2:19 pm

Oh, I see, your parentheses are messed up… Should be =1/(1+(EXP(2.693606-(4.029688*B1)-(1.357123*C1)))

Reply

Richie November 1, 2012 at 2:43 pm

Alright, so I ran the numbers for the NFC South. Maybe I am mis-applying the formula, or don’t understand the intent. But it looks to me like this system is just going to end up predicting that everybody should be close to .500.

I have:
Atlanta – 4.3 wins
TB – 3.0
NO – 3.4
Car – 2.9

Atlanta had a .525 probability of winning week 1. They had a .567 probability for week 2. Then it was: .601, .630, .654, .675, 0 (bye) .692. Add those up and it’s 4.3.

Since everybody starts the season with a current probability of .500, and every team is currently between .708 and .329, and those values slowly rose/fell to those levels, every team is going to have expected wins between about 2.5 and 4.3 through 8 weeks.

Am I using the information properly?
So what does this information actually tell us?

Reply

Neil November 1, 2012 at 3:06 pm

It’s not really intended to be retrospective like that. It’s predictive — it’s meant to temper what we might think about teams with certain records at a glance (OMG Atlanta 7-0!!) by heavily regressing those records to the mean. In other words, it tells us how much we really know about a team’s true quality from its W-L record. (The 2-year system from this post is just an extension of that theory, since Dave was curious.)

Continuing with the Falcons example, even being 7-0 (on the heels of a 10-6 season) doesn’t actually actually prove they’re anything more than a “true” 11-5 caliber team… Good, but not great. You’re not necessarily learning anything about the relative order of teams from this (although the 2-year version does a good job integrating last year’s performance into what we’ve learned so far this season), but you are learning about the relative talent levels of teams going forward. The spread of talent is nowhere near as wide as we might think it is from raw W-L records.

Reply

Mike November 2, 2012 at 4:42 pm

Very interesting. But, since points scored and allowed are a better predictor of future win/loss record (than current win/loss record) wouldn’t it be more accurate if you used those? Is there some sort of magic number, much like the +11 games of .500 football, that would apply there? Or would it just be average points per game x 11?

Reply

Leave a Comment

You can use these HTML tags and attributes: <strong> <em> <pre> <code> <a href="" title="">