Today I want to look at how traditional rushing statistics compare to rushing Expected Points Added, one of the main stats used over at Advanced Football Analytics. In my analysis, I used the EPA numbers for each team in each season from 2002 to 2013.

**Stickiness from year to year**

Yards per carry is not a sticky metric: by that, I mean, it is not very consistent from year to year. The correlation coefficient between a team’s yards per carry in Year N and yards per carry in Year N+1 was just 0.31. Sometimes the square of the correlation coefficient is described in terms of “explanatory power”: loosely speaking, this means roughly 10% of a team’s YPC average in Year N+1 can be explained by its YPC average in Year N.

Now, a lot of metrics aren’t sticky from year to year, because the NFL is a highly competitive league. In fact, Rushing EPA per play has a *lower* correlation coefficient from year to year at just 0.30. That’s a strike against EPA. On the other hand, Burke’s success rate metric has a CC of 0.39, which is more impressive. The CC for Net Passing Yards per Attempt year over year is 0.43.

**Correlations between EPA and Other Metrics**

Let’s switch from the same metric in different years to different metrics in the same year. The CC between EPA and rushing yards per carry is 0.77. That sounds pretty high, but the CC between EPA and rushing yards — that is, comparing an efficiency statistic to a gross one — is 0.70.

The CC between EPA and success rate is 0.80; I’m not quite sure what to think about that. If we add a 20-yard bonus for every touchdown, and calculate the CC between Rush EPA and this adjusted form of YPC, the CC is 0.83. But if we also provide a 9-yard bonus for first downs, that vaults the CC to 0.87. In other words, this provides support for the great work Brian Burke produced a couple of weeks ago. Using Adjusted Yards per Carry, with a 20-yard bonus for touchdowns and a 9-yard bonus for first downs, corresponds very well with AFA’s EPA metric.

**Regression Analysis**

What if use three variables — yards per carry, touchdowns per carry, and first downs per carry — to predict EPA? We get the following formula using a linear regression:

EPA/Carry = -0.49 + 0.058 * YPC + 1.29 * TD/Carry + 0.90 FD/Carry

The R^2 is 0.87, but the more important thing we can tease out of the results is the relationships among the coefficients. For example, the weight on the TD/Carry variable is 22.3 times as large as the weight on the Yards/Carry variable. And the weight on the First Downs/Carry variable is 15.5 times as large as the weight on the Yards/Carry variable.

Does this mean the 20 and 9 weights are wrong? Perhaps, although note that the correlation coefficient is the same using either weight. ^{1} This means that while the regression implies that different weights should be used, changing the weights doesn’t seem to make much of a difference. I suspect that the weights are higher here because first downs and touchdowns are correlated with other types of good play, so I’m okay for now keeping the weights at 20 and 9.

What if we perform the same regression, but use success rate as out dependent variable instead of EPA?

Success Rate = 16.8 + 0.92 * YPC + 45.9 * TD/Carry + 81.2 * FD/Carry

Look at how unimportant the YPC variable is to predicting success rate: going from 3.5 YPC to 5.5 YPC only predicts an increase in success rate of 1.8%, assuming first down and touchdown rates remain the same. The coefficient on the first down variable is 92.4 times as large as the variable on yards, while the variable for TDs is 49.8 times as large as the yards variable (presumably, this is because touchdowns are much more infrequent than first downs, so knowing the number of touchdowns a team scores won’t tell you much about the team’s overall success rate).

I don’t have too much else to add, so let me know your thoughts in the comments. I have a lot of reservations about using YPC, but I think AYPC is a big step in the right direction. And for those curious, the correlation coefficient between AYPC in Year N and AYPC in Year N+1 is 0.35.

- Okay, it’s slightly higher: 0.874 to 0.870. [↩]

{ 28 comments… read them below or add one }

If I remember from the GQBOAT project, a fumble is worth -35 yards. Is there any way you could incorporate that into the equation?

For recent seasons, yes. But historically, we don’t have a way of knowing which fumbles occurred on rushing plays and which did not.

One conceptual problem I have with TDs/carry and 1st down/carry is that they are easily gamed stats since I believe the majority of TDs/1st downs are short yardage plays – plays that you would expect a player (on average) to convert. Given a certain number of carries it is hard to game YPC.

For example, Adrian Peterson has a career 5.0 YPC and a career 1st down rushing % of 23.6% and a 4.2% TD%

OTOH, Tom Brady has a career 1.8 YPC and a career 1st down rushing % of 33.9% and a 3.2% TD%

If we use the 9 yard bonus for 1st downs and 20 yard bonus for TDs, given 100 carries you would expect Tom to have

1.8*100 + 33.9*9 + 3.2*20 = 549 adjusted? yards

and AD to have

5.0*100 + 23.6*9 + 4.2*20 = 796 adjusted? yards

That isn’t much of a difference considering one player is considered an all time great runner (breaking many top 5/10 lists) and the other posted a 5.28 40 at the combine and is considered unathletic even amongst QBs.

I think you would find YPC important if you were looking at EPA/baseline EPA based on the play call at that DnD.

Let me think about this one.

I wrote that comment late last night and I realize I might not have been very clear regarding what I meant by EPA/Baseline EPA.

IIRC mean, median, and mode of RB carries are ~4,3, and 2 respectively.

In a DnD situation of 2nd and 1, I think it is pretty safe to assume that we expect a rusher to pick up that 1 yard. If he picks up the median of 3 yards it becomes 12 adjusted yards (3 yards + 9 yard 1st down bonus). However, this rusher didn’t do anything extraordinary. He was pretty much average (really, I think 1-2 yards is a more likely outcome than the median based on the situation… but it doesn’t matter for illustrative purposes). I would guess that the expected adjusted yards rushing in that situation is around 9-10. The above runner only nets his team 2-3 adjusted yards above what is expected.

Another runner is faced with 1st and 10. He picks up 9 yards (again let’s assume a 3 yard gain is expected). His adjusted yards are only 9 (since he gained 9 yards but didn’t pick up a 1st or TD). However, he is 6 adjusted yards above the baseline (vs only 2-3 in the first example).

The first run is more important in a game theory sense – but it is also expected. Most RBs and QBs can pick up short yardage. The 2nd run isn’t as important in a game theory sense but it is likely *much* better than the expected outcome.

I am sure Burke has the data to do this sort of comparison. Since his EPA is based off drive stats. He would just need to break out rush vs. pass and he *should* break out rush by QB vs. rush by RB. I actually asked Aaron at FO to do this several years ago because I was arguing that the best play call on the 1 yard line (disregarding injuries) was a QB rush and that QBs are more successful than RBs in short yardage situations. This was his response:

We have each year’s data in a separate workbook so to go through all

the data at once would take a good amount of time. It’s an interesting

question, certainly. Probably deserves a bit of research. Are QBs more

successful than RBs in short yardage?

Before doing all that research, I at least looked at 2011 for you.

Here are the numbers for 2011.

From the 1

RB: 211 attempts, 50% success

QB: 40 attempts, 65% success

From the 2

RB: 96 attempts, 33% success

QB: 14 attempts, 8 successes (but six were scrambles — take out

scrambles and you get 8 attempts, 2 successes)

Thanks for the follow-up.

I agree that this is not designed to mimic EPA. I think there are a couple of things you’re leaving out of the analysis, though.

1) Failed runs in short-yardage situations. A failed run on 2nd-and-1 gains 0 yards; a failed run on 1st-and-10 might gain 3 yards. In general, and as another commenter pointed out, short-yardage backs are at a disadvantage when it comes to YPC. That’s why I like AYPC, which helps out short-yardage backs.

2) I think quantity is still king. So a RB A who may have a higher AYPC but fewer carries than RB B, isn’t going to be presumed better in my analysis.

1) Sure – but the bonus for 1st down/td is so large is that it makes YPC practically irrelevant (which you illustrated). There should be a baseline expectation to make the comparison meaningful.

2) I agree – more or less.

1) The issue here is what do we do for historical seasons?

EPA only goes back to 2000… I would think 14 years of data would give a good data set.

It’s easy to game YPC: have a ‘normal’ RB and a short yardage back. I can all but guarantee you the normal back carrying it on 1st and 10 and 3rd and long is going to have a much better YPC than the short yardage back. If used exclusively in those roles I bet the normal back would be at least around 4.3 YPC and the other 3.0 at best.

However I also think it’s easy to game Success Rate in a similar way (i.e. only run in short yardage) so I would like to see a Success Rate over Expected and then I think teams and players would really stand out. A perfectly average back/team should have a 0.0% SR over Exp regardless of the situations they are put in, but unadjusted YPC should vary significantly. This ties in to the point you made in your follow-up comment to Chase.

However I’m not too worried about the Brady/Peterson comparison you made. For one, there’s a reason AD has a lot more carries than Brady, which radically changes the difference in adjusted yards from giving each 100 carries. This goes back to Chase’s frequent point that a player shouldn’t be punished for getting a lot of carries since that says something positive about their talent (at least compared to the rest of the team).

For another, factoring in baseline yards is exactly what Burke’s EPA does (Expected Points Added above average). The primary difference is EPA’s baseline is all plays, making it more suited for team-level evaluations, whereas for Chase’s intended purpose you want a rushing baseline. The problem is that requires play-by-play data to know the down and distance when a player carried a ball, and Chase doesn’t have that from before 2000.

I don’t think it plays out that way in reality with any significant number of carries. If you had a stable of RBs all taking 50 carries total on the season then perhaps.

EPA isn’t above average. At least, not the way it is described on his site. It is purely the Expected points difference between DnD #1 and DnD #2.

Actually, I might have to take that back regarding the EPA being over average.

You’re right, I didn’t pick the right words; EPA is the difference in EP between Down 1 and Down 2.

What I meant was the EP at Down 1 anticipates some moderate gain, and so to earn a positive EPA and a “success” on a typical 1st and 10 it requires a gain of at least 5 yards. Last year the average 1st and 10 play gained 5.5 yards, but the average 1st and 10 *carry* had a YPC of 4.3 and only 33.5% gained 5 or more yards. So any carry on 1st and 10 is expected to be a failure and lower your EPA while averaging 4.3 YPC.

Compare with 3rd and 1. Last year teams gained an average of 3.6 yards on 3rd and 1, but only 2.8 YPC, yet succeeded on 69.4% of their carries.

Want a real life example? Donald Brown and Michael Tolbert. Same success rate (47%), number of carries (101), number of receptions (27), nearly equal in receiving yards, and 8 vs 7 total TDs. About as close as you can get, right? Yet despite equal success rates Brown gained 5.3 YPC while Tolbert only gained 3.6 YPC. A yard and a half per carry is a huge difference!

How did they get the same success rate? Because 21% of Tolbert’s carries came on 3rd or 4th down, averaging only 2.7 yards to go. He converted 67% of them. Only 8% of Brown’s carries came on 3rd down, and they averaged over 5 yards to gain. He converted one of his 8 attempts.

Murray had nearly twice as many carries, but his situations were almost identical to Brown’s. 5.2 YPC, but only 8% of his attempts on 3rd down, averaging 5.4 yards to go.

EP doesn’t anticipate a moderate gain on 1st down. Rather, for a 1st down gain to be successful (adding to EP) it must be moderate.

My issue, is that if you give a RB a 1st down carry you are really setting him up for failure. EP is great in a game theory sense – but not ideal in judging the performance of individuals (which is where I am assuming Chase is sort of going with this since he wants historical seasons) without some sort of baseline expectation. Although, I do think EPA it is much better than DYAR/DVOA of FO since many of those results don’t pass the eyeball test because it severely devalues long plays.

Where did you get the Brown/Tolbert breakdown? IMO, the real question there is if you take away the shorter 3rd/4th down runs from Tolbert until his average to go is similar to Brown or Murray how is his success rate/YPC affected?

As an aside, has anyone looked at the game charting data sold by FO? If was data that was easy to manipulate and consume I would pay for a couple seasons… but I don’t want to spend $30 for a season if I have to spend hours upon hours trying to get the data into a database.

I can’t comment on FO’s game charting.

I agree about setting a RB up for failure, and that it makes EP useful for evaluating teams but not necessarily runningbacks. It’s not their fault if their coach puts them in a bad spot. (Incidentally, EPA is handy for QBs because the baseline is low enough that it functions as “replacement level” QB production).

Tolbert/Brown success rate information came from Advanced Football Analytics, while the down and distance info is from PFR’s play-by-play database. As for Tolbert’s success rate on longer to go distances, it’s tough to say. I don’t have an easy way to determine success rate on 1st/2nd without estimating and counting by hand (3rd and 4th is clear – 1st down or TD only). On 1st and 2nd Tolbert’s to go distance is 7.9, still lower than Murray/Brown, but his YPC is still 3.7. I also estimated about a 33% success rate on those runs, many of which were short yardage on 2nd down and/or goal-to-go.

AFA: http://wp.advancedfootballanalytics.com/playerstats.php?year=2013&pos=RB&season=reg

PFR: http://pfref.com/tiny/8tSCT

Murray had 8.5 yards to go on 1/2, gained 5.3 YPC, and was successful about 45% of his carries.

Success rate can be gamed, but it depends on the definition of success rate. One way to avoid that is to only compare short-yardage runs to other short-yardage runs, although I’m not sure which sites do and don’t do that. I think you and I are on the same page.

Good comment, James.

Rushing first downs and rushing touchdowns are correlated variables: a rushing touchdown is also a rushing first down. That was a problem with the original post, which was why adding touchdowns to the how much is a rushing first down worth made touchdowns even more important.

That’s a good point, and one I should have realized earlier. I suppose we should make a rushing TD worth 11 yards, or just subtract the number of TDs from the number of FDs when calculating AYPC. Good catch! (and my bad!)

EPA success rate is probably the best stat for running backs, IMHO.

maybe multiply EPA/carry x EPA success rate?

The problem with EPA for evaluating rushers is it combines receiving EPA with rushing EPA, and furthermore rushing EPA is judged against ALL plays, not just rushing plays. The first means EPA times SR or anything else isn’t evaluating the metrics we want to isolate, and the second means almost every rushing play is negative EPA since rushes gain fewer yards than the league average play due to the disproportionate success of passes. Those combine to create a list centered around primarily pass catching RBs (hence Sproles, Woodhead, Vereen) and penalizing players for getting more carries!

I have actually asked/posted several times on Burke’s site for him to break out rushing vs. receiving.

Here, EPA/p x SR% and EPA x SR%, sorted by EPAxSR%:

Player Team EPA/p x SR% EPA x SR%

25-L.McCoy PHI 0.052 21.330

27-K.Moreno DEN 0.049 18.115

25-J.Charles KC 0.036 13.470

43-D.Sproles NO 0.075 13.128

39-D.Woodhead SD 0.054 12.303

34-S.Vereen NE 0.082 11.038

29-D.Murray DAL 0.037 10.508

31-D.Brown IND 0.056 10.112

44-J.Starks GB 0.084 9.158

22-M.Forte CHI 0.021 8.648

38-A.Ellington ARZ 0.045 8.200

35-J.Bell DET 0.022 5.338

45-M.Reece OAK 0.043 4.413

23-P.Thomas NO 0.019 3.982

28-A.Peterson MIN 0.011 3.440

24-M.Lynch SEA 0.008 3.286

24-R.Mathews SD 0.008 3.281

32-T.Gerhart MIN 0.053 3.085

22-F.Jackson BUF 0.009 2.884

29-R.Helu WAS 0.027 2.867

29-L.Blount NE 0.014 2.720

27-R.Jennings OAK 0.012 2.719

34-De.Williams CAR 0.011 2.635

23-R.Brown SD 0.029 2.142

44-A.Bradshaw IND 0.041 2.040

21-R.Hillman DEN 0.025 1.876

44-J.Snelling ATL 0.019 1.528

25-G.Bernard CIN 0.004 1.357

25-C.Ogbonnaya CLV 0.009 1.140

28-M.Ball DEN 0.005 1.125

35-M.Tolbert CAR 0.009 1.079

25-M.James TB 0.012 1.008

34-B.Jacobs NYG 0.008 0.498

27-E.Baker CLV 0.009 0.392

27-E.Lacy GB – 0.207

33-D.Thomas MIA – 0.172

21-R.Bush DET – (0.080)

29-K.Robinson NO (0.008) (0.752)

28-J.Stewart CAR (0.015) (0.811)

30-J.Todman JAX (0.010) (1.003)

21-J.Randle DAL (0.015) (1.033)

34-B.Brown PHI (0.013) (1.045)

30-B.Leonard TB (0.015) (1.261)

32-J.Rodgers ATL (0.008) (1.383)

33-T.Richardson CLV (0.031) (1.431)

36-B.Cunningham SL (0.028) (1.638)

33-C.Ivory NYJ (0.007) (1.656)

27-J.Dwyer PIT (0.027) (1.780)

29-M.Bush CHI (0.026) (1.793)

38-B.Bolden NE (0.020) (1.840)

22-M.Ingram NO (0.016) (2.055)

23-F.Jones PIT (0.041) (2.580)

28-C.Johnson TEN (0.008) (2.698)

44-M.Asiata MIN (0.052) (2.821)

23-A.Foster HST (0.019) (3.003)

23-S.Greene TEN (0.036) (3.160)

44-P.Hillis NYG (0.038) (3.511)

32-K.Hunter SF (0.034) (3.570)

30-Z.Stacy SL (0.012) (4.000)

28-C.Spiller BUF (0.016) (4.061)

28-D.Johnson HST (0.067) (4.118)

26-D.Richardson SL (0.046) (4.189)

34-K.Davis KC (0.038) (4.211)

42-B.Green-Ellis CIN (0.021) (4.521)

46-A.Morris WAS (0.017) (4.622)

32-M.Jones-Drew JAX (0.014) (4.690)

26-L.Bell PIT (0.019) (5.791)

43-B.Rainey TB (0.037) (6.012)

20-D.McFadden OAK (0.042) (6.020)

22-D.Wilson NYG (0.132) (6.720)

22-R.Turbin SEA (0.062) (7.004)

34-T.Richardson IND (0.034) (7.124)

39-S.Jackson ATL (0.035) (7.427)

22-S.Ridley NE (0.035) (7.569)

26-W.McGahee CLV (0.056) (8.189)

22-D.Martin TB (0.055) (8.462)

28-R.Mendenhall ARZ (0.039) (9.240)

44-B.Tate HST (0.042) (9.299)

26-L.Miller MIA (0.045) (9.926)

21-F.Gore SF (0.027) (9.968)

29-B.Powell NYJ (0.040) (10.026)

30-B.Pierce BLT (0.057) (10.611)

35-A.Brown NYG (0.066) (10.885)

27-R.Rice BLT (0.056) (16.754)

Chase,

You lose me at the end. You say AYPC is a step in the right direction, but correlation coefficient is only 0.35. Seems like very little incremental value for a metric that’s not as easily produced (or understood by subscribers/readers).

My question for you is whether you think this lack of predictability means forecasters should be more bold in their variance from year to year? For example, let’s say you’re scared to death about the Chiefs O-line losses and since the WR corps doesn’t appear any better, you fear that there just won’t be as much room to run in 2014. Most (myself included) would be tempted to lower the team’s YPC by a few tenths of a yard…but should we consider being much bolder? Or is that in effect just adding more randomness to what’s already a random exercise?

I’ll disagree with you that AYPC is not easily produced or understood, although I admit that perhaps I am not the target audience here

The CC is not necessarily the measure we want to concern ourselves with here. If, for example, the CC on AYPC was

lowerthan it was on YPC (which it isn’t), that wouldn’t be a big deal to me. That’s because I’m using AYPC as a backwards-looking, or retrodictive measure. So we aren’t concerned with “stickiness” just with being right, and crediting people for TDs and First Downs sounds right.Your second question is a good one. The Chiefs are a unique example, because Charles is such a singular talent. He has the highest YPC of any RB in NFL history by a good margin over Jim Brown, so I don’t really know what to say about him in particular. The Chiefs averaged 4.7 YPC last year, but some of that was the backups bringing the average down. I would not be tempted to lower Charles’ YPC by more than 0.2 because of OL issues, although that’s outside the issue of regression to the mean. For someone like Charles, figuring out his mean is pretty darn hard.

The NFL YPC average last year was 4.2, but the RBs with 1,000 rushing yards last year other than Charles averaged 4.46 YPC. Is that a biased sample? Yeah, but anyway, Charles would obviously be expected to have a mean higher than that anyway. I’d probably put Charles at something like 4.7 YPC.

Was using Rush EPA/P in a power rating formula and then read your recent post and ran numbers using AYPC and found it worked modestly better. I found a CC of .38 to Year N+1.

One question. What do you use to calculate the multivariate regression? I use Google sheets and can’t a function or script to calculate it anywhere.

install the Analysis ToolPak in Excel

I think this is all reinvention of the wheel. What’s wrong with EPA again? It rewards short yardage backs for picking up the necessary yardage, or even part of it (on first and second down). Short yardage backs are also lower variance, so I don’t really see any punishment here.

From codeandfootball:

“Touchdowns are more close in value to the NFL passer rating than THGF’s new passer rating. And although I’m critical of Chase Stuart’s derivation of the value of 20 for PFR’s AYA formula, the adjustment they made does seem to be in the right direction.

So where does the model break down?

Inside the 10 yard line. It doesn’t accurately depict the game as it gets close to the goal line. It’s also not down and distance specific in the way a more sophisticated equivalent points model can be. A stat like expected points added gets much closer to the value of an individual play than does a AYA style stat. In terms of a play’s effect on winning, then you need win stats, such as Brian’s WPA or ESPNs QBR to break things down (though I haven’t seen ESPN give us the QBR of a play just yet, which WPA can do).”

http://codeandfootball.wordpress.com/2011/10/21/the-valid-range-of-a-linearized-scoring-model/

{ 3 trackbacks }