≡ Menu

## Guest Post: Passing Volume vs. Passing Efficiency

Today’s guest post comes from Ben Baldwin, a contributor for Field Gulls and Bryan’s site, http://thegridfe.com. You can find more of Ben’s work here or on Twitter @guga31bb. What follows are Ben’s words.

Arguing on the internet

A common argument on the internet (e.g. Twitter, where I spent too much time) is that the efficiency of players like Dak Prescott and Russell Wilson in their rookie seasons (and subsequent seasons, for Wilson) was not impressive because they were not asked to throw the ball as much. Once they are asked to throw more often, the argument goes, we can expect their efficiency to fall off. Here is one of many, many examples:

Do quarterbacks really look good because they throw less?

We have evidence that in the NBA, players on a given team distribute shots among themselves in a manner that the marginal value of the next shot is roughly equal among players. This results in good players generally taking more shots than bad players in order to maximize the scoring of the team. Further increasing a given player’s volume is typically associated with a decrease in efficiency. This sort of logic is probably what people are leaning on when they assume that a passer’s volume increasing should cause a drop in efficiency. However, that conclusion doesn’t necessarily follow because each team only has one quarterback on the field at a given time: in the NBA, defenses can focus their attention on one player who shoots a lot, but in the NFL, there’s only one quarterback. If a team is going to pass, the defense knows who is going to be throwing the pass (trick plays aside). Another argument is that teams facing an opponent who passes a lot can focus their attention on the passing game rather than the rushing game, but is this really correlated with meaningful changes in efficiency? Let’s take a look.

Boring background stuff

• I analyzed every quarterback season in which a quarterback started at least 9 games from 2002 through 2016. This results in 436 distinct seasons played by QBs.
• I measure efficiency using yards per attempt, which has the benefits of both being very simple and doing about as well at out-of-sample predictions of team success as other metrics. Results are similar using passer rating or other efficiency stats. The mean Y/A season in the sample is 7.11.

Findings

Here is a basic scatterplot of the data. Again, each point represents a season played by a QB.

That looks like a giant blob with no real relationship, but already we can see that it is certainly not the case that the simple act of passing at low volume is sufficient to carry a QB to strong efficiency stats.

Next, here are all of the seasons played by QBs with at least 10 seasons in my data. I’ve added player labels to give a sense of what some individual seasons look like:

Again, it is not the case that the seasons with the fewest attempts are the seasons with the best efficiency.

For the remainder, I’m going to be looking at regression output. If this doesn’t sound interesting, skip to the end.

The first regression is a simple regression of a player’s yards per attempt on the average number of passes thrown per game and year effects (to account for efficiency rising over time):

The coefficient of .014 tells us that every 10 additional passes a player throws is correlated with an increase of .14 yards per attempt. In this simple look at the data, as with the scatterplot above, there is no evidence that increased volume is associated with a decrease in efficiency.

Perhaps the small, positive relationship between passes per game and yards per attempt found above is due to more experienced QBs throwing more and being more efficient. When I add controls for quarterback experience as well, the coefficient shrinks to essentially zero (-.004). Finally, it could be the case that a true negative relationship between volume and efficiency is masked by good quarterbacks being the ones who throw a lot and who are more efficient.

For the last exercise, I take the set of players who have played at least 10 seasons (Ben, Palmer, Brees, the Mannings, Rivers, Brady) and run a player fixed effects regression, which essentially compares each player to himself in his high volume versus low volume seasons. The point estimate becomes -0.029, which is small and not statistically significant from zero (the standard errors for the regression coefficient grow larger here because of the limited number of players under consideration). The way I would describe this result is that among the set of quarterbacks who have been in the league a long time, they have been slightly less efficient in the seasons in which they threw more.

Summary

Small sample sizes will always produce extreme results, and a player’s base rate/average is more likely to shine through over a larger number of attempts. That means quarterbacks with low-attempt seasons are more likely to have really high and really low efficiency numbers by virtue of the low sample size. That said, that’s a very different argument than a player’s efficiency numbers being high because the low number of attempts helps to boost his efficiency.

In a sample of QB seasons in the past 15 years, there is a small, positive association between volume and efficiency. This may be driven in part by skilled quarterbacks throwing more (on average) and being more efficient (on average). When looking at QBs with long careers, there is little evidence that they were meaningfully less efficient in the years they were asked to pass a lot relative to the years they were not. Comparing Brees in 2004 vs Brees in 2013 is a nice illustration — his passer rating was nearly identical despite a massive difference in attempts/game.

To sum up, looking at the last 15 seasons in the NFL reveals no statistical relationship between a QB’s volume and efficiency.

• sacramento gold miners

This topic reflects the influence of fantasy football in these discussions. Completions and yardage for some fans tends to be overrated, and wins underrated. The quality QBs can kill you with few attempts, or many throws, depending on the situation. I recall the concern some had about Bob Griese after 1974, when Miami had the WFL defections. But Griese was a star passer at Purdue, and even when the Dolphins had the great running attack, Griese still had to deliver when required. Although the post-1974 Dolphins weren’t as successful as before, that was more due to an aging defense, than any shortcomings by Griese.

• That’s a lot like Bart Starr. I’ve seen countless people discuss the Lombardi Packers as a run oriented team who only passed occasionally, allowing Starr to boost his efficiency numbers. In many of the games I have watched (or studied gamelogs of), the Packers opened the game with an almost even pass-run split in the first half, which is saying something in that era. Then, once the offense efficiently executed, and the defense stifled opposition, they ran out the clock with their strong running game. It seems as though many look back at the stats, see good rushing numbers, and just assume the runs came first instead of second.

• Need to calculate game scripts and pass identity ratios for that era!

• Tom

Right…somewhere, someone out there has play-by-play data for these early games…must find that person immediately!

• Renan

Ben, thanks for the article.
I would like to see the correlation between pass efficiency x good running game. How to define a good running game? Maybe we could compare pass efficiency with run volume and run efficiency.

• Thanks!

I haven’t seen a systematic study of passing efficiency vs rushing (volume / efficiency), but wouldn’t be super hard with the data I used for this (just need to add rushing).

• It seems that, at some point or in some way, you’d need to use an SRS style of system to account for the possibility of a strong passing game influencing the rushing game as well. Adrian Peterson always gets cited as having it tough because of his lack of a passing game, while a guy like Edgerrin James gets marked down for playing with Peyton Manning. I especially love seeing people talk about Peterson facing 8+ man boxes without mentioning how many blockers he had on those plays, but that’s another issue.

Almost everything we study is affected by another thing, and it can be super hard to tell where the causation needle points, or if there are feedback loops.

• In this post, I looked at which running backs played with the best passing games.

http://www.footballperspective.com/adrian-peterson-and-which-running-backs-played-with-best-passing-games-part-ii/

The average RANY/A was +0.179, which isn’t too high. Another thing to consider: how much of a good running game being driven the same hidden variables as a good passing game: i.e., smart coaches or a talented offensive line?

Peterson had it bad, but not as bad as Thomas Jones or Gale Sayers!

• sacramento gold miners

The former might be the most underrated RB ever. Written off as a first round bust after three disappointing seasons in Arizona, Jones rebounded with a tremendous career. Over 10,000 lifetime rushing yards, five 1000 yards plus seasons, with five different clubs.

• That’s why I don’t care for blanket statements on things like QB1 playing with a good RB or RB1 playing with a good QB. In some cases, they are partially responsible for the success of the other parts of their offense. I am sure playing with Peyton Manning allowed Edge to have greater success, as Manning was more likely than most QBs to check into the right play, as well as pose a serious threat in the air and give Edge a solid +/- in the box. Reciprocally, Edge forced teams to respect the run, but he was also among the most talented all-around RBs ever (thru 38 games, at least). He was a talented receiver, not on the Faulk level, but in the upper echelon among backs. He was also a very good blocker. And this wasn’t a case of a guy having to learn to block because Manning would have him benched if he didn’t know how; Edge was a terrific blocker in college as well.

Obviously, there are copious other examples throughout history, but we were already talking about those guys.

• Adam

If you do this, I recommend using success rate to measure rushing efficiency rather than YPC. I believe the correlation between YPC and passing YPA is zero, but that’s due in part to YPC being a terrible way to measure rushing efficiency.

• Frank Yi

The correlation between YPC and YPA is actually positive. However, when regressing win % on YPA AND YPC, the partial correlation with YPA is strongly positive, and the partial correlation with YPC is strongly negative. Yet, regressing win % on YPA and rushing att, both YPA and rush att are strongly positive. However, the reasons are easily explained: YPA is correlated with efficient offenses and “causes” winning, a team with the lead rushes the ball more to eat clock (regardless of ypc), and YPC would tend to be correlated with an efficient offense. In this scenario, it wouldn’t be a reach to say that YPA is the root cause (in very simplistic terms).

I don’t have the data for rush success rate, but I agree that would be the ideal way to go for the study.

• kevin trammo

I would also think that there might be impact by playing from behind as opposed to playing ahead. QBs who are behind do also tend to throw more, especially the bigger the deficit. and more desperation throws as well.

• Dave B

This is really the WRONG way to analyze this problem and the wrong conclusion I think. You need to look at individual QB’s and see how much their AY/A varies from their baseline average slicing their career game logs up into different volumes. I threw out games with less than 20 passes for reasons of small sample size and may not be full games.

These are somewhat arbitrary splits but I didn’t cherry pick them

EX: Tom Brady
20-29 passes (62 games 27 % ) AY/A : 8.16
30-39 passes (104 games 45% ) AY/A : 8.19
40-49 passes (43 games 18% ) AY/A : 7.37
50+ passes (20 games 8% ) AY/A : 6.7

EX: Peyton Manning
20-29 passes (61 games 23% ) AY/A : 9.04
30-39 passes (113 games 43% ) AY/A : 8.08
40-49 passes (66 games 25% ) AY/A : 6.59
50+ passes (17 games 6%) AY/A : 5.99

EX: Jay Cutler
20-29 passes (36 games 28% ) AY/A : 7.62
30-39 passes (69 games 53% ) AY/A : 6.77
40-49 passes (23 games 17% ) AY/A : 6.27
50+ passes N/A only 2 games

EX: Russel Wilson
20-29 passes (36 games 49% ) AY/A : 8.5
30-39 passes (33 games 45%) AY/A : 7.97
40-49 passes (4 games 5%) AY/A : 6.8
50+ passes N/A only 2 games

So to me it looks absolutely like their is a very STRONG RELATIONSHIP BETWEEN VOLUME AND EFFICIENCY for any given QB

The main reason for this is I think what someone posted already. If you are behind, you need to throw more passes. The defense knows you need to throw making it harder to pass. The reverse is true when leading, especially when you know you have a great defense to lean on, you can be more careful and don’t need to make risky passes. Wilson has gotten to play nearly half his games throwing less than 30 passes (more than 50% if you throw out the under 20 pass games I threw out for every QB). That is nearly double the amount of other QB’s

So a very bad QB throwing a small amount of times per season may still be worse than an elite QB throwing many. But definitely as any one QB’s workload goes up in any one game the more their efficiency drops.

• Well, doesn’t this run counter to the idea that losing QBs can rack up big numbers during garbage time? Do you think it’s harder or easier for QBs when they are behind and have to throw?

Also, I don’t think this is the right way to measure this, either. For example, let’s say Cutler threw two early INTs in a game, then played well the rest of the way. In that case, he likely ends up throwing 40+ passes because his team was behind, and it’s likely that his AY/A won’t be good, either: adding in -90 yards will cost him over 2 AY/A, so even if he is at 8.00 AY/A the rest of the way, this will go down as a 6.00 AY/A game. That makes it look like in a game where Cutler had to throw a lot, he had bad numbers.

The problem here is that games where QBs have to throw the most are often games where the QB played poorly early on, and vice versa.

• Dave B

Regular yards per attempt appears to show the same types of splits. For sure some of the games its a negative feedback loop where the QB plays poorly early and thus has to throw more often.

But it can also be the defense/special teams is giving up scores/yardage and the QB needs to keep throwing to keep pace rather than running more often and milking clock.

Regarding easier or harder it really depends on when you define garbage time. Someone looked at this pretty extensively once but I don’t remember the link anymore. The Blake Bortles effect of 2015. ha ha.

Just from a game theory perspective the more plays you throw as a percentage of your total plays the harder it should be to so as efficiently because the defense will adjust to stop you.

• Ben, I’m disappointed that you’re still trying to present this data as meaningful. It’s not.

It’s disingenuous to ignore the horde of variables affecting the data. As I pointed out previously, attempts per game have risen steadily from 2002-2016: 36.1, 34.3, 34.3, 34.5, 34.3, 35.4, 34.3, 35.4, 35.9, 36.3, 37.0, 38.0, 37.3, 38.1, 37.9. From 2003-09, teams consistently averaged 34-35.5 att/gm; since 2010, it’s never been lower than 35.9, and since 2012, it’s never been lower than 37.

Efficiency has risen correspondingly, with leaguewide NY/A of 5.9, 5.8, 6.1, 5.9, 6.0, 6.0, 6.2, 6.2, 6.2, 6.3, 6.2, 6.2, 6.4, 6.4, 6.4. This could be interpreted as supporting your conclusion: teams are passing more often, so they’re passing more efficiently. A more plausible explanation, though, would cite rule changes that favor passing, the growth of pro-style college offenses, and other environmental factors. Ignoring these factors distorts the data, and you’ve made no attempt to correct for that; you fail to even acknowledge it, which makes it look like you have an agenda.

This trend is fully sufficient to explain your findings. It is your responsibility as a statistician to examine individual data sets in the context of larger leaguewide trends in the data, to account for environmental factors that distort the numbers. You have failed to do this even after the problem was pointed out to you.

Your individual data sets (such as Brees) mirror leaguewide trends toward both greater efficiency and higher attempts. Failing to investigate this — or at least acknowledge it to your reader — is a serious misstep, and calls into question your motivation. It feels like you started with a conclusion and looked for data to support it, however tenuously. There are obvious environmental factors — most notably rule changes — influencing the numbers, all of which you have chosen to ignore.

When you compare ’04 Brees to ’13 Brees, you’re comparing Brees’ first half-decent season to his prime. You’re comparing him playing for the San Diego Chargers to the New Orleans Saints. His coaches are different, his blockers are different, his receivers are different, his home stadium is different. The rules have changed: defenseless receiver, Tom Brady rule, etc. It’s apples to oranges; we simply can’t draw a confident conclusion from that comparison.

Even shorter time differences can be problematic. Take Tom Brady. From 2002-06, he played on a team with dominant defense and lacking Pro Bowl receivers; he averaged 530 attempts per season. In ’07, he got Randy Moss and Wes Welker and started passing more, with great success. I don’t think that demonstrates anything meaningful about the correlation between attempts and efficiency. Shifting rosters, injuries, the efficiency of the run game, and other variables make this very difficult to measure. It seems to me that your counter-intuitive findings reflect these difficulties rather than the conclusion you imply.

You’ve muddled correlation and causation. Brady is a case where it seems obvious to me that increased efficiency led him to pass more often; you imply that he’s passing more efficiently not because of rule changes or because he has better receivers now than Troy Brown and David Patten.

I applaud your initiative in attempting to study this issue, but you imply that you’ve disproven any link between lower volume and higher efficiency, and that’s simply not the case. If one were to frame your hypothesis as “it is no easier to be efficient when passing at low volume”, we encounter so many variables that could distort the data that it’s impossible to draw any conclusion from the findings here. There just isn’t enough evidence here to refute a link between higher attempts and lower efficiency.