Today’s guest post comes from Ben Baldwin, a contributor for Field Gulls and Bryan’s site, http://thegridfe.com. You can find more of Ben’s work here or on Twitter @guga31bb. What follows are Ben’s words.

Arguing on the internet

A common argument on the internet (e.g. Twitter, where I spent too much time) is that the efficiency of players like Dak Prescott and Russell Wilson in their rookie seasons (and subsequent seasons, for Wilson) was not impressive because they were not asked to throw the ball as much. Once they are asked to throw more often, the argument goes, we can expect their efficiency to fall off. Here is one of many, many examples:

Do quarterbacks really look good because they throw less?

We have evidence that in the NBA, players on a given team distribute shots among themselves in a manner that the marginal value of the next shot is roughly equal among players. This results in good players generally taking more shots than bad players in order to maximize the scoring of the team. Further increasing a given player’s volume is typically associated with a decrease in efficiency. This sort of logic is probably what people are leaning on when they assume that a passer’s volume increasing should cause a drop in efficiency. However, that conclusion doesn’t necessarily follow because each team only has one quarterback on the field at a given time: in the NBA, defenses can focus their attention on one player who shoots a lot, but in the NFL, there’s only one quarterback. If a team is going to pass, the defense knows who is going to be throwing the pass (trick plays aside). Another argument is that teams facing an opponent who passes a lot can focus their attention on the passing game rather than the rushing game, but is this really correlated with meaningful changes in efficiency? Let’s take a look.

Boring background stuff

• I analyzed every quarterback season in which a quarterback started at least 9 games from 2002 through 2016. This results in 436 distinct seasons played by QBs.
• I measure efficiency using yards per attempt, which has the benefits of both being very simple and doing about as well at out-of-sample predictions of team success as other metrics. Results are similar using passer rating or other efficiency stats. The mean Y/A season in the sample is 7.11.

Findings

Here is a basic scatterplot of the data. Again, each point represents a season played by a QB.

That looks like a giant blob with no real relationship, but already we can see that it is certainly not the case that the simple act of passing at low volume is sufficient to carry a QB to strong efficiency stats.

Next, here are all of the seasons played by QBs with at least 10 seasons in my data. I’ve added player labels to give a sense of what some individual seasons look like:

Again, it is not the case that the seasons with the fewest attempts are the seasons with the best efficiency.

For the remainder, I’m going to be looking at regression output. If this doesn’t sound interesting, skip to the end.

The first regression is a simple regression of a player’s yards per attempt on the average number of passes thrown per game and year effects (to account for efficiency rising over time):

The coefficient of .014 tells us that every 10 additional passes a player throws is correlated with an increase of .14 yards per attempt. In this simple look at the data, as with the scatterplot above, there is no evidence that increased volume is associated with a decrease in efficiency.

Perhaps the small, positive relationship between passes per game and yards per attempt found above is due to more experienced QBs throwing more and being more efficient. When I add controls for quarterback experience as well, the coefficient shrinks to essentially zero (-.004). Finally, it could be the case that a true negative relationship between volume and efficiency is masked by good quarterbacks being the ones who throw a lot and who are more efficient.

For the last exercise, I take the set of players who have played at least 10 seasons (Ben, Palmer, Brees, the Mannings, Rivers, Brady) and run a player fixed effects regression, which essentially compares each player to himself in his high volume versus low volume seasons. The point estimate becomes -0.029, which is small and not statistically significant from zero (the standard errors for the regression coefficient grow larger here because of the limited number of players under consideration). The way I would describe this result is that among the set of quarterbacks who have been in the league a long time, they have been slightly less efficient in the seasons in which they threw more.

Summary

Small sample sizes will always produce extreme results, and a player’s base rate/average is more likely to shine through over a larger number of attempts. That means quarterbacks with low-attempt seasons are more likely to have really high and really low efficiency numbers by virtue of the low sample size. That said, that’s a very different argument than a player’s efficiency numbers being high because the low number of attempts helps to boost his efficiency.

In a sample of QB seasons in the past 15 years, there is a small, positive association between volume and efficiency. This may be driven in part by skilled quarterbacks throwing more (on average) and being more efficient (on average). When looking at QBs with long careers, there is little evidence that they were meaningfully less efficient in the years they were asked to pass a lot relative to the years they were not. Comparing Brees in 2004 vs Brees in 2013 is a nice illustration — his passer rating was nearly identical despite a massive difference in attempts/game.

To sum up, looking at the last 15 seasons in the NFL reveals no statistical relationship between a QB’s volume and efficiency.