## Tweaking the NFL’s Passer Rating Formula

Wilson scrambles and gets credit for it.

I hate passer rating. So do you. Everyone does, except for Kerry Byrne. Passer rating is stupid because it gives a 20-yard bonus for each completion, a 100-yard penalty for each interception, and an 80-yard bonus for each touchdown. In reality, there should be no (or a very small) weight on completions, a 45-yard weight on interceptions, and a 20-yard weight on touchdowns.

But let’s ignore those issues today. Reading Mike Tanier’s recent article inspired me to make see what passer rating would look like if we make three tweaks. I’m not going to change any of the weights in the formula, but just redefine the variables.

1) There’s no reason to exclude sack data from passer rating. I’ve stopped writing about how sacks are just as much (if not more) on the quarterback than other passing metrics, because I think that horse has been pretty well beaten by Jason Lisk and me.

2) Scrambles should be treated like completed passes. If Russell Wilson is about to be sacked, but escapes and run for 7 yards, why should that be treated any differently than if Peyton Manning is about to be sacked, but throws a seven-yard pass at the last second?

3) Lost Fumbles should be counted with interceptions. One could make a few advanced arguments here — we should use all fumbles instead of lost fumbles, or fumbles should be given an even stronger weight than interceptions (although consider that in light of this post), or that we should limit ourselves to just fumbles lost on passing plays. I’m going to play the simple card here, and just use lost fumbles data on the season level.

Passer rating consists of four metrics, all weighted equally: completions per attempt, yards per attempt, touchdowns per attempt, and interceptions per attempt. I will use the same formula with the same weights and the same variables, but redefine what those variables are. Here are the new definitions, with the additions in blue.

Completion percentage is now (Completions plus Scrambles) / (Pass Attempts plus Sacks plus Scrambles)

Yards per Attempt is now (Passing Yards plus Yards on Scrambles minus Sack Yards Lost) / (Pass Attempts plus Sacks plus Scrambles)

Touchdown Rate is now (Passing Touchdowns plus Touchdowns on Scrambles) / (Pass Attempts plus Sacks plus Scrambles)

Turnover Rate will replace Interception Rate in the formula, and is calculated as (Interceptions plus Fumbles Lost) / (Pass Attempts plus Sacks plus Scrambles)

The table below lists all of those metrics for the 32 quarterbacks who had enough pass attempts to qualify for the passer rating crown, along with Alex Smith and Colin Kaepernick, who just missed qualifying. Let’s look at the Robert Griffin III line.

He completed 258 of 393 pass attempts for 3200 yards, with 20 touchdowns and five interceptions. Those are the standard stats that make up passer rating, but he also took 30 sacks and lost 217 yards on those sacks. That makes Griffin’s numbers worse, but he also had 38 scrambles for 302 yards (which gets recorded as 38 completed passes for 302 yards), with no scramble touchdowns. Finally, he lost two fumbles. His new completion percentage is 64.2%, his new yards per attempt is 7.13, his new touchdown rate is 4.3%, and his turnover rate (which includes fumbles) is 1.5%. The final two columns show each quarterback’s passer rating under the normal system and their passer rating using these metrics, which I’ll call the FPPR for short.

RkNameCmpAttYdsTDIntSkYdsSCRSYd/TDFLCMP%Y/ATD RTTO RTPRFPPR
1Peyton Manning4005834659371121137427/0266.4%7.486.1%2.1%105.8100
2Aaron Rodgers37155242953985129318162/1462.6%6.716.4%1.9%10895.6
4Robert Griffin III25839332002053021738302/0264.2%7.134.3%1.5%102.493.4
5Matt Ryan422615471932142821019137/1266.6%7.025%2.4%99.193.4
6Alex Smith15321817371352413713111/0165.1%6.715.1%2.4%104.191.5
7Russell Wilson252393311826103320347298/2363.2%6.795.9%2.7%10091.3
8Drew Brees4226705177431926190315/0160.8%7.166.2%2.9%96.391.2
9Colin Kaepernick13621818141031611227225/1262.5%7.384.2%1.9%98.391
10Ben Roethlisberger2844493265268301821394/0360.4%6.465.3%2.2%9787.6
11Matt Schaub3505444008221227216210/0061.4%6.643.8%2.1%90.785
12Eli Manning3215363948261519136941/0158.5%6.834.6%2.8%87.282.9
13Tony Romo4256484903281936263965/0362.6%6.794%3.2%90.582.8
14Cam Newton280485386919123624434282/1356.6%7.043.6%2.7%86.279.3
15Joe Flacco3175313817221035227324/0456.2%6.353.9%2.5%87.778
16Andy Dalton329528366927164622919113/1458.7%5.994.7%3.4%87.477.6
17Carson Palmer3455654018221426199425/0558.7%6.463.7%3.2%85.376.9
18Josh Freeman306558406527172616120143/0254%6.74.5%3.1%81.676.8
20Ryan Fitzpatrick306505340024163016133185/0659.7%6.034.2%3.9%83.374.9
21Christian Ponder300483293518123218434241/2560.8%5.453.6%3.1%81.274.7
22Matthew Stafford43572749672017292121996/1458.6%6.262.7%2.7%79.874.7
23Philip Rivers3385273606261549311734/0759.2%5.714.5%3.8%88.674.3
24Jay Cutler255434303319143825022227/0456.1%6.093.8%3.6%81.371.8
25Andrew Luck339627437423184124634241/2553.1%6.223.6%3.3%76.570.5
26Michael Vick204351236212102815338298/0558%6.012.9%3.6%78.170.1
27Nick Foles16126516996520131739/1357.5%5.52.4%2.7%79.169.5
28Brandon Weeden297517338514172818614103/0155.6%5.912.5%3.2%72.668
29Blaine Gabbert16227816629622158950/0355.3%5.032.9%2.9%77.466.7
30Ryan Tannehill2824843294121335234759/0454.9%5.932.3%3.2%76.166.7
31Jake Locker177314217610112515125233/0455.5%6.22.7%4.1%7466.2
33Matt Cassel16127717966121910116124/0656.7%5.831.9%5.8%66.756
34Mark Sanchez2464532883131834209830/0851.3%5.462.6%5.3%66.954.5

The world certainly doesn’t need another passer rating formula. But I thought this would be a fun thing to do.

• Arif Hasan

I think Kerry Byrne did something like this and called it Real Passer Rating, but included runs, not just scrambles (I assume you used NFLGSIS to separate the two) and the biggest problem was overrating runs because of the completion bonus. That is, every run added 20 yards because of the completion bonus inherent in the Passer Rating formula.

Of course, it didn’t include sacks, which would then be “negative incompletions,” which could balance it out a little bit. I think getting rid of designed runs helps a lot too but I think it the formula could have just done better getting rid of completions and replacing them with a small bonus for first downs.

Completions are good and probably deserve a small bonus (4 attempts for 250 yards is probably not as good as 34 attempts for 250 yards), but you might be able to accomplish the bonus that completions provide by simply counting first downs (or rather giving bonuses for high first down rates).

You just tweaked the formula a little, which was your goal, and the ranks look generally pretty similar to the current formula. No real weight behind my comments, because all you meant to do was look at a small change. But man, those weights (and that completion bonus) just burns me so much that I can’t not mention it.

• In undergrad, I was a psychology major, which meant I was required to take statistics (which was one of the best classes I ever took). One of the very first things we learned, because it was used for our grading, was z-scores. Like all good nerdy football fans, I immediately began applying the method to the NFL, which quickly led to me creating my own passer rating formula.

I used z-scores for yards per attempt (which included rushing yards and attempts), completion percentage (which included all runs as completions), sack rate, interception rate, and fumble rate using every quarterback who had started at least 10 games, multiplied each score by 10 except for interception rate which got multiplied by 20, then added them up. Of course, later I realized that it was a horrendous formula in approximately one billion ways–it double counts a meaningless completion percentage, it puts a ton of weight on interceptions when they are much closer to random events than the other elements, etc. But what I still find interesting is that I, a fan who had no connection to the NFL statistical community and really knew nothing about football, knew to include sacks, fumbles, and rushing as part of a quarterback’s value while the NFL’s passer rating ignores them.

I still find it amazing that the league so heavily promotes a statistic that had what seemed like obvious flaws to a 19-year-old college student who knew nothing. I think fans just accept the passer rating as good because the league and announcers tell them it is and it’s such a complicated, bizarre formula that they can’t tell what’s wrong with it. However, I don’t understand the league’s love of passer rating. I can understand it a little bit from the perspective of 1971, when it was introduced: you needed to give people context for the number to have any meaning, but reliable sack data didn’t go back very far. So, you decide just to ignore it. But why does that persist 40+ years later? It’s still strange to me.

Leaving the self-congratulatory story to talk about the article itself, I thought it was interesting to look at the places where your formulation and the league’s differ the most. It looks like the players hurt most by FPPR compared to PR are Aaron Rodgers, Alex Smith, and Philip Rivers while those helped the most are Tom Brady and Eli Manning. The main difference maker thus seems to be sacks, with Rodgers (82 sack%+), Smith (70), and Rivers (81) ranking among the league’s worst in sack rate and Brady (118) and Manning (124) ranking among the league’s best.

• QB RATING PLUS QB RATING TIMES QB RATING MINUS FOUR HAS THE HIGHEST CORRELATION TO WINNING PERCENTAGE IN THE HISTORY OF STATISTICS.

THIS IS A COLD, HARD FOOTBALL FACT.

NO ONE DENIES THIS.

• Neil

Haha

• Bryan

Just for giggles, I did the math, and this magical stat does have a pretty good correlation to winning. I suppose the “mighty potentate of pigskin” is on to something. Maybe not though.

• Tim

Glad someone (Shattenjager) has actually spelled out their background with inferential statistics. I have a graduate degree in psychology and had to take three advanced classes related to statistics, modeling of concepts for statistical measurement, and tests and measurements (I’m sure you took that class, too). I too have fiddled with using normative scoring models (not much as my time is limited) including the idea of standard scores similar to the scale used for IQ tests (such as amean of 100, and whatever the standard deviation would yield). I still think this would be a valuable endeavor as long as different eras are accounted for. I believe Chase has addressed this issue and found it somewhat wanting, though the advanced passing formula ratings on PFR do have a normative standing score.

My deal is with how we label these concepts and also how they are modeled. Weighting different variables appears to be a debatable and even contentious issue (at least outside the domain of the readers of this blog). I would offer that we think of the rating formula be considered more of a QB rating system and not be called a passer rating system. There is more to be a QB than just passing- though passing is the dominant variable.

• JeremyDe

Quick question. Since fumble recoveries are roughly a 50/50 prospect, would it at all be beneficial to replace fumbles lost with (total fumbles/2)? That way it would (sort of) remove the luck of the bounce from the equation. Not sure if this would actually make any difference.

• Marko Markovic

I agree that fumbles lost is probably too random to include here. Recoveries are basically dependet on lucky bounces and not an inherent skill that can be taught or projected. Including all fumbles seems fair, since it includes all 50/50 situations of bad judgement where he gave the other team a great chance to get the ball.

Then again, the flipside of this would be to include dropped INTs too, probably the most undervalued variable in all passer metrics. (Especially since not one of them considers it.)

• mrh

Interesting how high Alex Smith rated. Tanier’s article mentioned his high sack rate last year and got me thinking and I believe I found something interesting. We all sense that Andy Reid did a good job with low-drafted QBs in PHI, consistently developing them and then trading them for 2nd round picks or such.

Let’s look at the career sack rates for Eagle QBs before their time in PHI, while in PHI, and after PHI.

For example, Vick pre-PHI had a 9.8% career sack rate, in PHI it’s been 6.8%.
McNabb in PHI 7.0%, post-PHI 7.8%
Garcia pre-PHI 3.6%, in PHI 3.1%, post-PHI 5.6%
Kolb in PHI 5.9%, post-PHI 9.3%
Feeley in PHI 4.2% (1st time), post-and-pre PHI 6.1%, in PHI 2.8% (2nd time), post-PHI 9.3%
McMahon pre-PHI 10.5%, post-PHI 8.4%
Young pre-PHI 5.9%, in PHI 6.6%

Young stands out as the exception but every other QB improved their sack rate when they came to PHI and saw it get worse when they left PHI. Obviously there are all sorts of factors involved besides Reid’s coaching, but I won’t be surprised if Alex Smith (8.3% career sack rate) is only sacked at a 6-7% rate this year.

• Jason

@Arif – “(4 attempts for 250 yards is probably not as good as 34 attempts for 250 yards)”
A minor quibble, here. Four attempts for 250 yards is a lot better than 34 attempts for 250 yards, because the first scenario means you get to add 30 running plays to the total.
I think we should say that 4/34 for 250 yards is not as good as 25/34 for 250 yards. If we assume that all plays were passes, then the mad bomber (4/34) probably got four TDs (or 3TDs and a field goal), but he needed at least 13 possessions to do so. The more proficient QB (25/34) likely achieved his yardage in just 3-5 possessions. If we assume a mix of runs, then the proficient QB is (likely) complementing the running game by providing many first downs; the mad bomber isn’t.

• Ben

Expected Points per Attempt is really a simple way to settle the debate on which rating to use.

• Ben

Clarification: EP/A itself is not simple, but it deals directly with how productive QBs are points-wise. All other rating systems are peripheral.

• Bryan

I think part of the problem of coming up with a definitive passer rating (not QB rating) is deciding whether to measure averages or value. Since 1932 the average pass thrown in the NFL (excluding AAFC/AFL) has gone for 0.586 completions, 6.31 yards, 0.0403 TDs, and 0.0322 picks. The current system is based off similar averages, but those averages don’t account for the TD=YDs*20 or INT=YDs*-50. When you finagle the current system to meet those requirements, you end up far away from the average numbers. Essentially, you are measuring two separate things.