## Interceptions per Incompletion (or POPIP)

The closest I'm willing to get with a baseball photo.

I leave the baseball analysis to my brothers at baseball-reference.com, but I know enough to be dangerous. There’s a stat called BABIP, which stands for Batting Average on Balls In Play. A “ball in play” is simply any at bat that doesn’t end in a home run or a strikeout. The thinking goes that luck and randomness is mostly responsible for the variance in BABIP allowed by pitchers to opposing batters. Pitchers can control the number of strikeouts they throw and control whether they allow home runs or not, but they can’t really control their BABIP.

Therefore, if a pitcher has a high BABIP, sort of like an NFL team with a lot of turnovers, he’s probably been unlucky. And good things may be coming around the corner. A high BABIP means a pitcher probably has an ERA higher than he “should” and that his ERA will go down in the future. In fact, you can easily recalculate a pitcher’s ERA by replacing the actual BABIP he has allowed with the league average BABIP. And that ERA will be a better predictor of future ERA than the actual ERA. At least, I think. Forgive me if my baseball analysis is not perfect.

Are you still awake? It’s Monday, and I’ve brought not only baseball into the equation, but obscure baseball statistics. Let’s get to the point of the post by starting with a hypothesis:

Assume that it is within a quarterback’s control as to whether he throws a completed pass on any given pass attempt. However, if he throws an incomplete pass, then he has no control over whether or not that pass is intercepted.

Essentially, we’re saying that all incomplete passes are “passes in play.” Therefore, a quarterback’s average of “Picks On Passes In Play” — the number of interceptions per incomplete pass he throws — is out of his control. This is just a hypotheses; how would we go about proving or disproving this theory?

Manning and Roethlisberger obviously discussing the intricacies of POPIP.

Let’s look at an example. In 2010, Eli Manning led the NFL with 25 interceptions, throwing an interception on 4.6% of his pass attempts, and on 12.5% of his incomplete passes. That same year, his draft classmate Ben Roethlisberger had just five interceptions on 389 attempts. Roethlisberger averaged only 1.3 interceptions per 100 passes, and threw an interception on just 3.4% of his incomplete passes.

Eli actually had a higher completion percentage than Roethlisberger. Maybe that’s a sign that completion percentage and interception rate aren’t strongly related, or maybe it’s a sign that interception rates on incomplete passes are random. In 2010, 7.5% of all incomplete passes were intercepted. If 7.5% of all of Eli Manning’s incomplete passes were intercepted, he would have thrown 15 interceptions in 2010 instead of 25, and therefore would have an estimated interception rate of 2.8% based on a league average POPIP ratio. For Big Ben, if 7.5% of his incomplete passes had been picked off, he would have thrown 11.2 interceptions instead of just five; that would give him an estimated INT rate of 2.9%.

Okay, so what does any of this mean? Manning had an actual INT rate of 4.6% but an estimated rate of 2.8%, while Roethlisberger was at 1.3% and 2.9%, respectively. Obviously this doesn’t impact what’s already happened. But just like how BABIP can help predict future INT, POPIP could help predict future INT rate — well, at least that’s the theory.

As it turns out, in 2011, Manning and Roethlisberger both had actual INT rates of 2.7%. That gives us two pieces of evidence that estimated INT rate based on POPIP is a better predictor of future INT rate than actual INT rate. But those are just two pieces of evidence.

Since 1970, there have been 813 quarterbacks to play for the same team in consecutive years, and throw at least 224 passes in Year N and at least 100 passes in Year N+1. Why those cutoffs? 224 passes in the current minimum number of attempts needed to qualify for passing crowns in rate statistics; meanwhile, I don’t want to lose out on including quarterbacks who were benched early the next year because they were bad. But we can play around with several cutoffs.

I went through and gave a win to the actual INT rate if the player’s interception rate in Year N+1 was closer to his actual INT rate from Year N than his estimated INT rate. On the other hand, if the player’s estimated INT rate in Year N was closer to the actual INT rate in Year N+1 than the actual INT rate, I gave a win to the estimated INT metric.

The results? 335 times, the actual interception rate proved to be the better predictor, while 478 times the estimated interception rate closer to the future rate. So 59% of the time, using estimated INT rates based on POPIP proved to be helpful.

If we make the quarterbacks throw 224+ times in both years, we’re left with 693 pairs of quarterback seasons. The estimated INT rate was a better predictor in 403, or 58%, of those pairs.

If we go back to only 1978 instead of 1970, the estimated interception rate was better 58% of the time out of 595 pairs. If we change the cutoff to 1990, the estimated interception rate was better 58% of the time on 385 pairs.

If we bump the attempts threshold to 350 in both Year N and Year N+1, since 1990, the estimated interception rate was the better predictor on 157 of 253 pairs, or 62% of the time. If we limit it to just the past ten years, we have 122 pairs of quarterbacks — and the estimated INT rate was better on 63% of those pairs. Change the cutoffs to 400 attempts both years and look at only quarterbacks over the last five years, and 60% of the time on 57 pairs the estimated interception rate was better.

I think you get the point. Estimated interception rate isn’t perfect — there are so many fluky interceptions that no one could come very close to predicting future interception rate — but I feel pretty confident in telling you that estimated interception rate is better at predicting future INT rate than actual INT rate. In that vein, it is similar to how Pythagorean winning percentage is a better predictor of future winning percentage than actual winning percentage.

We can also use POPIP to find quarterback outliers. We can calculate how many interceptions a quarterback was estimated to throw for his career along with how many he actually threw. The difference there might be pretty informative. The table below lists all quarterbacks since 1970 with at least 50 interceptions; as always, quarterback who played prior to 1970 are included but only their stats since 1970 are reflected in the table below. The table is currently sorted based on the quarterbacks who came in under their estimated number of interceptions the most. Let me use Donovan McNabb as an example.

The first row reads: McNabb for his career has 5,374 attempts and 2,204 incomplete passes. His career INT rate is 2.2%, and has thrown an interception on 5.3% of his incompletions. Based on league average POPIP, we would have expected him to throw an interception on 3.2% of his passes. For his career, he has 117 interceptions and we would estimate that he would have thrown 171.1 interceptions. As a result, McNabb has thrown 54.1 fewer interceptions than we would have estimated (this is what the table is sorted by). He has thrown one fewer interception per 100 passes than we would have projected.

RkNameAttINCINT RTINT/INCEST INT RTINTEST INTINT DIFFINT RT DIF
1Donovan McNabb537422042.2%5.3%3.2%117171.1-54.1-1%
2Roman Gabriel226710503.3%7%5.1%74116.2-42.2-1.9%
3Joe Montana539119822.6%7%3.3%139178.1-39.1-0.7%
4Neil O'Donnell322913642.1%5%3.3%68105.5-37.5-1.2%
5Mark Brunell464018792.3%5.7%3.1%108144.7-36.7-0.8%
6John Elway725031273.1%7.2%3.6%226261.8-35.8-0.5%
8Neil Lomax315313362.9%6.7%3.9%90123.8-33.8-1.1%
9Ken Anderson447518213.6%8.8%4.3%160193.2-33.2-0.7%
10Dan Marino835833913%7.4%3.4%252284-32-0.4%
11Doug Williams250712673.7%7.3%5%93125-32-1.3%
12Ken O'Brien360214922.7%6.6%3.6%98129.6-31.6-0.9%
13Ron Jaworski411719304%8.5%4.7%164195.5-31.5-0.8%
14Phil Simms464720713.4%7.6%4%157187.9-30.9-0.7%
15Jim Hart418320104.5%9.3%5.2%187217.9-30.9-0.7%
16Bernie Kosar336513712.6%6.3%3.5%87117.6-30.6-0.9%
17Roger Staubach291112493.7%8.6%4.7%107137.1-30.1-1%
18Rich Gannon420616732.5%6.2%3.2%104132.7-28.7-0.7%
19Jeff Garcia367614122.3%5.9%3%83110.2-27.2-0.7%
20Fran Tarkenton344513893.8%9.5%4.5%132154.4-22.4-0.7%
21Joe Theismann360215583.8%8.9%4.4%138158.9-20.9-0.6%
22Steve McNair454418112.6%6.6%3.1%119139.5-20.5-0.5%
23Randall Cunningham428918603.1%7.2%3.6%134154.3-20.3-0.5%
24Bill Kenney243011003.5%7.8%4.3%86104.5-18.5-0.8%
25Bert Jones255111214%9%4.7%101119.1-18.1-0.7%
26Kerry Collins626127743.1%7.1%3.4%196212.9-16.9-0.3%
27Jeff George396716692.8%6.8%3.3%113129.5-16.5-0.4%
28Drew Bledsoe671728783.1%7.2%3.3%206220-14-0.2%
29Jason Campbell21318352.3%6%3%5064-14-0.7%
30Joe Ferguson451921504.6%9.7%4.9%209223-14-0.3%
31Michael Vick253811162.8%6.5%3.4%7285.7-13.7-0.5%
32Greg Landry20929194.1%9.4%4.8%8699.6-13.6-0.6%
33David Garrard22818752.4%6.2%2.9%5467.1-13.1-0.6%
34Steve Bartkowski345615244.2%9.4%4.5%144156.9-12.9-0.4%
35Billy Kilmer20289404.5%9.8%5.2%92104.7-12.7-0.6%
36Jim Zorn314914804.5%9.5%4.9%141153.6-12.6-0.4%
37Kyle Orton22049202.6%6.2%3.1%5769.4-12.4-0.6%
38Steve Young414914822.6%7.2%2.9%107119.4-12.4-0.3%
39Jim Harbaugh391816133%7.3%3.3%117128.3-11.3-0.3%
40Jay Schroeder280813823.8%7.8%4.2%108118.4-10.4-0.4%
41Doug Flutie21519743.2%7%3.6%6878.3-10.3-0.5%
42Tony Banks235610783.1%6.8%3.5%7382.8-9.8-0.4%
43Bobby Douglass10305915.4%9.5%6.3%5665.3-9.3-0.9%
44Aaron Brooks296312903.1%7.1%3.4%92101.3-9.3-0.3%
45Jeff Blake324114143.1%7%3.3%99108-9-0.3%
46Tony Eason15646533.3%7.8%3.8%5159.9-8.9-0.6%
48Mike Pagel15097534.2%8.4%4.7%6370.6-7.6-0.5%
49Mike Livingston15907624.8%10.1%5.3%7784.2-7.2-0.5%
50Mark Rypien261311473.4%7.7%3.6%8895.1-7.1-0.3%
51Gary Danielson19328274%9.4%4.4%7884.9-6.9-0.4%
52Jim McMahon257310813.5%8.3%3.8%9096.6-6.6-0.3%
53Philip Rivers303711072.6%7%2.8%7884.5-6.5-0.2%
55Jeff Hostetler23389813%7.2%3.3%7177.1-6.1-0.3%
56Brian Sipe343914954.3%10%4.5%149155-6-0.2%
57Chris Miller289213123.5%7.8%3.7%102108-6-0.2%
58Matt Hasselbeck479719063%7.5%3.1%142147.9-5.9-0.1%
59Alex Smith19598223%7.1%3.2%5863.2-5.2-0.3%
60Craig Morton320114534.7%10.4%4.9%151156.2-5.2-0.2%
61Bubby Brister221210053.5%7.8%3.7%7882.8-4.8-0.2%
62Don Majkowski19058493.5%7.9%3.8%6771.5-4.5-0.2%
63Matt Schaub22798132.5%7.1%2.7%5861.9-3.9-0.2%
64Archie Manning364216314.8%10.6%4.8%173176.4-3.4-0.1%
65Dan Pastorini305514995.3%10.7%5.4%161164-3-0.1%
66Boomer Esiason520522363.5%8.2%3.6%184186.8-2.8-0.1%
67Troy Aikman471518173%7.8%3%141143.3-2.30%
68Joey Harrington253811143.3%7.6%3.4%8587.2-2.2-0.1%
69Len Dawson13445644.5%10.8%4.7%6162.8-1.8-0.1%
70Stan Humphries251610853.3%7.7%3.4%8485.3-1.3-0.1%
71Billy Joe Tolliver17078163.7%7.8%3.8%6465.1-1.1-0.1%
72Trent Green374014743%7.7%3.1%114115-10%
74Warren Moon682328353.4%8.2%3.4%233233.8-0.80%
75Steve Beuerlein332814343.4%7.8%3.4%112112.7-0.70%
76Marc Bulger317112022.9%7.7%2.9%9393.4-0.40%
77Gus Frerotte310614073.4%7.5%3.4%106106.3-0.30%
78James Harris11135215.2%11.1%5.2%5858.3-0.30%
79David Carr22649133.1%7.8%3.1%7171.1-0.10%
80David Whitehurst9804765.2%10.7%5.2%5150.80.20%
81Tommy Kramer365116394.3%9.6%4.3%158157.70.30%
82Scott Brunner10465345.2%10.1%5.1%5453.40.60.1%
83Charley Johnson13456385.3%11.1%5.2%7170.40.60%
84Daryle Lamonica9814905.6%11.2%5.5%5554.30.70.1%
85Steve Walsh13176043.8%8.3%3.7%5048.71.30.1%
86John Brodie10694705.1%11.5%4.9%5452.61.40.1%
87Tony Romo25929202.8%7.8%2.7%7270.41.60.1%
88Eric Hipple15467164.5%9.8%4.4%7068.31.70.1%
89Peyton Manning721025282.7%7.8%2.7%198196.21.80%
90Dave M. Brown16347423.5%7.8%3.4%5856.21.80.1%
91Kyle Boller15196583.6%8.2%3.4%5451.82.20.1%
92Jack Trudeau16447714.2%8.9%4.1%6966.62.40.1%
93Scott Mitchell234610453.5%7.8%3.3%8178.52.50.1%
94Derek Anderson14366803.8%8.1%3.7%5552.52.50.2%
95Drew Brees547918662.7%7.8%2.6%146143.12.90.1%
96David Woodley13006134.8%10.3%4.6%6360.12.90.2%
97Rick Mirer20439553.7%8%3.6%7672.93.10.2%
98Mark Sanchez14146323.6%8.1%3.4%5147.63.40.2%
99Erik Kramer22999823.4%8%3.3%7975.33.70.2%
100Elvis Grbac24459993.3%8.1%3.2%8177.23.80.2%
101Jim Everett492320823.6%8.4%3.5%17517140.1%
102Eli Manning392116303.3%7.9%3.2%129124.94.10.1%
103Kordell Stewart235810423.6%8.1%3.4%8479.84.20.2%
104Steve DeBerg502421504.1%9.5%4%204199.64.40.1%
105Gary Huff7883966.3%12.6%5.6%5044.45.60.7%
106Rex Grossman15626993.8%8.6%3.4%6053.76.30.4%
108Ben Roethlisberger331312233%8.2%2.8%10093.56.50.2%
109Steve Ramsey9214656.3%12.5%5.5%5850.97.10.8%
110Gary Hogeboom13255824.5%10.3%4%6052.67.40.6%
111Mark Malone16488094.9%10%4.4%8173.17.90.5%
112Vince Young13045493.9%9.3%3.3%5142.98.10.6%
113Bob Griese249110384.9%11.8%4.6%122113.98.10.3%
115Dave Krieg531122063.7%9%3.6%199190.88.20.2%
116Mike Phipps17999136%11.8%5.5%10899.58.50.5%
117Jake Delhomme293211913.4%8.5%3.1%10192.28.80.3%
118Bob Avellini11105506.2%12.5%5.4%6960.18.90.8%
119Dave Wilson10394885.3%11.3%4.4%5545.39.70.9%
120Vince Evans13906865.3%10.8%4.6%7464.29.80.7%
121Jay Fiedler17177093.8%9.3%3.3%6656100.6%
122Dan Fouts560423074.3%10.5%4.1%242231.410.60.2%
123Randy Wright11195175.1%11%4.1%5746.110.91%
124Marc Wilson20819964.9%10.2%4.4%10290.711.30.5%
125Jay Cutler25219803.4%8.8%3%8674.611.40.5%
126Jim Plunkett370117585.3%11.3%5%198186120.3%
127Ryan Fitzpatrick17447123.7%9.1%3%6553120.7%
128Rodney Peete234610023.9%9.2%3.4%9279.312.70.5%
129Tim Couch17146893.9%9.7%3.2%6754.212.80.7%
130Chris Chandler400516773.6%8.7%3.3%146133.112.90.3%
133Daunte Culpepper319911833.3%9%2.9%10692.513.50.4%
134Carson Palmer354513223.3%8.8%2.9%116102.313.70.4%
135Mike Tomczak233710894.5%9.7%3.9%10690.715.30.7%
136Bobby Hebert312112824%9.7%3.5%124108.615.40.5%
137Jake Plummer435018663.7%8.6%3.3%161144.416.60.4%
138Danny White295011894.5%11.1%3.9%132115.116.90.6%
139Jim Kelly477919053.7%9.2%3.3%175158.116.90.4%
140Brian Griese279610443.5%9.5%2.9%9981.817.20.6%
142Kurt Warner407014043.1%9.1%2.7%128109.418.60.5%
143Dennis Shaw9244357.4%15.6%5.2%6848.519.52.1%
144Vince Ferragamo16157135.6%12.8%4.4%9171.419.61.2%
145Trent Dilfer317214134.1%9.1%3.4%129107.821.20.7%
146Joe Namath17198596.7%13.5%5.5%11694.321.71.3%
147Richard Todd296713575.4%11.9%4.7%161138.222.80.8%
148Jon Kitna444217653.7%9.3%3.1%165138.526.50.6%
149Vinny Testaverde670129144%9.2%3.5%267233.833.20.5%
150Steve Grogan359317145.8%12.1%4.8%208173.834.21%
151Brett Favre1016938693.3%8.7%3%336300.235.80.4%
152Lynn Dickey312513785.7%13%4.5%179140.138.91.2%
153Ken Stabler379315235.9%14.6%4.3%222162.159.91.6%

It’s tempting to attribute much of the spread between actual and estimated interceptions to luck. No doubt, luck plays a big part, and is perhaps the biggest single factor. I don’t know if there is a lot of talent involved in keeping your POPIP low; both Peyton Manning and Drew Brees have almost exactly hit their estimated interception numbers. Greats like Brett Favre and Kurt Warner came in with a bit more interceptions than expected, while Donovan McNabb, Bernie Kosar and Ken O’Brien were great at having a low POPIP average. And, of course, we should need to guard against circular logic such as ‘Keeping your POPIP low is a skill because some of the best quarterbacks kept it low, and those quarterbacks are some of the best quarterbacks because they didn’t throw many interceptions.’

But just like we can look at the relationship between sack rate and interception rate to see a quarterback’s style of play rather than just consider stats as good and bad, we can do the same here. Commenter Red astutely pointed out that McNabb had a low interception rate even though he was inaccurate, and he did seem to throw a lot of bad passes towards his receivers’ feet and not by defenders’ hands. I don’t really think of Warner and Favre as similar quarterbacks, but both did seem to throw a fair share of interceptions.

Ken Stabler is the one that really throws me. His INT/INC rate is off the charts, even for his era. Let me know your thoughts in the comments. I’ll close with a cool chart, showing the INT/ATT and INT/INC rates for every year since the merger.

• Richie

Interesting idea, but my instinct is that there’s nothing to this.

Isn’t your predicted interception numbers basically just regression? If a guy has a lot of interceptions in year N, isn’t he usually going to have fewer in N+1 due to regression?

As for the general theory, I think there are lots of different types of interceptions. If you could filter those out it might be more informative.

First of all, if all hail mary-type interceptions could be entirely removed from the analysis, that would be helpful. Those aren’t the same, because an INT and an incompletion are equivalent to the QB at that point.

Also, if there was a way to remove interceptions where a defender clearly sneaks in front of a receiver to grab the pass, would be helpful. I’m not sure if that is really an uncontrollable situation for the QB. That pass probably wasn’t going to be an incomplete anyway. But the QB probably could have avoided it if he had been watching the defenders better.

Tipped passes seem the most obvious candidates for this analysis. Once a pass gets tipped, it seems much more likely that what happens next (completion, incompletion or interception) is random.

Anyway, those are just some initial thoughts.

• Chase Stuart

Agreed with all of those thoughts. Ideally, you would want to analyze each interception thrown by a quarterback (and really, each pass thrown by a quarterback) and grade the throw, not the result.

Re: your regression comment, that’s true. If I had time, my plan was to compare these results to a few other models (comparing EST_INT_RT to just leave average rate to predict future INT rate for each QB; comparing EST_INT_RT to a regression-adjusted model for each QB’s INT rate (maybe give him 50% of his own INT rate, 50% of league average), etc.). But Sunday’s games took too much out of me. I think this is just step one of analysis, not the final step.

• Richie

But Sunday’s games took too much out of me.

F&\$^ing Jets.

• Chase Stuart

That’s an interesting article, and a cool project by Football Outsiders but I don’t think it’s very similar to my article.

• Richie

I think, maybe, he was mentioning that in rebuttal to my comment about classifying interceptions.

So, it might be interesting to apply POPIP to their data (though I think they only have data for 2011).

• Yeah, no, I wasn’t linking to it saying we’ve done your thing before. POPIP is definitely unique, and I like its straightforwardness. I was responding to Richie’s recommendations.

• Danish

Very interesting, and a nice idea. Those to graphs are extremely similar, to the point where I suspect you have made a mistake in excel. I guess the lack of spike in INT/ATT in ’82 gets you off the hook 🙂

• Red

Could it be that accuracy and decision making are two independent components of QB play? Brett Favre is probably the best example of this. He had several seasons in which he completed over 65% of his passes, yet still had awful INT % in those years (2003 and 2008 come to mind). Favre was always known as a great thrower, but a poor decision maker, which is backed up by the discrepancy between his completion % and INT/INC %.

• Chase Stuart

Yes, it’s possible that accuracy and decision making are pretty independent, which would go along with your theory. I’m not sure what that says about McNabb, though — that he wasn’t accurate but made good decisions?

I also think we’re going to be dealing with really small sample sizes. Outside of the few outliers like McNabb and Stabler, I’m not sure if we can make too many definitive statements.

• Red

I agree that POPIP is mostly sample size noise, and for 90% of QB’s it’s probably random. Outliers like McNabb are interesting, because frankly he just doesn’t make any sense. He wasn’t very accurate or very smart, and he often played in pass-heavy offenses that required him to throw in sub-optimal situations. I really have no idea why his POPIP is so consistently low throughout his career. I’m too young to have seen Stabler play, so I couldn’t even begin to formulate an explanation for his high POPIP.

Just out of curisosity, how many pass attempts does a QB need for his INT % to become statistically significant? I would guess it has to be at least 2000+ attempts before INT numbers mean anything.

It would be very interesting to see an alternate version of your GQBOAT rankings, using expected INT’s instead of actual INT’s. I’m thinking that would give us a more accurate picture of who was good and who wasn’t, with the randomness of interceptions pretty much eliminated from the equation.

• DC

Stats can only go so far in this case. You’d have to look at the game film for each interception to assess blame. Did the quarterback miss his target? Did the receiver forget his route? Did defensive line deflect the ball? Did the ball bounce off the target’s hands?

• Pingback: The definitive analysis of offensive fumbles()

• Pingback: More work on POPIP and predicting INT rates()

• Derek

I know this is a late post, but I just came across your article and I like the idea behind it. I’m a fan of advanced baseball statistics and would like to see similar work in football. Here are my thoughts on INT/INC…

I like the concept of determining the repeatability of avoiding interceptions. I don’t think the research should start by comparing individual seasons though. The sample sizes are too small to produce meaningful data.

You might glean more useful data when examining careers. If POPIP is primarily luck, the career rates should be fairly close because we could expect each QB’s POPIP to regress toward the average over the course of his career.

I took a cursory glance at the numbers. Excluding the five highest and lowest values, the range of INT/INC is 6% to 12.8%. The average is 8.82%. Out of 143 samples, 58 are within 1% of the average, and 114 are within 2% of the average. That’s far from a complete picture, but my gut tells me there is too much variance between each QB for INT/INC to be a non-repeatable skill.

The big caveat though is that the pool of qualifying QBs is only 143 (after five excluding highest and lowest values). I’d like to see more research on the subject with a bigger sample of quarterbacks, though I don’t think you’d want to push the minimum career length any lower. I wonder if a look at college numbers might provide more information. Perhaps you could establish a baseline for team quality and minimum career passes attempted, and compare the INT/INT career rates of those QBs.

• Pingback: Safest passers, post-merger | NFL's Greatest()