Flaccoing?

Assume that it is within a quarterback’s control as to whether or not he throws a completed pass on any given pass attempt. However, if he throws an incomplete pass, then he has no control over whether or not that pass is intercepted.

If that assumption is true, that would mean all incomplete pass attempts could be labeled as “passes in play” for the defense to intercept. Therefore, a quarterback’s average number of “Picks On Passes In Play” (or POPIP) — that is, the number of interceptions per incomplete pass he throws — is out of his control.

After doing the legwork to test that assumption, I reached two conclusions. One, interception rate is just really random, and predicting it is a fool’s errand. Two, using a normalized INT rate — essentially replacing a quarterback’s number of interceptions per incomplete pass with the league average number of interceptions per incomplete pass — was a slightly better predictor of future INT rate than actual INT rate. It’s not a slam dunk, but there is some merit to using POPIP, because completion percentage, on average, is a better predictor of future INT rate than current INT rate.

So, why am I bringing this up today, at the start of Super Bowl week? Take a look at where Sunday’s starting quarterbacks ranked this year in POPIP (playoff statistics included, minimum 250 pass attempts):

Everything about New England’s offense was an outlier this year, so I’m never surprised to see Brady at the top of any list. But what does it mean that Flacco and Kaepernick come in at 2nd and 3rd in (fewest) interceptions per incomplete pass? Well, the initial reaction is that perhaps both quarterbacks were really lucky, particularly Flacco. Kaepernick ranked 11th in completion percentage and finished 4th in interception rate, thanks to his great POPIP. Meanwhile, Flacco ranked 21st in completion percentage and finished 5th in interception rate! That’s a particularly drastic split, although not necessarily one that places him in great company.

Things get even more interesting when you consider the types of throws each quarterback is making. In my second POPIP post, Mike Clay helped me quantify the relationship between interception rate and the length or depth of a throw. Look at the NFL averages below:

On throws behind the line of scrimmage, completion percentage is very high and interception rate is very low; as pass attempts get progressively farther away from the line of scrimmage, the interception rate increases and the completion percentage decreases.

That all makes sense, until we get to this next bit of information, courtesy of the ESPN Stats and Information department: ^[1]Via e-mail, ESPN also added this: Including the postseason, Kaepernick has the best completion percentage (52.6%), yards per attempt (17.0) and QBR (99.7) on throws 20+ yards downfield.

Including the postseason, Joe Flacco and Colin Kaepernick are #3 and #4 respectively in average length of throw. Flacco’s average pass travels 9.93 yards in the air, Kaepernick’s 9.66. Jay Cutler and Andrew Luck are #1 and #2.

So Joe Flacco throws a ton of deep passes. And his completion percentage is pretty low. No issues there. The weird part is that his interception rate is extremely low. It would be pretty simple for me to end this post with the line, “Joe Flacco has had a really low interception rate this year, despite being both inaccurate and a risky passer (as measured by his average length of throw). You can imagine how sustainable that is.”

Except here’s the weird thing: Flacco’s had a really good interception rate most of his career. ^[2]I’m not really sure what to do with this information, but it probably isn’t irrelevant. Flacco threw ten interceptions this year, and I checked the official play-by-play logs to see how … Continue reading It is, of course, possible that Flacco has simply been Wyatt Earp, and his POPIP will regress to the league average in the future. But that’s not the only possibility.

Commenter Red brought up a good point in the original POPIP post:

Could it be that accuracy and decision making are two independent components of QB play? Brett Favre is probably the best example of this. He had several seasons in which he completed over 65% of his passes, yet still had awful INT % in those years (2003 and 2008 come to mind). Favre was always known as a great thrower, but a poor decision maker, which is backed up by the discrepancy between his completion % and INT/INC %.

I suppose that would make Flacco a good decision maker but an inaccurate quarterback? I’m not sure if those are the elements that lead to the fifth best interception rate in the NFL, but it’s worth pondering. Roman Gabriel, Doug Williams, Neil O’Donnell, Neil Lomax, Roger Staubach, and Donovan McNabb threw far fewer interceptions than you would predict based on their completion percentages. Subjectively, Flacco doesn’t really remind me of a McNabb or an O’Donnell, a player who works well within the system and doesn’t take many chances. Joe Montana, Mark Brunell, Ken Anderson and Neil Lomax also had pretty low POPIP ratios, too, although I’m not sure if there’s a common thread among those quarterbacks, either.

So, is Flacco lucky, or is he the type of quarterback who will always have a high average depth of throw, a low completion percentage, and a low interception rate? And if so, why is that the case (perhaps because he’s usually playing in favorable game scripts)? My intuition is that he’s lucky, although he’s been lucky for quite a few years now. If that’s the case, you never know when Lady Luck will turn on you, and Super Bowl Sunday would be as likely a day as any other. But it’s worth pondering whether there is a legitimate reason Flacco’s been able to keep his interception rate so low.

[Update: I e-mailed with Mike Clay after publishing this post, and Mike showed me some numbers indicating that Flacco historically has had a really low interception rate on deep passes. In the second footnote to this post, I noted that only 2 of his 148 passes that were labeled “deep” were intercepted. After Mike pointed out his historical data, that made me take another look at the 2012 numbers. I think deep means greater than or equal to 15 yards, so based on the aDOT/INT table, you would expect 8-12 interceptions on so many deep throws. Considering that Flacco threw only two, and has a history of not throwing interceptions on deep passes, perhaps there’s something there.

On the other hand, I hesitate to give him too much credit, because I would think “trying not to throw interceptions when you go deep” isn’t a skill that only Joe Flacco has acquired. Part of it may also be related to the game script idea hinted at above, in that more of Flacco’s deep passes may come when his team is winning and he doesn’t need to be as risky. All quarterbacks throw more interceptions in losses, but he threw 6 INTs in 5 losses (ignoring the CIN game) and only 4 in 10 wins (13, if you count the post-season). Essentially, it may be that Flacco is throwing passes like he’s losing but since he’s winning, he can be more conservative with them. I’m also not sure how that impacts how we would predict his future interception rate: if his team is trailing, he may revert to the league average in that metric, but if his team continues to be a Super Bowl contender, maybe it won’t.]

The closest I'm willing to get with a baseball photo.

Therefore, if a pitcher has a high BABIP, sort of like an NFL team with a lot of turnovers, he’s probably been unlucky. And good things may be coming around the corner. A high BABIP means a pitcher probably has an ERA higher than he “should” and that his ERA will go down in the future. In fact, you can easily recalculate a pitcher’s ERA by replacing the actual BABIP he has allowed with the league average BABIP. And that ERA will be a better predictor of future ERA than the actual ERA. At least, I think. Forgive me if my baseball analysis is not perfect.

Are you still awake? It’s Monday, and I’ve brought not only baseball into the equation, but obscure baseball statistics. Let’s get to the point of the post by starting with a hypothesis:

Assume that it is within a quarterback’s control as to whether he throws a completed pass on any given pass attempt. However, if he throws an incomplete pass, then he has no control over whether or not that pass is intercepted.

Essentially, we’re saying that all incomplete passes are “passes in play.” Therefore, a quarterback’s average of “Picks On Passes In Play” — the number of interceptions per incomplete pass he throws — is out of his control. This is just a hypotheses; how would we go about proving or disproving this theory?

Manning and Roethlisberger obviously discussing the intricacies of POPIP.

Eli actually had a higher completion percentage than Roethlisberger. Maybe that’s a sign that completion percentage and interception rate aren’t strongly related, or maybe it’s a sign that interception rates on incomplete passes are random. In 2010, 7.5% of all incomplete passes were intercepted. If 7.5% of all of Eli Manning’s incomplete passes were intercepted, he would have thrown 15 interceptions in 2010 instead of 25, and therefore would have an estimated interception rate of 2.8% based on a league average POPIP ratio. For Big Ben, if 7.5% of his incomplete passes had been picked off, he would have thrown 11.2 interceptions instead of just five; that would give him an estimated INT rate of 2.9%.

Okay, so what does any of this mean? Manning had an actual INT rate of 4.6% but an estimated rate of 2.8%, while Roethlisberger was at 1.3% and 2.9%, respectively. Obviously this doesn’t impact what’s already happened. But just like how BABIP can help predict future INT, POPIP could help predict future INT rate — well, at least that’s the theory.

As it turns out, in 2011, Manning and Roethlisberger both had actual INT rates of 2.7%. That gives us two pieces of evidence that estimated INT rate based on POPIP is a better predictor of future INT rate than actual INT rate. But those are just two pieces of evidence.

Since 1970, there have been 813 quarterbacks to play for the same team in consecutive years, and throw at least 224 passes in Year N and at least 100 passes in Year N+1. Why those cutoffs? 224 passes in the current minimum number of attempts needed to qualify for passing crowns in rate statistics; meanwhile, I don’t want to lose out on including quarterbacks who were benched early the next year because they were bad. But we can play around with several cutoffs.

I went through and gave a win to the actual INT rate if the player’s interception rate in Year N+1 was closer to his actual INT rate from Year N than his estimated INT rate. On the other hand, if the player’s estimated INT rate in Year N was closer to the actual INT rate in Year N+1 than the actual INT rate, I gave a win to the estimated INT metric.

The results? 335 times, the actual interception rate proved to be the better predictor, while 478 times the estimated interception rate closer to the future rate. So 59% of the time, using estimated INT rates based on POPIP proved to be helpful.

If we make the quarterbacks throw 224+ times in both years, we’re left with 693 pairs of quarterback seasons. The estimated INT rate was a better predictor in 403, or 58%, of those pairs.

If we go back to only 1978 instead of 1970, the estimated interception rate was better 58% of the time out of 595 pairs. If we change the cutoff to 1990, the estimated interception rate was better 58% of the time on 385 pairs.

If we bump the attempts threshold to 350 in both Year N and Year N+1, since 1990, the estimated interception rate was the better predictor on 157 of 253 pairs, or 62% of the time. If we limit it to just the past ten years, we have 122 pairs of quarterbacks — and the estimated INT rate was better on 63% of those pairs. Change the cutoffs to 400 attempts both years and look at only quarterbacks over the last five years, and 60% of the time on 57 pairs the estimated interception rate was better.

I think you get the point. Estimated interception rate isn’t perfect — there are so many fluky interceptions that no one could come very close to predicting future interception rate — but I feel pretty confident in telling you that estimated interception rate is better at predicting future INT rate than actual INT rate. In that vein, it is similar to how Pythagorean winning percentage is a better predictor of future winning percentage than actual winning percentage.

We can also use POPIP to find quarterback outliers. We can calculate how many interceptions a quarterback was estimated to throw for his career along with how many he actually threw. The difference there might be pretty informative. The table below lists all quarterbacks since 1970 with at least 50 interceptions; as always, quarterback who played prior to 1970 are included but only their stats since 1970 are reflected in the table below. The table is currently sorted based on the quarterbacks who came in under their estimated number of interceptions the most. Let me use Donovan McNabb as an example.

The first row reads: McNabb for his career has 5,374 attempts and 2,204 incomplete passes. His career INT rate is 2.2%, and has thrown an interception on 5.3% of his incompletions. Based on league average POPIP, we would have expected him to throw an interception on 3.2% of his passes. For his career, he has 117 interceptions and we would estimate that he would have thrown 171.1 interceptions. As a result, McNabb has thrown 54.1 fewer interceptions than we would have estimated (this is what the table is sorted by). He has thrown one fewer interception per 100 passes than we would have projected.

It’s tempting to attribute much of the spread between actual and estimated interceptions to luck. No doubt, luck plays a big part, and is perhaps the biggest single factor. I don’t know if there is a lot of talent involved in keeping your POPIP low; both Peyton Manning and Drew Brees have almost exactly hit their estimated interception numbers. Greats like Brett Favre and Kurt Warner came in with a bit more interceptions than expected, while Donovan McNabb, Bernie Kosar and Ken O’Brien were great at having a low POPIP average. And, of course, we should need to guard against circular logic such as ‘Keeping your POPIP low is a skill because some of the best quarterbacks kept it low, and those quarterbacks are some of the best quarterbacks because they didn’t throw many interceptions.’

But just like we can look at the relationship between sack rate and interception rate to see a quarterback’s style of play rather than just consider stats as good and bad, we can do the same here. Commenter Red astutely pointed out that McNabb had a low interception rate even though he was inaccurate, and he did seem to throw a lot of bad passes towards his receivers’ feet and not by defenders’ hands. I don’t really think of Warner and Favre as similar quarterbacks, but both did seem to throw a fair share of interceptions.

Ken Stabler is the one that really throws me. His INT/INC rate is off the charts, even for his era. Let me know your thoughts in the comments. I’ll close with a cool chart, showing the INT/ATT and INT/INC rates for every year since the merger.