## Was Belichick Correct In Going For It in the AFC Championship Game?

Matt, a Patriots fan and friend of mine, sent me an email after the AFC Championship Game, asking me to analyze New England’s decision to go for it in the following situation:

At the Denver 16-yard line, 4th-and-1, 6:03 remaining in game, trailing by 8 points

Play: Tom Brady pass complete short left to Julian Edelman for -1 yards (tackle by Chris Harris and Aqib Talib)

In real time, I thought this was a no-brainer. You have to go for it. That’s because, in general, 4th-and-1 is a “go for it” down. But after some deep review, I’m not so sure.

Down by 8 points seems to cut in favor of needing a touchdown, but  not quite as much as you might think. That’s because announcers — and maybe coaches — seem to treat being down 8 points as not much different from being down 6 or 7 points. But being down 8 is not the same thing as being down 7, as the Patriots painfully discovered later in the game. And being down 7 is a far cry from being down 6, where a touchdown wins the game. Down by 6, I think it’s clear that you go for it.1 But down by 8, it’s easy to play for one score, which relies on you both making a 2-point conversion (50/50) and winning in overtime (50/50). Taking the field goal cuts the lead to 5, and then you can win the game in regulation with a touchdown.

Let’s look at some numbers.

• Down by 6: According to Brian Burke’s win probability model, if the Patriots were down by 6 points, the decision is clear: New England’s win probability would be 47% with a conversion and 17% with a miss; given the 74% expected conversion rate (more on that in a minute), the Patriots would have a 39% chance of winning if they chose to go for it, and 22% if they kicked the field goal.
• Down by 7: If the Patriots were down by 7, the decision is still in favor of going for it: the expected conversion rate doesn’t change, of course, but the win probability drops from 29% with a successful conversion, and 9% with a failure, for a 24% weighted average.  By kicking a field goal, New England’s win probability would be 22%2.
• Down by 8: Now, the probabilities drop a bit, to 21% in the event of a conversion and 7% in the event of a miss, for a weighted-average win probability of 17%.3   A field goal attempt? That has a win probability of 18%, courtesy of a 20% WP if successful and 7% with a miss.  As a result, down by 8 makes for a big difference, and here, swings the odds.

Now, a difference of one percent is well within the margin of error, so let’s look into this a little more closely.  First, the field goal odds, which assumes an 86% conversion rate on the field goal attempt.  That is way too low.  Why? During the 2015 regular season, kickers made 166 of 173 (96%) kicks from between the 13-19 yard line.  Given that Gostkowski is one of the top three field goal kickers in the league, and that Denver is the easiest place to kick, it seems like his odds of converting were no less than 96%.4 Sure, he missed an extra point earlier in the game, but that doesn’t change the analysis. Frankly, he probably hits from 33 yards away here more than 96 times out of 100.

If Gostkowski hits, he likely follows that up with a touchback. Based on Brian’s win probability calculator, Denver would have a 79% chance of winning if the Broncos had the ball at their 20, up by 5, with 5:55 remaining. 5  Factor in the 4% chance of a miss, and we can estimate that the Patriots had a roughly 20% of winning if they elected to kick.

Now, what about the 18%? That assumes a 74% conversion rate.  On 3rd-and-1 or 4th-and-1 from the opponent’s 30 to the opponent’s 10 yard line, teams converted 71% of the time this year.6  From 2011 to 2014, that rate was 66%.

But here’s something to consider: rushing plays are more successful than passing ones in short yardage.  Rushing plays converted 73% of the time this year, while passing plays converted 64% of the time. From 2011 to 2014, rushing plays converted 69% of the time, while passing plays converted 55% of the time.  I am not sure if the higher numbers in 2015 are representative of a smaller sample size (and therefore sample error) or a trend, given how pass-friendly the 2015 season was.

New England being New England, you have to figure the Patriots odds were better than average.  On the other hand, the Broncos pass defense was awesome in general, and really awesome yesterday in particular.  Estimating New England’s odds of converting on 4th-and-1 knowing that they chose a pass play is difficult, but I do feel pretty confident in saying it was less than 74%.  I think a 2-out-of-3 likelihood makes sense here.7

Given that New England wasn’t going to run (a decision I agree with)8, let’s say the Patriots had a 67% chance of converting.  Using Burke’s numbers, that means a weighted average winning percentage of 16%.  So we’re talking about a 4% swing (rather than a 1% swing using the earlier assumptions). That’s not insignificant (thought of another way, the decision dropped New England’s odds of winning by 20%), but it’s probably within the margin of error.

On the other hand… there may be some errors with Burke’s model. Let’s look at his Win Probability calculator (rather than the 4th down calculator): Denver would have a 93% chance of winning if the Broncos had the ball, up by 8, at their own 17, with about six minutes to go.  This is the New England miss option, which jives with the 7% number.

On the other hand, if you have 1st-and-10, down by 8, with 5:30 to go, at the opponent’s 10-yard line, the win probability is 29%.  At the 11, it’s 29%. At the 12, it’s 29%. At the 13, it’s 29%. At the 14, it’ 29%. But at the 15, it’s 14%!  That’s obviously a bug in the model, but it may be a big one.  If we say the Patriots has a 29% win probability with a make, and a 7% with a miss, that puts the Patriots right at 21% with a 67% conversion rate. Drop the conversion rate to 63%, and the win probability drops to 20%.  In other words, this one is really, really close.

The NYT 4th Down bot thought it was a good call, putting New England’s win probability at 23% if the Patriots went for it and 21% if they kicked the field goal.  The bot assumes a 68% conversion rate, so a 23% blended average could be the result of say, 29% with a success and 7% with a miss.

So this one is well within any margin of error.  But even that result is interesting to me, since I thought it was clear that the Patriots should have kept the offense on the field. I think the notion that New England should have gone for it because they were so close to the end zone and had struggled all day is misguided; this is the Patriots, and teams routinely have their best offensive drive of the day when trailing at the end of games.  I thought they should go for it for the normal reasons: it was only one yard, and a miss would back Denver up. But I don’t think I quite processed the gravity of how much being down by 8, rather than 7, was a factor here.  I think I’m generally much more aware of that issue than most, but even this one slipped by me because of the default idea of “go for it on 4th-and-1.”  If nothing else, this analysis made me think.

1. The irony is not lost on me that some coaches would view this as a reason to kick, because you can make it a one-field goal game. []
2. This is outside the scope of this article, but it’s interesting that the win probabilities are considered the same here. That’s because down by 4 is actually better than being down by 3, according to the win probability models, in these situations, because it encourages aggressive play by the offense.  On the other hand, in the unlikely event of a miss, obviously it’s much better to be down by six.  These seem to largely wash out in the math. []
3. That seven percent might seem low given what happened, but recall that before the Patriots last offensive snap of the season — facing 4th-and-goal at the Denver 4-yard line — New England’s winning percentage was roughly 12.5%.  So everything that happened after the Patriots 4th-and-1 play (which included two Denver three-and-outs) only raised the team’s win probability by 5-6 percent. []
4. I’ll note that since field goal rates have been consistently rising, if you look at numbers over the last 5 years, you need to consider those a very conservative floor. []
5. You may notice that the “successful” conversion yields a 20% WP; that’s because the model assumes the ball at the 22; that makes sense for most games, but not one in Mile High.  That two-yard difference drops the win probability by a point. []
6. Why the 10 and the 30? The closer you get to the goal line, the harder the conversion gets, so stopping at the 10 makes sense.  And to increase my sample size, I went out to the 30, rather than the 22. []
7. Believe it or not, no team threw a pass on 3rd-and-1 or 4th-and-1 against Denver during the entire regular season.  New England, meanwhile, had just one such play.  That came against the Jets, and Brady was sacked. []
8. Another thing to consider: running in that situation is more likely to convert at home than on the road. []
• James

“teams routinely have their best offensive drive of the day when trailing at the end of games”

This made me think. Isn’t that often because teams are forced to pass more and passing is generally more efficient than running? If so, I think the Pats were already playing close to optimally, and their success the final couple of drives was largely due to randomness/both of Denver’s starting safeties getting injured, so I’m not sure if they would benefit as much late as other teams.

• evo34

Is this [Stuart’s quote] even accurate? Or is it that we tend to remember late-game drives and forget about the routine failures and subsequent kneel-downs by the leading team?

• Well, I only said routinely 🙂

• Good point, James. That’s definitely part of it – and teams have no choice but to play more aggressively while trailing. For NE, I’m not sure much would change.

But the other huge part of it is having 4 downs rather than 3. And we saw that come up on NE’s final drive. What would have otherwise been a 3-and-out resulted in a touchdown.

• Trepur

I made this revelation in the Packers-Vikings week 17 game, about how suboptimal going for it on 4th and long is when trailing 7 or 8 points inside of field goal range.

Packers were in 4th and 13, and the situation became worse as they spent their 2nd time out to prevent a delay of game.

As soon as the Packers took their time out, it got me thinking. Why not take the delay of game, kick a field goal, try and force a stop (with 2:20 left and two time outs rather then one) and then get a TD to win the game?

The logic was:
1. converting a 4th and 13 is hard.
2. Even if you convert, it doesn’t win you the game, it only ties the game. They then have 2:20 and 3 time outs to kick the game winning field goal and should they fail, it becomes a 50/50 toss-up in OT.

Since you had to score twice either way, might as well take the field goal now.

I then had the same thought process in the Patriots-Broncos game, but the situation for the Pats was more dire, as a touchdown wouldn’t tie the game, but would still need a 2-point conversion afterwards.

I did still think going for it on 4th and one was the right call, it’s easy enough to convert that it’s definetely worth trying to avoid needing to get into the redzone a second time (especially given how much the Pats struggled to move the ball).

But on 4th and six, I applied the logic above (with the added step that if you consider the 2-point score, you need to score 3 times with going for it, but only twice when taking the field goal) and argued with a friend that the FG was the right call.

• Over the last three seasons I believe the Pats conversion rate via pass on 3rd/4th and 1 is about 53%, which is slightly less than the league average. Id think that should translate to being under league average even as you go in the opponents 10-30 yardline range so Id drop them down from the 66.7% chance you were giving them on a conversion especially considering the defense. Gronkowski is generally their go to guy in that situation as well as passes to the backfield so the call designed to go to Edelman Id think that drops it further too.

In hidsight I guess the lack of confidence in their line and depleted group of running backs probably overinfluenced that decision. Teams had been successful in those spots running against the Denver defense this year.I still think going for it is the right decision but Im not sure if that particular play stood a high chance of conversion against a team playing such solid defense.

• sacramento gold miners

I probably would have gone for it as well, due to the short distance, and the Pats didn’t know how the rest of the game would play out. New England was losing the physical battle, and the lack of a offensive balance was costly against a quality defense like Denver’s. We also have to keep in mind these short passes often don’t have the type of blocking support seen in the run game, and you can’t begin to compare the power of a good size RB to a WR like Edelman.

• Honest question on a minor point: Is a 33 yard field goal really any easier in Denver than anywhere else?

• I think so. All kicks are easier. It may be akin to say, a 30-yard FG elsewhere, but that still makes it easier. The difference is probably larger at longer distances, though.

Gostkowski was 17/17 on kicks inside of 40 yards in the regular season, and 52/52 on extra points. Note that extra points are now at the 15, and the Patriots were at the 16, so that makes that data pretty valuable. Given that he was 69/69, I’d say his odds of making that kick were pretty damn good.

• Yeah, 53 in Denver would have to be easier than at sea level to a greater extent than 33. (Elevation matters on almost guaranteeing the touchback too.)

I’m also sure having Gostkowski instead of a closer-to-average kicker should have been a significant part of the calculus, but maybe even Belichick was a tad susceptible to some recency bias in making his decision.

• Richie

“but maybe even Belichick was a tad susceptible to some recency bias in making his decision.”

Are you implying that Belichick lost trust in Gostkowski? I can’t believe that was a consideration.

• Do you mean “lost trust” as in “I don’t trust him to make it” or “I don’t trust him to make it as much as I usually do”? I wouldn’t believe the former either, but it could have been the latter (and not even as an explicit thought).

Either way, when I use weasel words like maybe, even, tad, and some, that indicates a lack of confidence in the supposition rather than an insidious implication.

• Rick

If you think Belihick didn’t kick the FG because he was afraid Gostkowski would miss such a short FG attempt, I think you’re just flat wrong. You’re conjuring an explanation (“recency bias”) out of nothing. Belichick is not a stupid man. He’s not going to think that a 37-yard FG attempt is dangerous just because Gostkowski missed a PAT.

• You’re making the initial citation into a more definitive statement than it was.

• Mark Moore

Well, I must admit I was in the ‘you have to go for it’ camp because, like you say, 4th-and-1 is usually a no-brainer. Easy to forget that forcing overtime is not the same as winning (though it seems so obvious when you write it) and that there is huge value in scoring now to mean that a touchdown wins.

• Matt

Thanks Chase. The other thing that had me thinking the Patriots made the wrong decision was that the Patriots defense in the second half was doing a good job at stopping the Broncos. If the Patriots make the field goal and get a touchback (both likely scenarios) then the only difference if they went for it on fourth down and failed is four yards plus the defense had done a good job so I don’t think it was likely that Denver would take the ball and score so the Patriots shoukd have had one more opportunity with the ball for the win.

Rare to question these decisions as the Patriots usually do the right thing but this is one time where I think Beli made the wrong call.

• Rick

You’re really glossing over both the difficulty of stopping the Broncos from getting only one first down and of driving down the field with so little left on the clock. That the Patriots were able to do both of these things doesn’t mean that the decision beforehand should have assumed they would be easy to accomplish.

• Richie

“But being down 8 is not the same thing as being down 7”

That’s weird, because being down 14 IS the same as being down 7.

Sincerely,
Andy Reid

• Richie

I also thought “go for it” was a no-brainer, and find this analysis interesting. “Settling” to pursue a tie with an 8-point deficit appears to be the wrong choice (due to the 50/50 of converting the 2-pointer and then the 50/50 of winning in OT).

One other point in favor of “kick the FG” that I don’t think you touched on is that the play was a very long 1 yard, closer to 2 in my memory. I think 4th and 2 converts about 10% less often than 4th and 1 in recent history.

• Tom

As someone rooting for the Broncos, I didn’t want the Pats to go for it at all…I know their offense was getting beat up all day, but man, they are so efficient (perhaps I’m historically biased on this), I’d never underestimate their ability to gain a yard (or a few yards, etc.). Sometimes I feel like the “right” decision is the one the other team doesn’t want you to make, if that makes any sense.

• Richie

Yeah.

I think Cardinals fans would have been terrified had the Packers lined up to go for 2 at the end of regulation last week.

• Tom

Richie – yeah, that’s exactly what I’m talking about. I bet they all heaved a HUGE sigh of relief when they saw Rodgers walk off the field and Crosby walk on…

• Rick

It isn’t entirely correct to use Brian Burke’s probabilities, which are derived from seeing all teams against all opponents, and saying that they should be exactly correct in estimating the Patriots’ win probability in this situation. This is something you notice when making the adjustment for Gostkowski’s FG percentage. But the same reasoning applies to the question of how well the Patriots can move the ball against the Broncos’ defense.
Aside from the fumble recovery, that first FG non-attempt in the 4th was decided the first time the Patriots had gotten that close to the end zone. And the Patriots needed to get into the end zone at some point. You cannot condition the likelihood of returning to that position on the field simply by averaging all offenses against all defenses. You have to condition on the Patriots’ offense against that Broncos’ defense. And things were not going well for the Patriots’ offense that day.
There was no reason for Belichick to feel confident about returning to that position again in the last five minutes. That they were able to was partly miraculous (and partly due to the fact that the Broncos went all conservative and didn’t get any more first downs.) Two things could have gone wrong if the Patriots had kicked a FG (well three, if we add missing the FG, but I’m assuming it’s good). First is that the Broncos get the kickoff and never give the ball back. Second is that the Broncos give the ball back but their defense stops the Patriots from getting into the Red Zone (which is the result we should have expected based on the game had been going.)
I don’t think it’s entirely fair to use WP calculations to say that Belichick was “wrong”. He understood the situation based on how he felt his team was doing on that day. He’s not coaching Generic Offense against Generic Defense. There are conditional probabilities that cannot be ignored here, even if we don’t know exactly what they are.

• Anonymous

Another thing in favor of kicking the FG is that Denver may still win the game in regulation even if the Patriots got the TD and 2 point conversion. Teams optimize a lot more when they are tied than if they are winning. If you assume that NE may have used another minute of the clock when driving the extra 14 yards, that still gives Denver a huge surplus of time to get a score on their next drive and leave NE down by 3 or 7 with very little time to go. On the other hand, if Denver has a 5 point lead, that means they’ll have the 4 minute drill where they’ll run the ball up the middle 3 times in a row when the defense knows it’s coming and Denver is plenty happy with a 3 and out as long as it takes 1:30 off the clock.