≡ Menu
Mike Smith, thinking about kicking or punting.

Mike Smith, thinking about kicking or punting.

With just under five minutes left in last Sunday’s game against the Giants and his team trailing 27-20, Mike Smith went for it on 4th and 1 from his own 29 yard line. As was the case on repeated 4th down attempts the last time his team visited MetLife Stadium to face the Giants, the decision to be aggressive did not work out well. Matt Ryan was sacked for a nine-yard loss that effectively ended the game. If his previous behavior is any guide, Smith may learn the wrong lesson from that outcome and choose not to go for it again when the next similar opportunity arises. Smith illustrates better than any other coach the potential for fourth down failure to lead to future fourth down timidity.

Before those two failed Ryan fourth down sneaks against the Giants in that 2011 playoff game, Smith actually was one of the more enlightened coaches on fourth down strategy. From 2008-2011, Smith was the third-most aggressive coach of the last twenty years, at least according to Football Outsiders’ Aggressiveness Index. Dating Smith’s turning point is a little tough. He got burned going for it in Week 10 of the 2011 regular season, when he tried a sneak on 4th and inches from his own 29 in overtime against the Saints. He punted in a couple of situations where he usually went for it late in the 2011 season, but then was aggressive closer in against the Giants. By the 2012 regular season, Smith hadn’t just abandoned his prior tendency for aggressive strategy. He entirely reversed it. In 2012, he was the least aggressive coach in football, only going for it once in 91 qualifying fourth-down tries. He was similarly passive in 2013. His fourth down decision last Sunday was surprising given that trend.

To see Smith’s evolution on fourth down strategy, consider his decisions on 4th and 3 or less when between the opponent’s 10- and 40-yard lines. To consider only situations where there was a real choice while keeping things as simple as possible, I look only at first-half decisions along with third-quarter decisions where the margin was ten points or less.

YearWeekQuarterOpponentATL ptsOpp ptsYdlineTo goPlayWent for it?
200813DET2414323FG goodN
200822TB017383PuntN
2008132SD67201RunY
2008171STL30213FG goodN
2008182ARI07121FG goodN
200923CAR2113151RunY
200923CAR2113211RunY
200932NE310343PassY
2009112NYG710191RunY
2009141NO63122FG goodN
2009172TB33182FG goodN
201012PIT03283FG noN
201032NO714262PassY
201032NO714112PassY
201042SF714101RunY
201091TB00342PassY
201092TB177133FG goodN
2010122GB33363PassY
2010151SEA07181RunY
2010172CAR140261RunY
2010191GB00131RunY
201111CHI33303FG goodN
2011103NO610261RunY
2011112TEN130372PuntN
2011133HOU310343PassY
2011161NO30163FG goodN
2011182NYG00241RunY
2011182NYG20421PuntN
2011183NYG210211RunY
201211KC103162FG goodN
201222DEN130242FG goodN
201261OAK00253FG noN
201291DAL03193FG noN
201292DAL36363PuntN
2012101NO100192FG goodN
2012112ARI313332FG goodN
2012152NYG170192FG goodN
2012181SEA00213FG goodN
201332MIA100342FG goodN
2013141GB00352PuntN
2013162SF103171FG goodN

It is pretty amazing that the same guy is coaching the same team before and after 2011. Smith went for it 17 out of 29 times in his first four years with the Falcons, including many situations where most coaches would not have gone for it. After 2011, Scaredy-cat Smith went for it 0 out of 15 times on 4th and short between the 10 and the 40 in these situations. Note that these numbers do not include Week 5 and so ignore the first-half field goals the Falcons kicked from the 2 and 3, respectively.

The Falcons’ pattern of success on those fourth down plays paints a pretty clear picture for what happened. Below, I highlight only those 17 plays from the previous table where the Falcons went for it on fourth down.

YearWeekQuarterOpponentATL ptsOpp ptsYdlineTo goPlaySuccessful?
2008132SD67201RunY
200923CAR2113151RunY
200923CAR2113211RunY
200932NE310343PassY
2009112NYG710191RunY
201032NO714262PassY
201032NO714112PassY
201042SF714101RunY
201091TB00342PassY
2010122GB33363PassY
2010151SEA07181RunY
2010172CAR140261RunN
2010191GB00131RunY
2011103NO610261RunY
2011133HOU310343PassN
2011182NYG00241RunN
2011183NYG210211RunN

The Falcons were successful the first 11 times they went for it in this situation under Smith. They failed for the first time in Week 17 of 2010 against the Panthers. They then succeeded two more times, including in the 2010 playoff loss to the Packers, making it a run of 13 for 14 for Smith in that situation. They then failed their next three times, including the two failures in that 2011 playoff game. And they haven’t gone for it since.

Vince Lombardi and Other Coach Overreactions

Mike Smith isn’t the only good coach to overreact to failures on fourth down. Maybe the greatest coach of all time learned a similarly unfortunate lesson from fourth down stops. In the 1960 NFL Championship game, Vince Lombardi twice went for it on 4th down deep inside Philadelphia territory. After failing both times and losing the game 17-13, Lombardi blamed the loss on himself for not taking the three points. David Maraniss writes about Lombardi’s postgame laments in “When Pride Still Mattered,” his biography of the coach:

He was proud of his men, Lombardi said. They had given every ounce of effort. But that was more than he could say for himself. He had cost the team six points, the difference between winning and losing, by not going for easy field goals… (Lombardi said) ‘I learned my lesson today. When you get down there, come out with something. I lost the game, not my players. That was my fault.’ (p. 265)

Remember that Lombardi’s kicker, Paul Hornung, was 8 of 13 that year on kicks under 30 yards. Hornung would be so bad four years later that his foot played a large role in torpedoing the Packers’ season.1 Short field goals were no sure thing with Paul Hornung. Even if they were, Lombardi made the right decision. Still, Lombardi, just like Smith, drew exactly the wrong lessons from the outcomes of his earlier decisions.

We can use linear regression to see how previous success on fourth down influences the decision across all recent coaches. Looking at the regular season from 1991-2013, I considered the decisions made on 4th down and 3 or less from inside the opponent’s territory in the first half or in the third quarter with the game within ten points. The first regression I checked out was just looking at the impact of the most recent outcome. Controlling for the team, the distance to go, and the line of scrimmage, a success on the previous fourth-down try meant that a team was about 2.1% more likely to try the next time. This effect is significant statistically any way you slice it, but relatively small.2 Teams appear to have been a little more likely to go for it on fourth down when they succeeded the previous time, on average.

But maybe failing a few times in a row is what’s more traumatic. I looked at the number of times a team succeeded on its previous three tries in that situation. On average, teams that failed three times in a row were about 4.7% less likely to try the next time as teams that succeeded on their previous three tries.3  Repeated previous failure seems to have a larger impact on future decisions.

It’s important to point out that we might expect the effects of the most controversial fourth down calls to be bigger than these failures in the first three quarters of regular season games, too. We’ve already seen how the Falcons’ fourth-down failures in 2011 coincided with Mike Smith going from a coach who usually went for it on fourth and short in opposing territory to someone who almost never does. Other famous fourth down failures may have impacted other coaches, too, although we need to be cautious due to small sample sizes. After failing on 4th and 1 from his own 29 against Philadelphia with two minutes left in a 17-17 game, Barry Switzer went from 60% (3 for 5) aggressiveness on fourth-and-short plays to 25% (6 for 24).4 After the 4th and 2 failure against the Colts, Bill Belichick went from 62% aggressiveness (40 of 65) as Patriots coach before that to 48% after (13 of 27).

This much smaller change puts Mike Smith’s remarkable Reverse Riverboat transformation in proper perspective. After his 2011 failures, Smith went from going for it 59% of the time before to 0% after. Falcons fans could worry that Mike Smith will learn better than almost any other coach the wrong lesson from that day’s failure.5

Takeaways

Mike Smith’s lessons from failing on fourth down are particularly vexing given the amount of success he had being aggressive. His overall success rate on those plays as Falcons coach (13 for 17, 76%) means that even the outcomes of his fourth down decisions helped his team increase their win probability over the course of those games. That two high-profile failures in that playoff game against the Giants could overrule years of previous success speaks to how difficult it is to fight conventional wisdom, even when that prevailing viewpoint is horribly wrong.6

If a few failures in a row can cause even a previously aggressive coach to become timid, moreover, that will push things towards an unfortunate outcome. Many coaches like John Fox who start off conservative never experiment with fourth down aggressiveness. Coaches like Mike Smith who start out relatively aggressive will run into a streak of failures at some point that may cause them to stop trying.7 It is inevitable that fourth-down strategy will evolve to eventually be more optimal, but these are strong headwinds to overcome.

  1. In 1964, Hornung missed the most kicks, 26, in NFL history. His average (.316) was lower than Roberto Clemente’s (.339). []
  2. The p-value for the effect is .02. For the methodologically inclined, the regression included a set of fixed effects for team, distance, and the yard line for the decision. This amounts to just controlling for these variables in a flexible way. []
  3. The p-value here is .005. []
  4. That play, you may remember, actually failed twice since the two minute warning wiped out the initial stuff. []
  5. On Sunday against the Bears, Smith also faced a 4th-and-1 from his own territory.  Trailing by 6 late in the 3rd quarter, Smith chose to punt on 4th-and-1 at his own 29. Granted, this is what just about every coach does in this situation. But Atlanta did not go for it on 4th down against Chicago until the game circumstances mandated it. []
  6. Chase note: And, of course, playoff performances having a disproportionate impact on viewpoints is not limited to 4th down decisions. []
  7. It’s too soon to tell, but we may have lost Riverboat Ron after that fourth-down failure against the 49ers in the playoffs last year. In terms of the situation I looked at, so far Rivers has a punt on 4th and 2 from the Tampa 41 and a run on 4th and 1 in that same week 1 game. Rivera’s aggression even last year was confined primarily to 4th and 1, anyway, so we need more evidence before determining whether the Riverboat still roams. And, of course, there was Sunday’s conservative decision in overtime that may have cost his team the win. []
  • I could see Smith maybe being a little shy about taking risks after his very public failures, but i didn’t realize just how drastic his change was. That is nuts. I’d be interested to see how an “enlightened” coach like Chip Kelly compares.

    • We have a lot of evidence on Belichick before and after the 2009 4th and 2 against the Colts. By my measures, he changed but much less than Smith. So maybe enlightened coaches at least stick to their guns a little more. One day, we’ll get a guy (maybe Kelly is it) who is all process and no outcome.

  • Ty

    Humans are naturally conditioned to believe in the outcome, rather than the process. If they (humans in general) get burned by the outcome, they are less likely going to do that specific outcome, even if the process is correct.

  • Richie

    I’m always curious if management has any impact on these decisions. Does Arthur Blank feel embarrassed by that playoff loss in New York, and ask Smith to quit making such “crazy” decisions?

    • We have a lot of evidence on Belichick before and after the 2009 4th and 2 against the Colts. By my measures, he changed but much less than Smith. So maybe enlightened coaches at least stick to their guns a little more. One day, we’ll get a guy (maybe Kelly is it) who is all process and no outcome.

    • That’s a neat thought. It would be interesting if that depended on the owner. Maybe a Jim Dolan-esque owner would ask Smith to do that, but a self-made entrepreneur type (who presumably knows all about calculated risks) wouldn’t more often.

  • Pingback: Coffee and Cigarettes: Seahawks links for Wednesday - Sports Go()

  • Pingback: Reads Listens Views 10/17/2014 | The Rookie Scouting Portfolio()

  • Anonymous

    Almost every time, the announcers will talk about how it was so great of a decision to bring the punter or field goal kicker. That’s one of the main reasons for the fourth down caution. Nobody remembers if your decision to punt on fourth and 2 from midfield in the second quarter (or heck even early in the fourth quarter) costs you the game. Everybody still remembers Belichick’s supposedly terrible decision in a mid-October regular season game six years ago, though.

  • Anonymous

    Lombardi could have learned the exact opposite lessons when he was on the 1958 Giants coaching staff. The Giants punted up by 3 late in the game facing fourth and inches from the Eagles 40. A first down would have essentially ended the game. The Eagles proceeded to drive for the tying field goal. The Giants had another fourth and 1 on their first possession in overtime. Again they punted. The Eagles drove for the winning touchdown.