≡ Menu

Today’s guest post/contest comes from Thomas McDermott, a licensed land surveyor in the State of California, a music theory instructor at Loyola Marymount University, and an NFL history enthusiast. As always, we thank him for his hard work.

One way we can tell if a quarterback is “clutch” – meaning, he plays well when he absolutely has to – is by looking at his 4th Quarter Comebacks (4QC) and Game Winning Drives (GWD). Below are definitions for both1:

    4th Quarter Comeback: In the 4th quarter, the quarterback leads a scoring drive while being down one score or less that results in the game being tied or his team taking the lead. As long as the QB’s team wins the game, the QB gets credit for the 4QC, even if his scoring drive wasn’t the game-winning drive.

    Game-Winning Drive: In the 4th quarter or overtime, the quarterback leads a scoring drive that results in his team taking the lead (meaning, breaking a tie or overcoming a deficit) for the last time.

While I believe that both of these stats do a pretty good job of revealing “clutch” quarterbacks (in any case they point us in the right direction), they don’t tell us much about the quarterbacks’ actual role in said stats. Did he just suddenly find himself on the opponent’s 1-yard line after a turnover, hand the ball off to the RB who scores a TD, winning the game? Or what about all those “missed” 4QC/GWD’s: situations where the quarterback heroically leads his team down the field for the go-ahead score, only to then watch on the sideline as his team loses the game when the opposing team scores on their last drive. Remember, the quarterback does not get credit for a 4QC if his team does not win the game.

What I thought might be helpful, and interesting in any event, is to look at a quarterback’s performance within the defined situation spelled out above for 4QC/GWD’s – namely, in the 4th quarter or overtime, when the game is tied or the quarterback’s team is trailing by one score (8 points since 1994, 7 before that). This might give us some insight into how a QB played when it really mattered, regardless of who actually won the game.

The metric I’ll use to evaluate performance will be a variant of Bryan Frye’s Total Adjusted Yards per Play. I say “variant” because I’m not including 1st down data, which Bryan does2, and I’m giving a 15-yard bonus for successful 2-point conversions.3 Here’s the formula:

(passing & rushing yards + (touchdowns * 20) + (2ptconv * 15) – (interceptions * 45) – (fumbles lost * 25) – ( sack yards)) / (pass attempts + rush attempts + sacks)

One final note before the table: a strength of opponent adjustment is entirely appropriate for a study like this, and regrettably, I didn’t do it.  That’s on the list for next time.

Finally, the table. Here’s how to read it:

Jay Cutler had 11 games where he and the Bears offense found themselves in our defined clutch situation; in those situations, he had 56 completions in 95 attempts for 773 yards, 5 passing touchdowns, 2 interceptions, and was sacked four times for 16 yards. He had 5 carries for 39 yards and a touchdown, he lost one fumble (from a sack by Von Miller!), and had one 2-point4 conversion. This makes for 816 Total Adjusted Yards in 104 action plays, giving Cutler 7.8 TAY/P, which is the fifth highest in 2015. If we take his TAY/P and compare it to the league average of 5.2 yards, Cutler’s performance is 2.7 yards above average; multiplying that by 104 action plays gives him 277 yards of Clutch Value, the most for 2015 (and 26 yards ahead of “Clutch-Master” Eli Manning).

RkPlayerGCmp-Att-YdsPass TD-IntSks-YdsCar-Yds-Rush TD-Fum LostAction PlaysTAYTAY/PTAY/A OvAvgClutch Value4QCGWD
1Jay Cutler1156-95-7735-24-165-39-1-11048167.82.727744
2Eli Manning1038-72-5514-01-62-15-0-0756408.53.325112
3Drew Brees855-85-7103-25-312-9-0-0926587.2218112
4Blaine Gabbert325-42-3681-04-266-66-1-0524488.63.417822
5Cam Newton526-40-3733-03-196-21-0-1494108.43.215634
6Matthew Stafford736-57-4462-12-143-31-1-1624537.32.113123
7Marcus Mariota634-55-3533-22-83-90-1-0604407.32.112922
8Russell Wilson731-44-3502-12-1513-83-0-05941371.810733
9Teddy Bridgewater322-30-3142-06-312-29-0-2383027.92.810511
10Carson Palmer627-44-3311-11-41-8-1-0463307.229125
11Philip Rivers1051-88-6693-28-562-0-0-0985976.10.98912
12Ben Roethlisberger535-50-3774-22-161-7-0-0533586.81.68322
13Andy Dalton532-54-4533-23-154-8-0-1613916.41.27533
14Tom Brady630-53-3683-05-311-2-0-1593746.31.26822
15Brian Hoyer312-24-1882-10-01-6-0-0251897.62.45911
16Sam Bradford535-51-4302-25-340-0-0-0563466.215602
17Josh McCown631-43-3923-34-282-2-0-0493066.21.15211
18Peyton Manning522-45-3353-23-130-0-0-0482926.10.94344
19Kirk Cousins641-64-4073-20-01-0-0-0653775.80.64023
20Ryan Tannehill726-48-2792-05-413-40-0-0563185.70.52822
21Joe Flacco1052-102-6592-31-106-5-1-01095795.30.11433
22Jameis Winston928-54-3251-13-194-34-2-1613305.40.21423
23Matt Ryan1052-78-6112-44-325-24-0-0874635.30.11244
24Tyrod Taylor616-33-2202-21-87-58-0-0412205.40.2712
25Andrew Luck311-23-1052-10-05-33-0-0281334.8-0.4-1212
26Brock Osweiler434-64-3891-13-195-8-0-0723534.9-0.3-2022
27Aaron Rodgers636-67-4163-33-164-35-0-0743604.9-0.3-2422
28Blake Bortles1142-87-5104-210-456-42-0-11034724.6-0.6-6234
29Nick Foles415-31-1611-11-82-11-0-1341143.4-1.8-6211
30Matt Cassel625-48-3142-34-150-0-0-0522043.9-1.3-6600
31Matt Hasselbeck323-36-2330-22-131-4-0-0391343.4-1.8-6812
32Ryan Fitzpatrick736-61-4253-61-55-35-0-0672453.7-1.5-10323
33Derek Carr1256-96-5795-54-382-5-0-11023963.9-1.3-13344

Some initial comments:

  • The minimum number of drop backs is 24. I’ve gathered the clutch situation data for all quarterbacks since 1994, and the standard deviation for Clutch TAY/P starts to get reasonable at around 24 drop backs in a season.
  • Of course, this stat can still get a little wonky due to “fluke” plays and small sample sizes. Eli Manning made one clutch play in the Giants-Dolphins game in Week 14: an 84-yard catch-and-run TD pass to Odell Beckham Jr., which netted him 104 TAY/P.
  • It goes without saying, but let’s say it anyway: while these numbers might reveal quarterbacks who played well in clutch situations, it’s entirely possible they wouldn’t have needed to “come through in the clutch” had they just played better in “non-clutch” situations.
  • You might notice that Cutler and Derek Carr have the same number of 4QC/GWD’s (4) and yet Cutler is at the top of our ranking and Carr is at the bottom. What gives? Shouldn’t a high Clutch Value correlate well with 4QC’s and GWD’s? Well, yes and no. There’s a lot of randomness here, and as noted above, a QB can play well in the clutch and his team can still lose the game, or a QB can play pretty badly and still win the game. But, generally, better QB clutch play does correlate well with wins (or, 4QC’s for our study). So, for 2015, the correlation is zero, but the single season correlation factor since 1994 is 0.38, and if we correlate using career numbers (again, since 1994), it’s 0.56.
  • Eli Manning played pretty well in the clutch in 2015, but you might not be able to tell from his 4QC/GWD’s. He led his team on two possible game-winning drives, against the Patriots in Week 10 and the Panthers in Week 15. In both cases, he watched from the sidelines as those teams’ quarterbacks’ led their offenses on game-winning drives.
  • As poorly as Derek Carr played in the clutch (admittedly, his numbers are mostly driven down by his disastrous two-pick 4th quarter performance against Kansas City in Week 13), he, like Manning, could have had 2 more 4QC/GWD’s had the Oakland defense stopped Jay Cutler in Week 4 and Landry Jones in Week 9.
  • Fun fact: the Broncos defense recovered 5 quarterback fumbles (at least 4 were forced) in clutch situations in 2015.

Would love to hear any comments, especially whether or not you think this stat can tell us anything useful (I’m on the fence). I’ve already collected the single season and career numbers, if there’s enough interest, I’ll post those as well. Thanks for reading.

[Editor’s Note 1: We should state the obvious that the idea of “clutch” may not be repeatable on any meaningful scale at the professional level. That doesn’t make this less interesting, but just interesting in a retrodictive rather than a predictive way.]

[Editor’s Note 2: Ryan Fitzpatrick remains unsigned, IMO, in large part because of his ranking on this table. He had six interceptions on just 67 passes in clutch situations, including two enormous ones when down by 7 with 2 minutes to go against Houston and down by 5 with 2 minutes to go against Buffalo in the final game of the year (he also threw a key interception earlier in the quarter). He had two desperation interceptions in the final 25 seconds of the fourth quarter, which aren’t as bad, but also had two big failures on 4th down against Buffalo (in November) and Houston (one of which was also an interception). Those three games were half of the Jets losses in 2015 — and Fitzpatrick played terribly against the Eagles in a fourth loss — which is why it’s easier for Jets management to take a hard stance on Fitzpatrick now. Whether that is appropriate is up for debate, but it’s a lot easier to not throw big money at a quarterback who had a big season when that big season was tainted with some notable failures in “clutch” situations.]

  1. As many of you already know, Scott Kacsmar did a ton of great work in this area; his “Quarterbacks and fourth quarter comebacks” article at the old PFR Blog provides all the information you’ll ever need about 4QC/GWDs. []
  2. Including 1st down data is entirely appropriate for this study, probably even more so than for season or career data, since getting a first down on a possible game-winning drive is pretty important. I’m not including it here because I feel that if we give a bonus for 1st downs, there should be a penalty for failed 3rd and 4th down conversions, and I haven’t done enough research yet for that (at the moment, I’ve got a failed 3rd down conversion being equal to losing about 16 yards). []
  3. I haven’t done any research on this yet; for the moment, 15 yards “feels” right. []
  4. Note: To save room on the table, 2-point conversions were omitted but are still part of the formula. There were only four in 2015, converted by Cutler, Rivers, McCown, and Mariota. []
  • Question about the addition of 2pt conversions. Are you including incomplete passes or failed runs for 2pts in the denominators as well, or can a QB only gain positive value from a 2pt attempt?

    • Tom

      Bryan – thanks for asking, this should be cleared up. The QB can only gain positive value from the 2-point attempt (at least for now anyway). So Cutler had 1 successful 2-point conversion (a pass), he gets the 15 yard bonus, but it’s not counted against his attempts (he has 56 attempts, not counting the conversion). I believe I did this so that it it had more “bonus value” if that makes sense. Maybe on the next run I’ll count it.

  • Clint

    So even with that big Hail Mary against the Lions, Rodgers still wasn’t good in the clutch. The media never seems to talk about this.

    • Tom

      Yeah, he’s got 3 TD’s and 3 picks which isn’t horrible, but not good. Still, these numbers are oblivious to the order in which things happened – he could have thrown a pick on a drive (while in our clutch situation), but then threw the game-winning TD on the next (this happened with Derek Carr). His numbers would look bad, but he won the game…so there are some issues with this stat. The more you mess with it, the more you get into the Win Probability territory, but that’s another topic altogether.

  • sacramento gold miners

    Too bad we’re not including the playoffs here, both Rodgers and Big Ben had memorable comebacks this past season.

    • Tom

      Yep, agreed. Since I’ve got the apparatus up, here are the top 5 playoff QB’s:
      1. Carson Palmer, 15 plays, 2 TD’s, 1 INT, 158 TAY, Clutch Value 80 (this reflects his game against GB, he didn’t have any clutch situations against Carolina)
      2. Aaron Rodgers, 9 plays, 1 TD, 0 INT, 115 TAY, CV 68
      3. A.J. McCarron, 14 plays, 1 TD, 0 INT, 118 TAY, CV 45

      P. Manning is #4, Brady #5, Big Ben #6.

      Ben played a bit better in the clutch in the playoffs (8-11-63-0-0, 1 sack for -13) than these numbers indicate since he made some key first downs, I think the lack of a TD pass and the low number of attempts drags the CV number down.

      • sacramento gold miners

        Carson Palmer really melted down in his first conference title game last season, and it was interesting to hear Troy Aikman foreshadowing his problems. Early in the broadcast, Aikman alluded to Palmer seeming a bit “off”(not his exact words), This was long before the game got out of hand, but it’s a good example of how someone who played the game, and excelled in those circumstances, can tell the difference. Like many fans, I assume Palmer would play well versus Carolina, even if the Cards lost.

  • Josh Sanford

    Can you divide the Clutch Value by the number of games in order to determine which QBs did the most with the number of opportunities that they were given?

    • Due to an earlier error in the table, I have deleted Bryan’s response to you here. But I have run the numbers for you:

      Blaine Gabbert – 59.3
      Teddy Bridgewater – 35.0
      Cam Newton – 31.2
      Jay Cutler – 25.2
      Eli Manning – 25.1
      Drew Brees – 22.6
      Marcus Mariota – 21.5
      Brian Hoyer – 19.7
      Matthew Stafford – 18.7
      Ben Roethlisberger – 16.6
      Russell Wilson – 15.3
      Carson Palmer – 15.2
      Andy Dalton – 15.0
      Tom Brady – 11.3
      Sam Bradford – 11.2
      Philip Rivers – 8.9
      Josh McCown – 8.7
      Peyton Manning – 8.6
      Kirk Cousins – 6.7
      Ryan Tannehill – 4.0
      Jameis Winston – 1.6
      Joe Flacco – 1.4
      Matt Ryan – 1.2
      Tyrod Taylor – 1.2
      Andrew Luck – -4.0
      Aaron Rodgers – -4.0
      Brock Osweiler – -5.0
      Blake Bortles – -5.6
      Matt Cassel – -11.0
      Derek Carr – -11.1
      Ryan Fitzpatrick – -14.7
      Nick Foles – -15.5
      Matt Hasselbeck – -22.7

      • Josh Sanford


      • Adam

        Blaine Gabbert must be the new Tebow – horrible for 55 minutes and great in the final five.

  • Tom

    Chase – to your point about Ryan Fitzpatrick, I have to think that his clutch performance must be in play in his discussions with the Jets. I’m still looking at the numbers, but right now I have him as being dead last in career Clutch Value, out of 90 quarterbacks since 1994 (minimum 150 drop backs). He has 12 TD’s and 25 INT’s…the interception number in this metric is somewhat deceiving because of the high number of desperation throws at the end of games, etc., but man, that still is not good.

  • Adam

    This metric seems more useful at the career level than for single seasons due to sample size issues, so I’d like to see the career numbers. Also think the INT penalty should be reduced because a lot of picks thrown in these situations are desperation heaves on 3rd and 4th down that serve as glorified punts.

    • Tom

      Regarding the interceptions, I’m thinking the same thing. Another option would be to weed out just those picks (or weight them less)…might be possible (albeit, time consuming) with PFR’s Play Index. I’m working on the career numbers and Favre had a year where he threw 10 picks in the clutch, but two of them were what you described – end of game heaves that none of us would actually consider an interception.

      • Richie

        I wish the NFL could classify these interceptions differently. Or maybe they are so rare that it wouldn’t make sense. But it’s almost like “catchers indifference” on a steal in baseball, or sacrifice outs.

        The intent of the play should be factored in. On the other hand, maybe when the Hail Mary touchdown is scored, it should count differently as well.

        • Tom

          Yeah, that’s where things get all muddy…if we deem a play to be “fluky” do we count it? So Aaron Rodgers throws a miraculous Hail Mary that wins the game, do we not count it as a TD since it didn’t appear (to me anyway) to be a “skill” throw (i.e., he just chucked it)? And if a defender picks off a Hail Mary, does he not get credit for it?

          All that being said, if we went down this road, I’m thinking the interception should stand (meaning we count it), but the penalty is less (maybe 25 yards).

          Of course, this is where Win Probability shines…an interception at the end of the game when the QB’s team has about a 2% chance of winning would only “cost” the QB 0.02 WPA (as opposed to the pick Russell Wilson threw a few years back which cost upwards of 0.85 WPA).