## Andrew Luck and Quarterback Help

Luck confuses defenders, statisticians.

It’s no secret that Andrew Luck’s efficiency numbers aren’t quite up to par with his reputation. Over the past two seasons, Luck ranks just 18th in ANY/A, far behind some of the other young quarterbacks in the NFL. Nick Foles, Russell Wilson, and Colin Kaepernick rank in the top 6th in that metric, Robert Griffin is 11th, Cam Newton is 14th, and even Andy Dalton is 16th. Luck tends to fare much better in ESPN’s QBR than in ANY/A (and Andy Benoit wrote an interesting pro-Luck piece yesterday), but today I wanted to try to quantify another issue: quarterback help.

A quick disclaimer: there are probably a zillion different ways to quantify quarterback help. This is certainly not not not the best way, but it’s the way that was easiest and most intuitive to me. On the scale of “this feels right” to “rigorous quantitative analysis” this certainly falls closer to the former end of the scale. But it’s Friday and we’re having fun, so here’s what I did.

1) Calculate how many standard deviations from average each team was in Points Allowed (negative means fewer points allowed).

2) Calculate how many standard deviations from average each team was in Pass Ratio (negative means more run-heavy).

3) Add the two standard deviations to see how much each team relied on each quarterback’s arm.

Here were the 2013 results. According to this, no quarterback was asked to do more than Matt Ryan. Here’s how to read the table below: The Falcons allowed 443 points last year, which was 1.05 standard deviations more than the average team. Atlanta also passed on 68.7% of all plays, which was 1.99 standard deviations above average. Add those together, and the Falcons get a grade of +3.04, the most in the NFL in 2013.

RkTeamPts AllSt Dev PAPass RatioSt Dev PRQB Help
1Atlanta Falcons4431.0568.7%1.993.04
2Cleveland Browns4060.4867.7%1.812.29
3Dallas Cowboys4320.8864.9%1.262.14
4Jacksonville Jaguars4491.1462.9%0.892.03
5Chicago Bears4781.5860.1%0.351.93
6Washington Redskins4781.5859.1%0.151.73
7Minnesota Vikings4801.6258.2%-0.011.6
8Houston Texans4280.8262%0.711.53
9Oakland Raiders4531.256.3%-0.390.82
10New York Giants3830.1361.4%0.60.73
11Denver Broncos3990.3760.1%0.350.72
12Miami Dolphins335-0.6165.1%1.310.71
13Green Bay Packers4280.8257.3%-0.20.62
14Pittsburgh Steelers370-0.0761.5%0.610.54
15Detroit Lions3760.0259.6%0.250.27
16Baltimore Ravens352-0.3561.2%0.550.21
17Tampa Bay Buccaneers3890.2257.2%-0.220.01
18New Orleans Saints304-1.0863.8%1.05-0.03
19Indianapolis Colts336-0.5960%0.33-0.26
20New England Patriots338-0.5658.7%0.08-0.48
21Tennessee Titans3810.155.2%-0.59-0.49
22Arizona Cardinals324-0.7759.3%0.19-0.58
23St. Louis Rams364-0.1656%-0.44-0.61
25New York Jets3870.1951.7%-1.28-1.08
26Buffalo Bills3880.2151.1%-1.39-1.18
27San Diego Chargers348-0.4154.2%-0.8-1.2
28Kansas City Chiefs305-1.0757%-0.24-1.31
29Cincinnati Bengals305-1.0756.2%-0.41-1.48
30Carolina Panthers241-2.0551.7%-1.28-3.32
31San Francisco 49ers272-1.5747.5%-2.08-3.66
32Seattle Seahawks231-2.247.7%-2.04-4.24

Again, this is a quick and dirty formula. Peyton Manning is obviously asked to do quite a lot, but because he was so good at it last year, Denver wound up throwing on “only” 60% of their pass plays. But I think this formula does do a decent enough job for our purposes: it says Wilson, Kaepernick, and Newton had the most help from their running games and defenses. And it says the Browns relied heavily on their quarterback, which is one reason why Cleveland finished 4-12.

Luck’s Colts ranked 19th last year in this metric, and 9th in 2012. I suppose what’s made Luck so widely-praised is the success the team has had despite asking him to shoulder such a large burden. You don’t come to this site to read about #QBWinz, but it’s pretty unusual for such a young quarterback to shoulder such a large burden and win a lot of games. I am the first to say that #QBWinz are stupid, but since it’s Friday and we’re having fun, let’s play around with some more numbers.

Since 1950, there have been 39 quarterbacks who have started at least 10 games in each of their first two seasons.1 For each quarterback, I calculated his QB Help score during his first two seasons along with his winning percentage.2 So Luck gets a QB Help score of +0.6 with a winning percentage of 0.688. Meanwhile, Wilson gets a Help Value of -8.5, but produced a winning percentage of 0.750. Not surprisingly, there’s a pretty clear inverse relationship between QB Help and winning percentage. But arguably no quarterback has balanced these two metrics as well as Luck. In the picture below, QB Help is on the Y-Axis, and winning percentage is on the X-Axis:

Let’s take those numbers in graphic form. Here’s how to read the table below. Luck’s first two years were in 2012 and 2013, and he went 22-10 (posting a 0.688 winning percentage). That’s the 3rd best winning percentage of the 39 quarterbacks in this sample. He had a QB Help grade of +0.6, which is the 18th highest number. If you average those two ranks — and averaging ranks is a terribly flawed idea, but I’m short on time and this is mostly made up stuff anyway — Luck has the lowest average rank in these two categories of any of the 39 quarterbacks.

QuarterbackYearRecordwin%Rank Win%HelpRank HelpAvg Rks
Andrew Luck2012-201322-100.68830.61810.5
Peyton Manning1998-199916-160.5143.61012
Charlie Batch1998-199911-110.5142.11414
Joey Harrington2002-20038-200.286285.8315.5
David Woodley1980-198117-8-10.6734-1.42715.5
Jim Plunkett1971-19729-190.321264.7616
Bob Griese1967-19688-14-10.37243.9816
Jim Zorn1976-19776-180.25337.8117
Tobin Rote1950-19516-170.261315.5417.5
Drew Bledsoe1993-199415-130.536130.12217.5
John Elway1983-198416-80.6675-2.13017.5
Phil Simms1979-19809-150.375232.21318
Matt Ryan2008-200920-100.6675-2.73319
Ben Roethlisberger2004-200522-30.881-7.83819.5
Dennis Shaw1970-19714-19-10.188386.5220
Tony Banks1996-199710-190.345251.71520
Robert Griffin2012-201312-160.429200.42020
Troy Aikman1989-19907-190.269292.51220.5
Andy Dalton2011-201219-130.59412-1.72920.5
Bernie Kosar1985-198616-100.6159-2.53220.5
Russell Wilson2012-201324-80.752-8.53920.5
Jeff George1990-19916-220.214354.6721
Vince Young2006-200717-110.60711-2.33121
Kerry Collins1995-199616-90.647-3.63521
Christian Ponder2011-201212-140.46219-0.22421.5
Ryan Tannehill2012-201315-170.46918-0.52521.5
Cam Newton2011-201213-190.4062102322
Byron Leftwich2003-200413-140.48116-1.52822
Joe Flacco2008-200920-120.6258-5.83622
Randy Johnson1966-19674-18-10.196373.8923
Mark Sanchez2009-201019-120.61310-7.33723.5
Rick Mirer1993-199411-180.37922-0.62624
Fran Tarkenton1961-19624-19-10.188383.51124.5
Steve Fuller1979-198012-130.4817-3.23425.5
Lamar McHan1954-19555-16-10.25330.51926
Blaine Gabbert2011-20125-190.208360.81726.5
David Carr2002-20037-200.259320.32126.5

As I wrote about last year, Wilson’s career has been extremely similar to Ben Roethlisberger’s through two seasons. Considering the QB Help for Wilson and Roethlisberger, it’s remarkable that Luck has the third best winning percentage of any quarterback in this group. Wilson and Roethlisberger had the two best records and the two highest help scores. The two quarterbacks who received the most help after them? Joe Flacco and Mark Sanchez.

1. (Note: I excluded Carson Palmer, Steve DeBerg, Jim Kelly, Joe Kapp, Joe Pisarcik, Tom Flores, Warren Moon, and Jeff Garcia, each of whom would otherwise qualify, since none of them were true rookies during their first year. []
2. Note: In the interest of time, I just took simple averages for both metrics. []
• Arif

Wouldn’t it make more sense to use “pass identity”? If only because Pass% is in some way already account for points allowed.

Regardless, fun work and something to chew on.

• Chase Stuart

Perhaps. But this was what my brain told me to do!

• Arif

Curse your move towards intuitive statistics!

• Ian

Also, QB rushes shouldn’t be lumped together with other rushes; that should bring the bottom three teams closer to the average.

• andy

It is interesting to see how the AL’s phenomenon turns out. He is a very good qb and maybe will become a great one in the future. But based on the last two years’ metrics, he does not “have” it yet. It is kind of interesting that all the pundits were “banking” on this guy to be the HOF, even before taking the first snap. Besides ESPN’s qbr metrics, he is “averaged” according to all other saber metrics. It is kind of funny that we can argue all the advanced metrics and how they can be differentiated but when you compare them to individual player, they are quite reliable in the context of where each player ranked accoringly, not exact match but very close. AL is an exception to this rule. He ranks high in one but averages on others. So I really don’t know what to make of it. Is he just an outlier? You just mentioned about the most “recognizable” metrics- ANY/A and ESPN (do you know the variables and methodologies that they used? I’ve heard it is proprietary and “lock” in the vault somewhere) It would be intersesting to see as an open-source but I doubt it if it happens. Check out PFF on their qb’s situational throws and “scorecard”. He is pretty much in middle of the pack as average.

• Ty

Andrew Luck has been an anomaly for me. I know he is good (but not great), but as you say, the passing metrics (ANYA, NYA) say that he is merely average. I think some of that has more to do with the talent around Luck. Indy’s offense isn’t that talented (the receiving corp are okay, but the offensive line is bad), which causes Luck to take lots of hits (thus, lowering his NYA and ANYA); then again, most QB sacks are on the QB than on the offensive line (see Tim Tebow in 2011 and Peyton Manning in 2012).

I think he should eventually be great, and I think he is better than average, but the talent around him really holds him back from “breaking out”, and I’m not too high on the Colts front office, based on the decisions that they have made.

• If you break down Luck’s success by passing distance compared to league average its certainly no better than average. Last time I did that was November of last year he was around -80 yards. His turnover ratios were good though and balanced that out to bring him right to average. Regardless the day he is up for an extension Irsay is going to pay him some outlandish figure close to \$25 million a year.

• Ben

One stat that’s absolutely crazy is Luck’s Air Yards per Attempt. He only actually threw the ball 4 yards farther than Russell Wilson did, and that was on more than 100 more attempts.

• Tom

I also find the Andrew Luck “anomaly” interesting…his efficiency numbers are somewhat average, as noted, but his efforts lead to wins, nonetheless. Our eyes tell us this (I’m thinking of the KC wildcard game, even though he was partly responsible for them falling behind in the first place), but some advanced stats reflect it as well: Brian Burke’s website has him ranked #7 in 2013 and #4 in 2012 in Win Probability Added per game (regular season). And although his ANYA and WPA ranks were closer in 2013 (#13 and #7 respectively), we did see that “high-low” ranking thing in 2012: he ranked #19 in ANYA (5.0, which was slightly below the average) and #4 in WPA/game.

• Richie

Dan Marino missed your sample by one start (he started 9 games in ’83 but played in 11). Where would he appear on this chart? I’m guessing above Roethlisberger, and to the right of Kerry Collins.

• Dave

I think the “dissconnect” with Luck is that QBR also is factoring in his rushing contributions which efficiency wise have been the among… if not best in the league. Two years in a row he has been at ~40% DVOA rushing the ball.

Also looking at the actual QBR breakdown…. http://espn.go.com/nfl/qbr/_/year/2012/sort/cwepaPassesCondensed

In 2012 he was 13th in ESPN’s PASS EPA but was #1 on penalty EPA by a large margin.
In 2013 he was 17th in ESPN’s PASS EPA but was #3 on penlaty EPA again with quite a bit of distance from most of the other QB’s

Finally here is the big one that matches with the DVOA finding. the ESPN page has 5 columns PASS EPA, RUN EPA, SACK EPA PENALTY EPA, and TOTAL EPA. You’ll notice that summing the first 4 columns doesn’t equal the final one. Thats because the pass EPA is only for plays with PASS ATTEMPTS and the RUN EPA is only for DESIGNED RUNS.

If you take the TOTAL EPA- the sum of the 4 columns you get what I will call “SCRAMBLE EPA”

I’m sure you can guess what is coming here. He was #1 in scramble EPA in both 2012 and 2013. Add that with his high penalty EPA both seasons and you can see why he is doing better in QBR than in the traditional metrics.

• In the QB stats I came up with, Luck ranked 4 in volume (Adjusted Total Yards) and 10 in rating (TQR) last year. I think the volume rank suggests I high and effective usage rate. Flacco, for example, had more plays but far fewer AY than Luck.

• Nick

I think it would be quite interesting if you could somehow combine the top table with the efficiency stats… basically, efficiency controlled for how help they got.

• Jason Drake

Fun stuff.
It seems like measuring wins and accounting for points allowed invites more noise into the results. If Andrew Luck led his offense to 28 points against the Bengals, does it matter that Cincinatti scored 42 instead of 35? Arguably, the bigger gap would’ve invited more prevent defense and made Luck’s job easier. Russell Wilson had 300+ yards and 3 TD’s against the Saints in the regular season, with the Saints scoring just 7 points. Would his performance have been more difficult if the Saints had scored 14? Or 21?

I think it would be interesting to do this using just the quarterback efficiency and quarterback help. I did something similar, using per-play stats instead of standard deviations. I started with a quarterback efficiency similar to ANY/A (but accounting for quarterback runs and carefully excluding kneel-downs). Then I calculated quarterback production per EVERY PLAY that his team ran. That figure is basically proportional to the QB’s total production (but adjusted for teams which run more or fewer plays).

Then I simply calculated a geometric mean between the two.

For basic efficiency (production per quarterback play), Wilson came out ahead of Kaepernick and both were ahead of Andrew Luck.

But the geometric mean, which accounted for Luck’s greater play percentage, gave Luck a (very, very) slight lead over Wilson with Kaepernick just a little behind both.