## Matt Stafford has a 17-28 record, which has little predictive value

The guy on the right was a loser until he wasn't.

Longtime readers of this blog know that I’m not a big fan of Matt Stafford. Last year, when most people were praising his breakout 2011 season, I questioned whether he was as good as his backers claimed. And, of course, his 2012 performance only raised more questions.

Stafford has a 17-28 career record, which in light of his recent contract extension, has caused people to criticize the Lions for giving big money to a player who is not a “winner.” There are legitimate reasons to criticize Stafford, so why would people fall back on statements like this? I’m sure Lions fans wish the team had won more games under Stafford, but that’s in the past. The real question — and the one faced by Lions management before giving him the extension — is whether his current career record has any predictive value when it comes to his future record.

Since 1960, there have been 77 quarterbacks1 who started at least 25 games in their first four seasons and then 25 more games in years five through eight. There’s some survivor bias in the sample — if you stick around for 25+ starts in years five through eight, you’re probably a pretty good quarterback — but there’s not much we can do about that. If you run a regression using winning percentage through four years as your input and winning percentage in years five through eight as your output, you get the following best-fit equation:

0.450 + 0.20 * Old Win %

The correlation coefficient is a tiny 0.04, and the p-value on the “Old Win %” variable is 0.09. Putting aside the questions of statistical significance, there is no practical effect. Stafford has a 0.377 winning percentage, which means this formula would predict him to win 52.6% of his games from 2013 to 2016. Joe Flacco won 68.75% of his games in his first four seasons; this would say he should be expected to win 58.7% of his games in years five through eight. In other words, someone with a great winning percentage should be expected to win only one more game per season than someone with a terrible winning percentage. And that’s even assuming the results are statistically significant, which many would say they are not.2

I know some folks tend to “skim” the math section, so let’s put the advanced tools away for a second. Here are all 77 quarterbacks, listed from biggest increase in winning percentage from years five through eight compared to years one through four. Steve Young, Dan Fouts, and Bob Griese are the three quarterbacks with the biggest jumps in winning percentages. Young, for example, won just 28% of his games in years one through four (fraud!) and then 73.3% of his games in years five through eight (Hall of Famer!).

QuarterbackYear 11-4 Win%5-8 Win%Diff
Steve Young19850.280.7330.453
Dan Fouts19730.2690.680.411
Bob Griese19670.4570.830.373
Dan Pastorini19710.2750.6150.34
Don Meredith19600.370.6440.274
Rodney Peete19890.4050.6790.273
Tom Flores19600.3930.660.267
Peyton Manning19980.50.750.25
Alex Smith20050.40.6430.243
Phil Simms19790.4120.6490.237
Troy Aikman19890.50.7290.229
Trent Dilfer19940.460.6840.224
Tony Banks19960.3770.60.223
Jay Cutler20060.4530.6750.222
Joe Montana19790.5450.7220.177
Brett Favre19920.5780.750.172
Jake Plummer19970.360.5250.165
Steve McNair19950.5260.6840.158
Chris Miller19870.2620.4120.15
Bubby Brister19860.4140.5560.142
Randall Cunningham19850.5270.6670.14
Steve DeBerg19780.1940.3240.129
Fran Tarkenton19610.3430.4720.129
Jake Delhomme19990.5450.6670.121
Jeff George19900.2860.3920.106
Steve Bartkowski19750.4440.5450.101
Vinny Testaverde19870.3480.4440.097
Kordell Stewart19970.5630.6510.089
Brian Griese19990.50.5880.088
Mark Brunell19950.5250.610.085
Richard Todd19760.4050.4820.077
Kerry Collins19950.490.5640.074
Eli Manning20040.5450.6090.064
Ken Anderson19710.5450.5960.051
Jim Plunkett19710.3750.4190.044
Donovan McNabb19990.6460.680.034
Archie Manning19710.3020.3330.031
Drew Brees20020.50.5310.031
Bert Jones19730.5610.5830.022
Brian Sipe19740.50.5160.016
Boomer Esiason19840.50.5080.008
Ben Roethlisberger20040.7090.707-0.002
Dave M. Brown19940.4520.444-0.007
Joe Namath19650.580.571-0.009
Tommy Kramer19770.4690.457-0.012
Jim McMahon19820.70.688-0.013
Bobby Hebert19850.6220.6-0.022
Drew Bledsoe19930.5420.5-0.042
Daunte Culpepper20000.4880.439-0.049
Jeff Blake19940.4390.389-0.05
Bill Kenney19800.50.441-0.059
Jim Zorn19760.4290.364-0.065
Stan Humphries19900.6670.588-0.078
Greg Landry19680.6030.519-0.085
Jack Kemp19600.7740.683-0.092
Ken O'Brien19840.5320.438-0.094
Jay Schroeder19850.6920.592-0.1
Michael Vick20010.6530.548-0.105
Jim Hart19670.5480.434-0.114
Steve Grogan19750.6270.512-0.115
Carson Palmer20040.5250.4-0.125
John Elway19830.6790.543-0.135
Eric Hipple19810.5670.423-0.144
Joe Ferguson19730.5710.419-0.152
Jon Kitna19970.5450.391-0.154
Neil Lomax19810.5670.402-0.165
Dan Marino19830.7190.55-0.169
Neil O'Donnell19910.6120.442-0.17
Philip Rivers20060.7810.594-0.188
Aaron Brooks20000.5090.297-0.212
Bernie Kosar19850.6380.394-0.244
Jim Everett19860.6040.298-0.306
Marc Bulger20020.6360.255-0.381

In Young’s case, he went from the worst team in the league to the best, which helps to explain why his winning percentage went through the roof. That’s also a good counter to the argument of using records to judge quarterbacks. In Stafford’s case, his mechanics have come under fire, and that’s probably a fair criticism. He obviously needs to improve, but it’s not like he was bad last year. He posted league-average rates in both NY/A and ANY/A in 2012, although he only ranked 22nd in my modified version of passer rating. With Stafford’s numbers, you have to remember that he’s nowhere near as good as his gross numbers make him appear, but that doesn’t make him a bad quarterback. He had Calvin Johnson but not much else last year, and I expect his rate numbers to bounce back in 2013. Without Titus Young and with Reggie Bush, there are reasons to expect a bounce-back year for Stafford. As for why he lost so many games last year? Well, I’ve already covered that.

1. Because their first seasons as starters came after age 25, I decided to eliminate Jeff Garcia, Trent Green, Warren Moon, Kurt Warner, Ed Brown, Tony Romo, Mark Rypien, and Jim Kelly from this study. []
2. And, of course, it doesn’t mean that that one extra win is because of the quarterback. Presumably, like in the case of Flacco, those quarterbacks who win games early are on good teams, and those teams are more likely to stay good than the bad teams. []
• Judging quarterbacks by record is among the dumbest things anyone does in sports, which is saying something. If we could get rid of quarterback and pitcher records, I think approximately 1/2 of the worst blather that goes on in sports would disappear instantly.

• JeremyDe

I would contend it was more than 1/2, but then I remember all the ridiculous blather associated with ring worship.

btw…good word, blather.

• Neil Paine

I obviously agree with the thesis that QB W-L records are meaningless and idiotic, but I’m not entirely sure the survivorship bias issue can be waived away so easily. You briefly touched on this in the post, but almost by definition, only teams that show improved success will keep a guy around for 25+ starts in years 5-8, particularly if he had a bad W-L% in years 1-4. I think the regression is almost completely capturing this effect & this effect only — it’s not seeing the guys whose records started bad and would have stayed bad if given the chance, because those guys get weeded out well before 25 starts in years 5-8 (prospect QBs get progressively less rope as they get older; think Joey Harrington).

• Chase Stuart

True — is there a way around the survivorship bias issue?

Also, consider that the 19 QBs who won the most games… they won 67% of their games in Years 1 through 4 and then 56% in Years 5 through 8.

• James

I think if you threw in a quality cutoff everything works fine. I suggest reviewing the QBs who meet the first half of your criteria (25+ starts in first four seasons, age 25 or younger) but don’t meet the second half (25+ starts in seasons 5-8) to see if there is a common thread between them. My guess would be most of them performed poorly, so you could say the regression only applies to QBs above a certain threshold.

My suggestion would be a cutoff ANY/A+ of ~90 for years 1-4. I’d appreciate it if someone would check my work, but from what I can tell of the 47 QBs who have 25 early career starts and an ANY/A+ above 91, only 7 of them don’t also have 25+ starts in years 5-8 (excluding 4 current starting QBs): Derek Anderson, Don Majkowski, Craig Erickson, Byron Leftwich, Gus Frerotte, Pat Haden, and Randy Johnson. Most of those players had their careers derailed by injury (Leftwich replaced by Garrard, Majkowski replaced by Favre, Erickson had Tommy John surgery, Anderson had a series of minor injuries), Haden only had 25 starts as a backup pressed into service via injury and racism, and I can’t find any information on Randy Johnson. Frerotte seems to be the biggest exception – no major injuries that I know of but lost his starting job despite above average play. In any case, it seems like the regression safely applies to QBs with ANY/A+ above 91.

On the other hand, look at the QBs with ANY/A+ below 90: Harrington, Russell, Carr, Couch. Those QBs lost a lot of games and had they continued starting they would probably change the results of the regression, but they also aren’t representative of Matthew Stafford so it’s not appropriate to include them in his sample population. After all, there’s a reason why you wrote this post about Stafford and not Mark Sanchez or Sam Bradford.

As an aside, it might be interesting to look at the guys down in this group that then went on to have successful careers, such as Bradshaw, Aikman, Plummer, Collins, and Plunkett. Interesting group there.

• Michael

Selection bias is almost certainly going to make any results of your regression full of error. Instead of regressing previous winning percentage on future winning percentage why not do a survival analysis. After doing that you can then determine the probability that someone with x% in their first four years lasts as a start to year 5, year 6, etc.

• Chase Stuart

Interesting idea, but I would think that would bring a lot of different biases into place. A lot of solid QBs on bad teams might lose their jobs, but I don’t know what that would tell us about W-L records.

In any event, we can safely determine that Stafford will be starting 25+ games for the Lions over the next 4 years barring injury.

• Pingback: 2012 Rearview Adjusted Net Yards per Attempt()

• stateofmifan

I’m amused at how the media hype influences QB picks, and how stupid fans and coaches buy into the hype! Stafford at Georgia had good receivers and all day to throw. Take OSU’s Troy Smith, Heisman man hype, got his fanny decimated in bowl game and his value tanked. Denard Robinson, initial Heisman hype, getting approx 400 yds/game himself and he would notoriously throw ints or incomplete passes in the red zone. Problem is any decent QB can complete passes given all day to throw. Stafford has arguably one of the best receiving corps, all being 6’5″ or better and he still overthrows or plain misses them. Another big hype was 1st pick Jamarcus Russell, just think 2nd pick was Calvin Johnson that same year. Look at Drew Stanton, won 2 of the 3 or 4 starts in Detroit and can’t get a chance, while Hoyer, Henne, etc. all get starts. Failure to recognize talent or the lack of in most NFL teams.