≡ Menu

Guest Post: Adam Steele on True QB Talent

Adam Steele is back for another guest post. You can view all of Adam’s posts here. As always, we thank him for contributing.

Introduction: QB True Talent

One of the mandates for being a football analytics guy is to create a quarterback rating system, so today I’m going to throw my flag onto the field. However, I’m taking a different approach by asking a different question. I’m not trying to measure individual or team accomplishments, nor am I calculating value or attempting to predict the future. My goal is to answer a simple question: At a fundamental level, how good was he? As far as I’m aware, nobody has made a systematic attempt at answering this. Before we go any further, I need to add a vital disclaimer. My formula is statistically derived, and does not account for supporting cast, coaching, or anything else we can’t measure directly. So when I use the label “True Talent”, it really means “A rough estimate of true talent, based solely on statistics.”

I’ll save the gory micro details of True Talent’s calculation for another post, but today I want to outline how it works and ask for feedback on how to improve it. First, I’ve attempted to isolate what I believe are the four pillars of QB play: Passing Dominance, Passing Consistency, Ball Protection, and Rushing Ability. These categories are weighted by a) their importance within the framework of the overall QB skillet, and b) the level of control a QB has in converting the skill into results. The score for each category is era-adjusted, balanced by volume and efficiency, and scaled so zero is equal to replacement level. The overall True Talent score is simply the sum of the four category scores, minus five (the replacement level bar is higher for overall QB play compared to each of the pillars on their own). The overall score is expressed as percentage above or below replacement level. I’ve purposely rounded all figures to whole numbers to remind readers that these numbers are estimates, not precise measurements.

My plan is to eventually apply True Talent back to the 1940s, but for now we’re going to look at the last twenty seasons (1996-2015). I want to nail down the methodology before I go all the way back through history. Normally I wouldn’t subject readers to an arduously long table, but in this case I think it’s warranted. I want you to see how all levels of QB fare in my system, not just the best and worst. This table includes every QB season since 1996 with at least 100 dropbacks. I encourage you to sort by each category, by season, and by player to really get a feel for True Talent.

11Peyton ManningIND200458.630.525.47.70.0
21Peyton ManningIND200648.020.621.010.41.1
31Tom BradyNE200746.022.916.110.02.1
41Peyton ManningDEN201342.921.
51Peyton ManningIND200541.719.619.57.50.0
61Aaron RodgersGB201141.421.413.08.93.2
72Drew BreesNO201140.718.
81Randall CunninghamMIN199839.722.312.67.82.1
91Daunte CulpepperMIN200038.220.913.62.85.9
101Carson PalmerARI201536.220.517.03.70.0
111Steve McNairTEN200336.219.315.44.52.0
123Tom BradyNE201136.114.317.07.22.6
131Kurt WarnerSTL200135.319.818.32.20.0
141Peyton ManningIND200934.915.617.56.70.0
152Philip RiversSD200934.615.616.76.90.4
161Kurt WarnerSTL199934.519.
172Steve YoungSF199833.712.312.76.67.2
181Aaron RodgersGB201433.514.814.24.94.7
192Matt HasselbeckSEA200533.514.514.08.61.5
202Peyton ManningIND200033.418.612.06.61.2
213Kurt WarnerSTL200033.421.018.3-0.90.0
223Drew BreesNO200933.
232Daunte CulpepperMIN200433.
243Carson PalmerCIN200532.016.612.67.70.0
252Tony RomoDAL200730.917.
262Philip RiversSD201330.612.616.16.60.3
271Peyton ManningDEN201230.616.
284Jeff GarciaSF200030.512.
291Chad PenningtonNYJ200230.
303Vinny TestaverdeNYJ199830.011.814.96.51.7
311Philip RiversSD200829.816.612.54.21.5
323Peyton ManningIND200729.816.
334Chris ChandlerATL199829.715.814.12.91.9
341Philip RiversSD201029.612.916.74.90.1
352Peyton ManningIND200329.513.512.09.00.0
362Cam NewtonCAR201529.411.
372Peyton ManningIND199929.214.412.86.30.7
384Matt SchaubHOU200929.213.912.77.60.0
392Steve McNairTEN200129.
403Nick FolesPHI201328.914.
415Brian GrieseDEN200028.912.
421Brett FavreGB199728.415.811.04.72.0
432Tom BradyNE201228.39.513.29.01.6
445Doug FlutieBUF199827.912.411.85.43.3
452Tony RomoDAL200627.717.
465Brett FavreMIN200927.610.811.710.20.0
472Aaron RodgersGB201027.311.
486Rich GannonOAK200027.310.
493Tom BradyNE201027.
502Tony RomoDAL201427.
514David GarrardJAC200726.910.
526John ElwayDEN199826.815.
534Ben RoethlisbergerPIT200526.814.
543Trent GreenKC200326.79.812.57.91.5
553Ben RoethlisbergerPIT200426.611.713.45.31.2
562Trent GreenKC200226.510.710.27.23.4
572Steve YoungSF199726.47.813.07.82.7
582Peyton ManningIND200826.111.811.37.90.1
593Drew BreesNO201226.114.313.03.70.0
606Tom BradyNE200925.912.
614Donovan McNabbPHI200425.812.
625Brett FavreGB200425.514.
636Tom BradyNE200425.513.
644Matt RyanATL201225.410.
653Peyton ManningDEN201425.
663Drew BreesNO200825.
673Russell WilsonSEA201524.911.
687Trent GreenKC200424.811.414.43.30.7
693Mark BrunellJAC199724.
703Drew BreesNO200624.713.
715Ben RoethlisbergerPIT200724.610.610.44.44.2
725Tom BradyNE200524.610.
737Trent GreenSTL200024.314.
748Elvis GrbacKC200023.713.
757Ben RoethlisbergerPIT200923.710.610.06.21.9
764Jay CutlerDEN200823.512.
775Chad PenningtonMIA200823.410.
783Jeff GarciaSF200123.
794Ben RoethlisbergerPIT201423.312.
804Jake PlummerDEN200323.
818Aaron RodgersGB200923.
821Mark BrunellJAC199623.
834Boomer EsiasonCIN199723.211.410.46.40.0
842Vinny TestaverdeBAL199622.911.
853Steve McNairTEN200222.
863Steve BeuerleinCAR199922.311.
874Peyton ManningIND201022.310.
883Steve YoungSF199622.
895Aaron RodgersGB201221.
904Drew BreesNO201321.911.
915Daunte CulpepperMIN200321.
924Andy DaltonCIN201521.713.
934Peyton ManningIND200221.710.
946Robert GriffinWAS201221.
958Drew BreesSD200421.412.
964Rich GannonOAK199921.45.610.36.54.0
974Brett FavreGB199621.310.
985John ElwayDEN199621.
996Jake PlummerDEN200521.
1007Russell WilsonSEA201221.
1015Andrew LuckIND201421.
1025Rich GannonOAK200221.
1034Donovan McNabbPHI200621.
1044Rich GannonOAK200121.
1059Tony RomoDAL200921.
1065Josh McCownCHI201320.
1075Carson PalmerCIN200620.813.
1085Ben RoethlisbergerPIT201020.
1099Jake PlummerDEN200420.812.
1105Peyton ManningIND200120.610.
1116Marc BulgerSTL200220.611.
1124Tony RomoDAL201120.
1136Drew BreesNO201420.510.
1146Derek AndersonCLE200720.412.
1155Eli ManningNYG201120.313.
1167Matt HasselbeckSEA200220.
1175Mark BrunellJAC199919.
1186Matthew StaffordDET201119.
1197Carson PalmerCIN200719.810.
1206Matt HasselbeckSEA200319.
1215Ben RoethlisbergerPIT201519.614.
1225Kordell StewartPIT199719.
1237Matt RyanATL201119.
1246Matt RyanATL200819.411.
1258Philip RiversSD201119.49.812.81.60.2
1266Jeff GeorgeMIN199919.315.
1277Trent GreenKC200519.
1286Brett FavreGB200119.312.710.11.50.0
12910Marc BulgerSTL200419.
1307Tom BradyNE201419.
1317Troy AikmanDAL199819.
1328Brett FavreGB200719.
1336Damon HuardKC200619.06.713.33.90.0
1349Matt SchaubHOU201118.
1356Josh FreemanTB201018.
13610Michael VickPHI201118.
1378Mark BrunellJAC199818.
1387Philip RiversSD200618.
1397Michael VickPHI201018.
1408Michael VickATL200218.
1416Jameis WinstonTB201518.
1429Brett FavreGB199818.
1436John ElwayDEN199718.
14410Steve McNairTEN199817.
1457Kurt WarnerARI200817.810.
1469Jeff GarciaSF200217.
1476Russell WilsonSEA201317.
1488Jake DelhommeCAR200817.511.
1497Tom BradyNE201517.
1508Matt RyanATL201417.
15110Kerry CollinsNYG200217.
15211Ben RoethlisbergerPIT201117.26.712.03.30.3
1538Matt SchaubHOU201017.06.310.64.80.3
1547Chris ChandlerATL199717.
1559Matt RyanATL201016.
1569Aaron RodgersGB200816.810.
1578Marc BulgerSTL200616.84.86.410.00.6
1588Dan MarinoMIA199716.
1598Kirk CousinsWAS201516.
16012Cam NewtonCAR201116.
1617Brad JohnsonWAS199916.610.
1628Doug FlutieBUF199916.
1638Drew BreesSD200516.
16411Jake DelhommeCAR200416.
1659Jay CutlerDEN200716.
1669Marc BulgerSTL200516.
1678Colin KaepernickSF201216.
1687Kordell StewartPIT200116.
16910Matt HasselbeckSEA200716.
1706Gus FrerotteWAS199616.
17112Chad PenningtonNYJ200415.
1729Jeff GarciaSF199915.
1738Jay FiedlerMIA200115.
1747Brett FavreGB200315.810.
1757Aaron RodgersGB201315.810.
1767Dan MarinoMIA199615.
17713Matt HasselbeckSEA200415.
1789Cam NewtonCAR201215.
1798Aaron BrooksNO200315.
18011Drew BreesNO200715.
1819Ryan FitzpatrickHOU201415.
18210Kurt WarnerARI200915.
1838Colin KaepernickSF201315.
1849Marc BulgerSTL200315.
18512Kurt WarnerARI200715.313.27.9-0.70.0
1869Kerry CollinsNYG200015.
18711Jay FiedlerMIA200215.
1889Drew BreesNO201515.
18910Eli ManningNYG201215.
19010Jeff GarciaSF200315.
19111Eli ManningNYG200915.
1929Drew BledsoeNE199714.
19310Matt CasselKC201014.
19411Jon KitnaCIN200314.
19510Mark BrunellJAC200014.
19610Tyrod TaylorBUF201514.
19711Ryan FitzpatrickNYJ201514.
19810Donovan McNabbPHI200814.
1999Jake PlummerARI200114.
20010Brad JohnsonMIN199714.
20110David GarrardJAC200514.
2029Cam NewtonCAR201314.
20311Matthew StaffordDET201214.
20411Kyle OrtonDEN201014.
20510Joe FlaccoBAL201414.
20610Tony RomoDAL201314.
2078Jeff BlakeCIN199613.
20813Sage RosenfelsHOU200713.910.18.5-0.60.9
20912Matt SchaubHOU201213.
21011Matt SchaubHOU200813.
21111Warren MoonSEA199713.
21211Drew BledsoeNE199813.
2139Brad JohnsonMIN199613.
21410Steve McNairTEN199913.
21513Ben RoethlisbergerPIT201213.
21611Steve McNairTEN200513.
21711Chris ChandlerATL199913.
21814Tony RomoDAL201213.
21912Eli ManningNYG200813.
22011Steve McNairTEN200013.
22111Eli ManningNYG201413.
22210Troy AikmanDAL199613.
2239Steve McNairBAL200613.
22412Eli ManningNYG201013.113.37.2-2.40.0
22514Jeff GarciaTB200713.
22610Michael VickATL200612.
22712Drew BledsoeBUF200212.
22811Drew BledsoeNE199612.
22910Mark BrunellJAC200112.
23012Jeff GeorgeOAK199712.
23113Brian GrieseDEN200212.
23214Tommy MaddoxPIT200212.611.47.4-1.30.1
23312Joe FlaccoBAL200912.
23413Carson PalmerOAK201112.510.
23511Andy DaltonCIN201312.
23613Drew BreesNO201012.
23712Derek AndersonCAR201412.
23815Donovan McNabbPHI200212.
23913Philip RiversSD201412.
24012Ty DetmerPHI199612.
24112Brad JohnsonTB200312.
24213Vince YoungTEN200912.
24314Russell WilsonSEA201412.
24415Ryan TannehillMIA201412.
24512Jay CutlerCHI201312.
24612Doug FlutieBUF200011.
24713Donovan McNabbPHI200011.8-
24812Matthew StaffordDET201511.
24914Shaun KingTB200011.
25012Bubby BristerDEN199811.
25111Ben RoethlisbergerPIT200611.
25212Elvis GrbacKC199911.
25313David GarrardJAC200811.
25413Scott MitchellDET199611.
25516Brett FavreGB200211.
25614Brian GrieseTB200411.
25712Aaron BrooksNO200511.
25813Byron LeftwichJAC200511.
25913Jon KitnaSEA199911.
26014Donovan McNabbPHI200910.
26115Matt RyanATL200910.
26213Matt RyanATL201510.
26315David CarrHOU200410.
26413Andrew LuckIND201310.
26514Jay CutlerCHI201510.
26613Elvis GrbacKC199710.
26713Tom BradyNE200310.
26815Alex SmithSF201210.
26912Tom BradyNE200610.
27016Kyle OrtonDEN200910.
27117Kelly HolcombCLE200210.
27214Vinny TestaverdeNYJ200310.
27314Joe FlaccoBAL201010.
27418Mark BrunellJAC200210.
27511Brad JohnsonTB200110.5-0.12.510.22.9
27616Brian HoyerCLE201410.
27714Brett FavreGB199910.
27814Gus FrerotteWAS199710.
27915Gus FrerotteDEN200010.
28014Kurt WarnerARI200510.
28117Alex SmithKC201410.3-
28214Jeff GarciaTB200810.
28315Matt CasselNE200810.
28419Brad JohnsonTB200210.
28515Michael VickATL200510.
28615Steve McNairTEN199710.
28712Doug FlutieSD200110.
28815Randall CunninghamMIN199910.
28916Byron LeftwichJAC200410.
29013Charlie BatchDET19989.
29115Donovan McNabbPHI20039.
29214Matthew StaffordDET20139.
29315Tony RomoDAL20109.
29416Chad PenningtonNYJ20039.
29517Carson PalmerCIN20099.
29616Jake DelhommeCAR20059.
29718David GarrardJAC20099.
29816Ryan FitzpatrickBUF20109.
29917Mark BrunellWAS20059.
30015Tom BradyNE20139.
30115Blake BortlesJAC20159.57.24.2-1.84.8
30216Aaron BrooksNO20009.
30314Jeff HostetlerOAK19969.
30417Tim RattaySF20039.
30520Aaron BrooksNO20029.
30617Vince YoungTEN20109.
30714Brad JohnsonMIN19989.
30815Kerry CollinsCAR19969.
30915Donovan McNabbPHI20079.
31016Glenn FoleyNYJ19979.
31117Jay FiedlerMIA20009.
31218Carson PalmerCIN20109.
31316Josh FreemanTB20129.
31413Chad PenningtonNYJ20069.
31518Eli ManningNYG20058.
31618Brett FavreGB20008.
31716Drew BledsoeNE19998.
31819Matt MooreCAR20098.
31913Daunte CulpepperMIN20018.84.14.4-0.86.1
32019Steve BeuerleinCAR20008.
32116Todd CollinsWAS20078.
32216Brian HoyerHOU20158.
32314Jeff GarciaPHI20068.
32417Andrew LuckIND20128.
32517Troy AikmanDAL19978.
32616Ben RoethlisbergerPIT20138.
32718Tommy MaddoxPIT20038.
32816Mike TomczakPIT19968.
32914Donovan McNabbPHI20018.42.1-
33017Alex SmithKC20158.
33115Aaron BrooksNO20018.
33216Tony RomoDAL20088.311.23.7-1.60.0
33318Carson PalmerARI20148.
33418Trent DilferTB19978.
33515Erik KramerCHI19988.
33616Dan MarinoMIA19988.
33717Carson PalmerARI20138.
33815Matt HasselbeckSEA20068.
33919Colin KaepernickSF20148.
34021Jon KitnaCIN20028.
34118Marcus MariotaTEN20158.04.66.2-1.53.7
34217Jeff BlakeCIN19998.
34317Quinn GrayJAC20078.
34418Matt SchaubHOU20078.
34517Boomer EsiasonARI19968.
34618Michael VickPHI20137.
34719Jim HarbaughIND19977.
34820Paul JustinIND19977.
34919Kerry CollinsOAK20057.
35018Neil O'DonnellTEN19997.
35119Jason CampbellWAS20077.
35216Jason CampbellWAS20067.
35317Peyton ManningIND19987.
35422Kordell StewartPIT20027.
35519Alex SmithKC20137.
35617Aaron BrooksNO20047.
35717David GarrardJAC20067.
35820Matt RyanATL20137.
35920Cam NewtonCAR20147.
36020Donovan McNabbPHI20057.
36118Billy Joe TolliverNO19987.
36218John FrieszSEA19967.
36319Jay FiedlerJAC19997.
36416Trent GreenKC20017.
36517Chris ChandlerATL20017.
36619Steve McNairHOU19967.46.51.9-0.14.1
36717Trent EdwardsBUF20087.
36820Rob JohnsonBUF20007.
36921Erik KramerCHI19977.
37021Kordell StewartPIT20007.
37121Drew BledsoeDAL20057.16.75.2-0.10.3
37220Vince YoungTEN20077.
37320Dan MarinoMIA19997.
37420Jason CampbellWAS20097.
37518Jon KitnaDET20067.
37619Trent GreenWAS19987.
37718Billy VolekTEN20046.
37819Byron LeftwichJAC20036.95.83.9-0.12.3
37919Eli ManningNYG20156.
38020Jay FiedlerMIA20036.
38120Dave KriegCHI19966.
38218Tyler ThigpenKC20086.
38319David GarrardJAC20106.86.23.9-2.84.5
38414Alex SmithSF20116.
38519Kerry CollinsTEN20086.
38619Carson PalmerCIN20046.
38720Jason CampbellWAS20086.5-
38821Philip RiversSD20076.
38920Trent DilferTB19986.
39022Scott MitchellDET19976.
39121Jake PlummerARI19986.
39221Charlie BatchDET19996.
39320Jay CutlerCHI20106.
39422Jeff BlakeNO20006.
39523Drew BreesSD20026.
39621Robert GriffinWAS20136.
39721Kent GrahamARI19966.
39815Andy DaltonCIN20116.03.8-
39920Philip RiversSD20156.
40021Jon KitnaDAL20105.
40122Chris ChandlerHOU19965.
40218Jay CutlerCHI20125.
40318Trent DilferSEA20015.
40419Joe FlaccoBAL20125.
40524Tom BradyNE20025.
40622Gus FrerotteDET19995.
40723Kerry CollinsNYG19995.63.27.1-0.50.8
40819Brian GrieseDEN20015.
40916Kyle Orton2TM20115.
41021Drew StantonARI20145.
41120Sam BradfordSTL20125.
41223Stan HumphriesSD19965.
41321Chad HenneMIA20095.
41423Steve BeuerleinCAR19975.
41519Matt LeinartARI20065.
41624Jim HarbaughIND19965.
41722Ryan TannehillMIA20135.
41821Shaun HillSF20085.
41922Kirk CousinsWAS20145.26.34.6-1.10.5
42017Jason CampbellOAK20115.
42124Mike TomczakPIT19995.
42220Alex Van PeltBUF20015.
42320Vinny TestaverdeDAL20044.
42425Troy AikmanDAL19994.
42526Ray LucasNYJ19994.
42623Andy DaltonCIN20144.
42720Rex GrossmanCHI20064.
42822Kerry CollinsTEN20104.
42924Jake PlummerARI19974.
43023Mark SanchezNYJ20104.
43121Derek CarrOAK20154.
43223Vinny TestaverdeNYJ20004.
43321Jake DelhommeCAR20034.66.14.1-0.70.1
43422Rob JohnsonBUF19984.
43524Mark SanchezPHI20144.54.85.6-1.30.4
43622Tarvaris JacksonMIN20074.
43721Carson PalmerOAK20124.
43824Jeff GeorgeWAS20004.
43921Michael VickATL20044.40.5-0.1-0.49.4
44022Andy DaltonCIN20124.
44125Rich GannonKC19974.
44218Jay CutlerCHI20114.
44322Brandon Weeden2TM20154.
44422Quincy CarterDAL20034.
44526Vinny TestaverdeBAL19974.
44623Aaron RodgersGB20154.10.4-
44721Tom BradyNE20014.
44824Chad HenneMIA20104.
44919Joe FlaccoBAL20114.
45023Doug FlutieSD20034.
45121Vince YoungTEN20064.
45220Matt HasselbeckTEN20114.
45325Matthew StaffordDET20144.
45421Chad HenneMIA20113.
45522Brad JohnsonMIN20053.
45625Daunte CulpepperMIN20023.84.34.8-7.77.3
45723Chris SimmsTB20053.
45825Shaun HillDET20103.
45925Brad JohnsonWAS20003.
46022Charlie BatchDET20013.
46122Steve McNairTEN20043.
46226Teddy BridgewaterMIN20143.
46322Eli ManningNYG20063.
46423Neil O'DonnellCIN19983.42.7-
46527Mike GlennonTB20143.35.7-
46623Jon KitnaDET20073.34.63.6-0.30.4
46727Brian GrieseDEN19993.
46825Frank ReichNYJ19963.37.93.8-3.80.4
46923Ryan FitzpatrickBUF20123.
47028Billy Joe TolliverNO19993.15.01.7-1.83.2
47124Rich GannonKC19983.
47224Kerry CollinsNYG20033.
47323Ryan FitzpatrickTEN20133.14.02.3-1.63.4
47423Vinny TestaverdeNYJ20013.
47527Jeff BlakeCIN19973.0-
47625Jason GarrettDAL19983.05.5-
47724A.J. McCarronCIN20153.
47826Colt McCoyCLE20103.
47926Frank ReichDET19983.
48028Nick FolesPHI20142.
48124Brett FavreGB20052.92.95.0-0.70.8
48226Drew BledsoeNE20002.9-
48323Jake DelhommeCAR20062.
48426Paul JustinIND19962.
48528Eric ZeierBAL19972.
48625Ryan TannehillMIA20152.
48724Kerry CollinsNYG20012.73.24.6-0.50.4
48822Tarvaris JacksonMIN20082.
48922Ryan FitzpatrickBUF20112.
49023Joe FlaccoBAL20082.
49129Bobby HoyingPHI19972.
49225Jamie MartinSTL20052.
49325Michael VickATL20032.
49424Seneca WallaceSEA20082.61.5-
49527Kent GrahamNYG19982.60.7-
49624Chris RedmanATL20072.
49726Jim MillerCHI20022.
49826David CarrHOU20032.
49926Josh McCownARI20052.
50024Sam BradfordSTL20132.
50123Kerry CollinsOAK20042.
50225Chad PenningtonNYJ20072.
50325Kyle OrtonCHI20082.
50428Steve BeuerleinCAR19982.24.35.1-2.20.0
50529Kordell StewartPIT19982.2-1.3-
50630Neil O'DonnellNYJ19972.10.6-
50724Mark BrunellWAS20062.
50829Damon HuardMIA19992.1-
50925Elvis GrbacBAL20012.
51025Jake LockerTEN20132.01.7-
51127Bobby HebertATL19962.03.15.8-3.11.2
51226Teddy BridgewaterMIN20152.0-
51328Steve BonoKC19961.9-
51427Steve BeuerleinDEN20021.85.93.4-2.70.2
51531Ty DetmerPHI19971.
51622Chris RedmanATL20091.
51730Kent GrahamNYG19991.
51829Kyle OrtonBUF20141.
51925Byron LeftwichJAC20061.
52026Ben RoethlisbergerPIT20081.63.03.2-1.01.3
52124Jake LockerTEN20121.52.4-
52229Jim KellyBUF19961.55.21.5-2.01.8
52325Ryan TannehillMIA20121.
52426Trent GreenKC20061.51.05.2-0.40.7
52527Brett FavreGB20061.
52632Tony BanksSTL19971.
52730Rodney PeetePHI19961.
52823Jay CutlerCHI20091.44.40.1-0.92.8
52926Matt CasselMIN20131.
53026Luke McCownTB20071.
53131Tony BanksBAL19991.23.0-
53227Andrew LuckIND20151.23.90.1-0.83.1
53330Jeff BlakeCIN19981.21.5-
53430Charlie WhitehurstTEN20141.22.0-
53527Kyle BollerBAL20071.
53626Christian PonderMIN20121.1-
53728J.P. LosmanBUF20061.02.7-
53828Trent GreenMIA20071.01.83.6-0.10.7
53932Jim MillerCHI19991.
54029Tim RattayTB20061.05.2-
54127Randall CunninghamDAL20001.04.2-
54228Jeff BlakeBAL20020.
54327Matt McGloinOAK20130.95.22.2-1.60.1
54431Austin DavisSTL20140.
54530Jake PlummerDEN20060.
54627Seneca WallaceCLE20100.
54728Jason CampbellOAK20100.
54823Mark SanchezNYJ20110.
54924Matt MooreMIA20110.74.62.6-1.90.3
55027Kevin KolbARI20120.7-
55129Tim CouchCLE20020.63.1-
55231Elvis GrbacSF19960.62.5-
55333Shane MatthewsCHI19990.6-
55427Brett FavreNYJ20080.51.95.8-2.60.4
55528Josh McCownCLE20150.
55627Brooks BollingerNYJ20050.5-1.8-
55724Joey HarringtonDET20040.5-1.0-
55825Donovan McNabbMIN20110.4-
55926Rob JohnsonBUF20010.4-
56029Brock OsweilerDEN20150.
56130Trent DilferSEA20020.
56227Jim MillerCHI20010.21.0-
56326Josh FreemanTB20110.
56429Eli ManningNYG20070.
56534Trent DilferTB19990.
56632Trent DilferTB19960.
56725Drew BledsoeBUF20040.
56828Tim CouchCLE20000.13.6-
56931Jamie MartinSTL20020.11.94.1-0.80.0
57030Matt HasselbeckIND20150.
57128Daunte CulpepperMIN20050.00.52.6-1.23.1
57232Quincy CarterDAL20020.
57328Michael VickPHI2012-
57429Gus FrerotteMIA2005-0.11.6-
57531Ryan Mallett2TM2015-
57628Michael VickATL2001-0.1-0.2-
57733Rodney PeetePHI1997-
57829Donovan McNabbWAS2010-
57927Patrick RamseyWAS2003-0.20.7-
58033Craig EricksonMIA1996-
58124Bruce GradkowskiOAK2009-0.21.9-
58233Rodney PeeteCAR2002-
58335Kordell StewartPIT1999-0.3-
58431Jim HarbaughBAL1998-
58527Dan OrlovskyIND2011-
58634A.J. FeeleyPHI2002-
58728Sage RosenfelsHOU2008-
58829Matt HasselbeckSEA2001-0.4-1.8-
58928Rex GrossmanWAS2011-
59035Shane MatthewsWAS2002-0.50.3-
59136Jim HarbaughSD1999-
59230Drew StantonDET2010-0.5-
59329Brandon WeedenCLE2012-0.5-
59428Brian HoyerCLE2013-
59529Colt McCoyCLE2011-0.6-
59630Tony BanksWAS2001-0.60.0-
59729Mike GlennonTB2013-0.61.6-
59832Eric ZeierBAL1998-0.6-
59933Glenn FoleyNYJ1998-
60030Matt SchaubHOU2013-
60128Jeff BlakeARI2003-0.72.1-
60231Alex SmithSF2010-0.71.0-
60330J.P. LosmanBUF2007-0.7-
60431Marc BulgerSTL2007-
60534Rodney PeetePHI1998-0.8-
60631Alex SmithSF2006-0.80.6-
60732Matt LeinartARI2007-0.9-0.1-
60831Case KeenumHOU2013-1.01.9-
60932Jay CutlerCHI2014-
61033Joey HarringtonATL2007-1.0-2.6-
61129JaMarcus RussellOAK2008-1.10.4-
61232Joey HarringtonMIA2006-
61329Charlie BatchDET2000-1.2-0.8-
61430Troy AikmanDAL2000-
61530Brian GrieseTB2005-
61633Drew BledsoeDAL2006-
61735Steve StenstromCHI1998-1.3-
61832Case KeenumSTL2015-
61930Dan OrlovskyDET2008-1.31.8-
62034Damon HuardKC2007-
62131Jake PlummerARI2000-1.31.8-0.5-0.12.4
62231Joey HarringtonDET2005-
62331Jon KitnaCIN2001-
62432Matt HasselbeckSEA2010-
62534Dave BrownNYG1997-1.4-0.7-
62632Randall CunninghamBAL2001-
62732Terrelle PryorOAK2013-1.5-2.1-0.4-0.16.1
62830Matt HasselbeckTEN2012-
62933Quincy CarterDAL2001-1.61.9-
63029Rick MirerOAK2003-1.6-2.2-
63133Geno SmithNYJ2014-
63234Rick MirerSEA1996-1.6-0.7-
63332Trent DilferCLE2005-
63437Billy Joe HobertNO1999-1.70.5-
63533Anthony WrightBAL2005-1.7-0.6-
63634Mike McMahonDET2001-1.7-1.3-
63733Jason CampbellCLE2013-1.70.3-
63834Christian PonderMIN2013-
63935Trent EdwardsBUF2007-1.8-
64025Alex SmithSF2009-1.81.9-
64130Tim CouchCLE2003-1.81.2-
64232Jim HarbaughSD2000-
64336Warren MoonSEA1998-
64438Cade McNownCHI1999-1.80.9-
64535Chad HenneJAC2013-1.9-2.2-
64636Geno SmithNYJ2013-
64733Johnny ManzielCLE2015-1.9-
64826Josh McCownARI2004-1.9-0.9-
64935Kent GrahamARI1997-1.9-1.1-
65030Tim TebowDEN2011-1.90.0-
65137Joe FlaccoBAL2013-2.0-0.2-
65231Kelly HolcombCLE2003-2.0-
65333Sam BradfordSTL2010-2.0-
65432Joey HarringtonDET2003-2.0-0.4-
65533Chris ChandlerATL2000-
65634Kellen MooreDAL2015-
65737Steve BonoSTL1998-
65836Stan HumphriesSD1997-
65927Kurt WarnerNYG2004-2.1-
66036Kellen ClemensNYJ2007-2.1-
66133Kordell StewartCHI2003-2.1-0.5-
66237Matt MooreCAR2007-
66334Kurt WarnerARI2006-
66431Gus FrerotteMIN2008-
66535Warren MoonMIN1996-
66635Blaine GabbertSF2015-2.3-0.2-
66731Tarvaris JacksonSEA2011-2.30.2-
66835Tim CouchCLE2001-2.31.1-
66934Tony BanksBAL2000-2.3-
67034Tim HasselbeckWAS2003-
67132Kevin KolbARI2011-
67233Matt CasselKC2011-2.40.5-
67335Shane MatthewsCHI2000-2.5-
67436Kent GrahamPIT2000-2.5-1.0-
67536Tony RomoDAL2015-
67634Seneca WallaceCLE2011-2.6-0.7-
67732Brian GrieseTB2008-2.6-
67836Dave BrownNYG1996-2.6-
67937Jim EverettNO1996-2.61.9-
68036Jake PlummerARI2002-2.6-0.2-1.7-0.24.6
68128Jeff GarciaCLE2004-2.61.4-2.0-0.43.4
68237Joe FlaccoBAL2015-2.70.6-
68335Brad JohnsonMIN2006-2.7-
68437Brock HuardSEA2000-2.8-
68538Cleo LemonMIA2007-2.9-3.7-
68639Shaun KingTB1999-2.9-0.1-
68735Kellen ClemensSTL2011-2.9-0.7-
68836T.J. YatesHOU2011-2.9-
68938Kellen ClemensSTL2013-
69038Jon KitnaSEA1998-
69137Christian PonderMIN2011-3.0-0.1-0.4-1.33.9
69236David CarrHOU2006-3.0-
69334Jeff GarciaDET2005-3.0-1.1-
69437Danny WuerffelWAS2002-
69529Tim RattaySF2004-3.0-
69626Marc BulgerSTL2009-3.1-
69738Neil O'DonnellNYJ1996-3.1-0.9-
69837Charlie FryeCLE2006-3.2-1.8-
69935Rich GannonOAK2003-3.2-0.7-
70034Matthew StaffordDET2010-3.2-0.1-
70130Jon KitnaCIN2004-
70236Doug JohnsonATL2003-
70335David CarrHOU2005-3.3-4.6-
70433Matt HasselbeckSEA2008-3.3-
70531Kyle BollerBAL2004-3.3-3.5-
70638Seneca WallaceSEA2006-3.33.6-0.4-3.82.3
70727Brady QuinnCLE2009-3.4-2.7-
70834EJ ManuelBUF2014-3.4-1.1-
70939Tony BanksSTL1998-3.4-2.0-
71028Ryan FitzpatrickBUF2009-3.50.0-
71131Philip RiversSD2012-
71239A.J. FeeleyPHI2007-
71339Todd CollinsBUF1996-3.62.8-1.4-0.20.3
71435Troy SmithSF2010-3.61.4-1.3-0.92.3
71532Patrick RamseyWAS2004-3.6-
71638Chris RedmanBAL2002-3.60.2-
71739EJ ManuelBUF2013-3.60.1-
71836Derek AndersonARI2010-3.70.5-
71938Sam BradfordPHI2015-
72029Kerry CollinsTEN2009-3.70.0-
72130Trent EdwardsBUF2009-3.7-2.0-
72237Drew BledsoeBUF2003-3.8-
72340Stoney CaseBAL1999-3.8-0.7-
72431Byron LeftwichTB2009-3.91.7-
72538Tony BanksHOU2003-3.91.5-
72640Jeff GeorgeATL1996-
72738John SkeltonARI2011-3.91.6-1.1-1.62.2
72840Matt Flynn3TM2013-
72935Shaun HillSTL2014-
73037Rex GrossmanWAS2010-
73140Daunte CulpepperOAK2007-
73238Stoney CaseDET2000-4.1-2.1-
73339Colin KaepernickSF2015-4.1-3.1-
73436Kyle BollerBAL2005-4.20.2-0.4-0.21.2
73541Brian GrieseCHI2007-
73640Koy DetmerPHI1998-4.4-1.6-
73733Brad JohnsonTB2004-4.40.0-
73836Jake LockerTEN2014-4.50.1-0.1-2.22.8
73934Carson PalmerCIN2008-4.5-
74039Jay CutlerDEN2006-
74139Cade McNownCHI2000-4.5-0.8-
74239Joey HarringtonDET2002-4.6-0.9-
74340Peyton ManningDEN2015-
74441Steve BeuerleinCAR1996-
74539John BeckWAS2011-4.6-
74632Matt HasselbeckSEA2009-4.6-1.2-
74736Chris WeinkeCAR2001-4.6-1.7-
74837Billy Joe Hobert2TM1997-
74941Tony GrazianiATL1999-4.8-
75037Shane MatthewsCHI2001-4.9-0.63.1-2.50.0
75142Tim CouchCLE1999-4.9-1.9-
75239Drew BreesSD2003-5.0-1.9-
75338Danny KanellNYG1997-5.0-0.2-
75437Derek CarrOAK2014-5.0-1.1-
75540Chris ChandlerCHI2002-5.1-0.9-
75640Daunte CulpepperMIA2006-5.3-
75738Bruce GradkowskiOAK2010-
75840Curtis PainterIND2011-5.4-1.2-1.1-0.12.0
75933Seneca WallaceSEA2009-5.5-
76041Danny WuerffelNO1998-5.5-2.6-
76137Ryan FitzpatrickSTL2005-
76238Colt McCoyWAS2014-
76332Chad HenneJAC2012-5.71.2-
76441Eli ManningNYG2013-
76539Zach MettenbergerTEN2014-5.72.2-0.7-2.30.0
76642Brandon WeedenCLE2013-5.80.7-1.1-0.70.3
76741Mark SanchezPHI2015-5.8-0.40.0-0.70.4
76841Vince YoungPHI2011-
76942Don MajkowskiDET1996-5.90.0-0.7-0.60.4
77041Rob JohnsonTB2002-5.9-3.0-
77140Chris ChandlerCHI2003-5.9-1.5-
77235Derek AndersonCLE2008-5.9-0.1-0.5-0.70.3
77339Todd CollinsBUF1997-6.0-0.4-
77436Marc BulgerSTL2008-6.0-2.0-
77541Chris WeinkeCAR2006-6.00.1-
77643Erik KramerCHI1996-6.1-1.4-
77734Mark BrunellWAS2004-6.1-2.5-
77843Rick MirerNYJ1999-6.2-0.9-
77934Shaun HillSF2009-6.2-2.5-
78038Charlie FryeCLE2005-6.2-1.1-1.2-0.11.2
78140Billy Joe Tolliver2TM1997-6.2-1.10.3-0.50.0
78235Matthew StaffordDET2009-6.3-1.3-0.7-1.32.1
78336Daunte CulpepperDET2009-6.3-1.4-
78437Mark SanchezNYJ2009-
78543Thaddeus LewisBUF2013-6.40.0-0.7-0.80.0
78639Tim RattaySF2005-
78742Aaron BrooksOAK2006-6.7-3.5-
78840Trent DilferBAL2000-
78933Matt CasselKC2012-6.7-0.12.8-7.12.7
79041Jeff HostetlerWAS1997-
79140Mike McMahonPHI2005-6.7-0.7-2.2-0.81.9
79242Patrick RamseyWAS2002-
79342Vinny TestaverdeCAR2007-7.0-1.5-
79441Scott MitchellCIN2000-7.1-2.1-1.3-0.41.7
79543Derek AndersonCLE2006-
79639Kevin KolbPHI2010-7.2-0.5-0.1-2.81.2
79742Jimmy Clausen2TM2015-7.2-0.7-
79842Jeff GeorgeOAK1998-7.3-1.30.0-1.10.0
79934Nick FolesPHI2012-7.4-1.9-
80043Rex GrossmanCHI2007-7.4-2.7-
80144Josh Freeman2TM2013-7.4-2.3-
80242Sam BradfordSTL2011-7.4-3.0-
80341Anthony WrightBAL2003-
80435Blaine GabbertJAC2012-7.4-1.0-
80544Chris SimmsTB2006-7.5-1.0-1.1-0.70.3
80637Ryan FitzpatrickCIN2008-7.5-5.2-
80740John SkeltonARI2010-7.5-1.5-
80843Kerry CollinsIND2011-7.6-2.6-
80944Brodie CroyleKC2007-7.6-2.2-
81045Steve McNairBAL2007-7.7-3.7-
81143Ray LucasMIA2002-7.8-1.00.0-4.12.3
81244Glenn FoleyNYJ1996-7.8-0.8-2.1-0.60.7
81342Brian GrieseMIA2003-7.8-0.1-0.8-1.90.0
81445Billy Joe HobertOAK1996-
81538Anthony WrightDAL2001-7.91.5-2.3-2.80.8
81643Kerry Collins2TM1998-
81744Tyler PalkoKC2011-8.2-2.2-0.7-0.50.1
81838Jon KitnaDET2008-8.3-0.8-2.2-1.10.8
81943Josh McCownARI2003-8.5-1.1-0.2-4.52.4
82044Doug PedersonPHI1999-8.7-1.9-1.4-0.40.0
82142Kerry CollinsCAR1997-8.7-0.11.8-6.10.7
82244Kyle BollerBAL2003-8.8-0.5-0.9-2.50.2
82338Matt CasselKC2009-8.8-0.2-
82441Charlie WhitehurstSEA2010-8.9-1.6-
82541J.P. LosmanBUF2005-8.9-2.1-
82642Jon KitnaSEA2000-
82744Elvis GrbacKC1998-9.0-0.3-2.0-1.90.2
82839Ryan LeafDAL2001-9.1-1.6-1.4-1.00.0
82945Danny KanellNYG1998-9.2-1.8-
83040Blake BortlesJAC2014-9.3-6.5-
83143Danny KanellATL2000-9.4-2.7-1.5-0.20.0
83245Akili SmithCIN1999-9.4-2.9-
83344Mike McMahonDET2002-9.40.4-1.2-6.12.5
83440Ty DetmerDET2001-9.5-0.5-0.4-4.20.6
83543Matt CasselDAL2015-9.5-0.7-3.4-1.41.1
83646Jake PlummerARI1999-9.5-1.10.2-5.31.8
83746Sean SalisburySD1996-9.5-0.9-3.6-0.30.3
83835Eli ManningNYG2004-9.6-2.0-
83936Ken DorseySF2004-9.6-2.0-2.4-0.20.0
84046Gus FrerotteSTL2007-
84145Kirk CousinsWAS2013-9.70.2-1.8-3.50.4
84247Donovan McNabbPHI1999-9.7-7.1-
84339Jake DelhommeCAR2009-9.8-0.30.3-5.70.9
84441Michael VickNYJ2014-10.0-4.3-
84544Austin DavisCLE2015-10.1-3.0-2.6-0.40.9
84642Todd BoumanNO2005-10.1-0.70.2-4.80.2
84740Kyle BollerSTL2009-10.2-7.5-
84848Dave BrownARI1999-10.2-3.4-
84943Craig WhelihanSD1997-10.3-1.0-1.9-2.60.2
85047Tony BanksSTL1996-10.42.6-1.2-8.92.1
85145Danny KanellDEN2003-10.6-2.4-1.0-2.20.0
85247David CarrCAR2007-10.7-4.9-
85342Brett FavreMIN2010-
85443Trent Edwards2TM2010-11.1-4.3-
85548Alex SmithSF2007-11.2-5.1-
85637A.J. FeeleyMIA2004-11.2-1.9-1.4-2.90.0
85742Josh McCownTB2014-11.21.5-2.3-8.12.7
85843Kelly HolcombBUF2005-
85945A.J. FeeleySTL2011-11.6-5.6-
86043Robert GriffinWAS2014-11.6-4.5-1.8-2.32.0
86144Jake DelhommeCLE2010-12.2-1.0-3.5-2.70.0
86245Bruce GradkowskiTB2006-12.3-5.0-
86336Mark SanchezNYJ2012-
86446Kurt KittnerATL2003-13.0-5.2-
86547Jesse PalmerNYG2003-13.0-5.5-
86649Steve StenstromSF1999-13.1-4.2-3.5-0.40.1
86745Nick FolesSTL2015-13.1-3.2-
86837Brady QuinnKC2012-13.2-4.0-
86944Kyle OrtonCHI2005-13.4-3.2-4.9-0.30.0
87039Daunte CulpepperDET2008-13.50.8-2.9-6.80.4
87150Erik KramerSD1999-13.6-1.5-0.1-7.00.0
87245Kurt WarnerSTL2002-14.0-0.50.0-8.90.4
87338Chad HutchinsonCHI2004-14.2-5.5-3.5-0.30.1
87440J.T. O'SullivanSF2008-
87546Caleb HanieCHI2011-14.3-2.9-2.7-5.51.8
87645Vinny TestaverdeNYJ2005-
87741Josh FreemanTB2009-
87846Donald HollasOAK1998-
87939Jay FiedlerMIA2004-14.9-2.3-2.3-6.30.9
88044Alex Van PeltBUF1997-15.31.1-1.3-10.50.4
88146Zach MettenbergerTEN2015-15.4-3.0-2.6-4.80.1
88238John SkeltonARI2012-15.7-3.8-2.7-4.20.0
88344Doug PedersonCLE2000-15.8-6.0-
88445Matt MooreCAR2010-16.0-0.60.7-11.50.3
88546Chad HutchinsonDAL2002-16.1-1.9-4.8-4.90.6
88645Danny WuerffelNO1997-16.3-1.2-3.5-6.90.3
88741Spergon WynnMIN2001-16.8-5.2-4.8-3.31.5
88840Luke McCownCLE2004-16.9-2.0-6.4-4.00.5
88946Rick MirerCHI1997-16.9-6.6-2.7-3.81.1
89049John BeckMIA2007-17.4-3.7-4.8-4.60.6
89147Blaine GabbertJAC2011-17.5-7.2-
89241Jonathan QuinnCHI2004-17.6-7.6-
89347David CarrHOU2002-17.9-5.6-5.7-4.63.0
89442Derek AndersonCLE2009-18.3-3.0-4.8-6.51.0
89545Akili SmithCIN2000-18.9-7.7-8.3-1.23.2
89643Josh JohnsonTB2009-19.3-0.9-2.9-12.82.4
89750Josh McCownOAK2007-19.70.2-1.5-15.62.2
89851Trent DilferSF2007-20.0-4.3-2.8-8.00.1
89939Ryan LindleyARI2012-20.2-6.8-6.5-1.90.0
90047Craig WhelihanSD1998-20.2-1.0-2.6-12.20.6
90146Andrew WalterOAK2006-20.2-2.6-2.8-10.00.0
90241J.P. LosmanBUF2008-20.7-4.50.0-13.22.0
90346Jimmy ClausenCAR2010-21.4-9.1-6.9-0.40.0
90446Ryan LeafSD2000-21.70.4-4.1-13.10.1
90547Heath ShulerNO1997-22.0-0.6-1.3-15.20.0
90644Keith NullSTL2009-22.0-4.8-3.4-8.70.0
90748Ryan LeafSD1998-22.9-5.8-2.5-10.10.5
90842Craig KrenzelCHI2004-23.2-6.2-4.6-7.80.4
90949Bobby HoyingPHI1998-23.6-13.3-
91045JaMarcus RussellOAK2009-24.4-7.6-7.7-5.41.3
91146Alex SmithSF2005-35.8-7.7-7.2-16.60.6

If that was too many names for you to swallow, here’s a much shorter table displaying the top five QB’s from each season, as well as the worst:

2015Palmer (36)Newton (29)Wilson (25)Dalton (22)Roethlisberger (20)Mettenberger (-15)
2014Rodgers (33)Romo (27)P Manning (25)Roethlisberger (23)Luck (21)Griffin III (-12)
2013P Manning (43)Rivers (31)Foles (29)Brees (22)J McCown (21)Gabbert (-19)
2012P Manning (31)Brady (28)Brees (26)Ryan (25)Rodgers (22)Lindley (-20)
2011Rodgers (41)Brees (41)Brady (36)Romo (21)E Manning (20)Gabbert (-18)
2010Rivers (30)Rodgers (27)Brady (27)P Manning (22)Roethlisberger (21)Clausen (-21)
2009P Manning (35)Rivers (35)Brees (33)Schaub (29)Favre (28)Russell (-24)
2008Rivers (30)P Manning (26)Brees (25)Cutler (24)Pennington (23)Losman (-21)
2007Brady (46)Romo (31)P Manning (30)Garrard (27)Roethliberger (25)Dilfer (-21)
2006P Manning (48)Romo (28)Brees (25)McNabb (21)Palmer (21)Walter (-20)
2005P Manning (42)Hasselbeck (33)Palmer (32)Roethlisberger (27)Brady (25)A Smith (-36)
2004P Manning (59)Culpepper (33)Roethlisberger (27)McNabb (26)Favre (26)Krenzel (-23)
2003McNair (36)P Manning (29)Green (27)Plummer (23)Culpepper (22)Je Palmer (-13)
2002Pennington (30)Green (26)McNair (23)P Manning (22)Gannon (21)Dv Carr (-18)
2001Warner (35)McNair (29)Garcia (23)Gannon (21)P Manning (21)Wynn (-17)
2000Culpepper (38)P Manning (33)Warner (33)Garcia (31)Griese (29)Leaf (-22)
1999Warner (35)P Manning (29)Beuerlein (22)Gannon (21)Brunell (20)E Kramer (-14)
1998Cunningham (40)Young (34)Testaverde (30)Chandler (30)Flutie (28)Hoying (-24)
1997Favre (28)Young (26)Brunell (25)Esiason (23)K Stewart (19)Shuler (-22)
1996Brunell (23)Testaverde (23)Young (22)Favre (21)Elway (21)Banks (-10)

I’m going to refrain to sharing my opinions on what you see above, because I want your unbiased feedback. What do you like about True Talent? What needs tweaking or a complete overhaul? Does the scale work for you? What type of QB does it systematically underrate or overrate? Thanks in advance for your input! And a special thanks to Bryan Frye for providing me with the data I’ve long been searching for.

  • Josh Sanford

    I’d like to give feedback but I don’t think I understand the constituent parts of the rating. I will have to hold my thoughts until the next post.

    • Adam

      The consistency score is based on 1D%, a proxy for how often the QB makes successful plays.

  • I’ll have to look more deeply before giving a real comment, and I’m still hesitant till I know how the sausage is made. For now, how do you plan on extending this metric beyond 1992 (assuming air yards/YAC splits are accounted for) or 1991 (first downs)? I assume some sort of regression model to estimate first downs.

    • Adam

      I was trying to keep the post short, but it’s kind of silly to leave out important details.

      Dominance uses a modified version of Y/A, accounting for air yards, sacks, and TD’s. Ball protection is simply the % of plays resulting in a fumble or interception. Rushing is Y/A with a bonus for 1D and TD. All categories are “smoothed” by adding x number of replacement level plays – 200 for Dominance and Consistency, 400 for Protection, 100 for Rushing.

      For air yards prior to 1992, I’m currently using a 60/40 split for all QBs until I find a better way to estimate it. I’m using a regression for 1D before 1991, which I normally wouldn’t be comfortable with, but since this is supposed to be approximate I think it works alright (certainly open to change, though!)

      • WR

        You’ve been saying for a while that you had a new metric in the works, and it’s clear that you’ve put a lot of thought into this. Are you sure it’s a good idea to rate replacement-level performance as a zero? Surely the average replacement-level season is worth more than that. And is the consistency score based on first downs earned divided by downs faced? I know Manning does very well by that metric but for example, Brady has better career numbers on third down. So which is the guy you want if you’re facing a crucial 3rd and 6? I’m not sure.

        And are the ratings adjusted for defense? Some of the best single-season passing numbers in recent years have come from QBs who faced a weak slate of pass defenses.

        • Adam

          I’ve been playing around with various ideas for years, but really started working on this particular metric about six months ago, so it’s a work in progress. Why do you think replacement level should be worth more than zero? To me, being easily replaced is the very definition of not providing any value. I definitely believe “average” QBs have positive value, because if you look at the entire pool of QBs in the league, the 16th best is actually far above average.

          The 1D% is derived from total dropbacks, not third down attempts. There is a real skill for picking up first downs on first or second down, and IMO avoiding third down entirely is better than being good at converting it. Also, third down conversion rates are pretty random from year to year, so I don’t think that would be an accurate estimate for talent.

          I don’t use an opponent adjustment, but I do agree the metric would be better with one. Frankly it’s such a logistical ordeal for a tech cave man like me that Id rather just concede a bit of precision. However I do intend to add an approximate weather adjustment based on the QB’s home stadium. As I’m sure you’ve noticed, dome QB’s are overrepresented in the top 50 or so of this list.

          • WR

            OK, now that I’ve thought it through, replacement level may not be higher than zero. If you give Orlovsky or Weeden 300 pass attempts, the overall value will probably be higher than zero, but perhaps not by much. So I agree with you. I also agree that getting a first down before facing third down has real value. What I was referring to specifically is that on third down pass attempts, Brady has an adjusted yards per attempt figure of 7.53, Manning 6.61. That’s a big difference, though how useful it really is, I’m not sure.

            I think when we talk about the value of first down percentage, we have to consider context. Is it better for a team to produce a scoring drive that takes 15 plays, kills the clock, and picks up a bunch of first downs? Or is it better to go 80 yards in two plays with a quick strike offense? If we’re putting a lot of weight on 1D percentage, that becomes an important question.

            I like defense adjustments, which is one of the reasons I like DVOA and DYAR. I’ve worked on the dome/cold weather thing myself. I do think it’s a meaningful factor, but it’s very difficult to tell by how much. Cold weather QBs see their stats go down in December and January, but then again, a couple of Brady’s best-ever games were played in the snow, 2009 v. Tennessee and 2010 v. Chicago. If snow and cold made it harder to throw, that shouldn’t be possible.

            • sacramento gold miners

              There will always be sub-freezing examples of a great QB having an outstanding game in terrible weather, but generally speaking, it will more difficult when conditions are less favorable to the passing game. It’s also tougher for wide receivers to cut, and hold onto the ball in bad weather.

              • WR

                Oh, I agree that cold weather hurts passing numbers. I was just pointing out that it’s not safe to assume that just because a game is played in the snow, that means that passing will be difficult. One thing a lot of people don’t realize is that pass rushing is hard in the snow, because the defensive players can’t get traction. I haven’t done an exhaustive study, but there’s a very clear pattern that cold weather QBs see their rate stats go down in December and January, and dome and warm weather QBs do not.

                What’s interesting about this is that in the case of Peyton and Tom Brady, this factor explains all of the statistical difference between them. Manning’s career anypa figure is 7.17. Brady is at 7.00, but if you filter out his much weaker numbers in Dec-Jan, Brady goes up to 7.16. And those figures include Manning’s historically good sack totals. I haven’t had time to do the research, but it would be interesting to go through careers game by game, and remove all of the cold weather games played late in the season, to see how big the effect would be on guys like Brady, Rodgers, Manning, and Brees.

            • Adam

              I honestly don’t know if AY/A on third down attempts is a meaningful indicator of anything. My default position is to assume splits happen until proven otherwise. In this case, it could be that Brady has faced longer third downs than Manning, but other than that I can’t think of a good explanation.

              “If we’re putting a lot of weight on 1D percentage, that becomes an important question.”

              I don’t think running a quick strike offense has much impact on 1D%. If a QB goes 2/4 for 80 yards on a given drive, his 1D% would be 50%. Meanwhile, a 15 play drive will probably pick up six or seven first downs, which is less than 50% even if every pass is completed. This isn’t like DVOA where plays are counted as successful for gaining six yards on 1st and 10.

              As far as weather goes, I don’t think cold by itself has much effect on passing numbers. It’s the wind and rain that really make it hard to throw, because those conditions affect the grip and trajectory of the ball. As counterintuitive as it may sound, I don’t think snow has much effect, because the slippery field makes it much harder for defensive players to maintain their positions or react quickly.

    • Adam

      Regressions tend to break down at the extremes, which is where I have reservations. For example, I estimate that Dan Marino in 1984 made first downs on 41.7% of his dropbacks. Does that seem reasonable to you?

      • I was about to type something similar. Regressions are neat when we are trying to do things like minimize deviation from the mean, but they really suck when it comes to the things we really want to use them for. By that, I mean they give too much credit to the worst players and too little credit to the best players (these are the players we’re interested in to begin with; we don’t create these metrics to see who rates as the most average). The simplest way to provide evidence for this claim is to use your regression equation on Peyton Manning and see what it spits out. Then do the same thing to Alex Smith. You already know the real values, and you’ll see pretty quickly how much it diminishes the production of Manning and inflates that of Smith. Now imagine doing that for every single dude ever to play before 1991.

        • Adam

          Off the top of my head, I did run the regression for Manning’s 2013, and his first downs dropped from 289 to 257. That’s…way too much. Maybe it would be better to drop the estimated numbers and simply put more weight on NY/A for those years.

          • Stat collection is always an issue with football. Baseball guys have a century of good stuff to work with, while we are basically stuck at about a quarter of that. That’s one reason I did separate posts on TAY/P and New TAY/P. We want to use and present the data where we have it, but we have to figure out how to make something worthwhile for the eras prior to that.

            • Adam

              The question then becomes: Do we use less data and rely on fewer assumptions, or more data with more assumptions?

              In this case, maybe there is a desirable middle ground. For example, I could combine a regression with team first downs, in hopes that one adds signal while the other reduces noise. If a QB took 90% of his team’s dropbacks, we can estimate his 1D count with very high accuracy. Conversely, with a QB who only took 25% of the dropbacks, the regression would have to carry more weight.

              • Maybe for teams with weird splits (e.g., three or four guys with 10-30% of the passes apiece), you can use the regression based on their available stats to determine an estimated first down rate, then use those estimates in concert with a cap set at the known value of team passing first downs. I think that’s sort of what you mean, but I’m basically just thinking out loud here.

                • Adam

                  That’s probably a better idea than what I was considering. My idea is to blend the regression and the team 1D% with a ratio based on the % of team dropbacks used by each QB. If a QB took 75% of his team’s dropbacks, I’d weight the regession at 25% and the team 1D% at 75%.

                  You’re right, those baseball guys have it so easy!

  • Bruddog Tecmo

    Consistency feels a a bit like double counting as its going to be hard to dominate without being consistently good in a season. And without that part it seems like an adjusted yards per dropback that factored in fumbles and rushing yards would be just as good as this. I don’t like when the final units are sort of nonsensical. I mean how much better is a 58.6 then a 38.7.

    • Adam

      As I’ve given this more thought, I agree that dominance and consistency are redundant in most cases. It would probably make more to sense to simply add a 1D bonus into the Y/A component.

      Originally I had this set on a 0-100 scale, but ended up changing it mostly because I didn’t want to copy QBR. The reason I didn’t use a yardage based scale is because I’m not trying to measure value…although I suppose you could still express a talent estimation in terms of yards per play above replacement level.

      I chose not to combine rushing, fumbles, etc into one yardage number because some aspects of QB are more signal while others are more noise. Even though an INT might be worth -45 yards from an explanatory standpoint, the share of the INT penalty that can be attributed to the QB’s ability level is much smaller.

      Thanks for reading and offering feedback!