Adam Steele is back to recap his Wisdom of Crowds work. As always, we thank him for that. Football Perspective wouldn’t be what it is without contributions like this from folks like Adam.

---

I’d like to thank everyone who voted in this year’s Wisdom of the Crowds, and I also appreciate your patience in waiting for the long overdue recap article. I’m not much for small talk, so let’s get right to it.

Originally, my plan was to simply tally the scores and use the totals for the QB ranking. However, it quickly became evident that this wasn’t going to work, as we had very large discrepancies in how voters allocated their points. Some people awarded 25 points to their pick for best ever, while others didn’t give any QB more than six points. It would be just plain wrong for one voter’s GOAT to be weighted four times more than the next voter. My solution (helmet knock to commenter hscer ^[1]I highly encourage you to check out hscer’s collection of Sporcle quizzes., since he came up with it) is to tabulate points in proportion to the highest score on each ballot. Thus, a QB who scores five points on a ballot with a 25 maximum receives 0.2 ranking points, while a five-pointer on a ballot with a maximum of six is awarded 0.83 ranking points. This levels the playing field for all ballots, and in my opinion yields a far more honest result than the simple tally method. Since the abstract concept of ranking points is tough to put in proper context, I’ve translated them into Share %, which is the percentage of possible points earned. We had 51 legal ballots submitted this year, so Share % = ranking points / 51.

Results

In order to qualify for a WOC ranking, a quarterback had to be listed on a minimum of three ballots, leaving us with 36 qualifying QB’s. The table below lists the quarterbacks’ Share %, ballot appearances, “pantheon” appearances (ballots where he received at least 0.5 ranking points), and ballots where he received the highest score (including ties). I also included the ranking each QB earned in the 2015 edition of this exercise, as well as the number of positions gained or lost from 2015 to 2017.

In a result that should surprise absolutely no one, Tom Brady is declared the 2017 Greatest QB of All Time. Brady had plenty of GOAT support even before the 2016 season, and winning his fifth ring in dramatic fashion surely added to his cadre of proponents. Peyton Manning and Joe Montana round out the top three, with a major drop off between them and fourth place Johnny Unitas. Interestingly, Brady, Manning, and Montana all received exactly 47 pantheon votes; Brady distanced himself by getting almost as many GOAT votes as PFM and Joe Cool combined.

Check out the large gap between 14^th place Drew Brees and 15^th place Bart Starr – this makes for an obvious demarcation point between the upper-tier HoF’ers and everybody else. Fascinatingly, these same 14 quarterbacks also made up the top 14 spots two years ago. This result is remarkable considering only 16% of the voters from 2015 also submitted a ballot in 2017, and the scoring method was completely different. I’m quite encouraged by this, as it proves that the rankings are robust regardless of who is voting and how we tally the scores (at least among FP quality readership; the alternative will be discussed later).

The other noteworthy development from 2017 is the systematic rise of pre-merger quarterbacks, which I consider this a heartwarming turn of events. Otto Graham jumps three spots, Luckman and Tittle gain four, while Van Brocklin and Jurgensen make huge leaps of seven and eight positions, respectively. I can’t point to any obvious reason why the old timers have gained esteem in the eyes of FP readers, but I do feel safe in saying that these guys are now closer to where they belong. Of course, these jumps had to come at someone’s expense. For reasons beyond my comprehension, Kurt Warner and Jim Kelly each freefall 11 positions in the rankings. This is especially puzzling  for Warner, since he was just inducted into the Hall of Fame (I happen to agree with the lower rankings for Warner and Kelly, but that’s just one man’s opinion).

Comparing Methodologies

I enjoy running these WOC experiments for many reasons, but one of the principle purposes is to provide a baseline for comparison. In this table, I once again list the 2017 and 2015 WOC rankings. Additionally, you’ll see the rankings from an entirely different WOC format, an ongoing poll hosted at ranker.com. We also need a statistically derived baseline, so I included Bryan Frye’s VALUE metric (based on TAY/P above average), and Brad Oremland’s QB TSP (based on career value above replacement). The designation “-” is shorthand for Missed Cut, which I assigned to quarterbacks who didn’t qualify for ranking in any of the respective methods (in the case of VALUE, I consider any QB with negative career value to have missed the cut). The three QB’s with a dash mark under TSP are too old to be eligible for ranking. The table includes every QB who cracked the top 50 in any of the methods.

As I alluded to earlier, the ranker.com list represents a sharp departure from the rankings generated by FP readers. Their scoring method is much simpler, using a thumbs up / thumbs down system. And more importantly, their voters appear to be far less informed than the football savvy folks we have at FP. I don’t what the voting demographics look like at Ranker, but my guess is a heavy dose of casual fans, homers, and people who are relatively new to football viewing. I’ll give them credit – the top of the Ranker list is quite reasonable. But beyond that, their rankings follow a trail through ring counting territory, then descend into the abyss of recency bias and blatant fanboyism. I mean, Tim Tebow makes the top 100, while Sammy Baugh misses the cut entirely. As boneheaded as that may sound to the FP crowd, I have a feeling the Ranker list is a fairly accurate portrayal of the average football fan – the kind who tend to make arguments based on emotion rather than facts.

Returning the to sanity of VALUE and TSP, we see a number of QB’s whose ranking diverges wildly from their standing in the WOC lists. This is not surprising, since we “know” that some QB’s are misrepresented by their regular season statistics, both for better and for worse. Bart Starr fares much better in WOC than he does in the cold world of stats, as do John Elway, Troy Aikman, and Terry Bradshaw. Conversely, FP readers believe that Drew Brees and Ken Anderson are overrated by their statistics, as are Matt Ryan, Trent Green, and John Hadl (and I think they’re right). The beauty of sports analysis is the fact that there is no right answer, that is takes a multitude of different approaches to inch us closer and closer to the truth.