≡ Menu

Drafting Diamonds in the Rough

Guest blogger Andrew Healy, an economics professor at Loyola Marymount University, is back and the author of today’s post. As a reminder, there a tag at the site where you can find all of his great work.


Small school defender takes down big school quarterback

Small school defender takes down big school quarterback.

Asante Samuel. Jahri Evans. Robert Mathis. These three players share something in common that offers a hint to finding steals in the middle rounds of the draft. All three eventually made Pro Bowls. Each was drafted in Round 4 or later. And each played for a notable football powerhouse in college: Central Florida [1]In 2003, Central Florida went 3-9 in the MAC. While the Blake Bortles Knights may not be a football powerhouse, either, the 2013 UCF team that went 12-1 in the American Conference bears little … Continue reading, Bloomsburg, Alabama A&M.

The success of these smaller college players relative to their marquee school competitors turns out to be a much more general phenomenon. In the middle rounds of the NFL draft, players from outside the traditional power conferences have been more than twice as likely to eventually make the Pro Bowl as players from the most famous programs. On defense, small school players have been even more likely to make the Pro Bowl than their major school counterparts.

Let’s use the 2003 draft as an example. Only 5% (6 out of 116) of the major college players selected in round 4 or later eventually made it to a Pro Bowl. At the same time, 12% (6 out of 50) of small college players would eventually be selected for Hawaii. At the very least, if you were watching the draft and wanting to know what the chances are that your team drafted a future star, those chances increased in the middle of the draft when your team picked a player from a school like Bloomsburg than when it picked another player from the SEC.

In fact, it’s hard to think of anything else that can match the impact of simply picking small-school players as a way to find stars in the middle rounds. The data suggest that this logic has even applied at the top of the draft for comparisons such as those between Buffalo’s Khalil Mack and South Carolina’s Jadeveon Clowney. But the big gains from focusing on smaller football schools have come from finding the gems that the draft buzz mostly bypasses. Consistently, general managers have wasted picks on players from major conferences, missing chances to find difference-makers―particularly on defense―from schools such as Northern Colorado and Idaho State. [2]Bonus points for getting those players. Aaron Smith was a 4th round pick out of Northern Colorado in 1999 and Jared Allen was a 4th round pick out of Idaho State in 2004.

The Data

I look at all players drafted from 1998-2007, stopping at the later year to give players time to make a Pro Bowl. The measure of excellence is making a Pro Bowl, but I’ll also look at All-Pro selections. I ignore players listed at special teams positions (P, K, and KR), although it’s possible you could make a Pro Bowl as a special teamer after being drafted at an offensive or defensive position. I also did not include fullbacks because it became so easy to make the Pro Bowl at that spot.

Major conferences are defined according to the traditional BCS definitions: Big East/American, Big 10, Big 12, Pac 10, SEC, and ACC. Notre Dame is also included with these bigger (in terms of football) schools. A school such as Wake Forest gets defined as a big football school by this measure and it probably shouldn’t be, but adjustments from this definition would be judgment calls and so this simple rule seems best.

Note that almost none of the middle-round small-school Pro Bowlers during this time period come from schools such as Boise State that were big football schools at the time. The two possible exceptions are Brett Keisel of BYU (drafted in 2002) and Paul Soliai of Utah, who was drafted in 2007 before Utah joined the Pac-12.

Comparing Average Success Across Schools

Small school players get drafted later than big school players, so we need to control for draft position to get a fair comparison between them. Later, I’ll use regression to do that. Here, I’m just going to break down results according to ranges of draft position. The chance of making the Pro Bowl is much higher in the early parts of the draft, so I’ll break things down there according to selection number rather than just the round.

The table below looks at the first three rounds of the draft. Overall, the chances of drafting a Pro Bowler tend to be higher for small school players in the first three rounds. The small school samples are limited in the first round, but the share of small school players who make a Pro Bowl is higher throughout than for big schools. Out of all the rounds, the 2nd round is the only one where we see a small trend the other way.

 Small schoolsBig schools
Round# of selections% Pro Bowlers# of selections% Pro Bowlers
1 (Pick 1-10)771.4%9355.9%
1 (Pick 11-20)850.0%9141.8%
1 (Pick 21-32)1145.5%9929.3%
2 (Pick 33-48)2623.1%13124.4%
2 (Pick 49-64)3511.4%11215.2%
38310.8%2449.0%

The largest differences, and the clearest benefit from drafting players from smaller schools, come in the middle rounds. The table below shows the differences in rounds 4-7. In round 4 over the ten-year period, teams have been about three times more likely to draft a Pro Bowler when picking from a small school rather than a big one. 12.9% of small school draftees in Round 4 have made the Pro Bowl, compared to just 4.1% of big school players.

 Small schoolsBig schools
Round# of selections% Pro Bowlers# of selections% Pro Bowlers
49312.9%2474.1%
51088.3%2283.5%
61353.7%2203.6%
71572.6%2941.7%

In round 5, we see a similarly large difference. Round 5 players from small schools have been more than twice as likely as big school players to make a Pro Bowl. Altogether, across rounds 4 and 5, despite 475 non-special teams players being drafted from big schools, just 18 (3.8%) have made a Pro Bowl. On the other hand, out of just 201 players drafted in those rounds from small schools, 21 (10.5%) made a Pro Bowl. If you wanted to find a future star in rounds 4 or 5, you would have increased your chances by more than double by looking at the Northern Colorados and Alabama A&Ms of college football rather than the USCs and Alabamas.

[Chase note: It is at this point that I decided I needed to stop reading the article.  I trust Andrew, but found his claims too remarkable to just blindly accept. So I decided to open up my database to confirm. I removed punters and kickers but kept everyone else in the database.  To my amazement, the numbers not only seem legit, but perhaps even under-reported.  The average player selected from the 4th or 5th round from a Big School made 0.06 Pro Bowls, compared to 0.22 Pro Bowls for players from non-major schools!]

Regression Results: Controlling for Draft Position in a Flexible Way

To figure out the average bonus small school players offer compared to large school players, we can use linear regression to control for draft position. In the regressions, I predict whether a player became a Pro Bowler with a cubic polynomial in draft position and whether the player went to a major school. The regression results indicate that, looking across rounds and controlling for draft position, players from small schools are about 3 percentage points more likely to become Pro Bowlers. [3]We get almost the same result if we include higher powers of the pick number. We also get similar results if we use a logit instead of a linear regression. The standard error for the estimate is in … Continue reading

All rounds ( N = 2427 (0.014)):

[math]Pro Bowl = f(Pick, Pick^2, Pick^3) + 0.030 *Small School [/math]

The three percentage point bump for small school players is a substantial boost. Across all rounds of the draft, about 11.8% of the main position players made a Pro Bowl. Compared to this baseline, teams increase their chances of drafting a Pro Bowler by about 20% by drafting a small school player.

We can see more of this pattern by breaking things down according to the early and later rounds. If we look at rounds 1-3, nothing statistically significant emerges. The point estimate follows the overall pattern, but the result is not clear, in part due to the relatively small number of small school players drafted in the first three rounds.

Rounds 1-3 (N = 947 (0.034)):

[math]Pro Bowl = f(Pick, Pick^2, Pick^3) + 0.021 *Small School [/math]

On the other hand, in rounds 4-7, we get a very clear impact of picking small school players, an effect that is even more striking given the much smaller share of players who make the Pro Bowl in those rounds compared to earlier ones.

Round 4-7 (N = 1480 (0.011)):

[math]Pro Bowl = f(Pick, Pick^2, Pick^3) + 0.033 *Small School [/math]

We see that, controlling for the selection, small school players are 3.3 percentage points more likely to make the Pro Bowl. [4]The t-statistic is 3.04 and the p-value is .002. This represents about a doubling of the chance that a major school player makes the Pro Bowl. Just 3.1% of major school players drafted in Rounds 4-7 at the main positions made the Pro Bowl. The model predicts that around 6.4% of small school players drafted in those same positions would have made the Pro Bowl.

All-Pro Appearances

So focusing on small school players offers a much better way to draft a future star according to Pro Bowl appearances. And it doesn’t look like this is just about Pro Bowls. Instead, it’s pretty clear that small school players perform better more generally than major school players, once we control for draft position, with these differences primarily driven by the middle rounds, particularly 4 and 5.

Small school players drafted in rounds 4-7 are also about twice as likely to appear on an All-Pro team as their major school counterparts. Controlling for draft position, small school players are about 1.3 percentage points more likely to make an All-Pro team, relative to a baseline where 1.5% of major school players made an All-Pro team.

All-Pro (N = 1480 (0.008)):

[math]All-Pro= f(Pick, Pick^2, Pick^3) + 0.013 *Small School [/math]

Particularly given the relatively small number of players who made an All-Pro team, we can look at this another way by considering the number of appearances a player made on an All-Pro team. Controlling for draft position, players drafted in the middle rounds from small schools have an average of .036 more All-Pro selections than major school players. The mean number of All-Pro selections for major school players is .022, so small school players are predicted to have more than twice the number of All-Pro selections as their major school counterparts. [5]The small school players drafted in rounds 4-7 who made an All-Pro team are (with the number of appearances in parentheses): Adalius Thomas (2), Asante Samuel (3), Brandon Marshall (1), Cortland … Continue reading

Number of All-Pro Appearances (N = 1480 (0.015)):

[math]All-Pro Appearances = f(Pick, Pick^2, Pick^3) + 0.036 *Small School [/math]


The Best Defense Comes from Small Schools

One other interesting pattern in the data is the offense/defense breakdown. All of the above effects are driven by the defense. If we look just at offense, there’s basically no difference between big and small schools, which mimics what Chase found using a different methodology last year.  However, there are large gaps for defensive players.

Take the regression from before for rounds 4-7. Now let’s break it down separately for offense and defense:

Round 4-7, Offense only (N = 749 (0.016)):

[math]Pro Bowl = f(Pick, Pick^2, Pick^3) + 0.003 *Small School [/math]

Round 4-7, Defense only (N = 731 (0.015))::

[math]Pro Bowl = f(Pick, Pick^2, Pick^3) + 0.060 *Small School [/math]

The last gap is pretty enormous. Even if we don’t control for the spot the player is selected―which works against small school players since they get drafted later―we see the huge differences between small and large school defensive players. Out of 499 defensive players drafted in rounds 4-7 from major conference schools between 1998 and 2007, 10 (2.0%) made the Pro Bowl. On the other hand, out of 231 small school players drafted in those same rounds, 18 (7.8%) made the Pro Bowl. The gap for all-pro appearances is similarly large. There were a total of 10 all-pro appearances for the 499 large-school defensive players (.020 per player) and 17 all-pro appearances for the 232 small-school players (.073 per player) drafted in rounds 4-7 during this period.

Even though we have fewer than half as many draftees to pick from compared to major school players, look at the starting 11 we can field from small school players mostly picked in round 4 or later, with two round 3 draftees to fill in a couple of holes:

DE      Robert Mathis
DT      Paul Soliai
DT      Aaron Smith
DE       Jared Allen
OLB    Joey Porter (3)
MLB   Jeremiah Trotter (3)
OLB   Adalius Thomas
CB      Asante Samuel
CB      Cortland Finnegan
FS       Kerry Rhodes [6] Rhodes has actually never made a Pro Bowl, but he was second-team All-Pro in 2006. He did not count in the players from small schools who have made a Pro Bowl.
SS       Antoine Bethea

Note that if you go back a few more years, you can substitute La’Roi Glover (5th round, 1996, San Diego St.) for Soliai and Rodney Harrison (5th round, 1994, Western Illinois) in at SS, sliding Bethea in for Rhodes at FS. That is a pretty sweet defense, all built on middle-to-late round picks from small schools.

Conclusion

The data show that picks in the middle rounds of the draft have been substantially more productive when spent on players from smaller schools. Despite picking major-school players more than twice as frequently, teams have found as many stars from the smaller schools. On defense, they have actually found substantially more stars from schools such as New Hampshire than ones such as LSU. A defensive player taken in round 4 or later has been almost four times more likely to eventually make a Pro Bowl when that player comes from a school outside the traditional power conferences. Stars such as Jared Allen, Asante Samuel, and Robert Mathis are part of a larger pattern. Teams have found those essential mid-round steals by drafting players from smaller schools.

Why has there been this opportunity to do better by picking small school players? One possibility is that there was less information out there about those players, a gap that would have been decreasing as film and televised college games have become ubiquitous. That explanation makes some sense since the benefit to smaller school players emerges in the middle rounds, long after the Brian Urlachers (New Mexico) and Joe Greenes (North Texas) who were impossible to miss had been selected. However, with the sample going from 1998-2007, this explanation seems unsatisfying since teams have had relatively easy access to information about any college player.

The explanation that I think could make more sense is some kind of risk aversion, kind of like the bias that leads to punts on fourth down. Maybe teams in the middle rounds, not seeing clear standouts, felt that it’s safer to pick the player from Alabama instead of the one from Idaho State. Even though it’s anything but safer, general managers can say to themselves that they’re getting a player who’s a known quantity due to the college program he comes from. Picking the major school player might even be the kind of move that’s harder to criticize, putting the general manager in a similar position to the coach facing 4th and 3 at midfield, where the best choice for the team may not be optimal for the decision maker. Whatever the reason, the bias towards major school players in the middle rounds has left available potential stars to the teams that have chosen players from overlooked schools.

However, this potential opportunity may already be gone. Since 2008, six defensive players have made Pro Bowls and were drafted after round three. All six were actually from major schools: Kam Chancellor (Virginia Tech) and Richard Sherman (Stanford) in Seattle, Geno Atkins (Georgia), Henry Melton (Texas), Alterraun Verner (UCLA), and Greg Hardy (Mississippi). Across offense and defense, it’s eight Pro Bowlers for large schools (adding Carl Nicks and Jordan Cameron) versus four for small schools (Alfred Morris, True Receiving Yards champ Antonio Brown, Josh Sitton, and Julius Thomas, and not counting Jerome Felton, who plays FB), about the same ratio as players drafted altogether. Still, the biggest stars here are clearly the big school players.

Even though we need more years of data on all the players in these drafts, it is possible that the previous trend has shifted. Assuming that’s right, why might that have happened? One possibility is that ever more schools are getting national media attention, meaning that small schools aren’t so small anymore. [7]Another possibility is that NFL teams have changed their behavior. There has been almost no change over time, though, in the share of small school defensive players selected at certain points in the … Continue reading    Another possibility that seems even more plausible to me is that the increasing information on high school players means that great players are now less likely to be at small schools in the first place. Even though there will always be some great players who end up at small schools (see Watt, J.J.), maybe Jared Allen would have been recruited more heavily if he played now. There may now be fewer diamonds in the rough than there used to be. That idea suggests there might have been even more diamonds in the rough if we look at earlier years. And that looks like it might be exactly the case. Just looking at rounds 4 and later in some of these earlier drafts is kind of incredible. In 1989, there were five (non-kicker) Pro Bowlers from small schools and only one from a large school. In 1990, there were nine small school Pro Bowlers (including HOFer Shannon Sharpe) compared to just four from major schools. In 1991, it was eight small school Pro Bowlers compared to just two major school players. [8]Some of the late round diamonds in the rough may have become undrafted free agents in later years. For example, James Harrison (Kent State) and London Fletcher (John Carroll) are two small school … Continue reading All of this appears even though substantially more large school players are drafted in rounds 4-8. While the chance to find a small school steal was just on defense from 1998-2007, it seems like the opportunities may have been all over the field in earlier years.

References

References
1 In 2003, Central Florida went 3-9 in the MAC. While the Blake Bortles Knights may not be a football powerhouse, either, the 2013 UCF team that went 12-1 in the American Conference bears little resemblance to where the program was a decade ago.
2 Bonus points for getting those players. Aaron Smith was a 4th round pick out of Northern Colorado in 1999 and Jared Allen was a 4th round pick out of Idaho State in 2004.
3 We get almost the same result if we include higher powers of the pick number. We also get similar results if we use a logit instead of a linear regression. The standard error for the estimate is in parentheses.
4 The t-statistic is 3.04 and the p-value is .002.
5 The small school players drafted in rounds 4-7 who made an All-Pro team are (with the number of appearances in parentheses): Adalius Thomas (2), Asante Samuel (3), Brandon Marshall (1), Cortland Finnegan (1), Jahri Evans (5), Jared Allen (4), Jerry Azumah (1), Lance Schulters (1), Matt Birk (2), Michael Turner (2), Robert Mathis (1), Terrence McGee (2), and Trent Cole (1). Of these, McGee made it as a special teams player. Amongst major college players drafted in rounds 4-7, Dante Hall and Leon Washington made All-Pro teams as special teamers during this time.
6 Rhodes has actually never made a Pro Bowl, but he was second-team All-Pro in 2006. He did not count in the players from small schools who have made a Pro Bowl.
7 Another possibility is that NFL teams have changed their behavior. There has been almost no change over time, though, in the share of small school defensive players selected at certain points in the draft.
8 Some of the late round diamonds in the rough may have become undrafted free agents in later years. For example, James Harrison (Kent State) and London Fletcher (John Carroll) are two small school UDFAs who made Pro Bowls.
{ 13 comments }