You are offered the opportunity to win some money by picking two marbles, one each out of two different bags. First, you get to dip your hand into a bag of six red marbles, with each marble worth a different dollar amount. The dollar amounts are $10, $20, $30, $40, $50, and $60. You will next get to dip your hand into a bag of six blue marbles, with the marbles worth $60, $64, $68, $72, $76, and $80.
Simple math will tell you that the average red marble is worth $35 and the average blue marble is worth $70; it’s clear which bag has the better marbles, and you can also expect to walk away with about $105 once this game is completed. Now, our benevolent contest operator says he will make your life even better. In one of the bags, he will discard the bottom three marbles: he will remove either the $10, $20, or $30 marbles from the red bag, or the $60, $64, or $68 marbles from the blue bag. This, of course, is great news.
He then asks you a question: from which bag should he remove the bottom 3 marbles? To make your life a little easier, he has reproduced the values for the 6 red marbles for you in the graph below:
And here are the values for the 6 blue marbles.
Now that you have seen the values, from which bag do you want him to empty the worst three marbles? This is not a trick question, although it’s one that’s not necessarily easy or immediately intuitive to everyone. The blue marbles are obviously superior, and if you remove the worst marbles from that bag, you are guaranteed at least $72 from that bag alone. The average of the top 3 blue marbles is $76: that sounds pretty good. For the red bag, the average of the top 3 most valuable marbles is only $50. Clearly, you’d rather have a top-3 blue marble than a top-3 red marble.
On the other hand, you know you have to take two marbles: one from each bag. The average red marble is worth only $35, while the average blue marble is worth $70.
So what do you do?
Take a minute to think about your decision.
*****************************************
Keep thinking, because this is an important question!
*****************************************
Also, this is a question that does have a correct answer. You have enough information to solve it.
*****************************************
Hopefully you figured out that you want to discard the 3 worst marbles from the red bag. Let’s see why.
Option 1: discard the 3 worst marbles from the red bag.
If you do this, you can expect to make an average of $50 from the marble you pull from the red bag: that is because the $40, $50, and $60 marbles are left. You can also expect to make about $70 from the blue bag, which is the average of all 6 marbles in that bag. So, on average, you will walk away from this contest with about $120.
Option 2: discard the worst 3 marbles from the blue bag.
If you do this, you can expect to make about $35 when you put your hand into the red bag. Then, when you try your hand with the blue bag, you can expect to make an impressive $76. Unfortunately, this leaves you walking away with only $111.
Again, this might not necessarily be intuitive. You might think, hey, getting a top-3 blue marble (worth an average of $76) is more valuable than getting a top-3 red marble (ranging from $40 to $60). And that of course is true. But given the parameters of this experiment, you can see that is not how to analyze the problem, because you have to take one marble from each bag.
Conclusion: The value of a top-3 marble is determined not by how much it is worth, but by how much it is worth relative to all of the other marbles in the bag.
That’s a really powerful conclusion. Value is not determined in the abstract, and you can’t compare values from two different sets of bags. Value is determined by the variation of values inside the bag, not from the absolute values of the marbles in the two bags.
What this means is that if you have the choice, getting to secure a top-3 marble in the blue bag — worth a lot of money! — still isn’t as valuable as getting to secure a top-3 marble in the red bag, because the values in the blue bag are more compressed. For the mathematically inclined among us, the standard deviation of marbles in the blue bag is just under $7, while the standard deviation of marbles in the red bag is over $17.
Are you convinced now that if given the choice, the value of securing a top-3 red marble is superior even though a top-3 blue marble is better than a top-3 red marble? I hope so; if not, let me know in the comments.
For those still with me, allow me to transition things to football. In 1967, there were 16 teams in the NFL. In 1968, there were 16 teams in the NFL, which means there were 32 team seasons in those two years (ignoring the AFL). The graph below shows the passer ratings of those 32 team seasons in red. In addition, I have put the 10th-best team in a white dot with a red circle so you can where a top-10 passing offense falls on the chart.
Now, what about the 32 teams from the 2018 NFL season? That’s below in blue, ranging from 62.6 for Buffalo to 113.8 for Kansas City. Once again, I put the 10th-best passing team in a white dot with a blue circle. As you can see, and are no doubt surprised, the ratings are a lot higher in this sample.
Now, let’s say you are the Rooney family and you run an NFL franchise. And, let’s pretend in our hypothetical universe that the teams from 1967 and 1968 seasons are able to play together in one season of 32 teams. The Rooneys get to run an NFL franchise in 2018, and also in this mythical 1967-1968 season. In one of the seasons, you get a completely random team. In the other season, you are given the chance to guarantee yourself a team that finishes in the top 10 in passer rating.
In which season is it more valuable for you to secure a top-10 passing team? On average, the passer rating for the 32 teams in 2018 was 93; meanwhile, the average passer ratings for teams in the ’67-’68 seasons was just 67. So, of course, top-10 passers now are better than top-10 passers from ’67-’68.
But we just learned that value is not determined by how good something is, but by how good it is relative to all of the other marbles in the bag… err, all the other passing offenses. The standard deviation of passer ratings in 2018 was under 12, while the standard deviation for the teams in the ’67-’68 seasons was 14. And, as we learned, variance is what drives value, not absolute values.
Let’s think about it this way: The average of the top 10 passer ratings in ’67-’68 was 84, an increase over the overall average of about 17 points. But the average of the top 10 passer ratings in ’18 was 105, which is an increase over the overall average of 12 points.
So, assuming passer rating is a good proxy for the value of a passing offense, this means the value of a top-10 passing offense is lower now than it was in ’67-’68. What makes a passing offense great isn’t how effective it is in the abstract: it’s how much more effective it is than all the other passing offenses.