You run into this problem anytime there is an effective bound on success (as Chase touches on in the comments below). I read a recent FiveThirtyEight article positing that Serena Williams was probably not as good as other female tennis greats, because she has played in a relatively weak era for women’s tennis. Basically, according to their methodology, because of Williams’ weaker competition, for her to be keep pace with the greats of the past, she would have to win essentially every tennis match she plays, which is an almost impossible standard, given everybody has bad days.

I don’t think this is inherently “fair” or “unfair.” It just means there is no foolproof way to compare athletes across different eras, which, to me, only makes discussion like there more interesting.