We all remember the furor that erupted a couple of months ago regarding the Price and Wolfers paper that found a subtle racial bias in referee decisions in the NBA. Wolfers and Price were absolutely pilloried by the NBA and basketball insiders, all of whom claimed that NBA officials are consistently watched by trained assessors who have never observed any racial discrimination.
Astute economists here and elsewhere noted that the level of bias observed in the paper amounted to a maximum of .1 or .2 fouls per game per player, a number unlikely to be uncovered by even careful watching of games and video tapes. (Similarly, as noted in Moneyball, the difference between a .300 hitter and a .275 hitter is 1 base hit every two weeks. Hence, Billy Beane relies on his statisticians rather than his scouts.) Of course, the whole reason the field of inferential statistics has been developed is to help discover relationships that cannot be identified through casual (or even intense) direct observation. While there may be valid arguments why Price and Wolfers are wrong, the NBA's claim that "We can't see it with our naked eyes" was always ignorant in the extreme.
Now it comes out that one of the NBA's men in black was apparently on the take, a fact that went unnoticed by the NBA's trained observers until they got a call from the FBI. If the naked eye can't even pick up a guy trying to throw games for the mob, how can there be any hope, aside from well-designed statistical experiements like that of Price and Wolfers, of discovering subtle (and almost certainly, subconcious) racial bias in the NBA?