Wednesday, May 2, 2007

NBA Refereeing Bias

An article in the New York Times about a academic paper written by Justin Wolfers (Wharton School of Business, UPenn) claims that there is a statistically significant bias in foul calls based on the race of the referee and the player. The actual paper has yet to be published, so it's impossible to say if they study was done correctly but experts who have read it (see NYT article) claim that it was. We can't really say until we see the actual paper, and nor can anyone else. That's why the response people have had to the article is so amazing. Here are a few:

Charles Barkley, NBA hall-of-fame player: "There are more black players so obviously there will be more fouls on black players" This is a popular sentiment which I've heard three times in the last 2 hrs on ESPN. Do they really think this Penn Professor hasn't though of this, an accounted for it? I guess it's possbile, but according to the three experts the NYT consulted he did, along with a bunch of other effects that these guys haven't thought of.

Kiki Vandeweghe, NBA analyst for ESPN: [Paraphrasing] "The refs get the majority of the calls right. If this were happening, we would have noticed" He also was not the only person to make this argument, and what a horrible argument it is. The entire point of doing statistical analysis is that "noticing" small statistical effects is really hard. Also, a huge fraction of the calls in basketball could go either way so in this case even "getting it right" is subjective.

Mark Cuban, Owner of the NBA's Dallas Mavericks: "We’re all human. We all have our own prejudice. That’s the point of doing statistical analysis. It bears it out in this application, as in a thousand others." Exactly! Thank you Mr. Cuban. The study is either done well, or it isn't. If it's done well, it either shows a statistically significant effect, or it doesn't. That's all there is to it.

The NBA: [paraphrasing] "We have better data that we claim does not corroborate your results, but we won't let you see it so you'll just have to trust us that what you are saying isn't true" I'll take less detailed data and peer-reviewed analysis (which this isn't yet but will be, mind you) over the best data and secret analysis any day.


David J said...

Kiki's comment is either impressively dumb or very cynical. In fairness, though, even peer-reviewed academics aren't immune to painful misuse of statistical data. I just read an article about the obesity epidemic in which the author reports on studies in which anyone who died while obese was counted as dying due to obesity. (the author was arguing that the dangers of obesity are vastly overblown, and fueled by our culture's distaste for fatness more than good data to suggest that it is as dangerous as it is made out to be).

dave hiller said...

I'll second the notion that there are some really horrible statistics in peer-reviewed literature. In general, peer review is not worth nearly as much as people generally think. That being said, once the paper is readily available, then we don't just have to trust "peer review", and instead can judge the paper directly. Even if we lack the expertise for that, we can listen to detailed comments of those who are experts, instead of the all-or-nothing judgement of whether the paper was publishable or not.

Alan Rosenwinkel said...

I totally agree with point about bad statistics in peer-reviewed papers. I read one really egregious one once in a medical journal, but I can't find it now. Great story, huh?

Also, I mixed up the quote attribution initially, which I have since changed, so when David sais "Kiki's comment", It really was Barkley who said it. my bad :-)

Also, David, can you provide a link to the obesity article you were reading? I'd like to read it.

jaideep said...

speaking of sports related statistical analyses from Penn, what ever became of that guy who "found" that Eddie Murray was the most "cluctch" hitter in baseball history?

php hit counter