Pomeroy came out with a blog post earlier in the week in which he pretty much said that +/- is useless as a statistical tool.
I was initially disappointed that I was using and promoting a "bad" analytical tool on this blog, but upon investigation of his article, it seems that we're using it somewhat correctly -- at least, as correctly as he allows (plus some).
Basically, what Pomeroy shows in his blog post is that there is TONS of randomness that can occur inside of 35 possessions (half of a game). His computer-simulated player had games ranging from -13 to +13, but the catch was that all players (on both teams) were programmed to be equal in talent.
After some discussion, these were his conclusions:
1) In-game plus minus is useless. As you can see from the first trial above, there were games when my player looked very good and games when he looked bad. Sure there are things that a player does to help or hurt his team that are not reflected in the box score, but plus-minus for a single game is often not reflective of this.
Whew. I'm 1-1 so far. Here's what I said in my very first plus/minus post (versus Davenport):
"I probably wouldn't put too much stock into a single game's worth of numbers. There are several factors that would "artificially" push +/- up or down."
2) Multi-game plus-minus isn’t much better. I suppose there are cases where there’s enough of a difference over a few games to draw a meaningful conclusion about a player. But I think that pertains to the kinds of players for whom you already know the answer
OK, so I'm getting dinged here a little bit here. There's still so much noise in the numbers that it really won't settle down quickly enough to be used as a reliable metric.
3) Season-long adjusted plus-minus might have limited use. I haven’t talked about adjusted plus-minus, but in the simplest terms possible, it accounts for who a player is playing with and against. It’s useful in the NBA, but the NBA has many more games, more minutes per game, and its star players see a fair amount of time on the bench during non-garbage time.
Maybe I get another point here? Although he's still hinting that the college season is probably too short to get a really good read on the numbers, he's saying that the +/- stat's usefulness is as a season-long stat.
I think I've been stressing the season total charts every time I post them. Here's what I had to say about plus/minus in the post after the home loss to Aquinas.:
"But we know that one game of +/- doesn't tell us much of anything about how an individual played. It's more of a cumulative season stat."
I've been tossing around the idea of applying plus/minus with WOWY (with or without you) principles to compare each players +/- numbers with who is (or isn't) on the floor with them. But I don't think I'd do this until next season.
4) Respect randomness.
True. Very True.
In summary, plus-minus, while neat to look at, is a poor tool in college basketball analysis.
Aww man.
I think I'll continue using it though, because I think it's the closest thing we'll get to an individual defense "stat". We now know that this type of analysis is far from perfect, but I'm not ready to completely discard it yet. Let's throw a few big grains of salt into the mix and carry on.
The Olivet game play-by-play needs some extra attention to adequately decode, so I'll be a little bit late with that data.
