Christopher Allen shot me an email the other day pointing to his work on practical applications of 5-star rating systems. I found it to be an interesting read, which you might expect given my previous commentary on the subject. Highlights:

Thus even when a bimodal distribution is not a problem, on a 5-point scale the upward bias often results in only 2 or 3 meaningful data points. This is problematic because it minimizes differentiation. In many cases, a 5-star rating system where most of the ratings are either 3 or 4 is actually no better then just a thumbs-up/thumbs-down rating system.

However, given that 5-point scales are probably here to stay, we are forced to make the best use of them we can.

The thing that I really enjoy is that Christopher then makes the jump to what really matters: providing incentives for the user to rate and to do so well:

I only have 11% of my collection rated so far, but using this system I’m finding it a lot easier to manage my ratings. I’m already getting many benefits from it — I’m playing my music more often, my iPods typically have the music I want on them, and various music discovery services can use my ratings to help me identify new music I might enjoy. This provides the incentive to keep me entering meaningful ratings.

I have 100% of my collection rated, and I make sure to do so each and every week when I make new additions to my collection. Now, it’s easy to classify my doing so as simply another manifestation of my obsessive-compulsive nature, and I’ll allow the point … but still, taking time out for music, which is clearly something that I enjoy, is something that is a reasonable use of my time. What benefit do I get out of it? When I dump stuff off to my iPod, I’ve got a good subset of my music collection to listen to at the time.

Allen’s stuff is worth a read if this sort of thing interests you.