Democracy is hard

Feb. 27th, 2005 at 9:47 PM

I'm trying to come up with a system for Starfic whereby users vote on a 1-to-5 scale for how good a story is; those votes are then combined into a score, which the stories are sorted on. The problem I'm having is in figuring out how to convert votes to scores. Here are some of the issues I'm thinking about:

Is two 5s better than one 5?

Is two 4s better than one 5?

How about ten 4s vs. one 5?

How about ten 4s on a ten-day-old story vs. one 5 on a brand-new story?

Is a story with a 3 and a 5 different from a story with two 4s?

Is a story with twenty 5s and a 4 worse than a story with three 5s?

Are votes from registered users more reliable than anonymous users? Are votes from authors more reliable than readers?

One system I've rather quickly rejected is to simply add up all the star ratings--that would result in stories that got to the top staying there forever (as they'd be the most read and the most up-voted). A pure average-vote based system suffers from problems with sample size (one 5 is just as good as fifty) and spoiler votes (three 5s is better than six 5s and one 4, or for that matter sixty 5s and a troll's 1).

Thoughts, anybody?

Current Mood: contemplative
Current Music:Interpol - Evil

Comments

( Read 3 comments — Leave a comment )

nathanbp

Feb. 28th, 2005 12:19 pm (UTC)

I've seen f(average score, # of votes) functions in a couple different places, like AnimeNfo's Top 200 list. No clue on the details of how they work, but that sounds like more or less what you're looking for (or at least a starting place). Altho since KGNE ended up #2, their algorithm must be pretty bad >_>;;

Link | Reply | Thread

holylucifer

Feb. 28th, 2005 12:22 pm (UTC)

my $.02

I'd say average them, and let people know how many votes it's gotten. Though personally, I would use a 10 point system, as they're more accurate about judging things, and a 7-9 would be the same as a 4, but more specific. Registered and unregistered shouldn't matter, but separate the authors from the two categories, as they can be skewed, and when they see they are separated, they'll realize their votes are watched, and will be more honest about it.

A 5 means outstanding, a 3 means mediocre. Average them, and you get a good feeling which is a 4, so just do an average. Also, if it plagues you that much, put the score on a link, so that interested parties can look at all the votes individually, and see what individual people thought. People familiarize style, and when the same name pops up on a review, they'll look for that reviewer, and see what they thought.

Link | Reply | Thread