I'm trying to come up with a system for Starfic whereby users vote on a 1-to-5 scale for how good a story is; those votes are then combined into a score, which the stories are sorted on. The problem I'm having is in figuring out how to convert votes to scores. Here are some of the issues I'm thinking about:
Is two 5s better than one 5?
Is two 4s better than one 5?
How about ten 4s vs. one 5?
How about ten 4s on a ten-day-old story vs. one 5 on a brand-new story?
Is a story with a 3 and a 5 different from a story with two 4s?
Is a story with twenty 5s and a 4 worse than a story with three 5s?
Are votes from registered users more reliable than anonymous users? Are votes from authors more reliable than readers?
One system I've rather quickly rejected is to simply add up all the star ratings--that would result in stories that got to the top staying there forever (as they'd be the most read and the most up-voted). A pure average-vote based system suffers from problems with sample size (one 5 is just as good as fifty) and spoiler votes (three 5s is better than six 5s and one 4, or for that matter sixty 5s and a troll's 1).