More Advanced Averaging/Sorting

Building a ranking system...

[Tom_Cash]

Hi everyone,
I have a website that has a ranking system. So when a user reviews something and gives it a score, the mean average is taken.

So lets say I have widget 1 and widget 2.

Widget 1 has 6 reviews and a mean average of 7.8

Widget 2 has 1 review and an average of 9.

Therefore, widget 2 becomes higher in my sites ranking system.

I'm not too fond of this and would ideally like to iron it out.

Could someone please help me with ideas for balancing the system and/or with some more advanced math logic than mean averaging.

Any help would be great,
cheers!

[edited by: Tom_Cash at 9:07 am (utc) on Feb. 2, 2009]

[caribguy]

What about this:

Ranking from 1-10, average score is 5
Your most reviewed widget has 50 reviews, score = 7.8
Your 2nd widget has 10 reviews, score = 9.0

You could add 50-10 = 40 times the average score of 5

And end up with a weighted score of (40 * 5 + 10 * 9) / 50 = 5.8 for the 2nd widget

If that seems unfair, you could also compare the 10 newest reviews only.

[cgrantski]

People who do surveys etc for a living usually ignore a certain number of outliers, like the highest score and the lowest score. And they also ignore (i.e. don't display) average ranks for any items for which there are too few data points. You could display the reviews for those items, just not offer an average.

Also, there are lots of other ways to describe "typical score" in the statistics world. The median, the geometric mean, etc. Try a search on these terms to get to somebody's statistics glossary.

And you can also ignore the idea of a middle point and go with something like "percent of reviews where the rank was higher than 3."

A good service for your readers would be a histogram, i.e. a bar graph, of the ranks. That way, people can decide for themselves based on the pattern, and they can also decide for themselves if there are enough reviews to pay attention to.

[Tom_Cash]

caribguy: I'm not sure I follow / understand your method?

cgrantski: Cheers for the insight, i've got a bunch of stuff to read now after opening seveal new tabs from related articles, etc.

Thanks very much peeps.

[StoutFiles]

I would just display widget2 as a ranking of 5 until it has, say, 5 reviews.

psuedo:

if(NumberOfVotes >= 5)
display CurrentRank
else
display RankOfFive

Or instead of a five rank, you could display a message that it doesn['t have enough reviews yet. There are many ways you could go about this.

[caribguy]

Rather than starting with an empty slate for each new widget, you assign it a set number of default ranking values when the item is created.

item_123_rank = [ 5, 5, 5, 5, 5, ]
item_123_avg = 5

Each time a "real" rating is added by your visitors, you push it to the front of the stack

item_123_rank = [ 8, 5, 5, 5, 5, ]
item_123_rank = [ 7, 8, 5, 5, 5, ]
item_123_avg = 6

[Tom_Cash]

Thanks for all the replies people, much appreciated. :D

I think I like that idea, caribguy! Me and my partner were sat down yesterday for about an hour trying to think of a system that would work only half that well!

Thanks a lot!

Her idea was:

Give the widgets a rank by score and a rank by review count and average those. I agreed it was a good idea, but couldn't see how to implement it without resorting the entire the database with each new review.