Search:

SIYE Time:18:05 on 28th March 2024
SIYE Login: no

News
Major Changes in Star Ratings
SIYE had many questions, comments, and complaints about out Star Ratings to the point we did some investigating. We have come to some very disturbing conclusions. Our findings have resulted in some major changes to the Rating system that will affect everybody and every story.

1) The Problems

A. We found that some members engaged in the practice of Trolling. That is leaving multiple entries of the lowest score possible (1 = Troll) on targeted stories, drastically bringing down their Star rating. This was already covered in an earlier NEWS Article.
B. On the flip side, we also found that many authors and their friends were doing just the opposite, practicing Outstandings. That is leaving multiple entries of the highest score possible (10 = Outstanding), over-inflating their real worth, including their own stories.
C. The third issue is our own. All stories start with the default of having a 5-Star rating, or a score of 10, no matter how bad the story really is.

2) The Repairs

SIYE has performed the following to correct these problems:
A. A script was executed to remove those Trolling scores from the targeted stories we found.
B. A script was executed to remove multiple Outstanding scores by the same member from any story, leaving only one entry.
C. Any logged in SIYE member can leave numerous Reviews for any story. But leaving any Rating score is limited to only one entry per chapter per story. So, no more Trolling or Outstandings.
D. The ratings will start with the default score of 6, Acceptable. Any logged in SIYE member can leave one score: higher, lower, or leave it alone.
E. All Reviews and Ratings are now public. The name of the individual will be shown. No more anonymity.

3) The Results

SIYE feels this will put everybody and all stories on a more equal footing and eliminate cheating. Overall, Star ratings for most stories will go down due to the default score of 10 being changed to 6. However, all stories will go down equally, unless readers already gave specific scores higher than 6. Most of these changes are already in effect. If not, they will be in a few days. We hope this will clean up some disparities some members have found between stories. Not every reader uses the Star ratings as a judgment to read any particular story. But for those that do, they may feel better knowing that those Stars are a truer reflection from the readers before them.
Sir Ollivander on 2008.02.04 - 10:26PM ()

Comments



Ginny Guerra came out of the woodwork on 2008.02.04 - 10:51PM to say:

It's amazing the amount of work you put in keeping fairness around here. Thank you very much!



Sibling Creature came out of the woodwork on 2008.02.04 - 11:07PM to say:

Excellent. these changes should make the ratings system a lot more useful. The one thing that bothers me now is that the results are still skewed for any stories rated before the changes... It occurs to me that these might be normalised by ignoring any 10 or 1 scores entered before the changes. (assuming that anyone selecting any other score is much more likely to have been judging a chapter on its merits rather than just ignoring the field or putting it down) At the very least I think it would be interesting to see what effect such a change would have on the overall scores.



Sir Ollivander came out of the woodwork on 2008.02.05 - 12:11AM to say:

Sibling Creature - the majority of stories should have been adjusted. The one script that deleted the Trolling removed those that were targeted. The second one deleted multiple ratings by the same person, That affected all stories. I'll have to look to see if making the default of 6 corrected those past stories that had 10s. It should have.



Sibling Creature came out of the woodwork on 2008.02.05 - 02:38AM to say:

Sir Ollivander It would have corrected any deliberate trolling, yes but not the skewing effects of having 10 as a default rating where the reviewer did not specifically intend to rate the story.... If you look through even the current recent additions for example its rare to see a rating less than ~8. Obviously there is no way to know whether someone leaving a 10 intended to rate the story or not, but its for that reason that I think ignoring all prior 10 ratings may yield more useful results. I got the idea from the way in which competitive diving is scored. 7 Judges sit on the panel, and the highest and lowest scores awarded are discarded, so that no individual judge can skew the score in either direction.



melkior came out of the woodwork on 2008.02.05 - 02:46AM to say:

Sibling Creature: The default rating has been set to 6 almost 3 days ago, so all the recent Outstandings have been intended.



Sibling Creature came out of the woodwork on 2008.02.05 - 03:37AM to say:

Melkior: You make a good point... I may have to more carefully assess these stories where most of the reviews have been posted within the last 3 days. :-) With the new changes to the system it will probably pay to start paying more attention to the ratings when deciding whether to read a story. That said I believe my point is still valid for any review posted before that change took effect.



fake a smile came out of the woodwork on 2008.02.05 - 10:01AM to say:

You cannot assume that every person who rated a story a 10 prior to this change did so without intending to. The truth is, you don't know if it was intentional or not, so you cannot go back and change those ratings. It would be unfair both to the authors and the reviewers. Personally, I think there should be an option to leave a review without rating a story - and I think that should be the default. That way you know that every rating left was intentional; otherwise, I imagine we'll start seeing some absolutely glowing reviews with a rating of 6 attached.



kmagarden came out of the woodwork on 2008.02.05 - 10:59AM to say:

Can we no longer delete our reviews? I accidently made multiple reviews to a story yesterday because it didn't look like it was taking. So I gave the same review and rating 3 times to the same chapter. I tried to delete the extra - but it didn't seem to be an option any longer. . .



werekitten came out of the woodwork on 2008.02.05 - 04:52PM to say:

Question: Can we rate a story multiple times, for each chapter? I often give a story a high rating, then, a week later when the next chapter comes out, find that the author's done something I despise and give the story a low rating. Would I not be able to do this under the rules?



Sir Ollivander came out of the woodwork on 2008.02.05 - 08:08PM to say:

werekitten - "Can we rate a story multiple times, for each chapter?" No! That's what got the Rating system screwed up in the first place. You can rate once per chapter per story. If the story has four chapters, you can rate each chapter once, for a total of four times. What you can do is leave multiple reviews. The reviews we didn't touch.



werekitten came out of the woodwork on 2008.02.05 - 09:26PM to say:

Heh, I meant can we rate a STORY more than once total, if it had multiple chapters. I thought your new policy meant that we could only rate once, period, regardless of what happened in later chapters. Sorry for the confusion!



Sakabatou came out of the woodwork on 2008.02.05 - 09:40PM to say:

fake a smile: "Personally, I think there should be an option to leave a review without rating a story." YouTube does that, and I agree. As for star rating being shown, I like the idea so I what readers really think even if they didn't leave a review.



Dianne came out of the woodwork on 2008.02.06 - 12:33PM to say:

I wonder if lazy people will leave a lot of '6' ratings. This may be a bit worrying for challenges in particular. Stars are nerve wracking in challenges in particular but I guess that's what makes them challenging. It's unfortunate that a few people have made all this extra work for everyone but alas, that's life. Thanks for all your work!



Sibling Creature came out of the woodwork on 2008.02.06 - 04:03PM to say:

upon further consideration what I'd really like to see is a percentage of reviews made under the new system next to each story with a rating... that way I'd have an indication of how reliable the rating on any given story is likely to be. While I don't disagree with what fake a smile has said, the fact remains that I don't trust the ratings of any story reviewed under the old system.



lecook4 came out of the woodwork on 2008.02.06 - 05:24PM to say:

I agree and I'd like to see ratings and reviews as two separate things. You can leave a review without rating the story at all, but have another like where you could leave a rating once per chapter if you were do inclined.



lecook4 came out of the woodwork on 2008.02.06 - 08:15PM to say:

Sorry for the spelling in the last comment. Reading it makes me realize how I shouldn't do this on my Crackberry. It was supposed to say "link" not like and "so inclined" not do inclined. Oh well.



Sakabatou came out of the woodwork on 2008.02.06 - 11:10PM to say:

It's all good, lecook4. If it makes ya feel better, I forgot to put "know" in between "I" and "what" so the sentence reads, "I like the idea so I know what readers really think even if they didn't leave a review.



Miri came out of the woodwork on 2008.02.07 - 10:24AM to say:

I have to agree that having an automatic star rating, whatever the number of stars, could be a problem. I've been leaving reviews ever since becomming a member 2 years ago, and just learned with the recent problem, that every time I left a review but didn't hit the "stars", that there was an automatic 5 star rating given. Since the stars, unlike now, didn't show up on the review pages it was easy to miss that fact, although I don't know if that fact was given somewhere and I just missed seeing it. All I knew was what if said at the bottom of the review section--that you could leave a rating *or* review *or* both. I know that it does warn people now that there is a default rating of 3 stars and hopefully our newest members will spot that a s they learn their way around the site. But, other members may not spot it. I've certainly seen several "oops" type reviews where readers have gone back to correct the inadvertent rating. As for deleting doubled or incorrect reviews--I've never been able to get the "delete review" to work so I don't try anymore. Again (and I don't think it can be repeated too often ) thanks to all the people whogive their time and efforts to keep this site going.



GinnyMarie came out of the woodwork on 2008.02.07 - 05:33PM to say:

This may be the wrong place to inquire, but are the skins going to be fixed anytime soon?



General Kenobi came out of the woodwork on 2008.02.07 - 08:12PM to say:

Good work admins! Trolling... *sighs* We've had problems at bcfiles of immature users simply rating Potds and files as 1s, the lowest score, just to be stupid. I thought people here were more mature, I guess not. Finally something will be done to curb it. I'm all for fair scoring.



_kb_ came out of the woodwork on 2008.02.07 - 10:40PM to say:

"fake a smile" has come the closest to describing the real problem. The reviews and the rating needs to be totally separate, AND the default value needs to be "NOT RATED". Or in database terms, it needs to be the NULL value, and then NULL values are not used in the average rating calculation. By setting the default value to 6, you have the same problem you have now ("1.C"), you've changed absolutely nothing. This is a common every day database problem using tri-value logic; there is no magic here. ;-) Past (default 10) ratings are what they are and you can't do anything about them; don't bother trying. Put an "*" next to the values if you want to show that it was with the "old" system. That's about the best you can do. You can't change the past, but the fix into the future is very very simple. Go for the real fix Dino; I know you can do it! :-)



melkior came out of the woodwork on 2008.02.07 - 11:19PM to say:

Oh, alright Kevin... I'll set them to none... But I won't be using the NULL value, so nyah-nyah!



Dianne came out of the woodwork on 2008.02.08 - 09:28PM to say:

I like the option of not rated, but who am I?



melkior came out of the woodwork on 2008.02.09 - 07:33PM to say:

Well, then you'll be pleased to hear that that option is now set as default.



_kb_ came out of the woodwork on 2008.02.10 - 01:43AM to say:

Thanks Dino! See, that wasn't so hard, was it? :-) And now the problem is really solved; everyone can rejoice in better stats. (Hmm, I think I'm up too late. :-)



Sir Ollivander came out of the woodwork on 2008.02.10 - 12:46PM to say:

Dianne - who are you? You are a reader, you are an author, and you are a member of SIYE. And, therefore, one of the reasons SIYE exists. So, your opinion counts as equally as the next person. And we listen to all of you. That's who you are.


../back
‘! Go To Top ‘!

Sink Into Your Eyes is hosted by Grey Media Internet Services. HARRY POTTER, characters, names and related characters are trademarks of Warner Bros. TM & © 2001-2006. Harry Potter Publishing Rights © J.K.R. Note the opinions on this site are those made by the owners. All stories(fanfiction) are owned by the author and are subject to copyright law under transformative use. Authors on this site take no compensation for their works. This site © 2003-2006 ALL RIGHTS RESERVED. Special thanks to: Aredhel, Kaz, Michelle, and Jeco for all the hard work on SIYE 1.0 and to Marta for the wonderful artwork.
Featured Artwork © 2003-2006 by Yethro.
Design and code © 2006 by SteveD3(AdminQ)
Additional coding © 2008 by melkior and Bear