Overall Rating Issue

I have seen many users who have:

  • lower/NA ratings in LONG contest than mine.
  • lower/NA ratings in SHORT contest than mine.
  • lower/NA ratings in LUNCHTIME contest than mine.

Still, their overall rating is much higher than mine.

Also, while my SHORT contest rating increased by by a few points in the last (October Cook-off 2017) challenge, the overall rating decreased.

Could anyone explain this behaviour?

Overall rating is not a “direct average” or a “Simple relationship” with the 3 ratings of Long, Cook-off and Lunchtime.

Since its affected after every contest, its more…lets say, dynamic. You may perform extremely good in 2 long and lunchtime but not in cook-ff. That doesnt mean your rating should have an upper bound of your cook-off rating.

Also, while my SHORT contest rating increased by by a few points in the last (October Cook-off 2017) challenge, the overall rating decreased.

Thats because you performed really well in LONG contest and on that basis a better overall rank was expected from you. How it goes is, Cookoff rating give an expected rank X, overall rating give an expected rank Y. Now, what happens when you score a rank between this X and Y?

Your rating for one will decrease and other will increase. Thats exactly what happened.

1 Like

Nice explanation @vijju123

Okay, I get what you are trying to say. Still, itsn’t it ironic that people with all lower rating have higher overall rating? No Algorithm should be designed that way.

What you are saying is that I performed better in past all challenges so I should’ve performed better in the next one too, and thats why the rating fell. That’s OK. But, the person who is higher in overall ratings must have done poorly somewhere to get lower individual ratings? It is a bit confusing, though.

Could you provide a simple illustration or example.

Like, a user (https://www.codechef.com/users/tarun_1909), has the same long rating as me. Still I have less overall rating. Explain this somehow.

"No Algorithm should be designed that way."

Without knowing the algorithm and its purpose, you have no claim on here.

"What you are saying is that I performed better in past all challenges so I should've performed better in the next one too, and thats why the rating fell."

Yes.

"But, the person who is higher in overall ratings must have done poorly somewhere to get lower individual ratings?"

Or perhaps he didnt participate in that type of contest.

Look, take it this way. Overall rating is changed at every rated contest you participate in. In a month, assuming you give 100% participation, it changes 3 times. After every time, it gets fresh statistics like volatility, expected rank etc. In this way, its more “recent” or “updated”.

But contest specific ratings change only once a month, and your overall rating changes twice (100% participation) before that contest is held again. So in a way, this is a bit more “outdated” and more dependent on how you were back then rather than how you are now.

And this ought to answer why there are variations in overall rating and rating per contest.

Ultimately the difference is a result of frequency of changes and differences in statistics- since contest specific ratings are only concerned with how you do in one type of contest.

"Could you provide a simple illustration or example."

For something as complex as this, for which even I didnt go into those minute details, I cant. Sorry for that.

Doesn’t this makes the overall rating specific to a user. I mean you can’t compare two users based on just their overall ratings?

@spp____ Thanks for appreciation.

@rishabh.jain9196 - Saying if person X is better than person Y since he has a better overall rating is…well not always right either.

I think I am missing something in interpreting it. Whats wrong with rating being closer/more specific to you?

Is your question on “people cannot compare 2 users” or “codechef cannot compare 2 users [and hence cant find expected rank etc.]” ??

No Algorithm should be designed that way." -Thats my take on it. No hard feelings. Not looking for a fight here, lol. :-p

Although, I see what are you trying to say. But somehow that doesn’t “feel” correct. I guess discussing it still won’t change it. Still, You somehow conviced me on the rating drop issue, so that’s good!

Hey, I understand thats your take on it. I re-read it, it sounds a bit that way because of the formal tone in which I wrote it (w/o realizing it lol). The only thing I wanted to convey was, that we cannot actually comment on that thing before being in @admin 's shoes (or high-heels :p).

Can you pin-point what doesnt feel right? If you can clearly point it out, I can make/convince @admin to give a reply. Thanks! :slight_smile:

Guess, have to accept it “as it is”, now. :-p

I got your point, thanks.