Ratings accuracy.

Post by **gmiller** » Fri Apr 01, 2011 9:39 pm

I was inspired by this article:
http://chessbase.com/newsdetail.asp?newsid=7114

I wanted to see how well the net-chess ratings stacked up to what Elo's system said the should be. I actually expected it to be way off, but was pretty surprised to see the curve appears to match even better than the FIDE ratings do.

The numbers across the top is the difference in rating points, each label goes to the line just to the left of itself. For example, the label of -1500 is on the very left edge, so any difference of larger than -1500 doesn't appear on the graph.

The numbers along the left edge is the expected score, and the labels go with the line just above them.

The blue dots are the average score of all of the games with that rating difference. The red line is the ideal Elo predicted score.

To orient yourself with the graph, note that the red line goes right through the point 0,.5. So with a rating difference of 0, you're expected to have a score of .5 (a draw).

Post by **gmiller** » Fri Apr 01, 2011 10:18 pm

The raw data is here:
http://www.net-chess.com/ratings2.txt

It contains a column for the number of games played at each point difference too.

Compute the ideal Elo expected score by (C# code):

Code: Select all

    class Elo
    {
        public static double getExpectedScore(int ratingDifference)
        {
            return 1 / (1 + Math.Pow(10, ((-ratingDifference) / 400f)));
        }
    }

Post by **gmiller** » Fri Apr 01, 2011 10:59 pm

And here is the same graph when I don't eliminate time outs under 10 moves:

Post by **gmiller** » Sat Apr 02, 2011 12:19 am

And here's a lame attempt at adding a moving average. It does weight the average based on the number of games played. The green line is the average, and accounts for +/- 100 values;

iamachessstudent · Post by **iamachessstudent** » Sat Apr 02, 2011 10:10 pm

Very cool, Greg!
TY man

islanderfan · Post by **islanderfan** » Sun Apr 03, 2011 12:29 pm

maybe this will shut people up who cry all the freakin time about the ratings and all.

iamachessstudent · Post by **iamachessstudent** » Mon Apr 04, 2011 5:26 pm

WOW Brian, u made a decent post!
CONGRATS :::party party::

islanderfan · Post by **islanderfan** » Mon Apr 04, 2011 8:57 pm

It's what I said all along, just people here are a bunch of pansies.

iamachessstudent · Post by **iamachessstudent** » Tue Apr 05, 2011 6:34 am

hehehe
Well said anyhow and now you are right