Fixing the ELO ratings system

by Tom Baldwin

Back in 2008 Brian covered a well known ratings system first conceived by the man whose name it now bears, that article offers a great insight into both the good and bad points of Elo ratings, and is certainly worth a read in preparation for this article.

Recently I got thinking about Elo ratings again, and realized their limitations, that they do not consider the score of games, can be overcome. When a game's outcome is considered by the original ELO ratings system it is done so on the basis that the game ends with a binary result, a win or a loss, but that is not all of the information available. As we have seen, a team like last year's Falcons can appear far stronger than they truly are when only their win-loss record is considered, and that is because they were on the right end of luck. Close wins are much more about luck than they are skill, winning by ten points is a lot more of an indication of one team's supremacy than winning by one, but how can we quantify this? Well, we need to know the answer to quite a simple question: when two completely equal teams play each other, what is the chance that, by luck alone, team A beats team B by X or more points? In this instance it will always be by luck alone, since both teams are technically equal, their levels of 'skill' completely cancel each other out.