The Ratings Committee (RC) had a busy year, tackling a number of tasks
that led to some important milestones and implementations. The two
main accomplishments were the completion of testing the candidate
formulas to change the K-factor and effective number of games in the
regular rating system, and the implementation of the new blitz rating
system. As of the date of this report, the ratings committee is also
involved in finalizing modifications to the blitz and quick rating
system, the focus of which involves shared rating changes across the
regular, quick and blitz systems. We describe these accomplishments
below, in addition to several other minor issues that had ratings
committee involvement.
With substantial help from Mike Nolan, the committee completed the
testing of several changes to the rating system that affected the
K-factor in the established rating formulas. Due to computing
hardware failures at the USCF office, the final testing was delayed
until December 2012 - January 2013. We tested two aspects of the
formulas simultaneously, and examined them through a simulation
analysis. The first aspect we examined was the formula for
the "effective N" based on the player's rating that
ultimately determines the value of K in the established rating
formula. We considered two alternatives to the current system
formula, both of which were designed to increase the value of K by as
much as 45% in the 2000-2200 rating range, but keep K relatively
unchanged at low and high rating levels. The other aspect was to
revise the impact on K for games played under time controls of
G/60 - G/30 among strong players. The two alternatives to the
current system we considered involved variations of lowering the value
of K the higher one's rating. Along with these two types of changes,
we also investigated the simultaneous effect of varying the bonus
factor. In total, we considered a total of 18 different rating
formula changes to explore, coming from 3 formulas for "effective N"
(including the current system), 3 formulas for the value of K for
fast time controls in established events (including the current system
formula), and two different bonus thresholds (3x3x2=18). For each
of the 18 systems, all USCF-rated events were retroactively re-rated
starting in 2004, and two types of information were recorded.
The first was the final 2012 ratings of all players based on each of
the 18 different systems. To summarize this information, these final
ratings were compared to the ratings of the current system to
check for increases or decreases in particular rating ranges.
The second was a measure of the predictability of ratings
on a game-by-game basis, and this measure was summed for all
games from 2010 onward. The reason for only including games starting
in 2010 was that the changes to the system were implemented starting
in 2004, so that games played in 2010 and after would likely have
evidenced the effects of rating system changes noticeably six years
after implementation. The results of the testing concluded that (1)
candidate changes to the "effective N" formula made
predictive accuracy worse than under the current system and
created greater variation in ratings compared to the current system,
and (2) candidate changes to the K-factor for high-rated players made
negligible change to ratings, nor affected the predictability of
ratings in any substantive way. While our recommendation was to not
change the "effective N" formula, the Executive Board in
its February 2013 meeting passed a motion for a set of new
and untested formulas. Because these approved formulas were not
explicitly tested, the ratings committee is working with Mike Nolan on
testing the approved formulas for unintended consequences before they
are rolled out officially. We hope to complete this testing by the
late Spring, and make the rating system changes effective as of
mid-May.
In addition to the new regular ratings formulas, the RC was involved
in discussions to set up the blitz rating system. The new system is
modeled off the quick chess rating system. The main issues that
required RC involvement was to identify the features of the blitz
rating system that would need to differ from the quick chess system,
or from the regular system. The primary differences all had to do
with initializing ratings in the blitz system. For example, if an
unrated blitz player does not have a regular USCF rating, a FIDE
rating, or a CFC (Canadian) rating, but had a quick rating, then we
would use the quick rating based on 0 games as the initialized blitz
rating. If unrated blitz players have both USCF regular and quick
ratings, the system will give precedence to the regular rating in
initializing the blitz rating, though this may change once the quick
system is viewed as producing more reliable ratings. The new blitz
system was rolled out officially on March 1, 2013.
One of the tasks charged to the committee back in November 2009 was to
propose changes to the quick rating system that would enable ratings
to track players' abilities more reliably. Because many players that
play quick chess events do so infrequently, quick ratings are often
stale relative to players' current abilities. The proposed remedy to
this problem is to increase the impact of regular (slow) game events
on quick ratings, as well as increase the K-factor for quick events in
general, and even more so as a function of infrequent play. The
current system already has quick ratings impacted by games played in
the "dual-rated" time controls of G/30-G/60, but the proposed changes
will attempt to expand the scope of the impact. The proposed changes
will also create similar connections to the blitz rating system so
that changes in quick or regular ratings can impact blitz ratings as
well. To date, the RC is working with members of the executive board
to solidify the details of the modifications. We anticipate producing
a final implementation during the summer of 2013.
In response to the news in April that FIDE is changing their ratings
formula this year, a question was raised whether the FIDE-to-USCF
conversion needed to be updated. The last official update occurred in
2008, though a more recent informal conversion formula was determined
by the RC chair in 2011 but never adopted. Given that even the 2011
conversion is likely out of date, the office decided in conjunction
with the RC chair that a new conversion be determined. This task will
be carried out in the near future.
With the implementation of the blitz rating system, we were informed
by Mike Nolan that he has begun work on revising the rating system
programming, intended to significantly reduce the time it takes to
perform rerates. Based on initial tests, Mike explained that the
revised code does indeed speed up time considerably, but the code does
not fully account in its current form for inconsistencies in USCF
historical data, such as rating floors. This new version of the code
has therefore not been implemented, but is ongoing work that could
eventually lead to large gains in efficiency.
During the course of the year, several specific ratings-related issues
arose that generated discussions about particular fixes to ongoing
ratings problems. For example, the USCF office received complaints
about the resulting rating by a player who competed in a section of a
large tournament and then played extra games in the event. Under the
current rating system implementation, extra games in large events are
treated as a separate section, and each section in an event is rated
in sequence as if each were a different tournament. This means that
if a player is in two sections of a large tournament (the actual
section in which he played, and the "extra-games" section), the overall
rating calculation acts as if these were two separate tournaments.
Whichever section was rated second would have more impact on the
player's current rating. It has been suggested that all
games in a single event, even large multi-section events, should be
rated simultaneously. The RC will work with the USCF office to
investigate whether the simultaneous rating of games in single events
can be implemented.
Another issue that arose had to do with players who repeatedly compete
against much lower rated players, and gain 1 rating point per victory,
as the rating system calculates a 1-point increase for a win against a
much lower-rated player. This is done because ratings are stored as
integers. An idea that may be investigated is whether ratings should
be stored in floating point precision, and then published ratings
would be rounded to the nearest integer. Under this scenario, if a
player repeatedly defeated a much lower-rated player, it might take
5-10 wins (or more) before the player's published rating
increased by 1 point while the stored rating would increase by
fractional amounts.
Each year the RC performs a set of diagnostic analyses to monitor
trends in the rating pool. Overall rating levels have deflated from
the mid-1990s through 2000 when rating floors were decreased by 100
points without a counteracting inflationary mechanism. With the new
rating system implemented in 2001, ratings started to re-inflate. The
RC's goal has been to reinflate and then maintain rating
levels roughly where they were at the end of 1997. The focus of RC
work has been on players with established ratings who have been active
over the current and previous three years and who are aged 35-45 years
old in the current year. In recent years, the average rating for this
group has been steadily increasing, though we are still about 20
points shy of our goal. Because of the recent acceleration of the
increase, we recommended to increase the established rating system
bonus threshold from B=6 to B=8, and this change was made effective
August 2012. Now that a new set of formulas for the K-factor have
been approved by the USCF executive board, and that these changes are
likely to be somewhat inflationary, the recommendation of the RC is
not to make any changes to the bonus threshold and instead wait until
next year to assess the impact on the ratings of this stable group of
players.