It turns out that despite the lack of a notice on TennisLink, year-end ratings are in fact being published today and they have even done so well in advance of the traditional 5pm. With that, TennisLink is now enduring its peak load for the year as we all check to see if we or our prospective teammates have been bumped up/down so we can know how to assemble our teams for 2013.
I've been generating Estimated Dynamic NTRP reports for anyone interested in a preview of what their year-end rating may be and the details behind it and with a number of reports done, it is now time to compare what I estimated with what year-end ratings ended up being.
Of all the reports I did, 79% estimated the player to be at the right level. But I consider being within 0.05 to be an acceptable margin for error considering it is just an estimate and an additional 9% or 88% total fell within this range. Of the remaining 12%, I had 3% estimated too high and 9% estimated too low.
All those I estimated too high were self-rates that went to playoffs, so since I don't attempt to estimate the benchmark part of the calculation, I don't feel too bad about missing here. And I did correctly estimate a self-rate being bumped down so I don't think I have a general problem with overrating self-rated players.
Of those I estimated too low, one went to playoffs (benchmark calculation likely kicked in) and one was just 0.07 from being estimated to be bumped up and also went to playoffs (benchmark calculation kicked in), so I don't feel too bad about these.
So I'll take 88% accuracy where most of the misses have explanations that I actually commented on being possible in the report I generated.
Even though ratings are out, if you want to know why your rating is where it is, I am still generating reports so contact me if you are interested.
Update 12/20: More reports have been generated and the accuracy is now at 92%.
Where are you finding the link to the ratings?
ReplyDeleteJust visit the TennisLink web-site at http://tennislink.usta.com/Leagues/Common/Home.aspx and use the Find NTRP Rating Info search.
DeleteIf bump up and bump down lower than 12%, then every one can have 88% accuracy (just predict every stay with last year level).
ReplyDeleteSo you need to provide info about #/% of bump up and bump down to prove your algorithm works better.
Fair question. Note that most of the folks that are interested in a report believe they are near a bump up/down so generally aren't in the middle of a level where it would be trivial to be accurate. That said, here are more specifics.
DeleteTo date (I've done more reports since the original blog), I've estimated 12 bump ups and 10 were bumped up. One of those that wasn't was self-rated and the other was just 0.03 above the threshold.
I've also estimated 3 bump downs and 2 were bumped down. The one that wasn't was just 0.04 below the threshold.
So that is 12 of 15 bump ups/downs correct or 80% on this group alone. And if you grant my estimation a small margin of error, my estimates have arguably only missed on 1 bump up/down.
Is that accurate enough?