SoCal 18 & Over Sectionals may be another unflighted round-robin disaster

It is the time of year that Sectionals begin to be played, and some sections elect to use the the unflighted round-robin format to identify the teams to advance to a single elimination bracket.  In this format, each team plays the same number of matches against random opponents, and then the top-2 or top-4 teams advance to a final or semi-finals.  USTA National began using it several years ago, and more sections are using it for Sectionals when the number of teams they have doesn't nicely fit into an even number of sub-flights.

The major problem with the format is that when not done correctly, there is a chance of a team going undefeated and not advancing.  The way Nationals is run with 17 sections and each team playing four matches, there is usually a very small chance of it happening, and it has come close to happening twice but hasn't yet.  However, the way some sections implement it there has been a far higher chance of it happening and it has happened at least once in NorCal.

The typical problem is having too many teams not playing enough matches.  In NorCal's case they've had over 20 teams in some flights and played just three matches.  That is a recipe for disaster and a 5th place team that was undefeated got sent home.

Well, it appears it may happen again this weekend.  SoCal is holding its 18 & Over Sectionals and is using unflighted round robin with too many teams and not enough matches.  Specifically, using my simulation of the round-robin, the problematic levels are:

• 3.0 Women - 14 teams playing 3 matches for 4 spots - More than 3% chance of 5+ undefeated.
• 3.5 Women - 17 teams playing 3 matches for 4 spots - More than 5% chance of 5+ undefeated before the event, part way through it is now over 42%!
• 4.0 Men - 10 teams playing 3 matches for 4 spots - Less than 1% chance of 5+ undefeated, but there is a chance.

The other levels are either in sub-flights or don't have too many teams.

The 3.0 women and 4.0 men have very slim chances of it happening, but we will see how the 3.5 women finish up, but I think there is a good chance an undefeated team is sent home!

WTN lookup now available on ITF's site with additional features

After many e-mails from the USTA about it, the ITF World Tennis Number (WTN) was rolled out on usta.com about a month ago, and today I got an e-mail that it is now available on the ITF's worldtennisnumber.com site.

The player search page has a footnote that it is currently only showing USTA players.  I'm not sure why this is as the LTA launched WTN about six months ago so you'd think those players could be available too, but at least for now they aren't.

The search is interesting in that it seemingly only lets you search by surname (last name) or a tennis ID.  The type ahead last name search isn't terribly useful as there are often many players with the same last name and the player you are looking for may not be in the four they show.  So you have to finish your search and get the full list which can then be filtered by gender, country, and singles or doubles WTN.  This is an interesting way to search as if you knew the WTN, you probably wouldn't be looking!

But you can put in a range and narrow it down and presumably find who you are after, at which point you can click on them and get a page that is similarly branded to what the USTA sites shows, but with more detail!  The same animated WTN widget is there with singles and doubles WTN and Game zONe numbers, but on the overview page you additionally get:

• Match activity with a count of unique singles opponents, average singles interval, and average doubles interval
• Rating range that can show the range for singles and doubles for all time or the past year
• A chart of "player tournament matches" showing the count of wins and losses by month, filterable by singles or doubles, although it does appear to include USTA League matches too, not just tournaments
• A results summary with the win-loss record which can be filtered by singles/doubles/both and different time periods

Based on the results summary, it does appear it includes matches, for me at least, that were played in 2016, so I'm guessing that is roughly when the data starts the USTA provided to the ITF, which is consistent with what we've seen/heard elsewhere.  In fact, one can go to the Results tab and there, the date filter goes back to July 1st, 2016 so that appears to be the start date for WTN.

The Results tab is also where you can see more details with the specific matches including the players, their WTN, and the score.  They are actually grouped by event/league which is kind of nice, but oddly had a a stray match that wasn't in the right league.  The WTN shown does seem to be the WTN at the time of the match, so you can see how it changes.

Then there is a Head to Head tab where you can search for another player and see a head to head comparison of WTN ratings and if they've played, the records.  It seems to only do this for singles matches though so you can't find your stats with a partner or against a doubles opponent.  It will also show competitors in common which may be useful if you are scouting a future opponent.

All in all it is a nice presentation and it does provide more information than the USTA's site does and adds some additional useful context to help understand more about your WTN and playing history.

Should WTN play a role in USTA NTRP leagues?

The fourth week of the new ITF run WTN ratings have been published, and I will try to do some more analysis soon.  But a related topic came to mind I wanted to write about as well.

For USTA League players, the carrot that is USTA League Nationals, and the chance to go through the process to win your local league, Districts, Sectionals, and advance on to Nationals, is something that becomes a goal for a fair number of players and captains.  This can promote friendly and competitive play, but taken to extremes, can result in what is considered by some to be going too far and undesirable behavior.

The NTRP level based system the USTA uses implicitly gives an advantage to teams that have players with abilities in the upper part of their level.  A 3.5 team with players in the 3.40's or higher is going to beat a team with "average" 3.5s in the 3.20s and 3.30s more often than not.  And generally speaking, to advance in playoffs you need to have players playing above level and on their way to being bumped up, so a Sectionals/Nationals 3.5 team will likely have at least some players in the 3.60s, 3.70s, or higher.  And to win Nationals, you likely need a line-up that is all on their way to being bumped up.

Some of this can happen naturally, e.g. someone who was a legitimate 3.5 last year and got a 3.5C decides to get better and puts in the work with lessons, drills, and increased play, and they naturally get better and by year-end their dynamic rating is around 3.70 and they get bumped up at year-end.  If a captain can recruit and motivate a group of players with this potential for improvement, they can be a Nationals caliber team.

But strong teams can form unnaturally too, e.g. someone that is a 4.0C from 2 years ago took it "easy" or perhaps deliberately lost last year to get bumped down to 3.5C and joins with a few others that did the same to have a 3.5 super team that can be Nationals caliber.  While natural bump downs can happen, when they are manipulated like I noted, most agree this violates the spirit of the rules for sure, and perhaps even some specific rules.

Another way super teams form at times is with brand new players that are able to self-rate at the desired level.  The USTA has self-rate guidelines that dictate the minimum level for a player based on their playing history, but they are far from perfect and can't factor in everything that might identify how good of a player someone is, and there are routinely situations where someone with "4.0" ability is able to self-rate as a 3.5 and if several of these get together, or are sprinkled in with players described above, they can also form a Nationals caliber team.

Self-rating "too low" as allowed by the guidelines is not a violation of any written rule, but some would argue that a captain/player probably knows they are out of level, or will figure that out after a few matches, and should opt to only play up going forward once they figure it out.  But not everyone feels that way and there are pressures from teammates to not bail on them, this is their chance at Nationals after all.

Now, the USTA will say the three-strike DQ process is in place to catch these players, but in my opinion, the threshold are too high and the minimum matches too low and that allows players to hide and make it to Sectionals and Nationals when they really are clearly out of level.

With the advent of WTN, I think there is an opportunity for the USTA to do something to address the self-rate issue.  Many players that self-rate will have played in high school or juniors, and if WTN does what it is purported to, these players will have a WTN rating.  I would posit that an actual rating based on match results should trump, or at least play a factor in the self-rate guidelines such that a player with a WTN of N or better can self-rate no lower than 4.0 for example.

I would think this would help tighten up the self-rate process and avoid some of the blatant under self-rating that goes on, and get players at the right level sooner.

Now, for this to work, WTN needs to work and be equitable across all players and there are some early indications there may be some challenges there (womens singles, womens doubles, mens singles, mens doubles), but hopefully that will improve, and even as-is I think using WTN in the self-rating process could help.

Note, UTR has been around and arguably could have been used in self-rating for years now, but the USTA was never going to validate UTR by doing so.  They have WTN now, so make it happen!

What do you think?  Should a player's WTN play a factor in what they can self-rate at?

What does WTN do with professionals, and what is the Pro Zone?

The World Tennis Number (WTN) from the ITF is now being published on USTA player profiles, is advertised as a rating going from 40 for a beginner, to 1 for a pro.  But what exactly does that mean?  What are professional's WTNs?  And how can I relate a pro's WTN to your own?

First, from the various FAQs on WTN, there is the text "Pro players will be closer to 1".  This isn't explicit, but with the FAQ preceding this statement with the range being 40 to 1, one might think that the very top pro is a 1.0 and other pros would be in the 1.x or perhaps even 2.x range.

Second, in my perusal of WTNs, the smallest I've seen is 1.1, this for a male unrated (NTRP) player from Mississippi.  There are actually very few in the 1's, but the 2's has a fair number.  So if a pro is a 1, is this 1.1 on par with pros?

When I did my chart showing a hypothetical mapping from NTRP to WTN, I left a little room for pros (NTRP 7.0) to live in the 1-3 WTN range.  My guess at the mapping has been proven wrong with the ranges much larger with huge overlap, but if there is no room for the pros, what is done for them?

It turns out, the USTA at least, is not publishing WTNs for pros.  If you go lookup Jack Sock for example, you won't see a WTN showing for singles or doubles, but instead it has the meter maxed out and "Pro Zone" text in the middle of the widget.

It appears what happens is anyone that is an actual "pro", this based on having an ATP/WTA ranking, will simply be listed as a "Pro" and that appears to be what "Pro Zone" means.  This from an LTA web-site:

A world-wide rating system that ranges from 40 (recreational players) to 1 (pro players) - players with an ATP/WTA ranking will be listed as PRO

What I'm not sure of is if the matches professionals play are part of the WTN dataset or not.  You'd think they would be, in which case my guess is the WTNs could go below 1.0 and perhaps even negative.  Perhaps this is why they don't show them and instead just show "Pro Zone".

But what will happen when Jack quits playing and his ATP ranking goes away?  Shouldn't his WTN then become visible?  What will it be?

This exposes a bit of a challenge for any rating system where there are arbitrary upper and lower thresholds.  What happens if a 39.9 does so poorly that their calculated WTN goes above 40?  What happens when a player, recreational player or pro, wins so much their WTN goes below 1?  Does the whole range have to be recalibrated and everyone's WTN changes to keep the integrity of the 40 and 1 boundaries?

It appears the ITF (or USTA at least) is punting on the subject and just not showing pro's WTNs.  And beginners, at this point at least, aren't playing any matches that get into the system so we don't have to worry about that yet.

Although on this last point, I do see some NTRP 2.5s  with WTNs of 40.0, not many, but a few.  So it appears this boundary is enforced, at least from what is published.

And in another "Whacky WTN" moment, I found a male player who is a self-rated NTRP 5.5, and in singles they show as "Pro Zone", but in doubles they are a measly 22.3 which is the WTN you might see for an NTRP 3.5 or even 3.0 player!  Similarly I found a female player with no NTRP that is "Pro Zone" in singles but a WTN 17.4 in doubles.  And there are others that are Pro Zone in singles or doubles and a double digit WTN in the other.  And some of these wild differences have the high confidence blue checkmark as well.

I'm sure some of what I'm seeing are exceptions or edge cases, but I'm coming across a fair amount.  It is interesting to say the least.

Observations about WTN for USTA League players after two weeks of publishing

The USTA has now published the new World Tennis Number (WTN) for two weeks, so there is enough data, real and anecdotal, to begin making some observations.

As stated by the ITF/USTA, a player's WTN is supposed to be updated once a week on Wednesday and incorporate matches through the previous Sunday.  So far we've had the ratings published on Wednesdays so they are hitting that, but are they incorporating the matches they say there are?

That leads to the first observation which is it appears there may be a delay in matches being included.  I've seen several cases where players played matches between 6/5 (the previous Sunday) and 6/12 (the Sunday before the 6/15 publish) and all indications are those matches were not included in the calculations.  And yes, I checked, the matches were entered on time and prior to 6/12.

This may just be growing pains and it will get ironed out, but if a player's matches are not consistently included, it makes it difficult to follow one's WTN and derive any meaning or benefit from tracking it.

But on the other side, there are players that played no new matches, and their WTN changed from week one to week two.  This isn't right or wrong, but does perhaps give us some insight into the algorithm.  It is possible that the matches included actually changed due to catching up, despite none being played, and this would be another indication of some growing pains in the data collection, but it is perhaps more likely that the WTN algorithm simply allows for changes to a player's rating when they don't play due to prior opponents playing and their rating going up or down.

It is a classic debate with ratings algorithms on if one should only look at the rating of the opponent at the time of the match, or if their future results, and them getting better or worse, should have an effect on your rating.  As I understand it, the dynamic NTRP rating calculated throughout the year only looks  at the rating at the time of the match and your dynamic rating won't change if you don't play.  UTR on the other hand will have a player's rating change when they don't play (wildly so at times) and it appears WTN is following UTR's lead in this area.  I have not seen wild swings with WTN, but we only have one update's worth of data.

A big observation I perhaps should have led with is that in mapping NTRP to WTN, you end up with some extremely large ranges.  I showed this in my analysis of singles (men and women) and doubles (men and women) mappings, but the summary is that doubles has a wider range than singles, but even singles has one NTRP level mapping to 10+ WTN levels (doubles has 15+).  A side-effect of this is that some WTN levels have noticeable representation from 3-4 NTRP levels.  If WTN is to be believed, these players from the 3-4 NTRP levels would have a competitive match together.

Another observation is that some players have dramatically different singles and doubles WTNs.  Differences of 5 are not uncommon at all and there are some that are 10 or more different.  And from personal experience, some of the differences are way off and perhaps even backwards.

Note that WTN is arguably better than NTRP in that it calculates separate ratings for singles and doubles, and the majority of players are probably better at one than the other.  The question is if what the WTN is calculating reflects reality or not.  Perhaps it does overall but there are at least some exceptions that make you go hmmm.  But the separate ratings can also be an explanation for some, but not all, of these observations.

WTN has been positioned as a gender neutral rating, but from what we are seeing, in part due to the really wide ranges noted above, there are women that are two or three NTRP levels lower than some men, that have better WTN ratings than the men.  For example, a 3.5 woman with a WTN that is 2 levels better than a 4.5 man.  It seems improbable that this is accurate, but NTRP has its edge case issues too so I wouldn't throw WTN out because of this, but something seems off.

Note that WTN has said it uses data from four years back, and perhaps even started using data six or more years back, so I don't think it being new and lack of data is an explanation.  I would have thought the observations I'm making would have been made during development and addressed if possible.  Perhaps WTN doesn't consider any of these to be an issue.

Given all these observations, I'm not sure a USTA League player will find WTN terribly valuable.  It is clearly different than NTRP, sometimes significantly higher or lower than expected, and your WTN going up or down or being at a certain level may give no indication of whether you will be bumped up or not which is what a lot of players care about.  Since the USTA says NTRP isn't going away, WTN may be just a curiosity for players to follow for fun.

And since it appears there are some clear inconsistencies in the mapping and ranges, the promise of the WTN Game zONe being a good indicator of a competitive match may or may not be accurate right now.

What do you think?  How much attention are you going to pay to your WTN?

Week two of WTN for USTA players

It has been a week since the USTA began publishing WTN ratings for players, and right on schedule it appears the weekly update is being pushed out.  Kudos to the USTA and ITF for sticking to the schedule as advertised.

I've only been able to do some anecdotal analysis of the new ratings so far, more details to hopefully come soon, but I've noticed a few things.

• WTN ratings can change without a player playing additional matches
• Correspondingly Game zONe and confidence levels can also change without a player playing additional matches
• For these players, changes I've seen are generally in the single-digit hundredths, but there are some with larger changes, the highest I've seen is 0.7.

It is entirely possible that while a player did not play more matches, something was reported late to the USTA or ITF and so the algorithm sees it as an additional match and that is why a rating changed.  However, I suspect instead there is some factor in the algorithm that gives more weight to recent results, or has results in the past fall outside the window being considered, or there is an iterative/retroactive aspect to the algorithm and while a player didn't play, someone they played in the past did play and that affected their rating.

If you want to see all the analysis I've done on WTN recently, click here and scroll through the list of posts.

Mapping NTRP to WTN - Men's Singles

The USTA has started publishing the new World Tennis Number (WTN) for players on their profile on usta.com.  I did some initial analysis last week, but now I'm able to do a bit more thorough and complete analysis so this begins a sequence of posts with that.

My process for this analysis is to look at a sampling of the USTA League players who received a 2021 year-end rating/level and look at the distribution of WTN levels for each different gender, discipline, and level.

By limiting it to players that received a 2021 year-end level I am only considering players who got a recent NTRP rating so that I'm not using stale NTRP levels in the comparison.  Since NTRP levels don't change again until the end of 2022, a player could have gotten better or worse in the 6 months since ratings were published, but this seems like a reasonably consistent approach to doing this mapping.  Ideally this would be done using WTN ratings at the time the NTRP ratings were published, but we haven't had WTN until now so we do the best we can.

In this post, I'm looking at WTN's for men's singles.  What you will see in the chart below is the count of players at each WTN level in total and broken out by NTRP level.  So let's have it!

(You likely want to click on the image to see a larger version of it).

Looking at the general distribution, we see a typical normal distribution to the left of WTN 17, but there isn't a true peak in the middle, and to the right of 20 there are some strange drops.  It is certainly possible this is the skill level profile of men doubles players, and it could be that the players that would be in the long tail aren't playing competitive matches and so aren't in the WTN system.

We also see as you move towards single digits, there isn't the strange blip that the men's doubles had.

The above is including all players, but if we limit it to those with a high confidence WTN, the chart is at follows:

This looks a bit better, but a bit of a strange step from 14 to 16.

What is dramatically different is the number of players drops over 85%.  It would seem there are very few men that play enough singles to get a high confidence WTN.

But looking at the counts by NTRP level is what is perhaps more interesting and will give us an idea of how NTRP maps to WTN.

As you'd expect there is for the most part the expected normal curve within each level.  What is perhaps not expected is how wide each NTRP level is, although they are probably not as wide as the doubles players.

There are a reasonable number of 4.5s ranging from 8 to 18 with an average of 13.6 and standard deviation of 2.6.  If there was a direct and perfect conversion from NTRP to WTN, one NTRP level would correlate with about 4 WTN levels, so having a range of 11 levels, with some players even beyond that, seems a little high.

Similarly, there are a reasonable number of 4.0s from 11 to 22 with an average of 16.8 and standard deviation of 2.6.  Both of these are significantly better than the doubles and approaching what you might expect.

The other thing you can do with this chart is see how many NTRP levels there are at a given WTN level.  For example WTN 18 has a noticeable segment of players from 3.0 to 4.5.  If WTN were to be right, it would be saying the 3.0s and 4.5s with WTN 18 would have a competitive match.  That seems hard to believe, but perhaps there really are some edge cases where a 4.5 has a rating that is lagging their ability and the WTN algorithm, despite being updated weekly, hasn't caught up with the 3.0's improvement.  But this seems a bit of a stretch.

Which is right?  WTN or NTRP?  I can't say at this point, but stay tuned, I'll keep doing analysis.  And it is likely that neither is "right" or "wrong" and they are just different.

See the prior post for Women's Doubles, Men's Doubles, and Women's Singles.