Summary of USTA League / NTRP Webinar from June 15

The USTA held a Virtual Town Hall Meeting earlier this week which was a combination league update and information session on the NTRP.  The NTRP part was something of a repeat of a recurring session given on-site and in webinars over the past few years, I attended one held in Seattle three years ago and wrote a summary and this one reinforced much of what was said there, but had a few new items of information.

You can watch an hour long replay here, but here is a summary of the key takeaways I had.

There were a number of presenters including what appears to be some new leadership including Michael Hughes from National and Heather Hawkes who has run prior NTRP sessions.  John Niedfeldt-Thomas who chairs the Adult Leagues committee also spoke.

A few league updates were given at the start:

• The new team types (closed, closed but seeking, open) I wrote about in January were highlighted.
• The refer a friend program is now open to all.  Refer a new or lapsed member to join/renew and you could earn a $25 gift card through the end of this year (while supplies last).

Then it was on to the NTRP info session.

An oft asked question is about how the system deals with tanking.  The response was there is no way to no for sure which matches are tanked so they are not thrown out or otherwise identified right now.  They continue to look at it but have no mathematical definition and are worried about throwing out good matches.

Another popular question is about publishing ratings more frequently.  The response is that since the NTRP was created for league play which is seasonal and played at different times of the year in different areas, but culminates in Nationals in the Fall, publishing more often would disrupt team play and there is no way to do it that would work across all sections.

For years, NTRP tournaments have not been treated consistently, many sections didn't include them in NTRP calculations, but it appears nearly all sections are or will be including them in year-end ratings soon, the hold-outs are for technical reasons.

A common complaint is that the difference in ability between the top and bottom of an NTRP level range is too large, and the matches are not competitive.  The discussion here introduces the distinction between "competitive" and "compatible", the USTA saying that two players of the same level should be "compatible" but not necessarily every match will be "competitive".

Why not shrink the bands to make it so more matches are competitive?  The answer is that while that is possible in some more highly populated areas, doing so across the board would be difficult in smaller or more rural areas as there aren't enough players for critical mass at narrower levels, or in larger areas there are not enough courts to support the additional teams that might result from an increase in flights and teams.

A very popular question is why there isn't more transparency and publishing of dynamic ratings.  The USTA response is that they feel there is an over-emphasis on the rating, leading to ratings management and degradation in the league program and they are concerned that publishing ratings would make this worse.  It isn't a hard no however, they are always looking at ways to engage the player with more information.

Side note, obviously their lack of transparency when it comes to dynamic ratings gives me an opportunity to do reports for folks wanting to get an idea of where their dynamic rating may be, and more importantly see how their rating is changing and what is and isn't working well.  It also leads to the variety of other sites that publish ratings, that the USTA continues to say are not authorized and are wildly different from the actual rating the USTA calculates.  I believe it was said that the most used site is out-of-level for players 40-60% of the time, which is even worse than what I've observed when I've done some casual sampling and comparisons.

What about calculating separate ratings for singles and doubles?  The response was it would be a huge change and I think would wreak havoc with team formation when a player had different levels for the different disciplines.

What about UTR?  The USTA sees that UTR serves a different segment of the tennis playing community, juniors and collegiate players, but claims it falls apart when applied to other segments.  That is consistent with what I've heard and observed as league player's UTRs can be all over the place and change wildly.

A question was asked about using UTR as part of the self-rate criteria.  This is actually not a bad idea, but the response was it is not consistent enough to use reliably.

What about the World Tennis Number (WTN)?  I've written about WTN before, and the USTA doesn't see it as a competitor to NTRP, although it was said it is designed with all segments in mind and the USTA has been participating in its development along with the majority of federations who have agreed to share data.  How the USTA may use WTN is TBD.

Will matches played in 2020 count for ratings?  We know 2020 year-end ratings were not published, which has caused a number of problems, but if you played matches in that year, they will be counted for what is effectively a 2020/2021 rating period when ratings are published at year-end.

Since there were no 2020 year-end ratings, what about players that should have been bumped up or down?  The USTA did institute a perhaps one-year only appeal process where players can manually appeal and if the data from 2020 matches supports their request, it may be granted.  I wrote about this here.

Self-rating, and abuse of the process, is always a concern and a question was asked about going back to in person ratings clinics.  The response was that those clinics were more subjective than the current process and was abused in its own way, and the new process is more cost effective and efficient.

Last, a question was asked about if the USTA adjusts things to keep the distribution of players at each level at some desired percentage.  The answer was no, but that the ratings naturally fit a bell curve nearly perfectly and a manual review or adjustment to maintain this is not required.

Hopefully this summary is useful for you if you aren't able to watch the webinar replay.  What do you think?

How often do players that appeal their rating down get disqualified? Interesting Tennis League Stats

One of the challenges of USTA League is that players that are bumped up may feel they can't compete at the higher level and get frustrated with the game, or in some areas getting bumped up to a certain level, say 4.5 or 5.0, may limit or even kill one's ability to play league as there are no teams in an area at that level.

Similarly, players who aren't bumped up may feel their game has declined, or the situation in their area has had a reduction of players at their level, and they would like to appeal down to a lower level.

The "solution" the USTA has for these players is the option to appeal their rating.  And there are two ways to appeal, the auto-appeal or a manual/medical appeal.  And if you self-rate, as a new player or due to an expired rating, you can appeal as well.

For auto-appeals, after year-end ratings are published, there is a link on TennisLink to appeal your rating, and if your year-end rating meets various criteria, your request may be granted automatically right then and there.  The criteria can vary a bit year to year as the USTA can and does change it periodically, but it is basically that you didn't go to Nationals the prior year and your rating is close enough to the threshold.

If your auto-appeal does not work, you can file a manual appeal and it will be reviewed by an area/district/section committee tasked with reviewing appeals.  These are generally limited to medical appeals where a medical condition is permanent and affects some aspects of a player's game.  From what I've been told, you are not likely to get a medical appeal granted for a temporary medical condition that you are expected to recover from.

Now, before you go click the appeal button or file a manual appeal, beware that if your appeal is granted, you do become subject to strikes and 3-strike disqualification/promotion similar to that for self-rated players.

A natural question then is how often to players that appeal down get a 3-strike DQ?

Looking at data I have since 2014, it does not appear to be very often at all.  I only see 141 players that had an 'A' rating that subsequently were disqualified.  My data may not be complete or perfect, but I doubt there are many more than that and even considering we had a pandemic 2020, that is only about 25 per year.  Given there are around 5K-7K appeals each year, this is not a very high rate.

Why might this be?

I believe the reason is that players that appeal have an established rating that is lower than the appeal threshold, just over the threshold for their appealed to level, and the strike threshold is higher than that.  So if a player simply goes back to playing at their prior ability, they will generally remain below the strike threshold.

For example, while the USTA doesn't publish what all these thresholds are, lets assume a 3.5 is bumped up to 4.0 (their year-end rating is 3.51 or higher) but is just a few hundredths over (say 3.55) and this is within the appeal threshold and they can appeal down to 3.5.  Let's also assume the strike threshold is 3.7.  If this player simply goes back to playing at their 3.55 ability and their rating stays around that point, they will remain short of the strike threshold.

So, we can surmise that the majority of legitimate appeals probably don't result in a DQ.  A player simply plays similar to the prior year and they won't get strikes.  It is those that perhaps aren't justified that end up getting strikes and getting DQ'd.

One can probably debate whether giving players the option of appealing is really necessary.  While it can be used for the category of players I described above that are arguably legit, it can also be used by players that can compete at the higher level and are simply looking for an unfair advantage of being at the lower level.  Worse, players might have lost intentionally to manage their rating down to be able to appeal.  It is likely these that are DQ'd as going back to their real ability is higher than the strike threshold.

What do you think?

Are appeal players wildly out of level in your area?  Or are they basically the same as the strongest players at a given level?

Should the auto-appeal option remain?  Or is it ripe for abuse and all appeals should be reviewed or at least the criteria beefed up to weed out the non-legit uses?

Leave a comment and let me know.

Expired rating? Need to self-rate? Here is how it works

When the USTA didn't publish 2020 year-end ratings, they also didn't change the rules regarding expired ratings.  The result is that a number of players that would have gotten 2020 year-end ratings didn't, and as a result had their rating expire meaning they have to self-rate again to play in 2021.

For example, consider a player that played in 2017 and got a year-end rating, but didn't play the minimum number of matches in 2018 or 2019, but did in 2020 before/after the pandemic shut things down.  Normally this player would have gotten a 2020 year-end rating and been fine for 2021 but now has to self-rate.

And for older players that have a 2-year expiration, they could have played in 2018, not had enough matches in 2019, but did in 2020, so missed just one year, and in 2021 they have to self-rate again.

This seems like a miss on the USTA's part.  First of all they should have published 2020 year-end ratings, but if they weren't going to, they should have suspending expiring ratings for one year, at least for those that played in 2020.

But they didn't so we have the situation we are in now.  Between all the self-rates from 2020 that didn't get ratings and these players that have to self-rate that perhaps shouldn't have had to, we likely have record numbers of self-rated players.

This has contributed to the increased rate of 3-strike DQs, but also raised questions on how self-rating works.

I was reminded of this when someone pointed me to an Essential Tennis video where Ian goes through the self-rate process as he is planning to begin playing USTA League again.  He raises a number of questions or criticisms of the process, and I left a comment there explaining things, but I thought it would be a good idea to write it up here too.

The USTA has a set of guidelines and a self-rating questionnaire to help players identify the minimum level they can self-rate at.  Ian goes through the guidelines and questionnaire, and concludes that the guidelines would require him to self-rate at 4.0 or higher based on his Div II college experience, but he knows he is competitive with 4.5s so he'd like to self-rate at that level.  But when he completes the questionnaire, the system spits out 5.0 as the minimum!

Before explaining what happened, I'd point out that Ian saying he'd elect to self-rate at 4.5 instead of 4.0 is the right thing to do.  What the self-rate questionnaire spits out is not the level for a player, but the minimum, and the USTA expects players to self-rate higher if they feel it is appropriate and Ian is following this guidance.  That said, he could always self-rate at 4.0 and play up at 4.5 which gives him a safety net of being able to play 4.0 if he plays 4.5 and finds out he is in over his head (not suggesting he would be, just offering an example).

Now, what happened?  While Ian didn't fall into the exact situation I explained above of an expired rating that wouldn't have happened with 2020 year-end ratings, he did have a prior 5.0C rating from 2013 when he was bumped up from 4.5 (that my ratings agree with BTW).  The USTA has a rule that when you have an expired rating and self-rate again, you can self-rate no lower than your most recent prior rating, which means Ian's 5.0 sets that as the minimum.

Does doing this make sense?  I think so, as while the questionnaire helps get players with no USTA record at a reasonable level, a real playing record is almost always better, even if the rating is several years old.  Basically, real match results, even from 8 years ago, are more indicative of a players ability than how they answer questions about their high school or collegiate experience from 15-20 years ago.

But why ask all the questions then if the answers were just going to be ignored?  The answer is that not everyone with an expired rating falls into Ian's situation.  It is possible that since the player last played USTA, their playing history has "improved" and the guidelines would call for a higher minimum rating than their last rating.  Take for example a high school senior that plays USTA League the Spring before graduating and gets a 4.0 year-end rating.  But then they go on to college and play and get experience that the guidelines require a minimum rating of 4.5.  If the questions weren't asked and answered, but instead it just said a minimum of 4.0 from the prior rating, the self-rate process couldn't spit out the 4.5 that is now appropriate.

Ian also pointed out that while the guidelines have age as an input, the questionnaire never asked for his age.  This is because he was logged in to his account which has his date of birth so it already knew his age to factor that in.

The guidelines also have a lot of granularity regarding varying levels of collegiate experience which makes things look complicated.  Ian asks if all of that is necessary.  I think it is as someone could play at a junior college and a 4.0 would be appropriate, but someone who is a top player at a Div I school could very well be a 5.5 or 6.0.  To get players slotted in at the right level, you need a lot of questions to narrow in on that.

Now, Ian last played in 2018 which was 8 years ago now, so shouldn't the process factor that in and consider while he was a 5.0 in 2013 he might not be now?  That is a good point and perhaps it does in some cases, but he is at an age where a decline due to age is not a given, so the system errs on the side of not letting the player get too low a level automatically.

So what is Ian to do if he really thinks being a 5.0 is too high a level?  There are provisions for appealing the minimum self-rating that is spit out, and a player can make their case and have a person/committee review it and perhaps grant the appeal.  And appeals are sometimes successful so it isn't just theoretical.

Hopefully this helps explain how things work.  Feel free to leave a comment with any questions.

Update on 3-strike DQs in 2021 - Interesting Tennis League Stats

About two months ago I wrote about the trends with 3-strike DQs in 2021, in large part because 2021 is a unique year as it is really an extension of 2020 as no year-end ratings were published by the USTA for 2020.  At that time, the DQ rate in 2021 was up about 60% over the same period in 2019.  As more areas are playing, it seemed like a good idea to update the stats.

Through June 1st this year, I show 314 DQs while through the same date in 2019, there were 253.  That is up 24% which is no where close to the 60% we saw earlier, so has the rate of DQs slowed?

As I did last time, it isn't enough to just look at raw DQs, but we need to look at the rate per individual or team.  Since 2020 halted play in March, we can't really compare with it so we'll look at 2019 as I did above with the counts.

In 2021, there have been just over 69K team matches played in 18/40/55 and over leagues, while in 2019 there were just over 91K.  That puts the 2021 DQ rate at 0.45% and in 2019 just 0.28%, which means the rate in 2021 is 63% higher.

In 2021, there have been just over 169K players that have played in 18/40/55 and over leagues, while in 2019 there were just over 221K.  That puts the 2021 DQ rate rate at 0.19% while in 2019 it was just 0.11%, which means the rate in 2021 is 62% higher.

So, the rate continues to be in the 60% range, supporting my hypothesis that one of the issues from not publishing 2020 year-end ratings would be more players out of level, which leads to more DQs.

The good news is the system appears to be working and DQs are happening for those self-rated and appeal players that are now wildly out of level having well over a year to play at their original self-rating or appealed rating.  The bad news is this means there have been more out of level players in league play which has perhaps led to frustrations from players who feel they have to play out of level players.

I obviously still believe it was a mistake to not publish 2020 year-end ratings, but read my earlier blog article to see more details.

What do you think?  Or what is your experience in your area?