Possible solutions to grossly above level teams at USTA League Nationals

The first two weekends of 2023 USTA League Nationals has found more or less the same team winning two events (18 & Over and 40 & Over 3.5 men) without much competition, which has raised the question about how teams are formed and how far above level players are by the time Nationals comes around.

Many people would agree there is a problem, and the "fair at-level competition" line has been crossed.  If that is the case, what steps could be taken to combat it and bring more fairness to Nationals?

Before I outline some ideas, it is probably good to summarize the issue.

The issue is that certainly at Nationals, but also at Sectionals and earlier rounds of playoffs, and even in local league play, you can have players that are significantly out of level and this results in noncompetitive matches and taking away the opportunity for legitimate at-level players to advance into playoffs.  Come Nationals, it becomes who was able to circumvent the rules the best to assemble a team of ringers who will end up winning.

There is also a safety factor at lower levels as grossly above-level players being on the same court and hitting with pace of a level or two higher player may actually be dangerous for the fair at-level players on the court.

The above level players aren't just self-rates, there are also players that appeal down, and players with C ratings that also end up being grossly out of level so all need to be addressed in my opinion.

What I list below is a bit of a hodgepodge of ideas on things that could be done that would result in increased competition both at the local level and through playoffs to Nationals.  Many of these ideas are not new, and some have been shared with me and I'm shamelessly stealing (thanks though!), and I've shared them with various USTA staff and written about them on my blog before, but this will serve as a collection of them all for posterity.

I'm also listing them in an order that prioritizes ideas that I think would both have a real impact and are reasonably doable.

On to the list.

Lower Strike Thresholds

The USTA doesn't publish what the strike thresholds are, but it is commonly known that they are pretty high, especially at lower levels, perhaps even so high that players have to be closer to a double bump than at level before they even get a single strike.  I understand the USTA doesn't want to penalize new players with a promotion that means they can't play with their team and leaves more than enough room for players to improve before they get a strike, but high strike thresholds are a gaping hole in the system that astute captains use to their advantage.  Lowering the strike thresholds would make it a lot more difficult to navigate the season with out of level self-rates or appeal down players.

Increased Match Requirements for Self-Rates and Appeals

The USTA has always required a minimum number of matches be played to be eligible for Nationals, and this was three for years.  This meant that a self-rate or appeal could play the minimum three matches and would have to avoid a strike in only one of those matches to be eligible for Nationals, so there would be virtually no way to get 3 strikes and be promoted and ineligible.  Consider players can't be disqualified during a Nationals and this was a loophole to be exploited.

A few years ago the USTA increased the minimum matches for self-rates to four, but that still isn't enough for the same reason highlighted above.  This should be increased to at least six in order to require players to actually play enough matches to have a real chance of getting strikes if they are above level.

If this means that captains have to play self-rates at the expense of computer rated players, so be it, perhaps strong computer rated players will shy away from joining teams with a lot of self-rates, or captains will limit the number of self-rates they have rostered.  Nearly every Nationals team has an adequate number of team matches to get their players eligible when you look at regular season, local playoffs, Districts/States, and Sectionals.

Incorporate WTN in Self Rating

Today there are guidelines and a questionnaire for players who are self-rating, but the process is vague enough that players can self-rate lower than they should.  The USTA encourages to self-rate at the higher level if there is doubt, anticipating inevitable improvement as the player plays matches, but there is no way to enforce that.

What would be ideal is if there were a more objective way to look at a player's history other than questions about playing in high school or college.  The USTA in coordination with the ITF has introduced such a measure with the World Tennis Number (WTN) which should be calculating ratings for players from their junior (and perhaps even high school?) and collegiate matches.  If the USTA were to establish a way to map WTN to NTRP (which I'm sure they have) this could be used to help ensure players self-rate at an appropriate level.

Make Bump Downs Subject to Strikes

There are already rules for players that appeal down that make them subject to strikes, but why not just make all bump downs subject to strikes?  If they are a legit bump down, there is no chance they are going to have their rating go back up so high as to get to the strike threshold, so this rule would only really affect players that manipulated their rating to get bumped down.

See discussion in the item below as it all applies, this just limits it to bump downs rather than all C rated players.

Make All Players Subject to Strikes

Today, computer rated players are not subject to strikes.  This means as soon as someone gets the coveted C rating they are golden and don't need to worry about anything, they can play all out.  This normally isn't an issue if the player got their C fairly, but some players have either tanked matches to get bumped down and get a C, or they self-rated and managed scores to get a C, and in both these cases their C level is fraudulent.

If all players were subject to strikes, this strategy wouldn't work, or work as well, and players that get a C and suddenly can't lose could get strikes and be DQ'd/promoted.

And I'm not sure there is really a downside to this.  Players that legitmately get a C and play at that level aren't going to get strikes and so nothing changes for them.  It is only those players that game the system to get their C, or are improving exceptionally fast, that would get strikes and be affected, and isn't it these cheaters or grossly out of level players that we want to catch?

Make Bump Ups Sticky

A common strategy among players gaming the system is to go all out in year 1 when they are on a super team and get bumped up only to have a "bad" year in year 2 to get bumped down, so they can go all out again in year 3 on another super team.  Making bump ups sticky for at least one year so players can't yo-yo back and forth would help address this.

Doing this across the board could have a slightly negative effect on legitimate bump downs, and if that is a concern there could be a qualifier on how high the year-end rating was.  For example if a 3.5 is bumped up to 4.0 with a 3.55 year-end rating, they wouldn't have the bump up be sticky, but if a player was bumped up to 4.0 with a 3.75 year-end rating, theirs would be sticky.  I don't think this would be an issue as it is highly unlikely that someone who does so well to get up to 3.75 is going to lose all that ability in one year.

And I'd propose the stickiness persist as long as the player's year-end rating remains above the "sticky threshold".  In essence it would be saying a player that is an upper half 4.0 can't be bumped down to 3.5 in one year, they have to go through being a lower half 4.0 first.

Or another type of stickiness that is more focused would be for any player bumped up that went to Nationals to not be able to be bumped down in the next year, i.e. a Nationals bump up is sticky for a second year.

Eliminate Auto Appeals

Today players that have a year-end rating a small amount over the threshold between levels may auto-appeal down and have it granted.  There are some eligibility rules that may not allow some players to do this, but by and large most players just over the threshold can automatically be given an "A" rating at the lower level.  These players are subject to  strikes and a DQ/promotion during the year, but see above for issues related to this that allow players that are genuinely above-level to slide by all year.

I get the idea behind auto-appeals, some players are barely bumped up and want to still play with their own team, and the USTA is sort of admitting that ratings aren't perfect and someone that is a 3.52 isn't really that different from a 3.49, so why not let the 3.52 appeal down.  Or some players that are bumped up may be at a level where there is no play in their area and they wouldn't be able to play USTA League anymore.

The problem is this is yet another exploitable rule as some players will unfortunately tank matches to try to get bumped down, and even if they don't succeed in the bump down, if they are close enough they can still appeal.  So if you have someone that is really a 3.75 player and they tank matches and get down to 3.55, they can perhaps appeal down to 3.5 and now you have a 3.75 playing as a 3.5.

If you get rid of auto-appeals, the legitimate cases for appeals can still be handled through the manual process.  Yes, this would be an inconvenience for the staff having to process these.

Again, if the first two items above were implemented that may tighten things up a bit and auto-appeals could still be allowed.

Don't Allow Self-Rates at Nationals

An alternative to the above changes to self-rate rules would be to simply not allow self-rates to play at Nationals.  If out of level self-rates are a major issue, this solves it very simply.

I don't really like this one though for a number of reasons, and I think the two items above would suitably address the issue, but I list this one for completeness.

All Manual Appeals Handled by National

Today players can file manual appeals or medical appeals and these are heard by district or section committees.  Despite any guidelines National may publish, these appeals don't appear to be processed consistently, e.g. in one section self-rate appeals are granted very easily while in another they aren't.  Or worse, some captains are friends with committee members and tend to get their appeals granted more often than captains without an in.

A solution to this is to have all of these appeals heard by National so they are applied consistently and there is no chance of local bias.

Allow DQs During Nationals

Today at Nationals, players that are subject to strikes won't be disqualified during a Nationals event.  Players can and are promoted after Nationals, but they can play all matches at that Nationals and no matches will be reversed.

Prior to Nationals, it is up to sections to decide how to handle this, and some do the same as Nationals and only consider strikes after the event, and others administer strikes and promotions after each match and will reverse results.  I've been disappointed to see more sections doing the former as that causes a few issues:

1. It lets what may be a "cheating" team continue to benefit from an out of level player well beyond when they've been determined to be out of level, e.g. a player could get their 3rd strike in their first Sectionals match and play 4 or 5 more with no penalty.  This isn't fair to the other teams with fairly at-level players.
2. If the "cheating" team wins, which they very well may since they have a clearly and identified out of level player, that player won't be eligible moving forward so the team that won and the team that advances aren't really the same team, and may in fact be a significantly weaker team.  That isn't fair to the other teams, nor to the section which isn't being represented by the strongest available team.

Back to Nationals, the situation is similar.  If a player is identified as being clearly above level in their first match, why let them keep playing?  I get that a player has gone to considerable expense to travel, and a team in theory could go with the minimum number of players and a DQ would mean they'd have to forfeit courts, but why is it fair for properly at level players to have to play players that will be DQ'd?  And why should a team benefit from a to be DQ'd player?

Disqualify/Suspend Captains/Players for Obvious Tanking

I admit it is hard to conclusively identify a tanked match.  Players can manage scores in a match so it is close and they win and you can't be sure it was a managed score.

But there are times where it is blatantly obvious from both the score line and from the ratings generated from the match.  If a player is consistently winning and generating match ratings of 3.8, then suddenly can't win a set and barely win a few games and generating match ratings of 3.2, you know something is fishy.  Especially when there are clear patterns of seasons/leagues used for the tanked matches.

Today, the USTA seems loathe to take action in these cases as they can't 100% prove the match was tanked.  I would like to think the USTA doesn't need to prove this 100% but instead can consider the match ratings and pattern, especially when an entire team is doing it, and suspend players under the sportsmanship clauses in the regulations.

Change the Algorithm

I realize this set of suggestions is highly unlikely, but I include them for completeness.

There are several things that could be done with the rating algorithm to address some issues.

One is to introduce some anti-tanking controls where matches that are significantly unexpected get thrown out so tanking matches would not be rewarded.  Or similar to USGA handicaps, base a player's rating on their best 50% of results.

One could also introduce some deterministic rules to year-end ratings to preclude some gaming the system scenarios.  Like the bump up stickiness I mention above.

What do you think?  Is there any merit to any of these suggestions?  Which would have the most impact?  Or how would you change the order from what I listed?

Or do you think it is fine as-is being a free-for-all as everyone is on the same footing, and just let the captain the best at gaming the system win?

Edit: Added the suspension/disqualification suggestion after the initial post.

How often do players that appeal their rating down get disqualified? Interesting Tennis League Stats

One of the challenges of USTA League is that players that are bumped up may feel they can't compete at the higher level and get frustrated with the game, or in some areas getting bumped up to a certain level, say 4.5 or 5.0, may limit or even kill one's ability to play league as there are no teams in an area at that level.

Similarly, players who aren't bumped up may feel their game has declined, or the situation in their area has had a reduction of players at their level, and they would like to appeal down to a lower level.

The "solution" the USTA has for these players is the option to appeal their rating.  And there are two ways to appeal, the auto-appeal or a manual/medical appeal.  And if you self-rate, as a new player or due to an expired rating, you can appeal as well.

For auto-appeals, after year-end ratings are published, there is a link on TennisLink to appeal your rating, and if your year-end rating meets various criteria, your request may be granted automatically right then and there.  The criteria can vary a bit year to year as the USTA can and does change it periodically, but it is basically that you didn't go to Nationals the prior year and your rating is close enough to the threshold.

If your auto-appeal does not work, you can file a manual appeal and it will be reviewed by an area/district/section committee tasked with reviewing appeals.  These are generally limited to medical appeals where a medical condition is permanent and affects some aspects of a player's game.  From what I've been told, you are not likely to get a medical appeal granted for a temporary medical condition that you are expected to recover from.

Now, before you go click the appeal button or file a manual appeal, beware that if your appeal is granted, you do become subject to strikes and 3-strike disqualification/promotion similar to that for self-rated players.

A natural question then is how often to players that appeal down get a 3-strike DQ?

Looking at data I have since 2014, it does not appear to be very often at all.  I only see 141 players that had an 'A' rating that subsequently were disqualified.  My data may not be complete or perfect, but I doubt there are many more than that and even considering we had a pandemic 2020, that is only about 25 per year.  Given there are around 5K-7K appeals each year, this is not a very high rate.

Why might this be?

I believe the reason is that players that appeal have an established rating that is lower than the appeal threshold, just over the threshold for their appealed to level, and the strike threshold is higher than that.  So if a player simply goes back to playing at their prior ability, they will generally remain below the strike threshold.

For example, while the USTA doesn't publish what all these thresholds are, lets assume a 3.5 is bumped up to 4.0 (their year-end rating is 3.51 or higher) but is just a few hundredths over (say 3.55) and this is within the appeal threshold and they can appeal down to 3.5.  Let's also assume the strike threshold is 3.7.  If this player simply goes back to playing at their 3.55 ability and their rating stays around that point, they will remain short of the strike threshold.

So, we can surmise that the majority of legitimate appeals probably don't result in a DQ.  A player simply plays similar to the prior year and they won't get strikes.  It is those that perhaps aren't justified that end up getting strikes and getting DQ'd.

One can probably debate whether giving players the option of appealing is really necessary.  While it can be used for the category of players I described above that are arguably legit, it can also be used by players that can compete at the higher level and are simply looking for an unfair advantage of being at the lower level.  Worse, players might have lost intentionally to manage their rating down to be able to appeal.  It is likely these that are DQ'd as going back to their real ability is higher than the strike threshold.

What do you think?

Are appeal players wildly out of level in your area?  Or are they basically the same as the strongest players at a given level?

Should the auto-appeal option remain?  Or is it ripe for abuse and all appeals should be reviewed or at least the criteria beefed up to weed out the non-legit uses?

Leave a comment and let me know.

Merry Christmas from the USTA? Denied appeals now granted

The Great Southern Bump of 2019 resulted in a lot of men from the Southern section being bumped up when year-end ratings came out.  Some of these bump ups were frankly very questionable, it seems the USTA did a blanket adjustment to several levels of players, and naturally quite a few players felt the bump up wasn't appropriate and tried to appeal back down.

Most of what I heard is that there were very few appeals down granted, but yesterday I heard from several folks that e-mails from the USTA had been sent to some players with the following message:

We recently found a problem with our auto-appeal system. In our investigation of the issue, we found that you may have attempted an appeal and may have been impacted. The problem has been resolved, and if desired, your automatic appeal can be attempted again.

At least some of these players appealed again and this time it was granted!

I have no idea what the problem was or what was done to fix it, but it appears just in time for Christmas, the USTA is granting appeals that were not originally granted.

So, if you had your appeal denied, perhaps you can try again, and if it is now granted, please let me know with a comment here or on Facebook or by dropping me a note at ratings@teravation.net.  The more reports I get, the more I can do to try to figure out a pattern or what happened.

Merry Christmas!

Appeal Rules for USTA NTRP Ratings

With 2018 year-end ratings now published, players that aren't happy with where they ended up may take the steps to appeal their rating on TennisLink in the hopes of having it granted.  Some may appeal up so they can play on a team with friends, others may appeal down if they don't think they can compete at the higher level or flights aren't offered in their area at the higher level.

The USTA League regulations list a number of things regarding appeals, but some of the criteria used for appeals is not public.  I wrote a blog five years ago when some of the criteria was included in a communication from the USTA, but I believe the USTA can and does make changes to the criteria, possibly as often as each year, and those rules I wrote about are now dated and no longer valid.  So if you come across that blog, don't bank on it being accurate anymore.

What we do know is what is in the regulations and here is a summary of that.  Note that for the purposes of this article, I'm only addressing the standard auto-appeals.  Manual appeals for medical or other reasons are an entirely separate thing.

First, there are some special rules for players age 60 and over.  Here is the relevant language from the regulations.

2.05E Promotion of Players 60 or Over and 65 or Over 

2.05E(1) Any player who is 60 years of age or older prior to, or during, the calendar year in which such player plays his or her first local league match and has achieved the same rating level or lower for his or her three most recent year-end ratings, without benefit of appeal of the player’s year-end rating, will automatically be granted an appeal (A rating) if promoted, subject to 2.05E(3) below. NTRP Dynamic Disqualification procedures as outlined in 2.04B(3) apply. 

2.05E(2) All players age 65 or over, if promoted, will automatically be granted an appeal (A rating) of their current rating back to their previous valid year-end rating, subject to 2.05E(3) below. NTRP Dynamic Disqualification procedures as outlined in 2.04B(3) apply. 

2.05E(3) Any player who is clearly above level under the applicable Computer Methodology procedures will be denied an appeal of his or her year-end rating under 2.05E(1) and 2.05E(2).

This basically means players over 60 that have had the same C rating for their past three ratings will be automatically granted an appeal down if they are bumped up.

Players over 65 will always be granted an appeal down, except if their rating is too far into the range for the next level.  This latter part (2.05E(3) is new for 2019 and just closes a loophole where a player that really can play at the next level (their rating is above the "clearly above level") could appeal down, now they can't.

Ok, what about the rest of us?  That is covered by a pretty generic item in the regulations.

2.05C Appeal of Year-End Ratings. Year-end NTRP valid computer rating levels may be appealed or denied through TennisLink except for 2.05D Medical Appeals.

That doesn't offer a whole lot, it just says an appeal can be done using TennisLink.

As far as eligibility to appeal, there is this:

2.07 CHAMPIONSHIP PLAYERS. 

A championship player’s computer rating achieved as a result of play in National Championships may not be appealed down following the Championship Year it is received except as in Reg. 2.05D Medical Appeals and Reg. 2.05E Promotion of Players 60 or Over and 65 or Over.

This is just saying that players that go to Nationals are not eligible to appeal down at year-end of the year they go to Nationals, except for those age 60 or over (see above).

What else do we know about eligibility to appeal?  Not much.  In the past there have been documents that mention being within 0.05 of a threshold, or graduated allowances based on the number of matches played, but I have not seen any specifics of what the current criteria are.

If you go online to appeal and it is not granted, you will see a message something like this:

Automated Appeal Decision

Automated appeal requested up for <player name> is denied - Benchmark or out of appeal range.

That simply tells you you weren't eligible or are out of range.  Don't get confused by the "benchmark" terminology, I think that is just left over from when players did get "B" ratings for going to playoffs, but that rating designation is no longer used so you should just interpret it to tell you your appeal is denied with no other special qualifiers.

Note that in all cases, a player that has an appeal granted is subject to 3-strike DQ.  This is more or less the same as a self-rated player in that if your rating gets above the strike threshold three times, you are DQ'd and promoted to the next level immediately.

Should I learn more, I'll add to this blog entry, but I'm also interested in hearing from players that appeal on whether or not the appeal is granted, so don't hesitate to leave a comment or contact me.