Monday, October 9, 2023

Possible solutions to grossly above level teams at USTA League Nationals

The first two weekends of 2023 USTA League Nationals has found more or less the same team winning two events (18 & Over and 40 & Over 3.5 men) without much competition, which has raised the question about how teams are formed and how far above level players are by the time Nationals comes around.

Many people would agree there is a problem, and the "fair at-level competition" line has been crossed.  If that is the case, what steps could be taken to combat it and bring more fairness to Nationals?

Before I outline some ideas, it is probably good to summarize the issue.

The issue is that certainly at Nationals, but also at Sectionals and earlier rounds of playoffs, and even in local league play, you can have players that are significantly out of level and this results in noncompetitive matches and taking away the opportunity for legitimate at-level players to advance into playoffs.  Come Nationals, it becomes who was able to circumvent the rules the best to assemble a team of ringers who will end up winning.

There is also a safety factor at lower levels as grossly above-level players being on the same court and hitting with pace of a level or two higher player may actually be dangerous for the fair at-level players on the court.

The above level players aren't just self-rates, there are also players that appeal down, and players with C ratings that also end up being grossly out of level so all need to be addressed in my opinion.

What I list below is a bit of a hodgepodge of ideas on things that could be done that would result in increased competition both at the local level and through playoffs to Nationals.  Many of these ideas are not new, and some have been shared with me and I'm shamelessly stealing (thanks though!), and I've shared them with various USTA staff and written about them on my blog before, but this will serve as a collection of them all for posterity.

I'm also listing them in an order that prioritizes ideas that I think would both have a real impact and are reasonably doable.

On to the list.

Lower Strike Thresholds

The USTA doesn't publish what the strike thresholds are, but it is commonly known that they are pretty high, especially at lower levels, perhaps even so high that players have to be closer to a double bump than at level before they even get a single strike.  I understand the USTA doesn't want to penalize new players with a promotion that means they can't play with their team and leaves more than enough room for players to improve before they get a strike, but high strike thresholds are a gaping hole in the system that astute captains use to their advantage.  Lowering the strike thresholds would make it a lot more difficult to navigate the season with out of level self-rates or appeal down players.

Increased Match Requirements for Self-Rates and Appeals

The USTA has always required a minimum number of matches be played to be eligible for Nationals, and this was three for years.  This meant that a self-rate or appeal could play the minimum three matches and would have to avoid a strike in only one of those matches to be eligible for Nationals, so there would be virtually no way to get 3 strikes and be promoted and ineligible.  Consider players can't be disqualified during a Nationals and this was a loophole to be exploited.

A few years ago the USTA increased the minimum matches for self-rates to four, but that still isn't enough for the same reason highlighted above.  This should be increased to at least six in order to require players to actually play enough matches to have a real chance of getting strikes if they are above level.

If this means that captains have to play self-rates at the expense of computer rated players, so be it, perhaps strong computer rated players will shy away from joining teams with a lot of self-rates, or captains will limit the number of self-rates they have rostered.  Nearly every Nationals team has an adequate number of team matches to get their players eligible when you look at regular season, local playoffs, Districts/States, and Sectionals.

Incorporate WTN in Self Rating

Today there are guidelines and a questionnaire for players who are self-rating, but the process is vague enough that players can self-rate lower than they should.  The USTA encourages to self-rate at the higher level if there is doubt, anticipating inevitable improvement as the player plays matches, but there is no way to enforce that.

What would be ideal is if there were a more objective way to look at a player's history other than questions about playing in high school or college.  The USTA in coordination with the ITF has introduced such a measure with the World Tennis Number (WTN) which should be calculating ratings for players from their junior (and perhaps even high school?) and collegiate matches.  If the USTA were to establish a way to map WTN to NTRP (which I'm sure they have) this could be used to help ensure players self-rate at an appropriate level.

Make Bump Downs Subject to Strikes

There are already rules for players that appeal down that make them subject to strikes, but why not just make all bump downs subject to strikes?  If they are a legit bump down, there is no chance they are going to have their rating go back up so high as to get to the strike threshold, so this rule would only really affect players that manipulated their rating to get bumped down.

See discussion in the item below as it all applies, this just limits it to bump downs rather than all C rated players.

Make All Players Subject to Strikes

Today, computer rated players are not subject to strikes.  This means as soon as someone gets the coveted C rating they are golden and don't need to worry about anything, they can play all out.  This normally isn't an issue if the player got their C fairly, but some players have either tanked matches to get bumped down and get a C, or they self-rated and managed scores to get a C, and in both these cases their C level is fraudulent.

If all players were subject to strikes, this strategy wouldn't work, or work as well, and players that get a C and suddenly can't lose could get strikes and be DQ'd/promoted.

And I'm not sure there is really a downside to this.  Players that legitmately get a C and play at that level aren't going to get strikes and so nothing changes for them.  It is only those players that game the system to get their C, or are improving exceptionally fast, that would get strikes and be affected, and isn't these cheaters or grossly out of level players that we want to catch?

Make Bump Ups Sticky

A common strategy among players gaming the system is to go all out in year 1 and get bumped up only to have a "bad" year in year 2 to get bumped down, so they can go all out again in year 3.  Making bump ups sticky for at least one year so players can't yo-yo back and forth would help address this.

Doing this across the board could have a slightly negative effect on legitimate bump downs, and if that is a concern there could be a qualifier on how high the year-end rating was.  For example if a 3.5 is bumped up to 4.0 with a 3.55 year-end rating, they wouldn't have the bump up be sticky, but if a player was bumped up to 4.0 with a 3.75 year-end rating, theirs would be sticky.  I don't think this would be an issue as it is highly unlikely that someone who does so well to get up to 3.75 is going to lose all that ability in one year.

And I'd propose the stickiness persist as long as the player's year-end rating remains above the "sticky threshold".  In essence it would be saying a player that is an upper half 4.0 can't be bumped down to 3.5 in one year, they have to go through being a lower half 4.0 first.

Or another type of stickiness that is more focused would be for any player bumped up that went to Nationals to not be able to be bumped down in the next year, i.e. a Nationals bump up is sticky for a second year.

Eliminate Auto Appeals

Today players that have a year-end rating a small amount over the threshold between levels may auto-appeal down and have it granted.  There are some eligibility rules that may not allow some players to do this, but by and large most players just over the threshold can automatically be given an "A" rating at the lower level.  These players are subject to  strikes and a DQ/promotion during the year, but see above for issues related to this that allow players that are genuinely above-level to slide by all year.

I get the idea behind auto-appeals, some players are barely bumped up and want to still play with their own team, and the USTA is sort of admitting that ratings aren't perfect and someone that is a 3.52 isn't really that different from a 3.49, so why not let the 3.52 appeal down.  Or some players that are bumped up may be at a level where there is no play in their area and they wouldn't be able to play USTA League anymore.

The problem is this is yet another exploitable rule as some players will unfortunately tank matches to try to get bumped down, and even if they don't succeed in the bump down, if they are close enough they can still appeal.  So if you have someone that is really a 3.75 player and they tank matches and get down to 3.55, they can perhaps appeal down to 3.5 and now you have a 3.75 playing as a 3.5.

If you get rid of auto-appeals, the legitimate cases for appeals can still be handled through the manual process.  Yes, this would be an inconvenience for the staff having to process these.

Again, if the first two items above were implemented that may tighten things up a bit and auto-appeals could still be allowed.

Don't Allow Self-Rates at Nationals

An alternative to the above changes to self-rate rules would be to simply not allow self-rates to play at Nationals.  If out of level self-rates are a major issue, this solves it very simply.

I don't really like this one though for a number of reasons, and I think the two items above would suitably address the issue, but I list this one for completeness.

All Manual Appeals Handled by National

Today players can file manual appeals or medical appeals and these are heard by district or section committees.  Despite any guidelines National may publish, these appeals don't appear to be processed consistently, e.g. in one section self-rate appeals are granted very easily while in another they aren't.  Or worse, some captains are friends with committee members and tend to get their appeals granted more often than captains without an in.

A solution to this is to have all of these appeals heard by National so they are applied consistently and there is no chance of local bias.

Allow DQs During Nationals

Today at Nationals, players that are subject to strikes won't be disqualified during a Nationals event.  Players can and are promoted after Nationals, but they can play all matches at that Nationals and no matches will be reversed.

Prior to Nationals, it is up to sections to decide how to handle this, and some do the same as Nationals and only consider strikes after the event, and others administer strikes and promotions after each match and will reverse results.  I've been disappointed to see more sections doing the former as that causes a few issues:

  1. It lets what may be a "cheating" team continue to benefit from an out of level player well beyond when they've been determined to be out of level, e.g. a player could get their 3rd strike in their first Sectionals match and play 4 or 5 more with no penalty.  This isn't fair to the other teams with fairly at-level players.
  2. If the "cheating" team wins, which they very well may since they have a clearly and identified out of level player, that player won't be eligible moving forward so the team that won and the team that advances aren't really the same team, and may in fact be a significantly weaker team.  That isn't fair to the other teams, nor to the section which isn't being represented by the strongest available team.

Back to Nationals, the situation is similar.  If a player is identified as being clearly above level in their first match, why let them keep playing?  I get that a player has gone to considerable expense to travel, and a team in theory could go with the minimum number of players and a DQ would mean they'd have to forfeit courts, but why is it fair for properly at level players to have to play players that will be DQ'd?  And why should a team benefit from a to be DQ'd player?

Disqualify/Suspend Captains/Players for Obvious Tanking

I admit it is hard to conclusively identify a tanked match.  Players can manage scores in a match so it is close and they win and you can't be sure it was a managed score.

But there are times where it is blatantly obvious from both the score line and from the ratings generated from the match.  If a player is consistently winning and generating match ratings of 3.8, then suddenly can't win a set and barely win a few games and generating match ratings of 3.2, you know something is fishy.  Especially when there are clear patterns of seasons/leagues used for the tanked matches.

Today, the USTA seems loathe to take action in these cases as they can't 100% prove the match was tanked.  I would like to think the USTA doesn't need to prove this 100% but instead can consider the match ratings and pattern, especially when an entire team is doing it, and suspend players under the sportsmanship clauses in the regulations.

Change the Algorithm

I realize this set of suggestions is highly unlikely, but I include them for completeness.

There are several things that could be done with the rating algorithm to address some issues.

One is to introduce some anti-tanking controls where matches that are significantly unexpected get thrown out so tanking matches would not be rewarded.  Or similar to USGA handicaps, base a player's rating on their best 50% of results.

One could also introduce some deterministic rules to year-end ratings to preclude some gaming the system scenarios.  Like the bump up stickiness I mention above.


What do you think?  Is there any merit to any of these suggestions?  Which would have the most impact?  Or how would you change the order from what I listed?

Or do you think it is fine as-is being a free-for-all as everyone is on the same footing, and just let the captain the best at gaming the system win?


Edit: Added the suspension/disqualification suggestion after the initial post.

16 comments:

  1. Kevin, always interesting to get your perspective. We just finished men’s 4.5 18+ nationals. From my view, there was not a massive difference between the teams that finished at the top versus those at the bottom. The Arizona heat was a serious factor. That put even more strain on teams that rely mainly on 8 or 9 players. The score lines don’t tell the story of how close the matches were. As we see with players like Djokovic, it comes down to winning the big points.

    I saw far more gaming of the ratings when I was at 3.5 and 4.0. Personally I can’t comprehend players that give away games to stay at their current rating level. As a competitor, I want to keep moving up. I don’t see how there is any algorithm that will eliminate that kind of manipulation of ratings. One idea would be that if a player is bumped down for the year, they could be subject to strikes as you suggest above instead of being protected for the year.

    As a captain, I enjoy the challenge of setting courts to give us the best chance to win at least 3 courts. Yet at sectionals and nationals, the USTA could have it so that captains turn in their lineups and then the computer looks at dynamic ratings to set S1, S2, D1, D2, D3. That might truly reward the best team. Yet it might also make upsets less likely.

    Will the USTA ever go to an open rating system like UTR? I think they are going to have to adapt to the new world and make some adjustments.

    ReplyDelete
    Replies
    1. You are right, the problem is worse at lower levels, in part because more players get bumped up so there is a larger pool to pull from to ask to tank back down. 4.5s also tend to be more experienced players and this sort of tanking is more (but not completely) beneath them.

      I'm also torn on combating or allowing stacking. I like giving captains the ability to try to steal a win, but also don't like completely sacrificing courts with lopsided match-ups. I've considered offering a service myself for line-ups like you describe.

      Delete
  2. As someone with a background in quality control, I think “anomalous” scores should simply not enter the rating when they are in a number larger than, let’s say, 2/3 over the calendar year. Getting bumped up by simply “managing the scores” is way more difficult. I was also at the o18 4.5 nationals, and I agree with the post above. I have not seen dramatic differences in teams levels with many bouts decided by a super tiebreaker (which what should naturally happen at Nationals.)

    ReplyDelete
  3. These are really good suggestions. The only one I would add is having EVERY USTA MATCH count towards ratings (Adult, Mixed, Combo Leagues). If you want to track separate ratings by leagues, fine, but then you must play the level of the highest one earned. Bottom line for me, if you play in a USTA match, it should count!

    From your list, the one that’s easiest to implement is to simply lower the strike threshold. It’s inconceivable to me why it’s so high (i.e. 3.8 for a 3.5 player).

    Eliminating automatic appeals makes lots of sense as well. Have players appeal their ratings and have a committee review, just like medical appeals. Small group of folks, but still. . . .

    The most intriguing one is changing the algorithm. Great idea, but no chance USTA will ever consider it, in my opinion.

    Again, great suggestions. I wish there was a way to reach USTA Nationals in a way that would be meaningful to drive any sort of change.

    ReplyDelete
    Replies
    1. Good point on matches counting, and perhaps using each league to establish a rating and someone having to play at whichever is highest. That may be a little tricky to implement and may need to deal with minimum match counts and the like, but it would allow everything to count and not allow one league or season to be used to tank.

      I had the algorithm changes at the end for a reason, not likely to happen.

      Delete
    2. Every match should probably count, but I understand why mixed is separate. But I don't like the idea of separate ratings and just going with the highest rating, unless it's singles and dubbs(any kind of dubbs). If singles and dubbs are separate ratings, then they should be used separately.

      Lowering the strike threshold is the easiest and and should be done. Increasing the match limit seems like an easy solution, but some local leagues have as few as 5 matches or maybe even lower, and then sectionals might only be 1-2 matches. So, I don't think this is really possible. WTN for juniors/college players mapped to NTRP seems like a great idea. College players' levels can span anywhere from 3.5 to 6.0. But, they all have to basically self rate at 5.0 minimum coming out of college. Most college players aren't that good.

      I like the thought of C players getting potential strikes at least as far as catching the tankers but this seems a bit extreme. Overall might be good, but the one player that has a legit rating strikes out and DQ'd is going to happen and I don't like that. Plus, there'll be players who have a legit C rating and just improve a lot in one year, which I think is fine. But if C players can get strikes, then I think C players should have a higher strike threshold than S players do.

      I do like the automatic appeals. If someone is barely into the next level like 0.02-0.03, I think if they want to appeal then that's fine. Now the manual appeals being handled by one National committee is a great idea. There's one standard and no bias or different standards in every section. But, this committee needs to be fairly big, active, and trained accordingly to deal with every section.

      I don't like the notion of suspending players just based on scores. There's lots of variable that go into any and every match. The much better player on paper definitely loses sometimes in tennis and any sport. I don't like the notion of notorious tankers getting away with it continually either. Especially if it's obvious a certain player is doing it consistently, his/her district/section needs to step in. But, the whole country needs to be onboard. It's still not ok, but we can't have only 1-2 sections or even 15-16 sections onboard when even just 1 section isn't.

      Delete
  4. In our Tri-Level leagues this year, there's been blatant match throwing so that people's dynamic ratings get lowered. My team members left the courts disgusted by the blatant tanking. What would be a solution to address this?

    ReplyDelete
    Replies
    1. Changing the Algorithm as suggested by Kevin would help. But that's very unlikely it will happen. Maybe USTA should update the player ratings more often, semi-annually?

      Delete
  5. Vigorous and open mockery toward the offenders. Unified community pressure to stamp out anti-social behavior.

    ReplyDelete
  6. Very thorough, and I appreciate all of the detail. We DO have a problem with a particular captain in our state. 9 self rates consistently on teams of 15, etc. One suggestion has been to limit self rates which you already mentioned. The other was that perhaps you don't allow self rates to play in the post-season at all (i.e. earn your badge, so to speak?) Curious as to your thoughts.

    ReplyDelete
    Replies
    1. I mention not letting self-rates play in the post-season in my write-up, and that is an option to consider, but I hate to completely preclude a segment of players from playing in the playoffs and understand that might be a barrier for some new players looking to join league play. I think it is preferable to have other controls in place that I mention instead.

      Delete
    2. Yea, I don't like excluding S players from postseason either from the reasons Kevin mentioned amongst others. I understand the thinking behind it, but if the other controls or suggestions Kevin made were implemented, most of the problems would go away.

      Delete
  7. Kevin, take a look at what happened at women’s 3.0 nationals this last weekend. The NorCal team dominated by a statistical margin that shouldn’t be possible if the playing field was anywhere close to level or fair. They are part of a tennis org in the Bay Area called AAA that has already won nationals with.4 teams since just 2021. Curious what your thoughts are.

    ReplyDelete
    Replies
    1. NorCal was clearly a very strong team. They had far and away the strongest top-10 average (by 0.1), the question is how did they come about being that strong?

      They did have a lot of self-rates, had a player DQ'd, had some who self-rated at a level higher than 3.0, so there are some suspicions, but the SoCal team they beat has a very similar Shenanigans score and profile. So from that standpoint, NorCal didn't have an advantage.

      So how did they get rated so much higher? Were their self-rates egregiously self-rated as 3.0s? That is possible, I've often thought any self-rate bumped up means they self-rated too low, and NorCal has several that will be bumped up, but most of their good players were not self-rated this year, but were last year. Looking at records, there isn't obvious tanking so either they did it very subtly or they really were 3.0s last year. But if 3.0s put in a lot of work to improve their game, there can be rapid improvement.

      So did NorCal just put in more work and improve a bit more than all the other teams? Or was there something else going on? Hard to say.

      Delete
    2. Thanks for the reply. A quick look at the NorCal squad reveals that at least a few of the players played high level varsity HS tennis (line 1 doubles) in the last 10 years, which means that it was absurd (and in violation of usta guidelines) for them to self rate as 3.0.

      Delete
    3. Good research, and if true, grievances should have been filed. Note some did self-rate as 3.5s in prior years so perhaps did so correctly given the experience you cite. Then the question becomes were their 3.0C ratings legit or managed.

      Unfortunately the USTA does kind of depend on captains filing grievances to root these issues out. I'd like to see more done automatically, or the grievance filed by an LC if something is brought to their attention.

      Delete