Introducing the "Shenanigans Score", a metric to identify teams that may be gaming the system in USTA League play

I just wrote about a team that won two USTA League Nationals the first two weekends, highlighting how they went about constructing their roster by having a large number of players that had been 4.0s or even 4.5s in the past somehow find their way down to 3.5.

This "gaming the system" is not unique to this team, it happens in most every section but usually to a lesser degree than happened with this team.  So I thought it might be useful to come up with a way to try to identify these teams in advance and have come up with a Shenanigans Score to do this.

There are number of ways to construct a roster with above level players that give an advantage on the court and there are indications of this such as:

• Self-rated players that are hidden during the year
• Players that appeal down from the higher level
• Players that are disqualified on the team
• Players that have held a higher level in the past
• Players that had to self-rate at a higher level in the past
• Players that were disqualified in the past

All of these are an indication that some sort of ratings management or "shenanigans" are going on.

To try and objectively look at the situation, I've created a way to combine all these objective facts or metrics to compute a score which can be looked at as a total, but also per rostered player to identify those teams that may be suspect.

As reference, at the recent 18 & Over 3.5 men's Nationals, the teams had the following totals and scores:

• Texas - 20 / 1.2
• NorCal - 17 / 0.71
• Midwest - 12 / 0.71
• Southern - 13 / 0.65
• New England - 10 / 0.63
• PNW - 8 / 0.62
• Eastern - 12 / 0.60
• Caribbean - 8 / 0.57
• Missouri Valley - 7 / 0.50

The rest were below 0.50, the lowest being Florida at 0.18 and they were the only one under 0.20.

Texas was well ahead of the rest of the teams, and you a sizable group around 0.6-0.7 which would appear to be what is "normal".  Texas used this roster construction advantage to have a top-10 average well ahead of the rest which gave them an advantage in winning the title pretty easily.

The 40 & Over 3.5 men's Nationals had the following totals/scores:

• Texas - 22 / 1.3
• Caribbean - 12 / 0.75
• Hawaii - 9 / 0.64
• Missouri Valley - 8 / 0.62
• Mid-Atlantic - 11 / 0.61
• Eastern - 10 / 0.59
• SoCal - 8 / 0.50
• New England - 7 / 0.50

The rest of the teams were below 8 and 0.50.  The lowest was Northern at 0.05 but just four were under 0.20.

Texas far and away has the highest total and score, more than double all but Caribbean (and still well ahead of them), and with this roster construction advantage, had a top-10 average well ahead of the other teams, and sure enough, they won the whole thing pretty easily.

What about women's 3.5?  Here is the 18 & Over:

• Caribbean - 10 / 0.63
• SoCal - 16 / 0.62
• Missouri Valley - 7 / 0.44
• Texas - 6 / 0.35
• Hawaii - 5 / 0.33
• Mid-Atlantic - 7 / 0.32

The rest were below 0.30, two below 0.10 with New England at 0.04 the lowest.

Overall these are lower, an indication of fewer shenanigans for the women, but we still see the highest scores between 0.60 and 0.70.

And the 40 & Over 3.5 women:

• Texas - 11 / 0.79
• Florida - 6 / 0.55
• Hawaii - 7 / 0.50
• Missouri Valley - 6 / 0.50
• Middle States - 7 / 0.44

The rest were below 0.40, four under 0.10 and one with a 0.0 score (Eastern).

It is worth noting that the I think this Texas women's team is also from Houston where the two men's 3.5 teams are from, so is perhaps aware of the roster strategy and has the highest score we see outside those two men's teams.

Note that a high score does not mean a team is guaranteed to do well at Nationals.  Some high scoring teams above even did poorly at their respective events.  This is because any single self-rate or bump down is not a smoking gun, as some players self-rate correctly and fairly, and some players are legitimately bumped down and are at the right level.  And at some levels, you are going to have a lot of self-rates just by the very nature of the level being one where new players are common.

Further, some teams go to Nationals without their best players, or plan to play everyone for the experience versus playing on their best, and this has a big influence on who wins.  In fact, one captain contacted me recently saying they were missing three top players and was curious what the simulation would have said knowing that.  My original simulation had them 3-1 but accounting for the missing players it changed to 2-2, and guess what, they finished 2-2.

But I think this Shenanigans Score is interesting to look at, as when there is smoke there is often fire, especially when you have such outlier scores the Texas 3.5 teams had.

What do you think?