Saturday, October 12, 2019

Nationals week 2 has five sets of semi-finalists - What tie-breakers had to be used?

The second weekend of 2019 USTA League Nationals has made it to Saturday night, and that means we have semi-finalists all around.  With the random round-robin format, this means we likely had tie-breakers involved in deciding who advanced.

The 18 & Over 2.5 women had three teams finish 3-0 in Midwest, SoCal, and Southern.  There were five teams at 3-1, but Intermountain had the best record on courts and got the 4th spot.

The 18 & Over 3.0 women had the perfect scenario with four undefeated teams in Intermountain, Florida, NorCal, and Eastern.  A tie-breaker was used to determine 3rd vs 4th and NorCal got it having lost one fewer set than Eastern even though Eastern won two more sets than Northern.  My preferred use of sets won/lost differential would have put Eastern in 3rd rather than 4th.  Note that Eastern also won one more game than NorCal both having lost the same number.  I think the tie-breakers got it wrong here, but thankfully it was just for 3rd vs 4th.

My simulation only had Eastern as a semi-finalist candidate so some surprises here as often happens at 3.0.

The 18 & Over 3.0 men had two teams at 4-0 in Caribbean, and Eastern, Caribbean having a clearly better record on courts, 15-5 to 12-8.

There was a three way tie at 3-1 with Southern and Northern having better records on courts, 15-5 and 13-7 vs NorCal's 11-9.

Of the semi-finalists, my simulation had just Southern, 3.0 can be hard to predict.

The 40 & Over 4.0 women had four undefeated teams in PNW, SoCalSouthwest, and Intermountain.  That made things simple on who was in the semis, but the tie-breakers were involved in seeding.

PNW and SoCal were both 16-4 on courts but PNW lost two fewer sets so they took the #1 seed.  Southwest and Intermountain were 15-5 on courts but Southwest lost three fewer courts and took the #3 seed.

The simulation had Southwest and Intermountain in the semis.

The 40 & Over 4.0 men had two undefeated teams in Intermountain and Florida.  Intermountain had the best record by far 17-3 vs 12-8, and took the #1 seed.

The last two spots had five way tie at 3-1 with Midwest, Texas, Mid-Atlantic, Southwest, and Middle States.  The first two were tied on courts at 14-6, both lost 13 sets and so it went to games lost where Midwest lost one fewer game.  My suggested tie-breaker would have used sets won/lost differential where Midwest was two sets better so it would have been the same, but determined earlier in the sequence.  Two of the other teams were at 13-7 and one at 12-8.

The simulation had Intermountain and Texas and now they play in the semis.  And interestingly we have a rematch from Saturday night in the other semi.

Thankfully, no really huge tie-breakers and no situation where we got to the controversial and flawed games lost tie-breaker leaving a deserving team out of the semis, but it did arguably seed a team wrong.

Good luck to all the teams tomorrow!

Friday, October 11, 2019

Day 1 update for week 2 of 2019 USTA League Nationals

Day 1 is in the books and here is how things stand.

The 18 & Over 2.5 have five undefeated teams, the simulation says that won't continue and probably three teams finish that way with SoCal, Southern and Midwest most likely to advance with PNW and Caribbean vying for the last spot.

The 18 & Over 3.0 women have five undefeated teams, but just one, Eastern may finish that way, Middle States, Midwest, Florida, and NorCal vying for the other three spots.

The 18 & Over 3.0 men are already down to just four perfect records, the simulation says just one will finish that way in Mid-Atlantic, Southern, Eastern, and New England taking the other 3 spots.

The 40 & Over 4.0 women remarkably have seven teams at 2-0, and the simulation says there is a 9% chance we end Saturday with five perfect records and one team would be sent home.  There is a 46% chance of four undefeated teams, those most likely Southwest, Middle States, Intermountain, and Mid-Atlantic, but PNW lurking.

The 40 & Over 4.0 men have five undefeated, and the simulation just two are most likely to finish that way, Intermountain and Texas, with a likely 5-way tie for the last two spots at 3-1.  Look for Mid-Atlantic and Middle States to most likely pick those spots up, NorCal, Midwest, and Florida lurking.

Does the USTA's 3-strike DQ system work? Looking at DQ's after Nationals

Those that follow USTA League know that there is a process in place for new players to self-rate based on playing history, but to help ensure this process is not abused there is a 3-strike DQ system to identify players that may have self-rated too low and proactively promote them to the next higher level.

Note that the system identifies strikes when a players rating exceeds a level dependent threshold.  The USTA does not disclose what the threshold are (although I have a pretty good idea) but does say they are higher than just the top of the level the player is playing at.  For example, the top of the range for a 3.5 player is 3.50, but they may not get a strike unless their rating exceeds, say 3.75.  This is done to allow players to have some natural improvement without being "punished" with disqualification and promotion.  And a fair number of at-level computer rated players will improve during the year too so having some self-rates still eligible with similar ratings isn't all that out of whack.

Now, the system does "work", it will routinely DQ and promote a number of players each year, but does it really work well?

One measure of working well would be that any out of level players are identified and deemed ineligible prior to Nationals.  While players are not DQ'd during Nationals, if they've accumulated 3-strikes at the end of any Nationals event, they can be DQ'd and promoted then.

I would posit that if a player is DQ'd after they play at Nationals, then the system didn't work as that player that has proven to be clearly above level and you'd expect the system to determine that before they get to an event as important as Nationals and get to lay waste to rules abiding opponents that likely have no shot against the clearly above player.

Additionally, if a self-rate is double bumped at year-end but was not DQ'd prior to Nationals, I'd argue that is a failure of the system too.

I'm sure not everyone will disagree with the above statement, but lets go with it and see if the system "failed" after one weekend of Nationals.

There have been some players marked as DQ'd/promoted that played at Nationals, and by the following Friday I'd hope any that were DQ'd have been reflected, and as of the writing of this, three players were DQ'd after playing at Nationals.

One player from New England was DQ'd, they went undefeated all year losing only one set, including 2-0 at Nationals.  Now going undefeated does not necessarily mean a player should be or have been DQ'd, they may have played weak opponents.  And my ratings for this player show they likely didn't have more than one strike until their Sectionals, but arguably could have gotten their third there (it was very close).  But they did get their 3rd strike at Nationals with my ratings showing all their Nationals matches played rated two levels above the level they were playing!  If someone can do that, they probably shouldn't have been playing at the level they were and the system should have caught that.

Another player from SoCal was DQ'd, they went undefeated in 18+ until Nationals where they lost one match in the semis.  They did lose two Tri-Level matches during the year, but had half of their Nationals matches rate two levels above the level they were playing and they very well could be double bumped.  My ratings also say they perhaps should have been DQ'd prior to Nationals but they weren't.

The third from Missouri Valley was DQ'd, they went 5-1 at Nationals with half their matches rating two levels above the one they were playing, and my ratings similarly say they should have been DQ'd prior to Nationals but they weren't.  They are also a borderline double bump candidate.

So here we have three players the system did not catch that my ratings say should have been, two may be double bumped, and the USTA 3-strike DQ system did not flag them and make them ineligible.  Why not?

Note that none of these players clearly tanked matches, so it isn't like they gamed the system to avoid strikes and suddenly turned it on at Nationals.  They did well at Nationals, but had done well during the year too.

It is possible the strike threshold is higher than I think it is and that is why they didn't get strikes.  If so, I'd suggest the threshold is significantly too high.  Shouldn't players who can play several individual matches with ratings two levels higher be flagged as ineligible sooner.  And they all had similarly high rated results during the year.

It is possible that my ratings are too high and I'm making a big deal out of nothing.  But when these players were undefeated or nearly so, the only losses being to other questionable players, I don't think it is "nothing".

Did the players perhaps accumulate 3-strikes but it was swept under the rug?  I doubt this, as while strikes in local league and district/state/sectional playoffs are managed by local staff, I'd think National staff would review eligibility for Nationals and anything would be caught there.

Now, for 2020 the USTA has instituted a new rule to try and preclude teams from "hiding" self-rates and appeals and qualifying them for Nationals with few matches played, but it is important to note that all of these players played 12, 6, and 10 matches respectively, well over the new rule's 4 requirement, so the new rule would have had no impact on their eligibility.

What do you think?  Is this a problem?  Is someone being DQ'd after Nationals an indication the system is broken?  Should the system have found players like the three above ineligible prior to Nationals?  Is the strike threshold too high if players like these can make their way through to Nationals?

Wednesday, October 9, 2019

Simulating 2019 USTA League Nationals - 40 & Over 4.0 Women

As I did last year, I will be doing simulations of each Nationals to predict who the most likely four teams are to make the semi-finals, and also look at other interesting things that may occur.  Last year, the simulations did a pretty good job predicting who would advance to the semis including having all four teams predicted correctly several times.

Why do these simulations you ask?  The primary reason is that the format for USTA League Nationals is now a flight-less random round-robin where each team plays four other random opponents.  This introduces significant variations in schedule strength, the possibility of an undefeated team not making the top-4, and teams vying for the top-4 perhaps not having played head-to-head and unfortunate tie-breakers being used.  The simulations aim to educate folks on how it all works and look at what may happen.  Also see this write-up for some things to know about Nationals.

Nationals are approaching the second weekend of competition, the 40 & Over 4.0 women's event being held in Surprise, AZ.  Here is what the simulation says is likely to happen.

As a reminder, my simulations are done using my Estimated Dynamic NTRP Ratings and looking at the average rating for the top group of players on each team, the actual schedule each team will play, and then doing a million simulations of the matches with some random variation in each team's expected result.

First, with 17 teams playing a random four opponents, it will always be possible for there to be five (or more) undefeated teams.  It is possible to limit it to five with the right schedule, but the USTA did not do that this time around and 52 of the million simulations had six teams undefeated.  The chances of five is just 0.5% and four is just a modest 9%.

Four undefeated would be a nice and tidy result with no drama, but there is a 50% chance of just three so we are more than likely to have some tie-break scenarios.

That leaves a good chance, 87%, that there will be a tie for the last spot and it come down to tie-breakers.  That tie is likely at a 3-1 record.  The most likely size of the tie is three at 32%, but four has a 30% chance, five 17%, and two 15%.  The chances of larger ties diminish pretty quickly, but a 11-way tie is theoretically possible, but it appears a multi-way tie is a very high probability and it could be reasonably large.

Should there be a tie on team record, it comes down first to who has the best court record, then head-to-head (if applicable), then to who lost the fewest sets, then who lost the fewest games, and finally percent of games won.  There is a 20% chance it comes down to the sets lost tie-breaker, and there is a 80% chance that is between two teams and a 19% chance it is between three teams,  If the controversial tie-breakers are going to come into play, this is the likely spot.

The schedule strengths do vary a bit but not too much, the team with the easiest schedule having an opponent average of 3.96 while the team with the toughest schedule having an opponent average of 4.07.  This leads to an easier road for some teams than others, in the strongest team has the second easiest schedule.

So who is most likely to come out in the top-4?  Southwest, Intermountain, Middle States, and Texas head the list, Southwest the most likely to make it and the favorite to win it all.  Eastern, Southern, and Mid-Atlantic are ready to step in if anyone slips up.

For those interested, I offer a variety of reports to make Nationals more fun and help captains prepare.  I have a Simulation Report that has all of the details of the simulation including the average ratings for each team, each team's schedule strength, the most likely record for each team, and the chance of each possible record for each team.  I also offer reports to help teams scout opponents in more detail, both a Flight Report with full roster averages, top-8 averages and played by court averages for each team, as well as full Team Reports with detailed ratings for each rostered player and stats who who plays with who and on which court and how they do together.  Contact me if interested in any of these reports.

Simulating 2019 USTA League Nationals - 40 & Over 4.0 Men

As I did last year, I will be doing simulations of each Nationals to predict who the most likely four teams are to make the semi-finals, and also look at other interesting things that may occur.  Last year, the simulations did a pretty good job predicting who would advance to the semis including having all four teams predicted correctly several times.

Why do these simulations you ask?  The primary reason is that the format for USTA League Nationals is now a flight-less random round-robin where each team plays four other random opponents.  This introduces significant variations in schedule strength, the possibility of an undefeated team not making the top-4, and teams vying for the top-4 perhaps not having played head-to-head and unfortunate tie-breakers being used.  The simulations aim to educate folks on how it all works and look at what may happen.  Also see this write-up for some things to know about Nationals.

Nationals are approaching the second weekend of competition, the 40 & Over 4.0 men's event being held in Surprise, AZ.  Here is what the simulation says is likely to happen.

As a reminder, my simulations are done using my Estimated Dynamic NTRP Ratings and looking at the average rating for the top group of players on each team, the actual schedule each team will play, and then doing a million simulations of the matches with some random variation in each team's expected result.

First, with 17 teams playing a random four opponents, it will always be possible for there to be five (or more) undefeated teams.  It is possible to limit it to five with the right schedule, but the USTA did not do that this time around and 81 of the million simulations had six teams undefeated.  The chances of five are a fairly high 3.4% and four is a very high 22%.

Four undefeated would be a nice and tidy result with no drama, but there is a 44% chance of just three so we are more than likely to have some tie-break scenarios.

That leaves a good chance, 77%, that there will be a tie for the last spot and it come down to tie-breakers.  That tie is likely at a 3-1 record.  The most likely size of the tie is five and four, both at 31%, but three and six way ties are possible at 16% an 14% each, and a 7-way tie is more likely than a 2-way tie at 4% vs 3%!  The chances of larger ties diminish pretty quickly, but a 9-way tie is theoretically possible, but it appears a multi-way tie is a very high probability and it could be large.

Should there be a tie on team record, it comes down first to who has the best court record, then head-to-head (if applicable), then to who lost the fewest sets, then who lost the fewest games, and finally percent of games won.  There is a 23% chance it comes down to the sets lost tie-breaker, and there is a 73% chance that is between two teams and a 23% chance it is between three teams,  If the controversial tie-breakers are going to come into play, this is the likely spot.

The schedule strengths do vary a bit, the team with the easiest schedule having an opponent average of 3.97 while the team with the toughest schedule having an opponent average of 4.12.  This leads to an easier road for some teams than others, in fact two of the stronger teams got the easier schedules.

Note that as the size of the potential ties tells us, this is a very competitive group of teams at the top with the top-5 teams separated by just 0.04 on average rating.

So who is most likely to come out in the top-4?  Texas, Mid-Atlantic, Intermountain, and MoValley head the list, Texas the most likely to make it due to their schedule, but not necessarily the favorite to win it all.  Middle States, NorCal, and Northern are all lurking just waiting for any of the top-4 to slip up.

For those interested, I offer a variety of reports to make Nationals more fun and help captains prepare.  I have a Simulation Report that has all of the details of the simulation including the average ratings for each team, each team's schedule strength, the most likely record for each team, and the chance of each possible record for each team.  I also offer reports to help teams scout opponents in more detail, both a Flight Report with full roster averages, top-8 averages and played by court averages for each team, as well as full Team Reports with detailed ratings for each rostered player and stats who who plays with who and on which court and how they do together.  Contact me if interested in any of these reports.

Weather Report - 2019 USTA League Nationals Week 2 - Vegas, Arizona, OKC

The second weekend of 2019 USTA League Nationals is nearly upon us, and it is always a good idea to do a weather check to see if any issues or delays may arise.

There have been several weather issues the past few years with questionable sites, bad luck, and limited backups plans, but this year the USTA has picked sites far less likely to have weather issues and I'm told has much better back-up plans in place.  I think all of the players appreciate this, and I'm hoping for the best at all the events.

Last week's events were held in Vegas and Surprise Arizona and I didn't do a weather report, because, well, when does it ever rain there!  Ok, it does on occasion and when it does it can be heavy rains, so it is best.

So what is the forecast for this weekend?

First, Oklahoma City is hosting its first event and while there is an 80% chance of thunderstorms on Thursday, the weekend looks dry, albeit a bit windy and cooler.


In Vegas, everything looks comfortable and dry, with a bit of breeze perhaps.


And warmer, but dry, conditions in Surprise.


So it appears we have should have uninterrupted tennis at all three sites!

Monday, October 7, 2019

Simulating 2019 USTA League Nationals - 18 & Over 3.0 Men

As I did last year, I will be doing simulations of each Nationals to predict who the most likely four teams are to make the semi-finals, and also look at other interesting things that may occur.  Last year, the simulations did a pretty good job predicting who would advance to the semis including having all four teams predicted correctly several times.

Why do these simulations you ask?  The primary reason is that the format for USTA League Nationals is now a flight-less random round-robin where each team plays four other random opponents.  This introduces significant variations in schedule strength, the possibility of an undefeated team not making the top-4, and teams vying for the top-4 perhaps not having played head-to-head and unfortunate tie-breakers being used.  The simulations aim to educate folks on how it all works and look at what may happen.  Also see this write-up for some things to know about Nationals.

Nationals continue with week two this weekend, the 18 & Over 3.0 men's event being held in Oklahoma City.  Here is what the simulation says is likely to happen.

As a reminder, my simulations are done using my Estimated Dynamic NTRP Ratings and looking at the average rating for the top group of players on each team, the actual schedule each team will play, and then doing a million simulations of the matches with some random variation in each team's expected result.

First, with 17 teams playing a random four opponents, it will always be possible for there to be five (or more) undefeated teams.  With the teams and schedule we have though, there is hardly a chance at all, just 0.02% chance, and in fact there is just a 1.3% chance of four teams being undefeated which is quite low.

In fact, there is only an 18% chance of three teams being undefeated, so we are likely in-line for some serious tie-break scenarios.

That leaves a very good chance, 93%, that there will be a tie for the last spot and it come down to tie-breakers.  That tie is likely at a 3-1 record.  The most likely size of the tie is four at 29%, but five and three way ties are likely at 23% each, and a 6-way tie is more likely than a 2-way tie at 12% vs 6%!  The chances of larger ties diminish pretty quickly, but a 12-way tie is theoretically possible, but it appears a multi-way tie is a very high probability.

Should there be a tie on team record, it comes down first to who has the best court record, then head-to-head (if applicable), then to who lost the fewest sets, then who lost the fewest games, and finally percent of games won.  There is a 28% chance it comes down to the sets lost tie-breaker, and there is a 67% chance that is between two teams, a 28% chance it is between three teams, and a 5% chance it is between four teams.  If the controversial tie-breakers are going to come into play, this is the likely spot.

The schedule strengths do vary a bit, the team with the easiest schedule having an opponent average of 2.99 while the team with the toughest schedule having an opponent average of 3.08.  So not as much disparity as some of the other events.  Still, this leads to an easier road for some teams than others, in fact one of the stronger teams got the easiest schedule.

So who is most likely to come out in the top-4?  Southern, Mid-Atlantic, Texas, and Southwest head the list, Southern the most likely to make it but not necessarily the favorite to win it all.  Florida is very close, and Intermountain and Missouri Valley next with NorCal and SoCal in the hunt if things go their way.

For those interested, I offer a variety of reports to make Nationals more fun and help captains prepare.  I have a Simulation Report that has all of the details of the simulation including the average ratings for each team, each team's schedule strength, the most likely record for each team, and the chance of each possible record for each team.  I also offer reports to help teams scout opponents in more detail, both a Flight Report with full roster averages, top-8 averages and played by court averages for each team, as well as full Team Reports with detailed ratings for each rostered player and stats who who plays with who and on which court and how they do together.  Contact me if interested in any of these reports.