NorCal USTA Un-flighted Round Robin - What should have been expected?

I just wrote about NorCal using the new un-flighted round-robin format for 40 & Over Districts last month where a 4.0 mens team went undefeated and is not going to Sectionals as there were four others and they finished 5th.

That seems like a bad thing, so I went about researching if this was a freak occurrence, or if it could have been known in advance that it was likely to happen.

I'll say up front, just looking at a situation where there are 22 teams, each only playing 3 matches against random opponents, it screams to me that it was likely for there to be a lot of undefeated teams, but let's do some real analysis to see just how likely.

To do this, I'm using my Flight Simulation Report that uses my Estimated Dynamic NTRP Ratings for the players on each team to go through one million simulations of each team's schedule ,with some random variability into how each team will do, tallying up what the most likely record will be for each team and what the standings will be.  Also, as part of doing this, I can look at how often specific situations occur, like what percentage of the time there are varying numbers of undefeated teams.

So what did it all say?


The simulation said the most likely resulting records would have 4 teams all 3-0 which would have been nice and clean.  The 4 teams in this most likely scenario were San Carlos, Bay Club, Seascape, and Walnut Creek and it turns out 3 of these teams did all finish undefeated.  So the simulation isn't too bad!

The chances of any 4 teams finishing undefeated was 41%, so pretty likely, but a lot of room for other scenarios.  Here is how the others were predicted:

• Three - 20%
• Four - 41%
• Five - 30%
• Six - 5%
• Seven - 0.3%
• Eight - 0.01%

Yes folks, with this format and the schedule they had, there was the possibility of eight (8) teams ending up undefeated!  Yes, it is a very slim chance, and similarly for seven, but still technically possible.

The bigger thing is that a still substantial six undefeated had a 5% chance, meaning if you hold this event 20 times it is likely that you'd have six undefeated teams once.  Having a format where that is reasonably possible is just silly!

The chance of five undefeated was an even higher 30%, nearly as likely as the most likely scenario of four undefeated.  This is in fact what happened and clearly it was not a freak occurrence but should have been reasonably expected it could happen.  Add up the chances of five or more and you are over 33%, so this is saying one of these was expected to occur in one of every three times the event would be played.

With this many teams playing just three matches, the chances of a tie for fourth place (the last spot advancing to Sectionals) is very high, and when it occurs, it is more often that not going to be very big.  Basically any time there aren't exactly four undefeated, you are going to have a large tie at 2-1.  The number of teams tied could in theory be as large as 16, but is more typically expected to be between 5 and 10 teams, still very big and highlighting the inadequacies of the current tie-breakers, especially when just three matches are played as that gives very little information to differentiate with.

What about other flights?

The women's 4.0 flight had 20 teams and fortunately only had three undefeated teams.  But what did the simulation say was likely?

Here are the chances:

• Three - 36%
• Four - 38%
• Five - 11%
• Six - 1.3%
• Seven - 0.05%
• Eight - 3 of the million scenarios

We see the chances of of "bad" situations is quite a bit lower, this is in part to having two fewer teams, but also the strength of the teams and how the schedule was laid out.  Still, there was a significant chance, 12+%, of five or more teams being undefeated even if one of the more likely three actually occurred.  Hold this even eight times it is likely that once you will have a "situation".

Another 20 team flight was the 3.5 women.  This ended up having a nice and clean four undefeated teams, but what were the possibilities?

• Three - 38%
• Four - 35%
• Five - 10%
• Six - 0.7%
• Seven - 0.01%
• Eight - 1 of the million scenarios

This is quite similar to the women's 4.0 flight, and one of the most likely scenarios occurred.  But there was still an 11+% chance of five or more, so it is likely to happen one of every nine times.

In fact, there was a team that was 2-1 in 5th place that lost a team match 3-2 on a super tie-break loss.  Had they won that super tie-break, there would have been five undefeated teams and controversy would ensue again.

I did not go through the other flights, there were fewer teams there so less chance of problems occurring, although the chances were by no means zero.

I think the above shows though that it was a little short-sighted to implement this format the way they did.  So many teams playing un-flighted round-robin against only three opponents is ripe for this to happen and it seems having a format planned with these problems built in is not really fair to the participants.

Simply playing four matches per team would have significantly improved things and reduced, but not eliminated, the chances.  I wonder why that wasn't done?

What do you think?

NorCal USTA uses un-flighted round-robin for Districts, undefeated team doesn't advance!

I recently wrote that the new format used for Nationals last year, un-flight round-robin, is starting to be used for other earlier rounds of playoffs.  Several states in Southern are using it, but I also noticed that Northern California used it for their Districts last month and taking a closer look, found some interesting situations.

Specifically, Nationals generally has 17 teams (one per section) and each team playing 4 matches, and this is enough to have some potential disasters such as very large groups of teams going into standings tie-breakers to see who advances, or even undefeated teams not advancing.

NorCal appears to have used this format with as many as 22 (!) teams and each team only playing 3 matches.  In my mind, this is a recipe for disaster as with that many teams, there is no way to avoid the possibility of 5 (or more) undefeated teams (and only 4 move on to Sectionals), and it is hard to get meaningful differentiation in the standings playing only 3 matches, especially given teams would have wildly different strengths of schedule.

But instead of me just raising the warning flag, let's take a look and see what actually happened.

Men's 4.0

This flight had a whopping 22 teams and disaster did occur here with five teams finishing undefeated at 3-0 and it appears, one of them not advancing on to Sectionals.  Teams 3 thru 5 were all tied at 10-5 on courts and they obviously didn't play each other, so it went to sets lost and one team had lost 13 sets vs 12 and 11 for the other two teams, so they were the one left out.  FWIW, my proposal I submitted to the USTA of using sets won/lost differential would have yielded the same order in this case.

But this is still brutal for the team that is left out.  You go to Districts and don't lose a team match, and aren't going on to Sectionals.  They very well could have had a tougher schedule than the other four teams and that is why they lost an extra court or set or two.  It is hard to understand why a format would be chosen like this with the likelihood of this happening so high.

Update: See this where I looked at the possible and expected scenarios for this scenario and a few other flights.

Men's 3.5

"Only" 18 teams and the standings ended up with three undefeated teams at 3-0 and a seven way tie for the fourth spot at 2-1.  This meant the flawed tie-breakers used kicked in, although in this case one team had a clearly better courts record at 12-3 vs the next best 9-6, so they advanced.

Men's 4.5+

Just 14 teams, three finished 3-0 with a three-way tie for the fourth spot.  One team had a one court better record on courts won/lost so they advanced, but is interesting that they had the same sets lost as the fifth place team and actually lost more games, so would have lost out on the flawed tie-breaker had it gotten to that point.

Men's 3.0

Just 12 teams, the chances of a situation with undefeated teams not really there, but there was a four way tie for third with the teams finishing 4th/5th tied on courts at 8-7, but one lost one more set (and won fewer too) so did not advance.

Women's 4.0

A good sized 20 teams in the flight and three undefeated teams at 3-0 leaving an eight way tie for fourth.  Worse, five of these teams were tied on courts at 9-6.  One team had lost one fewer set, so they were the team that advanced.

Women's 3.5

Also 20 teams in the flight and they lucked out with four 4-0 teams advancing.  There was a big seven-way tie for fifth giving an idea what kind of tie-breaker could have come into play.

Women's 3.0

There were 16 teams, just two went undefeated with a six-way tie for third at 2-1.  Two teams were both 10-5 on courts with the others behind so these two advanced.

Women's 4.5+

Just 14 teams, three teams undefeated with a three-way tie at 2-1 for fourth, one of those teams having a better courts won/lost record.


So there was not controversy at every level, but we did have (to my knowledge) the first case of an undefeated team being left out in this format, and several other very large tie-breaks where strength of schedule played a key role in who advanced since it came down to just a single court or even set.

I wonder if the USTA/NorCal really gave any thought to the possibility of sending an undefeated team home?  It was almost inevitable with this many teams and playing just three matches doesn't give much data to differentiate on and is very susceptible to strong/weak schedules skewing things.  I would hope they did, but then they apparently decided they were ok with it.

What do you think?  Is so many teams and so few matches taking this format too far?  Did you play at NorCal Districts and have first hand experience or feedback?

Some USTA districts are using un-flighted round-robin for 2019 playoffs - Beware of the standings tie-breakers!

Last year, USTA League Nationals used a brand new format, un-flighted round-robin, instead of having three or four flights of teams each playing a round-robin.  This meant that rather than four flight winners advancing to the semis, the top-four teams in one big flight would advance.

This format was great in that it guaranteed each team four instead of three matches, and gave a team, that was really good and just got a bad draw and lost early, a chance of still making the semis, but did introduce some new factors in deciding the top-4 teams.   Specifically, how do you compare teams that didn't play each other at all, to decide a top-4.

This meant some rarely used tie-breakers got invoked a lot, and unfortunately, in my opinion at least, these were flawed and choose the wrong team to advance several times.  I felt strongly enough about this that I proposed changes to the regulations to fix the tie-breakers, but alas they were not adopted, yet at least.

What is new for this year though is that this un-flighted round-robin (UFRR so I don't have to keep typing it) appears to be being used for more playoffs.  I've heard of at least two districts employing it later this month, and there very well could be more.

In most cases at the district level, I'm guessing the prior format was two flights of four teams, each team playing three round-robin matches and the flight winners facing off in a final.

In a UFRR, all the teams are put into one eight team flight and everyone plays four matches.  The fourth match is considered a perk to the teams as is the ability to advance to the final if you would have otherwise finished second in your flight in the old format.

The problem is, we still have the old tie-breakers that are going to be used.  And, depending on how the schedule is done, it is possible for three teams to all go undefeated but only two of those teams can advance to the final, and the current "challenged" tie-breakers will be used to determine those two teams.

I won't go into the details of what the problems are with the tie-breakers yet again (read this for a good summary), but I hope they don't come into play and pick the "wrong" team to advance.  My fear is that given the use of this format and tie-breakers enough times, it will happen again.

What format are your playoffs using?  Is a UFRR being used and if so, how many teams and how many matches?  What do you think of the format?

Leave a comment here or on Facebook.

Update: NorCal used the format for 40 & Over Districts and it resulted in a team that went undefeated not advancing to Sectionals.  Read about it here.