USTA PNW weighs in on the new 40 & Over 1 singles / 3 doubles format - National exhibits bad math/statistics

I just received a communication from my League Coordinator regarding the format change for the 40 & Over league to use a 4-court 1 singles and 3 doubles format.

The gist of the communication is that the 4-court format is confirmed as being used in the Pacific Northwest section for 2020 leagues.  But it also provided a link to an FAQ that was prepared answering a number of questions.

Please do read the full FAQ if you are interested, but here are a few highlights.

First, a statement from National on why the change was provided.  Here that is:

Background: The regulation proposal 1.04A from the FL Section for the 40 & Over League “team match” championship format of one singles match and four doubles matches was tabled to gather further data. For the past 5 years, players have been surveyed as to the preferred 40 & Over League “team match” format. In the 2018 player survey, players were asked if they preferred 2S/3D or 1S/4D for this league and it revealed that 60% preferred 1S/4D. This survey did not offer the 1S/3D option. In April 2019, a survey was sent to players age 39 and over, asking which “team match” championship format they preferred, and results showed 25% for 1S/4D; 26% for 1S/3D; 39% for 2S/3D; ie. greater than 50% preferred only one singles match.  In May 2019, League program providers and volunteers were surveyed with 58% preferring 1S/3D. Section League Coordinators were asked their choice at the May 2019 Joint Staff meeting. Two SLCs preferred 2S/3D, while all others preferred 1S/3D.

Summary: With convincing input received from players, providers, and Section League Coordinators in favor of one singles match, we’ve learned that it’s time to consider a change in the 40 & Over League “team match” championship format. This will address the majority of our players’ preference for a one singles “team match” championship format and the challenges with limited court availability for many facilities, the magnitude of league offerings, and the complaints that teams have difficulty in recruiting singles players. The 40 & Over League program is the second largest program and represents just over 25% of the total participation this year. Improving the player experience for this population is critical to the continued success and growth of this program.

So there we have the official explanation of why the change was made.

I do find it interesting that of the three formats considered, the one that had by far the highest percentage, 2S/3D, at 39%, is the one not chosen.  Instead, they elected to effectively add together the percentages of the two other options since they were both offering one singles court and make the conclusion that most people wanted just one singles court.

The flaw in this logic is that while yes, the 1S/3D and 1S/4D both offer just one singles court and together had 51% preferring them, that isn't necessarily sound use of statistics.  It is simply not sound statistics to add together the results of two separate survey questions that themselves are composite questions, and conclude the common piece of the two questions is the reason everyone chose those options.  It is entirely possible that players chose one of those options not because of 1S, but for another reason.

For example:

• Someone may have voted for 1S/4D because it would get 9 players in a match and explicitly didn't vote for 1S/3D because it would get just 7 in a match, so combining the options is not what they would have wanted.  Their second choice would have been 2S/3D as it gets 8 players in a match.
• Someone may have voted for 1S/3D just because it was 4 courts, not because of 1S, and may have actually preferred a 2S/2D option for 4 courts, but that wasn't an option.  Had it been offered as an option, the survey would have been more equitable/sound rounding out the options, and the responses for 2S/2D and 2S/3D could be combined as both wanting two singles courts and very well been higher than the combined one singles options.
• The majority of 40+ players already play doubles primarily, just 25% of players in a 2D/3D format are playing singles, so any survey of this player base is naturally going to be biased towards more doubles courts.  Then offer multiple options that have just one singles court and only one option that is has two singles and there is no way the two singles courts was going to have more support than the sum of the one singles options.  Not taking this into account in interpreting the results is a major shortcoming.

So saying that over 50% prefer one singles court is not a conclusion one can or should make based on the survey they did and its results.  For the two reasons I highlight above and perhaps others, it is entirely possible and likely that the number that actually prefer one singles court is well below 50%.

[added after initial post] If one were to apply the same bad logic the USTA did though, we should add together the 25% and 39% to show that 64% of players prefer five courts, and thus one of 2S/3D or 1S/4D should have been chosen.  Yet this wasn't done and we get the option, 1S/3D, that arguably has the least support.

In my opinion, the USTA has exhibited a pretty bad grasp of statistics and surveys in how they elected to perform and interpret the results of this survey.  I don't know that this is the case, but it almost feels like they did the survey and then worked to find a way to come to the conclusion they wanted.

Note that with nearly 200 responses, the survey I did simply asking if people prefer the old 2S/3D format or the new 1S/3D format overwhelming said the old 2S/3D format is preferred with 83% in support of it as of my writing this.

The other odd thing is that the three options together add up to 90% and not 100%.  Sure, some people may have abstained from answering this question, but the numbers not adding up makes one go hmmm.

Getting back to the PNW FAQ, it goes on to discuss a few other things that I raised as issues in my analysis of the rules changes:

• Team match winners - The 4-court format can end in 2-2 ties, and while there are tie-breakers, PNW admits they won't break all ties and it says they are working on filling the gaps.  Kudos to PNW for working to address what National did not.
• Singles opportunities - PNW acknowledges that opportunities for those over 40 to play singles is cut in half, and says they will work to find and promote other opportunities such as Flex leagues and tournaments.
• Plus leagues - PNW also acknowledges the impact on plus leagues, especially with the change to only allow one plus player in 4-court matches.  They agree it is an issue on the opportunities for 4.5s to play singles and say they will work to do away with plus leagues in the future and offer an Open level for 5.0s and above in the future.

What do you think?  What are you hearing from your section regarding the change?  Will your local leagues follow this format and are steps being taken to mitigate the negatives that National did not address?