Do USTA League self-rated players self-rate correctly? Do men sandbag and women over rate?

2016 USTA League Nationals are going on in full force, and unfortunately that means a few cases of questionable self-rated players comes up.  This blog post is not to identify any individuals or teams that have manipulated the rules, but instead to start analysis of self-rated players and how accurately they self-rate.

To start, I've taken a look at the self-rated players in 2015 to see how many ended up being bumped up or down.  While it is possible for a player to self-rate correctly and be bumped up or down, you'd expect the majority to end the year at the level they self-rated if they were at the right level, so for the purposes of this analysis, that is the criteria we'll use.

In each of the charts below, there is a bar whose height is the number of self-rated players by level that were bumped down, stayed the same level, or bumped up at year-end.  The annotation on each bar is the percentage of players at that level for that category.

First, here is a chart showing these stats for all self-rated players in 2015, men and women.

We see that more players self-rated at 3.5 than any other level, around 10 thousand of them, and 69% of those stayed 3.5 at year-end and 15% each were bumped up and down.  The 3.0s saw 65% of the just over 9 thousand players stay 3.0s and 21% bumped up and 14% down.  At the 2.5 level, no one was bumped down (I think think the USTA ever gives someone a 2.0 year-end rating) but a whopping 39% of 2.5s were bumped up.

Moving above 3.5, we see the trend change and a higher percentage of players rating correctly with over 70% at each of 4.0, 4.5, and 5.0, but we also see more players being bumped down rather than up.

Based on this, one could conclude that players at the lower levels have a harder time self-rating correctly, likely because players tend to naturally improve more quickly at the lower levels, than at the higher levels.  And that natural improvement can also explain the larger number of players being bumped up than down at the lower levels, but the higher levels have more bump downs is interesting.  Perhaps this is because some of the self-rate guidelines require some players to self-rate higher than their ability just because they have to check the 'played in college' box.

Now, the above chart is including both men and women.  I don't think it is safe to assume the self-rate behavior is the same for both sexes, so I took a look at them separately.  And they were more different than I expected!

Here is the chart for the women.

First, we see more women self-rate at 3.0 than any other level with nearly 6 thousand.  We also see that the number of players that stay at the same level, i.e. self-rate correctly, never gets to 70%.

On the bumps up vs down, while there are a lot of 2.5s that get bumped up, it is a lower percentage at 33% than the combined 39% above.  And this trend continues with more 3.0s being bumped down than up, 18% to 14%, and the gap becomes larger at 3.5 with 21% being bumped down and 11% up.  This gap gets even larger at 4.0.

So one could make the observation that it appears women are more likely to over rate themselves as evidenced by more being bumped down that up at every level a bump down is possible.

And then the men.

The men have a lot more that self-rate at 3.5 and there are even more 4.0s than 3.0s.  And then men have right around 75% of self-rates at the 4.0 and higher levels do so correctly.

On the bumps up/down we see a complete turnaround from the women with more players being bumped up at 2.5, 3.0, and 3.5 and it only becoming equal at 4.0.  A whopping 70% of self-rated 2.5 men get bumped up and nearly a third of 3.0s do.  Even at 3.5, nearly 20% were bumped up.

This would seem to support the notion that men are more likely to rate too low and be "sandbaggers" in an effort to win and care less about having a higher rating next to their name.  This is particularly true at 2.5 and 3.0 where far fewer men self-rate correctly than the women, but even at 3.5 where a good 71% of players self-rate correctly, those that don't err on the side of rating too low.

What do you think?  Does this match your observations in your area?  Are the observations by sex fair generalizations of how men and women self-rate?

Stay tuned for more, I plan to look at this data by section to see if any tendencies stand out.