Wednesday, November 23, 2011

Stanford coach says BCS system is flawed; does he have a point?

Stanford's coach, David Shaw, took the opportunity of his weekly roundtable with the media to point out the flaws in the BCS.  Is there any validity to his criticisms?  Let's take a look at them one by one.

First, he says all he has heard is that the computers don't like Stanford.  Is this true and if so, is it justified?

Before the Oregon game, the BCS computer average had Stanford #8 with a high of #3 and a low of #10.  My regular algorithm had them #5 and my BCS algorithm #10.  The lower rankings were due primarily to their weak schedule to that point, #62 in the FBS according to my regular algorithm.  So, it isn't that the computers didn't like Stanford, it is just that they didn't like them as much as some other teams that played a tougher schedule to that point.  After the Oregon game, Stanford's schedule jumped to #36 in my regular algorithm, so had they won they would have been more liked by the computers and moved up.

Second, he says that to have Virginia Tech ahead of Stanford means the computers value the ACC more than the Pac-12.  That isn't necessarily the case, instead the computers value Virginia Tech's schedule more than Stanford's.  This is indirectly making a statement about the ACC and Pac-12 though, and unfortunately for the Pac-12, they didn't do well out of conference this year so this view is somewhat justified.

Regarding individual schedules, my BCS algorithm presently has Virginia Tech's schedule #34 and Stanford's #69.  Now, this will improve for Stanford after they play Notre Dame, but they are being hurt by playing 4-7 and #85 San Jose State and 3-8 and #94 Duke.  They are also being hurt by the BCS computers not being able to use margin of victory which is why my regular algorithm rates Stanford higher than my BCS algorithm.

As far as the Pac-12 vs ACC goes, my regular algorithm has the Pac-12 with a 72.241 average and the ACC with 69.602.  However, my BCS algorithm has the ACC ahead 78.170 to 77.501.  The difference again is due to margin of victory not being used in the BCS algorithm.  Since they don't look at margin of victory, they focus on just wins/losses and the ACC is 30-14 out of conference while the Pac-12 is 21-12, slightly worse.  This accounts for the difference.

Third, he mentions results of Virginia Tech and Stanford games against common opponent Duke.  Stanford won 44-14 at Duke while Virginia Tech won only 14-10.  This is a very valid point, but like I mentioned above, the BCS computers can't reflect that, which is to me the really big flaw in the BCS.  In my opinion, if you completely ignore the score but only look at who won/lost, you are unable to give a team enough credit when playing a weak opponent, and you also give a team too much credit when they have a close win over a weak team.

Fourth, he mentions quality losses, Stanford's to Oregon and Virginia Tech's to Clemson.  If one uses the BCS average, Oregon is #7 and Clemson #9, so this is a bit of a push.  However, if a system like mine is used that incorporates scores, Oregon is #4 and Clemson is #31 (their schedule is weaker than Stanford's was before the Oregon game), so Coach Shaw has an excellent point here.

So yes, Coach Shaw makes some good points, but the root of the problem is that the BCS computers can't incorporate margin of victory.  This means a team like Stanford that has won a bunch of games easily, played a decent but not great schedule, and lost their most important game, get penalized perhaps unfairly.  Boise State can make many of the same complaints.

So it boils down to if you want a system that focuses on wins/losses alone, or one that actually tries to rate the teams where they should be such that game results are predicted well.  The history of my algorithms, and Jeff Sagarin's by the way too, is that to predict games well a system has to use the score. Unfortunately, the BCS doesn't agree and we are stuck with the handicapped computers which will continue to result in the problems we have today.

Agree?  Disagree?

4 comments:

  1. I'm also run computer ranking site (www.CSSportsRank.com), and after experimenting with several different algorithms I have come to two conclusions:
    1) Margin of Victory is crucial to an accurate ranking, as long as you keep it under control. Winning by 35 for example, should be just as good as winning by 70, but is should be much better than winning by 1.
    2) Strength of schedule as traditionally calculated is okay, but there are better systems. I especially have a problem with the "opponents of opponents" part as that waters down the indicator.
    However, even though Margin-of-Victory is not allowed, there are ways around it... for example by giving credit for winning and then giving credit for defensive strength and offensive strength.
    As Jeff Sagarin has emphasized many times, models using MOV are king.

    ReplyDelete
  2. Cam, thanks for the comments, we definitely agree. I've seen the same thing with my two algorithms. Not including MOV simply makes it impossible to have as accurate a system.

    I'm not sure how one could give credit for defensive/offensive strength though without looking at MOV or some other stat which the BCS doesn't allow. As constructed, the BCS computers are simply handicapped.

    I took a look at your ratings and it appears you (IMHO) are giving too much weight to MOV or not applying diminishing returns enough. I don't think one can justify Boise State being #2 at this point, and I read your blog and see how you say they are artificially high because they played only 10 games. I'd hope a system would accurately rate a team regardless of games played?

    ReplyDelete
  3. Kevin. Explain to me why so many on message boards are going to these campaigns of "computers will jump OSU ahead of Alabama." I don't see it happening. Every major network is predicting a rematch. So therefore, either the talking heads are right or the computers are so off that they are about to rob the nation of the TRUE second best team (Alabama). Whats the math? Chances?

    ReplyDelete
  4. Matt, I would not expect Alabama to fall at all should they beat Auburn.

    Auburn is not great this year, but is still a 7-4 SEC team and is #36 in my regular system and #24 in my BCS algorithm, so Alabama will not be hurt in the computers by playing them and in fact may be helped a bit.

    Oklahoma State does get to play Oklahoma which is rated higher than Auburn, so one might think OkSt has a chance to move up, but they are already ahead of Alabama in most computers (they get the #2 computer spot already) so beating OU isn't going to improve their position with the BCS computers. To get ahead of Alabama in the BCS they'd have to get help from the voters, and presently they are a ways back.

    Even if OkSt gets all the clear #2 with the computers, their gap over Alabama is only 0.04 and they'd have to get within 58 and 30 points respectively in the Harris and Coaches polls to keep that lead. It would likely take Arkansas, Virginia Tech, and Stanford all losing for them to get that close and maybe even then they wouldn't, unless the voters decide they don't want a rematch and manipulate their votes to make sure Alabama drops.

    ReplyDelete