In November 2010 I wrote a blog about the impact that strength of schedule has on winning in the NCAA Football Bowl Subdivision. What I concluded was that strength of schedule does not have a statistically significant effect on winning percentage in FBS college football. So I thought that now would be a good time to revisit the strength of schedule argument, as this is the time in the season where teams have played a larger proportion of their games against poor performing FBS teams or against FCS teams. Thus this should be the time where schedule strength should have the greatest impact on winning percent. I collected the data for all 125 teams in the NCAA FBS (winning percent, points scored, points surrendered and a measure of strength of schedule) through the third week of the 2013 regular season, since this seems to be a time period for strength of schedule to have the greatest impact on winning percent.
To investigate what the effect of strength of schedule has on NCAA FBS winning percentage, I conducted a statistical analysis using linear regression. The regression just fits a line to the underlying data and estimates the impact and statistical significant (if any) of the independent variables on the dependent variable. For our purposes, the dependent variable is winning percentage and the independent variables are points scored, points surrendered and strength of schedule.
The results are that points scored is positive and statistically significant (has a statistical impact with at least a 95% level of confidence), that points surrendered is negative and statistically significant, and that strength of schedule is statistically no different from zero using a 95% confidence level. I have also adjusted for heteroskedasticity (unequal scatter in the data) and adjusted for the different FBS conferences and in each case the overall conclusion remains the same.
In other words, for the "league" as a whole schedule strength does not affect whether teams win or lose. Another way of thinking about this is that high quality teams win against high quality opponents and low quality opponents and low quality opponents lose to both high quality and low quality opponents on average. This schedule strength issue seems to be a reason to discount the performance of "inferior" teams - think Boise State and Utah a few years ago or most likely Louisville this season. Yet while this may be a prevailing attitude among many college football fans, especially fans of the "high class" conferences - this perception does not hold up to statistical scrutiny.
I agree with the ultimate conclusion that for the "league" as a whole schedule strength does not affect whether teams win or lose.
ReplyDeleteHowever, I think your conclusion can be confusing because most people just think of Win % and strength of schedule. The average person interprets your conclusion as 12-0 against high quality teams is the same as 12-0 against low quality teams. I think that is obviously false assuming the scores in each situation were the same.
I think a better communicated conclusion would be 12-0 against high quality teams with a small point differential is equivalent to 12-0 against low quality teams with a large point differential.
Sean - thanks for your comment. In the analysis I include points scored and points against (which is going to be the same as point differential). So point differential is controlled for and the measure that I use for strength of schedule does not hold up statistically for the "league". I will look into teams in different conferences or those at the top and those at the bottom in the future.
ReplyDeleteStacey, isn't there an endogeneity issue here? I mean, isn't points scored and against a function of who you play? Yes, if I outscore good teams by an average of 10 points or bad teams by 10 points, my probability of winning is the same, but I'm less likely to do the former. The reason you're finding strength of schedule to be insignificant is because point for and against is a better predictor which already takes into account the teams you play.
ReplyDeleteJason - I agree that points for and points against is a better predictor (that is how wins are defined). That is also part of the point, that looking at something else (such as strength of schedule) beyond points for and against is not giving you much. In this case, statistically SOS is zero. If we look at how points scored and points against are determined, we can link that directly to actions taken on the field (not strength of schedule). I hope to get the next version of the paper done soon and sent off to give an opportunity for more constructive comments - just like yours.
DeleteOK, but I remember my Junior High teacher telling our football team "There is one sure path to victory, just score more points than the other team". Easier said than done, right? One way to do it is to schedule bad opponents. So wouldn't this mean that SOS "matters"?
Deletealmost all teams play in a conference, hence some of those teams are likely to be above average and some not. If all teams were independent, then it is possible, but right now 119 are affiliated with a conference for a majority of the games played, so given this constraint, SOS doesn't matter in a statistical sense.
DeleteJason - one other thing to note is that good teams will tend to have a weaker strength of schedule since they cannot play themselves, and weaker teams will have a stronger strength of schedule since they cannot play themselves. This is just an artifact of the way in which leagues are structured.
DeleteIt seems to me that any problems with the results would stem from your model of strength of schedule. More specifically, ranking all FCS opponents as 126 makes the SOS measure problematic, especially when looking at a timeframe with such a high number of FCS opponents. I wouldn't say that, for example, NDSU and Florida A&M are a comparable schedule challenge.
ReplyDeleteI think it would be interesting to isolate for only non-FCS opponents and to see if the lack of statistical significance holds. I know that in the past it has once those games are balanced with a conference schedule, but I'm interested to see if it holds for this year.
Jake - thanks for your input. I just ran the numbers for last season, and strength of schedule is still statistically insignificant. I plan on blogging about this on Friday of this week as well as how I did the analysis.
ReplyDeleteSOS does matter when you have multiple teams vying to get to a BCS NC game.Example would be if Alabama,Oregon and Oklahoma are all undefeated then you should take SOS into account because more than likely bama and Oregon would be in an OU out even though most SOS rankings have OU at a concensus #2 and Bama avg at #73 and Oregon at #101.Even if the latter went undefeated and Alabama had 1 loss they would still put Alabama in the NCG which is completely unfair...
ReplyDelete