This is our fifth year of BennettRanking. This year we have expanded our ranks to include all major NCAA college sports; men's and women's; D1, D2 and D3. These sports include football, soccer, volleyball, field hockey, basketball, ice hockey, baseball, softball, and lacrosse.
We rank all teams in every division, every day throughout the season beginning week three or four. BennettRanks are calculated every day during the season for all 450 D3 men's basketball teams as well as all 351 D1 women's soccer teams.
Every year the methodology improves a little based on previous year results and observations. This year, we have been able to fully automate the formulas. BennettRanks are now calculated twice a day based on the results reported to the NCAA up to that time.[divider]
What was your background that enabled you to create BennettRank?
Now, many years ago, I created algorithms and technologies that ranked hospital performance. We ranked all 5,000 US hospitals on their quality of care, financial performance, patient satisfaction, and efficiency. Virtually all the hospital ranking systems today have some direct lineage to these early concepts and algorithms.
I was fortunate enough to be able to sell the technology and it's company and begin investing in small technology companies, of which today I am involved with over 40. [divider]
What got you started on BennettRank?
When my daughter began playing D1 Women's Soccer, I observed the subjective (Coaches Poll, College Soccer, Top Drawer Soccer) ranking systems appeared not to reflect what I was seeing on the pitch, specifically that strength of schedule was under-recognized.
The RPI was better in some respects but in my opinion had significant flaws. [divider]
What were the shortcomings of the RPI?
I saw three fundamental shortcomings of the RPI. First, the primary statistic used was a win or loss; it was binary. In soccer, you could win by 1 goal or 4 goals, it was all the same in the formula. The same with basketball; you could win by 1 or 50, it didn't matter. We believe strongly there is value in the margin of victory.
The catch is to frame it properly to give proper credit for a win, and not allow blowouts to overweight the game. As an example for soccer, we eventually settled on a minimum of 1.5 goals for a win with no more than 4 goals as the maximum allowed. Each sport has their own allowable corridor of minimum and maximum margins of victory.
Second, we believed that more recent games should be weighted more heavily in the formula than games played in the early season. Up to week 3 or 4 for most sports, games are often played out of conference and frequently out of division. These are games in which the teams is feeling things out, coaches are experimenting, and teammates are trying to figure out how to work with each other.
By week 3 or 4, things get more important with conference games beginning and the race to qualify for the tournament begins. As such, beginning week 4 we begin weighting each week's games 5% more than the previous week. By week 15 of the season, week 15 games are weighted 50% more than week 5 games.
The RPI takes all games played within the division the same. You could lose the first 10 games and win the next 10 games and the RPI would be the same as if the winning/losing streaks were reversed (assuming all else remains the same).
And third, there is a home field advantage and it can be measured. We give the away teams 3 points for football and basketball and .4 goals for soccer. E.G. for a draw in soccer, we would award the away team .4 goals and the home team -.4 goals in our formulas.
In addition to these three shortcomings, D2 and D3 have not had the benefit of an RPI; and there has been no ranking system, up to this time that has ranked all teams in a division. We are it![divider]
Can you give me more insight into how the BennettRank is calculated?
Yes, it is fairly sophisticated math involving iterative statistics but in it's simplest form; it is a function of the average adjusted margin of victory of a specific team and the average BennettRanks of that team's opponents.
Thus it is a function of a teams margin of victory and their strength of schedule; with the strength of schedule a function of the opponents BennettRank. [divider]
How come sometimes there are big differences between BennettRank and some of the Top 25 Polls?
When BennettRank and the Coaches Poll's are out of sync it is almost always because BennettRank gives more credit to strength of schedule and it usually is a team that is "unknown" very well to the pollsters. This is a team that has lots of wins by big margins, but not a team expected by the pollsters. The same happens in reverse. When we see teams highly ranked in BennettRank and NOT ranked by the Top 25 polls, almost always it is a team with losses, but usually narrow losses to highly ranked teams. [divider]
During the season, at what time do you begin producing BennettRanks?
This is totally dependent upon the number of games played. We like to see all teams playing 6 games before we flip the switch to begin automatically producing BennettRanks. This usually happens between week 3 and 4 for most sports, and week 7 for football. For football we have a different formula for the first 6 weeks which is incremental week over week based on where they were last week and their performance this week. [divider]
When are the BennettRank's calculated?
They are calculated twice daily, at 10:30 p.m. and at 7:30 a.m.. The most recent scores are uploaded from NCAA.com, and we rerun the rankings at these times. Rankings can change every day not only based on a specific teams performance, but also on the performance of all other teams in the division.
If you look at the rankings page on HERO Sports, and a game appears in the "streak bar" or on a team page; that game is included in the up to date BennettRank. [divider]
How well does BennettRank perform?
How well does any ranking system perform? We have our own internal view on this. We do not look over our shoulder at any of the other ranking systems; many are top 25 and subjective.
Early in the season, we take a peek just to make sure our algorithms are calibrated properly. At the end of the season we take "our" Top 25 and the Coaches Poll and RPI top 25 and track to see how well they do in the tournament. Suffice it to say, our results speak for themselves with numerous articles on this subject in the methodology section of HERO Sports.
For sure, statistically speaking, one could not say that neither the Coaches Poll or the RPI is better. Quite the contrary, we are near statistically significant in the reverse, that BennettRank has more predictive power in the tournaments. [divider]
How well do the various Coaches Polls perform?
We are quite surprised how well some Coaches Polls perform and equally as surprised at how bad some can be. Those that are bad place very heavy emphasis on record and seem oblivious to strength of schedule.
The RPI almost always comes in third behind the Coaches Poll and BennettRank in predictive ability at yearend. We find this surprising since the RPI is used for seeding in the tournaments thus have a strong tailwind in this analysis. [divider]
About the author
Gregg Bennett is founder of BennettRank and HERO Sports. He holds bachelors degrees in Sociology and Molecular Biology, and a Masters in Health Care Policy from the University of Washington. He also holds an MBA from Stanford University.
At one time, he held an active Certificate of Public Accounting Certificate (CPA). He is a serial entrepreneur, has started numerous companies and is currently involved with 40 small technology startups. He currently is Vice Chair of the Seattle Alliance of Angels, and Executive Chairman of HERO Sports.
You can reach him at [email protected]. He tries to answer all emails and thrives on the "discussion".
