The 2017 FCS season is just eight days away — so it's time to talk about preseason FCS rankings. On this episode of the HERO Sports FCS Podcast on the HERO Sports Podcast Network, your host Jim Oxely talks to HERO Sports editor Colt Kesselring about how BennettRank rankings are calculated, what they mean, and how the heck James Madison comes into the season ranked third (?!) in the FCS.
MORE: 2017 FCS Preseason All-America Team
What is BennettRank? How accurate is it?
BennettRank is an objective, predictive, data-driven ranking algorithm. It is sometimes called an "enhanced-RPI" measure, in reference the Rating Power Index used by the NCAA college basketball tournament selection committees.
BennettRank is used to rank every team from every division of NCAA football, basketball, soccer, volleyball, hockey, baseball and lacrosse, and has consistently predicted the results of more playoff and tournament games than any other computer ranking system or human poll.
Need some proof? Here's where each major FCS ranking system had the four eventual semifinal teams ranked the week before the the 2016 FCS playoffs:
| BennettRank | Coaches Poll | STATS Media Poll | |
|---|---|---|---|
| James Madison (Champion) | 3 | 5 | 5 |
| Youngstown State (Runner-Up) | 6 | T-12 | 13 |
| Eastern Washington | 2 | 4 | 3 |
| North Dakota State | 1 | 3 | 4 |
Each of BennettRank's top three teams made the FCS semifinals. Every final four team was ranked sixth or better by BennettRank. This is not out of the norm for BennettRank.[divider]
How is BennettRank Calculated?
The BennettRank calculation is, on its face, pretty simple. It is based on two things:
50% Adjusted Margin of Victory (discussed later)
50% Opponent BennettRank
But wait… how can BennettRank use opponent BennettRank as a part of the calculation? It needs other teams' rankings to calculate each team's ranking? Sounds like a real chicken/egg situation. And it is.
BennettRank is calculated using what's called an iterative funciton. When game results are final and scores are entered into the system, each team's adjusted margin of victory changes, which changes the raw power scores* and each team's BennettRank.
Then the fun starts. Since 50% of each team's BennettRank is based on the the BennettRank of every opponent they have faced, it have to run the calculation again (since their opponents' adjusted margins of victory have also changed, which changes their BennettRanks). This next run changes all power scores and BennettRanks very slightly.
Then we run the equation again and Power Scores/BennettRanks change a little more. Then we run it again, and again, and again — until the power scores that underly BennettRank change by less than 0.1% with each iteration.
*Power score is the number behind the rankings. Power scores are calculated for each team, then the computer sorts the teams based on this number and the resultant order is BennettRank.[divider]
So . . . Why is JMU No. 3?
Preseason rankings are extremely difficult for BennettRank. As a completely data-driven ranking algorithm, BennettRank has a rough time when it doesn't have any data!
To get around this problem, BennettRank takes last season's final rankings and modifies them based on a rough strength of schedule estimate. The proxy BR uses for each team's SOS is the median ranking of every team in their conference. Preseason rankings are 85% last season's final ranking, and 15% the median ranking of every team in a team's conference.
This is just a starting point. By the end of the season, it will all shake out the same no matter where teams start the season. Good teams will prove themselves good, bad teams will prove themselves bad, and the world will keep spinning. Fear not, JMU fans, this is just another opportunity to prove how good the Dukes really are.[divider]
BennettRank vs. the RPI
To understand why BennettRank is so good at predicting playoff games, it's helpful to compare it to something fans might already be familiar with: the Rating Power Index, or RPI.
The RPI is employed by the NCAA college basketball tournament selection committees to select and seed the fields for their tournaments, and it shares some commonalities with BennettRank. In fact, BennettRank could be considered a super-charged, predictive RPI.
The RPI calculation is actually pretty simple. It only considers three things: a team's winning percentage (25%), their opponents' winning percentage (50%), and their opponents' opponents' winning percentage (25%).
So if a team wins a lot of games against teams that also win a lot of games, the RPI figures they're pretty good.
It's a good data point, but it's hardly predictive.[divider]
WHAT MAKES BENNETTRANK DIFFERENT?
1. Adjusted Margin of Victory
BennettRank uses Adjusted Margin of Victory as its focus statistic (AdjMV), rather than just wins and losses. A 45-0 win tells us something very different than a 10-7 win — the 45-0 victor significantly outplayed their opponent, while the 10-7 winner probably squeaked by. The RPI gives both games equal weight, but BennettRank considers the margin of victory.
There is a limit to how much credit a team can receive for racking up points in a big win though — a blowout is a blowout, after all. This is where the "adjustment" comes in to the Adjusted Margin of Victory.
The AdjMV number has a floor and a ceiling, so as to not allow aberrant or outlier games to overly influence the rankings. The ceiling means teams don't get extra credit for pouring it on — turning a blowout an even bigger blowout.
On the other end, there is a floor. One-point and two-point wins are automatically adjusted to earn the same amount of credit as a theoretical 2.5-point win. After all, o point is more important than the first point a team scores above their opponent.
2. Recent Games Get More Credit
Momentum is important, so BennettRank gives recent games more weight. In the FCS rankings, the current week's games are given 4% more weight than games the previous week. By the end of the season, games are typically 50-75% more important than the first game of the season.
3. Home Field Advantage
Margin of victory is adjusted for home-field advantage.
WHAT MAKES BENNETTRANK BETTER?
In short: strength of schedule. The underlying raw BennettRank scores are based on two components: adjusted margin of victory and opponent strength of schedule. Our research shows human polls overvalue wins and losses and do not give enough weight to strength of schedule.
It is unreasonable for a pollster to see every game of every team they are expected to rank, and thus has limited means to quickly assess strength of schedule. The BennettRank computer has no such problem.
WHERE DID BENNETTRANK COME FROM?
This is the fourth season for BennettRank. Each year new sports are added and the underlying algorithms are tweaked as more data is analyzed and the BennettRank IT infrastructure becomes more sophisticated.
Gregg Bennett, the founder of BennettRank, created the original algorithm for D1 women's soccer after watching his daughter play at the University of Washington, when he observed that none of the existing polls and rankings seemed to correlate well with what he saw on the field, and none were really very good at predicting tournament results.
The coaches polls seemed to do a reasonable job of ranking the Top 25, but there appeared to be no good rankings systems for all the other teams in the division.
It occurred to him that the basic concepts of the RPI were solid, but they could be very much improved upon. Gregg identified what he considered the weaknesses of the RPI and developed, from scratch, BennettRank.
