[DEV] ISFL Forums
*Preliminary NSFL ELO Ratings - Printable Version

+- [DEV] ISFL Forums (http://dev.sim-football.com/forums)
+-- Forum: Community (http://dev.sim-football.com/forums/forumdisplay.php?fid=5)
+--- Forum: Media (http://dev.sim-football.com/forums/forumdisplay.php?fid=37)
+---- Forum: Graded Statistical Analysis (http://dev.sim-football.com/forums/forumdisplay.php?fid=153)
+---- Thread: *Preliminary NSFL ELO Ratings (/showthread.php?tid=6038)

*Preliminary NSFL ELO Ratings - Beaver - 11-15-2017

I'm posting this in the hopes of getting some feedback to improve the methodology so if you have any suggestions or questions I'd love to hear them and if you see anything that looks glaringly wrong point it out because there is a non-zero chance that I fucked up somewhere along the line. These are far from being finished (see the last section for some of the flaws) so don't get too butthurt if you think your team is lower than it should be, etc.

Alright so I've been working through historical ELO Ratings for the NSFL and I've settled on these parameters for now:
  • 1500 ELO will be average and each team starts out there, including expansion teams
  • Beating a better team will increase your rating by more than beating a worse team (ex: in Week 14 last season ARI beat LVL 45-0 but only gained 3 ELO points, in the first playoff round they beat OCO 27-7 and gained 16)
  • Margin of victory matters but has diminishing returns (ex: in Week 6 last season BAL was 104 ELO points better than SJS, beat them 20-17, and gained 7 ELO points; 2 weeks later PHI was 102 ELO points better than COL, beat them 40-20, and gained 26 ELO points)
  • Preseason games don't count and playoff games are weighted the same as regular season games
  • All games are zero-sum: if one team goes up 20 points, the other team goes down 20. This means that each week that every team plays the average ELO is 1500 and doesn't fluctuate.
  • 25% season-to-season regression (that is, the start of season ELO Rating is 75% of your previous end of season ELO Rating and 25% average)
[Image: E8CWcr5.png]

The line graph looked ugly as sin when I had bye weeks as breaks in the lines so the flat parts of the lines are typically when teams aren't playing. (PS: I hate whoever's idea it was to have bye weeks in Season 2)
S1W1 = Season 1 Week 1
S1P1 = Season 1 Playoff Round 1
Considering this is just preliminary I didn't fancy up the graph too much (x axis labels are uggo and I'm going to emphasize the demarcation between seasons more in the final version).

Current ELO Ratings:
1. Arizona - 1771
2. Orange County - 1628
3. Baltimore - 1571
4. Yellowknife - 1499
5. San Jose - 1490
6. Philadelphia - 1472
7. Las Vegas - 1329
8. Colorado - 1238

Some fun facts (keep in mind this is based on preliminary numbers):
  • Biggest ELO gain to date: Season 3 Week 5 when San Jose gained 80 points after beating Colorado 40-0
  • Biggest underdog to win: Season 3 Week 12 when Balitmore beat Arizona 26-23 with a ~19% win probability
  • Baltimore has the most wins as an underdog with 16
  • Arizona has won their only 2 games played as an underdog: Season 1 Week 9 against Orange County 19-7 and Season 1 Week 12 against Orange County 23-3
  • Las Vegas has been a favorite in just 5 of their 33 games played (15.15%), San Jose is just ahead with 7 in 46 (15.22%)
  • Yellowknife has lost as a favorite 16 times, most in the NSFL
  • Favorites overall are 115-71 (61.83%)
  • Yellowknife has been the steadiest team, never dipping below 1470 ELO (2nd highest floor behind ARI's 1501) and maxing out at 1576 ELO
  • Colorado has fluctuated the most, recording the 3rd highest peak at 1593 ELO (behind ARI's 1828 and OCO's 1670) as well as the lowest valley at 1238 ELO
  • San Jose has the lowest peak ELO of any team, maxing out at 1503 ELO in Week 2 of Season 1 and nearly matching that with 1501 in Week 4 of Season 2
Going forward some things I'm going to look at in an effort to improve these are:
  • Introduce home field advantage: this completely slipped my mind until I was typing this up so I'm posting all of this without taking home field advantage into account which is a glaring oversight, luckily Home/Away shouldn't be hard to pull from the data so it's just a matter of figuring out exactly how much home field is worth
  • Test the variables so make the ratings as predictive as possible (specifically how swingy the ratings are from game to game - don't want them too static but also don't want them to overreact)
  • Perhaps weight playoff games more than regular season games (I'm hesitant to do this because a team that makes the Ultimus and loses shouldn't have their rating crater below teams that didn't make the playoffs)
  • Whatever other ideas y'all may have

GRADED

*Preliminary NSFL ELO Ratings - 7hawk77 - 11-15-2017

Excellent write up.

Well done!

*Preliminary NSFL ELO Ratings - ErMurazor - 11-15-2017

Nice. I like this. You should include Arizona' preseason S1 ELO of -9000.

*Preliminary NSFL ELO Ratings - bovovovo - 11-15-2017

This is very awesome. I love the graph, I'm a sucker for data visualization Big Grin

I do think it's important to incorporate home field advantage because we know the sim puts a lot of value in it.


(11-15-2017, 06:19 PM)Beaver Wrote:Yellowknife has lost as a favorite 16 times, most in the NSFL

This confirms my world view



*Preliminary NSFL ELO Ratings - manicmav36 - 11-15-2017

This is awesome stuff! I'd love to see this through the end of the season. I imagine things will change quite a bit when you include home-field advantage.

*Preliminary NSFL ELO Ratings - AdamS - 11-15-2017

hot damn that's good work

*Preliminary NSFL ELO Ratings - 37thchamber - 11-16-2017

I suspect your k factor is a little on the high side. An 80 pt gain is a 160 pt swing. That's way too much variance for one result IMO and a possible cause for rank inflation. This would also allow you to weight playoff games more. I'd probably go for 24 in the regular season and 32 in the playoffs. Just need to make sure your margin of victory adjustments don't allow it to extend beyond those limits probably.

Home advantage is also a must. In any implementation of the Elo rating system where home field is a factor, it is used (typically by adjusting rankings in the calculation) because it hurts the rating system to exclude it (and you get rank inflation). I'd suggest a rank advantage of about 150 pts as a test. I think it was JuOSu who did it before but can't remember what adjustment was made.


Solid stuff though. You beat me to it with the margin of victory implementation (I was planning to use a variant of the system used for soccer) which was one of the things I noted as being missing from the last implementation we saw of Elo here.




(Also why does everyone call these ELO ratings? It's Elo. Elo is a dude's name.)

*Preliminary NSFL ELO Ratings - ADwyer87 - 11-16-2017

holy jamoly

*Preliminary NSFL ELO Ratings - JuOSu - 11-16-2017

Excellent stuff, though I disagree with some of your choices. You should go back and read my first post of my ELO ratings to see some of the different numbers I used. Otherwise great work.

*Preliminary NSFL ELO Ratings - Beaver - 11-16-2017

Thanks everybody, this is definitely a work in progress but it's a fun little project to play around with.

(11-16-2017, 03:39 AM)37thchamber Wrote:I suspect your k factor is a little on the high side. An 80 pt gain is a 160 pt swing.  That's way too much variance for one result IMO and a possible cause for rank inflation. This would also allow you to weight playoff games more. I'd probably go for 24 in the regular season and 32 in the playoffs. Just need to make sure your margin of victory adjustments don't allow it to extend beyond those limits probably.
Yeah that makes sense. I went with a k factor of 20 as a starting value since that's about what real life sports tend to be around so I think that 160 point swing was either just a massive outlier or my MOV adjustment is too lenient - bumping the k factor up to 24 turns that into a 192 point swing. The mean swing with these numbers is 42 points with a standard deviation of 32 so that 160 point swing was 3 and a half standard deviations above average. That standard deviation is a bit high for my liking but I'll revisit this after I make the adjustments, especially home field, as making the k value more precise is one of my priorities going forward.

(11-16-2017, 03:39 AM)37thchamber Wrote:Home advantage is also a must. In any implementation of the Elo rating system where home field is a factor, it is used (typically by adjusting rankings in the calculation) because it hurts the rating system to exclude it (and you get rank inflation). I'd suggest a rank advantage of about 150 pts as a test. I think it was JuOSu who did it before but can't remember what adjustment was made.
Yeah, this started out as an SHL project where I don't think there's a home advantage and I just completely blanked on including it. Unfortunately I'm a fairly casual NSFL member so I'm not super familiar with the sim we use. It looks like in an evenly matched game having home field would make win probability increase from 50% to 70% with a 150 point adjustment. Does that sound about right with the way the sim works? Time to do some testing.

(11-16-2017, 03:39 AM)37thchamber Wrote:Solid stuff though. You beat me to it with the margin of victory implementation (I was planning to use a variant of the system used for soccer) which was one of the things I noted as being missing from the last implementation we saw of Elo here.
I leaned heavily on this article (among others) in developing this and based my margin of victory adjustment on theirs (specifically footnote 2). This was fortunate for me because I had originally planned on going with something similar to the soccer system but I didn't really want a piecewise function out of my own laziness.

(11-16-2017, 03:39 AM)37thchamber Wrote:(Also why does everyone call these ELO ratings? It's Elo. Elo is a dude's name.)
That's a very good question. Habit, I suppose Wink

(11-16-2017, 05:15 AM)JuOSu Wrote:Excellent stuff, though I disagree with some of your choices. You should go back and read my first post of my ELO ratings to see some of the different numbers I used. Otherwise great work.
As usual you beat me to it lmao. I'll definitely dig that up, appreciate the heads up.