[DEV] ISFL Forums
*Human Offensive Tackles: 3% is Significant - Printable Version

+- [DEV] ISFL Forums (http://dev.sim-football.com/forums)
+-- Forum: Community (http://dev.sim-football.com/forums/forumdisplay.php?fid=5)
+--- Forum: Media (http://dev.sim-football.com/forums/forumdisplay.php?fid=37)
+---- Forum: Graded Articles (http://dev.sim-football.com/forums/forumdisplay.php?fid=38)
+---- Thread: *Human Offensive Tackles: 3% is Significant (/showthread.php?tid=20363)

*Human Offensive Tackles: 3% is Significant - iStegosauruz - 03-23-2020

[div align=\\\"center\\\"]Background and Methodology – [/div]
I simulated 25,223 games so that you wouldn’t have to.

Several weeks ago, I simulated 25,199 games and tested the true value of a 550TPE bot offensive tackle. Most teams pay about $9,000,000 per season for two of those offensive tackle bots, but what I found was that teams are not only wasting money, they’re wasting potential success. Human offensive tackles all the way down to 200TPE are just as effective as the offensive line bots that teams pay for.

There were a variety of reactions to that experiment, but one of the most common were requests to do a second part of the experiment and test human offensive tackles past 550TPE and compare them to the highest tier of offensive linemen bots that teams can purchase which top out at 750TPE. This study is the result of those requests.

I attempted to make this study resemble the last one methodologically as much as I could. I used the Orange County Otters as the control team again. I tested their results against all other nine teams in the league at both home and away. I ran two sets of those 900 game simulation blocks for both home and away at seven different TPE levels. This means there are two sets of data for both the Otters at home and away against all other nine teams in the league. The total number of games tested should be 25,200 but there were a few errors and hiccups in the simulation engine along the way that resulted in me overshooting that target by 23 games.

The TPE levels I chose were 650TPE, 750TPE, 850TPE, 950TPE, 1050TPE, and 1150TPE. I tested those TPE values with all tackles weighing 340 pounds. I then tested the 750TPE build again with tackles weighing 310 pounds – the most a bot can weigh – and used that as my control study.

The builds I chose for my tackles were the following:

[div align=\\\"center\\\"][Image: SdSUIK6.png][/div]

[div align=\\\"center\\\"]Results [/div]
The first simulations pitted the Otters as the home team against every other team in the league. I ran two sets of 100 simulations at each TPE/Weight point to avoid the simulation crashing. The results with the Otters as the home team resulted in the following:

[div align=\\\"center\\\"][Image: jf3e1Lb.png][/div]

There is a marginal difference between the control group – 750/310 – and all other variations. When both sets of data were combined it had the lowest winning percentage of any of the variations at 73.96%. The average human offensive tackle winning percentage was 75.16%. That is a 1.2% difference. In previous study the 550/310 control group had a 72.56% winning percentage. There is a 2.6% difference between it and the average winning percentage of all human offensive tackle TPE variations between 650TPE and 1150TPE. This is important because although the control in this study is a 750TPE bot offensive tackle, only one team in the league – San Jose – currently uses that tier of bot offensive linemen.

These results represented graphically showcase the small improvement in winning percentage a human offensive tackle gives over a 750TPE bot.

[div align=\\\"center\\\"][Image: PU4SGC4.png][/div]

The results improve significantly in favor of human offensive tackles when you incorporate the data where the Otters were the away team.

[div align=\\\"center\\\"][Image: krslGj0.png][/div]

The control group in the away simulations had a 47.89% winning percentage. The human offensive tackles averaged a 51.92% winning percentage. This is a difference of 4.03% which is a highly substantial difference in comparison to other factors that influence a team’s winning percentage.

These results represented graphically:

[div align=\\\"center\\\"][Image: xF1nrcF.png][/div]

I then combined the home data and away data into one set of data. This means that each grouping had 1800 total games in each run respectively.

[div align=\\\"center\\\"][Image: nT3qZpn.png][/div]

The control group had a 60.82% winning percentage. All other variations had a higher winning percentage than the control group. The lowest variation winning percentage was 62.44% for the 1050/340 variation. This is probably due to an outlier in the simulation engine, however even that variation with the outlier won 1.62% more games total than the control group did. The average human offensive tackle winning percentage was 63.54% which was 2.72% higher than the control group.

These results represented graphically:

[div align=\\\"center\\\"][Image: LH4w74M.png][/div]

[div align=\\\"center\\\"]Conclusions 1.0[/div]
1. Human offensive tackles above 650TPE are worth about 2.5% more wins than bot offensive tackles with 750TPE.
2. There are two current active human offensive tackles in the league that are above 700TPE. On average they make $2.5 million. Teams can save, on average, $2 million per tackle spot with human offensive tackles if they currently employ human offensive tackles.

[div align=\\\"center\\\"]Deeper Dive[/div]

Since I had data from 25,199 simulated games, I also decided to look at the statistical performance at each variable combination when it came to fumbles and yards per game.

The data collection looks the same for this as it did for the win percentage analysis. Two runs of 100 games at each variable combination for both home and away.

I first looked at fumbles:

[div align=\\\"center\\\"][Image: qRQ0cHy.png]

[Image: yw3RFlY.png]

[Image: 59a8DEG.png]

[Image: IJhCJdr.png][/div]

Although there is a trend of fumble lost percentage decreasing as TPE increases across the home runs the trend is not particularly strong, reaching 0.28% at the highest gap. In the away runs the trend seems to be that fumbles increase as TPE increases; however, I have no logical reason for why that is the case. There must be some discrepancy in the sim between home and away fumbles that I can investigate at a later point. The trend seems to balance out again when combining all runs – there is not a discernable pattern at that point.

After looking at fumbles I looked at yards per game to see if the teams performed as well offensively at each variable group.

[div align=\\\"center\\\"][Image: 4nQXdqQ.png][/div]


In general, when looking at total yards per game the variable combinations generally had higher total yards per game, rush yards per game, and pass yards per game at all TPE levels. The largest gap in total yards per game was between the control group and the 1150/310 variation. The gap in total yards per game between these two marks was 9.5. These variations also had the largest gap in rush yards – 4.36 yards per game – and pass yards – 6.53 yards per game. On whole, however, there is not a particularly large gap in the performance of any of the variable combination groups when compared to the control group. The team performed very similarly across all simulations with all variable groups.

Graphical representation of total yards per game across groupings:

[div align=\\\"center\\\"][Image: uq7KR7i.png][/div]

Graphical representation of rush yards per game across groupings:

[div align=\\\"center\\\"][Image: 9XzL3Po.png][/div]

Graphical representation of pass yards per game across groupings:

[div align=\\\"center\\\"][Image: GpCZq1h.png][/div]

[div align=\\\"center\\\"]Conclusions 2.0 [/div]
1. Very similarly to last time teams should be encouraging new members to pursue being offensive linemen. Each team should have a priority on having at least two human offensive tackles.
2. The weight difference in human offensive tackles over offensive tackle bots provides a small boost to a team’s winning percentage. The average boost is around 3%, which although small in a vacuum is a sizable increase for small variable changes. Teams make changes every week trying to find the best strategy and sometimes a 3% increase in your chances to win is the make or break point.
3. Human offensive tackles are consistently cheaper than bot offensive tackles against the cap. Teams can save on average $4 million total by converting their bot offensive tackles to human offensive tackles. This change frees 5% of the salary cap to be used on other positions. These savings are magnified for teams using a 750TPE bot to get anywhere near the results of the 650TPE to 1150TPE human offensive tackles I examined in these simulations.
4. There is a slight trend of increasing yards per game performance in rush yards, pass yards, and total yards for increasing amounts of TPE allocated to offensive tackles. Since human offensive tackles can surpass 750TPE – the highest level a bot can reach – human offensive tackles have the highest ceiling for improving a team’s offensive performance.

[div align=\\\"center\\\"]TL;DR - Save money, increase performance. Use human offensive linemen.[/div]

[div align=\\\"center\\\"]Random Plugs [/div]
1. Collecting this much data takes an incredible amount of time. Each of the 25,223 sims takes 1200 milliseconds. That is approximately 8.5 hours in sim time, not factoring in the time it takes to swap from team to team and to program in the TPE variations.
2. All of my data is always open source. I encourage you to use it for your own purposes. You can find the data for this study here.
3. There are tons of quality offensive line prospects in the upcoming NSFL draft. Give them some love.

*Human Offensive Tackles: 3% is Significant - NicholasTheGreat - 03-23-2020

Australia has the Australian of the Year award which is awarded for Achievement and contribution of eminent citizens.

I believe we need an NSFL Person of the Year award which should be awarded to Istegosauruz for his work on improving the lives of offensive linemen throughout the league.

*Human Offensive Tackles: 3% is Significant - br0_0ker - 03-23-2020

(03-23-2020, 08:13 PM)NicholasTheGreat Wrote:Australia has the Australian of the Year award which is awarded for Achievement and contribution of eminent citizens.

I believe we need an NSFL Person of the Year award which should be awarded to Istegosauruz for his work on improving the lives of offensive linemen throughout the league.

I believe there is a most dedicated user award

*Human Offensive Tackles: 3% is Significant - Isidore94 - 03-23-2020

Super impressed with this work! great job man I appreciate some serious stats analysis

*Human Offensive Tackles: 3% is Significant - Opera_Phantom - 03-24-2020

This can change the league in some aspects, really.

*Human Offensive Tackles: 3% is Significant - CalvinGolladay - 03-24-2020

Most dedicated user definitely fits here, with the amount of time spent collecting the data for these articles

*Human Offensive Tackles: 3% is Significant - infinitempg - 03-24-2020

He actually did it. The absolute madman.

Can we bring back those media awards? This is amazing work.

I feel like the win percentages at home were a bit of a wash but looking at the away numbers I was stunned to see that much of a difference! Fumbles lost are definitely a scratch - and same with the yards. I might go and do a correlation study with the data you provided, though I've got my own projects to do right now. I'd also be interested to see how lower TPE OL do in this study compared to all the bots and high TPE OL.

Absolutely amazing work.

*Human Offensive Tackles: 3% is Significant - iStegosauruz - 03-24-2020

(03-24-2020, 10:20 AM)infinitempg Wrote:He actually did it. The absolute madman.

Can we bring back those media awards? This is amazing work.

I feel like the win percentages at home were a bit of a wash but looking at the away numbers I was stunned to see that much of a difference! Fumbles lost are definitely a scratch - and same with the yards. I might go and do a correlation study with the data you provided, though I've got my own projects to do right now. I'd also be interested to see how lower TPE OL do in this study compared to all the bots and high TPE OL.

Absolutely amazing work.

Thank you! I agree with your analysis on it for the most part. HFA is pretty strong in the sim engine it seems to moving the needle on home win% is fairly difficult.

I'm actually going to look at the value of HFA for my next project. I've got some ideas on how to reduce the impact and boost the away win%. Gonna see if the theories hold.

*Human Offensive Tackles: 3% is Significant - revuhlution - 03-24-2020

Seriously amazing work. I was just talking with someone about how far down this NSFL rabbit hole I've seen people go. This is on another level. The quality of the statistical analysis is impressive. Thanks for doing all this.