Mandatory: Sim testing

[JiriHrdina]

OK folks,
I've posted a folder on the CPHL site for you to each download and see the results of the sim testing.
I've tested 5 different scenarios and for each scenario I've simmed 10 seasons.

Within the folder you'll find a folder for each scenario, and then 10 folders underneath with each of those seasons.

Within those folders there are a bunch of files - with the most important ones being the standings, and scoring leaders.

Each GM must review these results (or as much of the results as you want) and then tell us which scenarios they prefer.

The folders can be downloaded here
cphl.calgarypuck.com/download/Coach.zip

Ignore the fact that name of the zip file is Coach - that was just the easiest way to get the files on the FTP site.

I've set up a quick survey to capture your feedback:
http://www.surveymonkey.com/s/H32TXH8

This is mandatory as it represents a very important decision for this league going forward.

I have my own thoughts on each scenario including what I've observed but I'll hold off sharing those until after the survey results are in.

Questions?

Let's set next Friday for a deadline excluding any GMs that are away.

[devo22]

thanks for doing this! Lots to discover here, will take some time to really evaluate it properly ... will try to do it this weekend.

[JiriHrdina]

BTW Injuries and CON were turned off for all tests. Since I was simming full seasons in one go - the sim would stop halfway when a team had too many injuries to fill a roster.

[Da_Chief]

So far not so random as far as where the Isles finish.

[TurdFerguson]

Have not looked yet, but how many league championships for the Stars? 7...8?

[Swayze11]

Have not looked yet, but how many league championships for the Stars? 7...8?

We don't sim the AHL anymore, remember?

[Caged Great]

some of them have a lot of variation in the tests. One of them had me finishing dead last in one run through and winning the division in the next under the same conditions. That kind of randomness is probably not good. I ranked them all in terms of consistency of results based off of where teams should finish based off their rosters and how random the results were..

[Daradon]

Wow! Nice job, much thanks.

I'll get on it on Sun and Mon.

[Flickered Flame]

Just a question, how many settings did you adjust/change in each of the 5 different scenarios? And if you only changed one setting at a time, did you change it back to whatever it was before running to next scenario?

[JiriHrdina]

Each scenario represents combinations of key settings and the sim was reloaded from the start for each test

[mbeauchamp76]

As far as standings go here is the average point totals over 10 seasons per scenario for the Eastern Conference. Will hopefully get to the Western Conference soon. I found that there was more variation between the seasons within the scenario than between the scenarios themselves (taking into account the average). Some teams had as high as 40 point variations between seasons within the same scenario and like Matt said finishing anywhere from 1st to last in their division. Things average out somewhat over many seasons. Again there is a ton of factors remaining constant and no one will ever know how much that effects the overall results. Nonetheless this is a very interesting exercise. NYI finished 1st in the East 39 out of 50 tests and finished below 4th once.


http://i1166.photobucket.com/albums/q603/cphl9/10SEASONAVGPTSbySCENARIO-EAST_zpsefe6ae2f.jpg

[mbeauchamp76]

If you guys would like to see more of this let me know and I will be glad to post all scenarios for both conferences

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioAAnalyis-East_zps8e72632f.jpg

[silentsim]

If you guys would like to see more of this let me know and I will be glad to post all scenarios for both conferences

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioAAnalyis-East_zps8e72632f.jpg

Please do for West very interesting to see this

[mbeauchamp76]

Should have everything up tonight

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioBAnalyis-East_zpsa254a71c.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioCAnalyis-East_zps026ff959.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioDAnalyis-East_zps0f92ef51.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioEAnalyis-East_zps29e8a11c.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/10SEASONAVGPTSbySCENARIO-WEST_zpse9575e0a.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioAAnalyis-West_zps13f426f6.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioBAnalyis-West_zps88985af3.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioCAnalyis-West_zpsfcf3967d.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioDAnalyis-West_zpsdd7c48a7.jpg

[mbeauchamp76]

http://i1166.photobucket.com/albums/q603/cphl9/ScenarioEAnalyis-West_zps7cf67b11.jpg

[MJK]

THIS IS AWESOME! There is no way I can ever thank you enough for posting this and this is exactly what I needed to see (visual) to get my head around all these stats.

Using this I now feel i can make an informed decision.

You rock!

[GreenLantern]

Thanks for posting all those graphs, made things a lot easier.

[Cheese]

Clearly the SIM is not a fan favorite of the Wild.

[JiriHrdina]

Clearly the SIM is not a fan favorite of the Wild.

Yup but that also means it isn't some random thing. It is a clear problem with something I've done with the current construction of the team. Falls to me to figure what that is and how to fix it. But it means it is solvable.

We have about half of the league responding so far to the survey. Remember this is mandatory as it represents a big decisions for the league.

[MJK]

Jackets are in the same boat as the Wild pretty much but we are pretty sure we know what the issues are and have already used these results to add peices we feel will get us back in the playoffs.

Also where the Jackets finished in the tests is pretty much bang on where we thought we would be considering the roster we finished the season with and what OV's the test was based on.

[Hanna Sniper]

I'll be honest, I really cound.t decide at what angle to vate my opinion from. I tried to look at it two way...

1) have the best team have the chance to have the best records

2) Or have the bottom of the league unleash havoc and have better chance of upsets

[Knut]

For Whatever reason my survey will not let me make any changes to the order. I tried it on both IE and Chrome and on two different computers.

[Swayze11]

My vote will come in when I get home from vacation

[Daradon]

Made my submission.

To be honest, there doesn't seem like there is much variety in the results (which could be a good thing) we're just seeing standard deviation. I'm ok with whatever scenario is picked really.

[JiriHrdina]

Still missing some responses - I'll extend the deadline to Sunday night and post results Monday morning. It will reflect poorly on any GM who doesn't do this and didn't let us know.

[I wanna be like Miikka]

I'll do my submission later today, sorry for the delay.

Edit - done

[flamesfan6]

Oops. fell asleep early last night and forgot to do it. Will do it right now.

edit - done

[Flickered Flame]

Here is how I came to my selection:
I went through all the teams and determined who, in my opinion, are the playoff teams (top 8) for each conference. From there, I went through each scenario and tallied how many of my playoff teams made the playoffs in each of the scenario's simmed seasons.
In the end, there was one scenario that had my selected playoff teams make the playoffs just over 75% of the time. The other 4 scenario's didn't even come close to that percentage.

[Slava]

I found it quite difficult to choose. In the end I tried to choose the scenarios that I think gave fairly consistent results. There was a lot of variety of course, so I'm really interested in seeing what the tests were.

[MJK]

is it just me or were the average sim results pretty much bang on to who things should go? The problem is out of 10 there are 2-3 years where things are completely strange but for the most part things are good IMO.

[Swayze11]

back in the swing of things... Voted.

[Mitch]

Voted