The trials and tribulations of Glen Gulutzan

[Jiri Hrdina]

How do you objectively measure things like "response to adversity"?
And I think you are mistaken if you think that management doesn't look at some sort of advance stats to determine decisions (though it may not be corsi specifically)

[Itse]

Originally Posted by Cali Panthers Fan

Well I don't agree that there are far too many outliers, but that's me. You're free to find fault if you require more data points, that's your prerogative. Even if there are outliers, they usually correct themselves by next season if the same pattern exists. You don't see teams that are continuously in the outlier area, so that suggests that over small sample sizes (and I would qualify an entire season as a small sample) that you can have teams that buck the trend.

Let's analyze this for a second. What are outliers? Teams that have bad corsi but win a lot (let's call this type A), and teams that have good corsi but don't win a lot (type B).

What are "not outliers"? Teams that have good corsi and win a lot (type C) and teams that have bad corsi and lose a lot (type D)

So when you're saying that these outliers "usually correct themselves IF the same pattern exists", what you're saying is that "outliers" will "usually" become "not outliers", or in other words:

- Type A teams become C or D
- Type B teams will become C or D. (For example: Oilers will either start doing better or their corsi numbers will go down.)

Well, which is it?

I think we're already starting to see the problem with this claim; when you say "if they keep having this corsi they'll get better results", you're not actually predicting the thing that's interesting; is this team likely to get better within an interesting timeframe?

But this isn't the end of it, because even if we assume that shots towards the net have absolutely nothing to do with winning hockey games (or: corsi and winning are completely unrelated), the claim "outliers usually correct themselves" would still be statistically true simply because that's how odds work. (And thus it's not a sign of predictive power.)

For example; if we assume that a team has a 50/50 chance of having a good season and a 50/50 chance of having good corsi, a team that had a bad season with good corsi has about 75% chance of EITHER having a good season OR having a bad corsi next season.

Also, let's remember that a good statistical analysis usually starts with eliminating outliers from the data and not by only looking at the outliers, such as the no good Oilers or any cup winner.

The worse a team is, the more likely it is to do better simply because it's not likely to do worse (well this of course doesn't really apply to the Oilers, but you know what I mean). Also if we assume that there is some correlation between shots and winning (which there probably is), then the more extreme an outlier you pick as your example (for example a cup winner), the more likely it is that even weak and generally irrelevant correlations will start showing up.

If you want to use outliers to prove your point, you would really need to show that EVERY outlier CLEARLY shows what you're talking about. Vice versa, if even the extreme outliers (the best and worst teams in the league) fail to lineup with your hypothesis, then your hypothesis isn't doing very well.

Here's a practical exercise; pick your next playoff brackets simply with corsi stats. If there's a notable correlation, you should have good results. I've done this, and I didn't even get half the first round winners right.

Which brings us to another problem that regularly shows up in pro-corsi argumentation; using past corsi to "predict" past results. First of all this habit is extremely prone to cherrypicking (or in other words; cognitive bias). Second, this isn't predicting anything. In other words: if you say that "cup winner C had good corsi", this is not a prediction. It's not in any way interesting. Past cup winners are likely to have all sorts of good numbers. That doesn't say anything about their predictive power for the next season.

If you assume that corsi and winning have some correlation, then of course you will have a situation where teams that did well during a certain stretch also had a good corsi during that stretch. This is not interesting information. It's like saying that there's often a lot of water on the ground when it rains.

If you say that corsi is a good predictor of future success, you have to use past corsi to predict future success. For example, by making that a playoff bracket based on how the corsi, or at the very least developing a system which is very good at predicting future results where corsi is a significant part of the equation.

[JerryUnderscore]

Originally Posted by Enoch Root

Well you listed a couple yourself. As did I earlier. Special teams, breakouts, response to adversity, etc. These are the things that determine the fates of coaches (along with wins and losses), not having corsi slip from 50 to 48%

Okay, well special teams can be measured objectively.

The powerplay is current at 22.2%. Good for 10th in the league. Since that's above average does Gulutzan get credit for it?

The penalty kill is 30th in the league at 73.8%. Obviously that's not good. Is that Gulutzan's fault?


How should we measure breakouts? Would that be measured in passing percentage when the pass originates in the defensive end? Do you have some data on that?

As for "Response to adversity" I have no idea how to measure that. After the defensive breakdown in Dallas, all three of his healthy scratches were brought in while Versteeg, Lazar and Kulak sat. They won in Colorado.

So did the team respond to adversity?

After the embarrassment in Detroit the team came back and won in Philly. Did they respond to adversity?

After losing 6-0 to Ottawa they came back at beat Vancouver 5-2. Was that a response to adversity?

The Flames haven't lost more than two in a row all season. Does that mean they're responding to adversity?

[Cali Panthers Fan]

Originally Posted by JerryUnderscore

Okay, well special teams can be measured objectively.

The powerplay is current at 22.2%. Good for 10th in the league. Since that's above average does Gulutzan get credit for it?

The penalty kill is 30th in the league at 73.8%. Obviously that's not good. Is that Gulutzan's fault?


How should we measure breakouts? Would that be measured in passing percentage when the pass originates in the defensive end? Do you have some data on that?

As for "Response to adversity" I have no idea how to measure that. After the defensive breakdown in Dallas, all three of his healthy scratches were brought in while Versteeg, Lazar and Kulak sat. They won in Colorado.

So did the team respond to adversity?

After the embarrassment in Detroit the team came back and won in Philly. Did they respond to adversity?

After losing 6-0 to Ottawa they came back at beat Vancouver 5-2. Was that a response to adversity?

The Flames haven't lost more than two in a row all season. Does that mean they're responding to adversity?

That would be an acceptable measure for me. Good teams don't tend to have prolonged losing streaks. Picking yourself up the next night seems to be a decent metric.

I would also include comeback wins when trailing after 1 or 2 periods, but since that gets into the problem of being behind often in games, it's maybe a negative indicator of failing to show up on time.

I could go on. As you see, these get pretty messy, as all measures of things that aren't direct on-ice events get messy when you try to quantify them. That's why I give Corsi and Corsi in context (Corsi relative, Score adjusted Corsi) a break when they're trying to determine a picture of what's happening out there.

[JerryUnderscore]

Originally Posted by Cali Panthers Fan

That would be an acceptable measure for me. Good teams don't tend to have prolonged losing streaks. Picking yourself up the next night seems to be a decent metric.

I would also include comeback wins when trailing after 1 or 2 periods, but since that gets into the problem of being behind often in games, it's maybe a negative indicator of failing to show up on time.

I could go on. As you see, these get pretty messy, as all measures of things that aren't direct on-ice events get messy when you try to quantify them. That's why I give Corsi and Corsi in context (Corsi relative, Score adjusted Corsi) a break when they're trying to determine a picture of what's happening out there.

I agree. My point is simply that we need objective ways to measure the effectiveness of the coaching staff rather than "in my opinion they don't look ready to play".

[Enoch Root]

Originally Posted by Wild GM

How do you objectively measure things like "response to adversity"?

Not all things can be measured quantitatively. Sometimes you just make as assessment that they are performing up to expectations or they are not. This is how life works.

And I think you are mistaken if you think that management doesn't look at some sort of advance stats to determine decisions (though it may not be corsi specifically)

At what point did I ever suggest that management doesn't look at statistical analysis?

[Enoch Root]

Originally Posted by Cali Panthers Fan

That would be an acceptable measure for me. Good teams don't tend to have prolonged losing streaks. Picking yourself up the next night seems to be a decent metric.

I would also include comeback wins when trailing after 1 or 2 periods, but since that gets into the problem of being behind often in games, it's maybe a negative indicator of failing to show up on time.

I could go on. As you see, these get pretty messy, as all measures of things that aren't direct on-ice events get messy when you try to quantify them. That's why I give Corsi and Corsi in context (Corsi relative, Score adjusted Corsi) a break when they're trying to determine a picture of what's happening out there.

I am intrigued by this. Why would you give corsi a break? It isn't a child that is trying hard. It is a statistic - either it is strongly predictive, and therefore very valuable, or it is less so, or it is useless. Giving it a break serves no purpose whatsoever.

This goes back to the very common defense of current advanced stats being: well, they're the best we have. Yes, they are. So what? If they're not actually good, who cares if they're the best we have?

[kevman]

Originally Posted by kevman

This discussion got me curious. Here is the 5v5 CF% data plotted against regular season points for the last 7 full seasons. Interpret how you wish.

It looks like maybe the original image didn't work? Trying again...

[Enoch Root]

Originally Posted by JerryUnderscore

Okay, well special teams can be measured objectively.

The powerplay is current at 22.2%. Good for 10th in the league. Since that's above average does Gulutzan get credit for it?

The penalty kill is 30th in the league at 73.8%. Obviously that's not good. Is that Gulutzan's fault?


How should we measure breakouts? Would that be measured in passing percentage when the pass originates in the defensive end? Do you have some data on that?

As for "Response to adversity" I have no idea how to measure that. After the defensive breakdown in Dallas, all three of his healthy scratches were brought in while Versteeg, Lazar and Kulak sat. They won in Colorado.

So did the team respond to adversity?

After the embarrassment in Detroit the team came back and won in Philly. Did they respond to adversity?

After losing 6-0 to Ottawa they came back at beat Vancouver 5-2. Was that a response to adversity?

The Flames haven't lost more than two in a row all season. Does that mean they're responding to adversity?

You make some valid supporting arguments. So why is it so hard for you to accept that others can make valid arguments to the contrary?

[edslunch]

Originally Posted by kevman

It looks like maybe the original image didn't work? Trying again...

By comparison, this is what strong correlation looks like (Goal differential vs. win % since 1946):



http://hockeyanalytics.com/2008/01/t...key-analytics/

[kevman]

Originally Posted by edslunch

By comparison, this is what strong correlation looks like (Goal differential vs. win % since 1946):



http://hockeyanalytics.com/2008/01/t...key-analytics/

That is a strong correlation. But it isn't a very good predictor. It's no surprise that the team that scores more goals wins the game. That's kind of the point. It's also no surprise that the team that has won more games scored more goals than they let in.

If you look at the current standings you see that they already follow the goal differential closely. So if you were to use it as a predictor you would say that Chicago and San Jose should be doing better and Pittsburgh should be doing worse. Other than that almost every team is exactly where they should be.

Is there a connection between winning big and where you're expected to finish in the standings? Is Vegas with 31 points a far better team than Nashville with 31 points because they win bigger and lose closer? Is Pittsburgh worse than their record because when they lose, they lose bad?

To quote Itse, would making brackets from goal differential do a good job at predicting the playoffs?

[Jiri Hrdina]

Originally Posted by Enoch Root

Not all things can be measured quantitatively. Sometimes you just make as assessment that they are performing up to expectations or they are not. This is how life works.



At what point did I ever suggest that management doesn't look at statistical analysis?

Well I didn’t say quantitatively I said objectively because more than likely with that type of stuff people are just going to assess it to align with what they already believe. So if you think the coach is garbage you will say the team hasn’t responded to adversity. If you think he isn’t you will say they have. It’s entirely subjective

[edslunch]

Originally Posted by kevman

That is a strong correlation. But it isn't a very good predictor. It's no surprise that the team that scores more goals wins the game. That's kind of the point. It's also no surprise that the team that has won more games scored more goals than they let in.



If you look at the current standings you see that they already follow the goal differential closely. So if you were to use it as a predictor you would say that Chicago and San Jose should be doing better and Pittsburgh should be doing worse. Other than that almost every team is exactly where they should be.



Is there a connection between winning big and where you're expected to finish in the standings? Is Vegas with 31 points a far better team than Nashville with 31 points because they win bigger and lose closer? Is Pittsburgh worse than their record because when they lose, they lose bad?



To quote Itse, would making brackets from goal differential do a good job at predicting the playoffs?

Let’s talk at the end of the season when the blowouts and shutouts even out. I would bet $100 on a playoff bracket picked with goal differential vs any other method.

[kevman]

I would think by the end of the seasons wins would be the best method of putting together a bracket...

Joke aside, and please help me understand this, how is goal differential the best method of predicting future results?

Yes, of course the team that scores more goals than it's opponents wins more games. But is the goal differential to date the best predictor of future wins? If that's the case, since they're so closely correlated, I'd then argue that the current standings, after 20 games, is the best predictor of the final standings after 82 games.

Is there a tool to predict which team will begin to score more and which team will let in less goals than they have to date? Maybe there isn't. Maybe the standings so far are actually the best way to predict the future standings.

[Itse]

Originally Posted by kevman

To quote Itse, would making brackets from goal differential do a good job at predicting the playoffs?

In my limited experience, they do notably better than than corsi, and actually better than combined with corsi which was surprising.

I have not done in any way a statistically valid analysis, just toyed with them a little though. I encourage people to make their own experiments, especially since there's lots of ways to do this stuff.

The experience has made me personally highly sceptical of the usefulness of corsi for these predictions. I consider it to be like plus/minus for players; uninteresting / irrelevant on its own, but if you limit the context right and combine corsi with other information, it's a neat addition to what we already have.

It's not as good as zone starts, which is another stat that's started to gain attention at the same time as corsi. Zone starts puts a number on something that I at least have a really hard time tracking just by eyeballing it.

[JerryUnderscore]

Originally Posted by Enoch Root

You make some valid supporting arguments. So why is it so hard for you to accept that others can make valid arguments to the contrary?

I'm sorry, I don't mean this to be snarky, but what points have you made?

I initially provided some advanced stats to show that the team is performing well except in terms of shooting percentage. This would indicate that the system the coach has implemented is working and they just aren't getting the puck to go in. The low shooting percentage hardly seems like a coaching issue since they are continuing to get scoring chances as well as high danger scoring chances.

You responded by saying that you disregard advanced stats and then later suggested that no one has been able to provide you with a legitimate argument that Gulutzan is a good coach.

You then suggested that "Special teams, breakouts, response to adversity, etc" are appropriate metrics to judge a coach by. However, even if we want to accept that those are the ways to measure a coach's success, how do you do that?

Special teams can be somewhat cut and dry if you simply want to look at their success rate. However, breakouts and response to adversity are way more subjective.

I'm completely willing to listen to an argument for why Gulutzan is not a good head coach, but if the argument essentially comes down to "I just don't feel like he's that good" I don't see that as a valid argument. It's an opinion you're entitled to have but it's not a valid argument.

[Red]

Advanced stats have been proven to be wrong time and time again. We have a few teams defying them every season. In 30 years we should be able to have a good chunk of data to support the notion that X% of teams defy Corsi every year :-) Can we then ask what good corsi does to predict results?

Goal differential is a much better way to tell if a team is going places. How many top 10 teams in the NHL have had poor goal differential at the end of the season in the last 20 years?

Point is that you get a good goal differential by cutting your goals against because no team is scoring 400 goals in a season. And that is statistically proven.
So although we did beat Philly in a high scoring game, chances of repeating that time and time again are very slim. We got lucky.

So you can choose to give GG credit for good Corsi while ignoring goal differential, but how much will that really mean come April?

I got scolded for criticizing GG during a winning streak, but I still think I have a case here. GG's team was winning by playing like a Hartley team. The Washington game was great, but we have only played a few of those this years. GG was brought in to make this team defensively dominant. We added so much D depth and skill to help him achieve that.
Seeing that the Flames are bottom 10 in GAA, he gets an F.

[GioforPM]

Originally Posted by Red

Advanced stats have been proven to be wrong time and time again. We have a few teams defying them every season. In 30 years we should be able to have a good chunk of data to support the notion that X% of teams defy Corsi every year :-) Can we then ask what good corsi does to predict results?

Goal differential is a much better way to tell if a team is going places. How many top 10 teams in the NHL have had poor goal differential at the end of the season in the last 20 years?

Point is that you get a good goal differential by cutting your goals against because no team is scoring 400 goals in a season. And that is statistically proven.
So although we did beat Philly in a high scoring game, chances of repeating that time and time again are very slim. We got lucky.

So you can choose to give GG credit for good Corsi while ignoring goal differential, but how much will that really mean come April?

I got scolded for criticizing GG during a winning streak, but I still think I have a case here. GG's team was winning by playing like a Hartley team. The Washington game was great, but we have only played a few of those this years. GG was brought in to make this team defensively dominant. We added so much D depth and skill to help him achieve that.
Seeing that the Flames are bottom 10 in GAA, he gets an F.

The Flames just beat the Avs by playing a pretty tight defensive game.

As for the bolded parts, GG was never touted as defensive genius. IIRC he was more of an overall systems guy, and when you look at the times the Flames play well, they are moving the puck in a group (which he promotes). The "D depth" of which you speak was also not brought in because of their stay at home, defensive nature. Gio and Brodie were here already and Gio is an all around guy, while Brodie is not a defensive defenceman by any stretch of the imagination. Only Hamonic fits that description. Treliving's other adds on defencemen are puck movers (Hamilton) or bottom pairing generalists (Stone, Bart). IMO the Flames are not generally winning "Hartley style". Their best games are when they play a good five man breakout and support eachother.

[Textcritic]

Originally Posted by Red

...Goal differential is a much better way to tell if a team is going places. How many top 10 teams in the NHL have had poor goal differential at the end of the season in the last 20 years?

Point is that you get a good goal differential by cutting your goals against because no team is scoring 400 goals in a season. And that is statistically proven...

So, at the quarter-season pole, what predictions exactly does current goal differential make? This seems to me to be a good indicator of past performance, but I don’t see how we can extrapolate from this figure what should happen in the future.

[edslunch]

Originally Posted by JerryUnderscore

I'm sorry, I don't mean this to be snarky, but what points have you made?



I initially provided some advanced stats to show that the team is performing well except in terms of shooting percentage. This would indicate that the system the coach has implemented is working and they just aren't getting the puck to go in. The low shooting percentage hardly seems like a coaching issue since they are continuing to get scoring chances as well as high danger scoring chances.



You responded by saying that you disregard advanced stats and then later suggested that no one has been able to provide you with a legitimate argument that Gulutzan is a good coach.



You then suggested that "Special teams, breakouts, response to adversity, etc" are appropriate metrics to judge a coach by. However, even if we want to accept that those are the ways to measure a coach's success, how do you do that?



Special teams can be somewhat cut and dry if you simply want to look at their success rate. However, breakouts and response to adversity are way more subjective.



I'm completely willing to listen to an argument for why Gulutzan is not a good head coach, but if the argument essentially comes down to "I just don't feel like he's that good" I don't see that as a valid argument. It's an opinion you're entitled to have but it's not a valid argument.

I think part of the problem is we are arguing the wrong question. The fire Gulutzan crew are making it a Gulutzan is terrible argument. He’s not terrible - the Flames have a winning record under him for starters. NHL coaching is an elite club and you don’t get there if you’re terrible.

To me the question is, is he an excellent coach. The Flames based on talent and age should be a top 10 team, trending towards top 5. Is Gulutzan a top 10 or 5 coach? I think there are good arguments that he’s not demonstrating characteristics usually associated with coaching excellence - preparation, identity, making players better, .... - but there is no objective standard for what makes a great coach except winning.

So far my assessment is he is an average coach which may not be enough to take the team to the next level