Lies, Damned Lies, and Statistics

[Cali Panthers Fan]

Originally Posted by CorsiHockeyLeague

Maybe, but this early in the season, the current top 10 teams in the league aren't the ten best teams in the league - Winnipeg and Minnesota, while good teams, aren't likely top 10 teams, and the Ducks, Hawks and Bolts pretty clearly are. So suggesting that those three would qualify as "Flames should win vs" teams simply based on standings position seems wrong.

Hell, Vancouver's tied for 10th in points, and the only reason for that is because they got 3 of 4 vs the Flames the first week.

I hate this reasoning. If your record indicates that you're a top 10 team, then you are one until you aren't. Consequently, if you were supposed to be a top 10 team, but aren't performing like one, then you aren't. Simple. It's easy to determine who's doing well and who isn't: look at the damn standings.

I'm not saying it will end up that way, but there is zero reason to assume that teams who have a lot of points won't continue to perform to that level the entire year. We don't know what will happen. That's why they still play the games, otherwise the preview magazines in September would always be right. And there's always teams like the Wild who make a change and go on a ridiculous pace, but often that's an outlier and previous performance is the best indicator of future performance.

Who saw the Flames and Ducks being so god-awful early on? Not me, but it doesn't mean they should be deserving of being in the conversation of a top-10 team. On the flip-side, I happen to feel that the Ducks and Hawks were not going to have as much success this year, and I also felt that the Wild and Jets were going to be very, very good. Just because you don't agree with that doesn't mean you're right.

It's a league with a lot of parity. The separation between teams is razor thin. There are probably 3 really awful teams, 3 really exceptional teams, and a whole bunch of teams in the middle who's success is determined by things going well or things going poorly, with little to no reason as to why it works for some and doesn't work for others.

I can't say this enough: analytics are not a predictor of success in the standings. They are simply a tool to see how you're playing the game outside of the boxscore.

[Mathgod]

Stats aren't useless. They can actually be very useful when interpreted correctly. In this case, B is likely the best answer.

The stats I can't seem to find in the NHL advanced stats though are TOI/+ and TOI/-.

[driveway]

Originally Posted by Finger Cookin

Who is this magical, all star calibre goalie that will fall into our lap? Last year there were 10 goalies in the league who played in 10 or more games had a .925 or greater save percentage.

That's for all situations. When looking at goalie stats, it's best to look at 5v5 numbers, they're the most repeatable and the most telling.

We need a goalie who can play .925 when we're 5v5, there were 32 goalies in the NHL last year who played more than 1000 minutes and posted a .920 or better 5v5 save percentage including both Ramo and Hiller.

There were in fact only 16 goalies who played more than 1000 minutes and had a 5v5 save percentage lower than .920. The worst were Scrivens and Fasth, Scrivens had an .899 and Fasth was .895.

Even Fasth's abysmal number is .027 better than what our goalies are putting up this year. That, again, works out to about 8 fewer 5v5 goals against already this season.

Here's an idea of how bad our goaltending has been. We're on pace to allow 323 goals against at even strength. Last year the Oilers allowed 276 GA total.

[Finger Cookin]

Originally Posted by driveway

That's for all situations.

Yes. The post I was responding to didn't specify anything differently.

If we need goaltending in the top 2/3 of 5v5 save %, and two of our goalies delivered that last year, it's reasonable to me to think that they can get there again. Neither has hit the age when goalie performance starts to drop off a cliff. And none of them have come close to 1000 minutes yet this year.

I'd imagine with most goalies in the league, over the course of a season, you could find a sample of their best 300 or worst 300 minutes and it would look appropriately exciting or terrifying.

[driveway]

Originally Posted by Mathgod

Stats aren't useless. They can actually be very useful when interpreted correctly. In this case, B is likely the best answer.

The stats I can't seem to find in the NHL advanced stats though are TOI/+ and TOI/-.

What do you mean by TOI/+ and TOI/-? Do you mean how much time is a player on the ice per goal-for and per goal-against?

If you go to War-On-Ice.com and set the "Columns-to-display" button to "Goal-Based" two of the columns are GF60 and GA60, which are how many goals-for and goals-against the player is on the ice for per 60 minutes of ice-time, which is I think the stat you're looking for.

Currently, T "Jesus" Brodie is 3rd in the league with 6.56 GF60.

However, among players with at least 60 minutes played, the three worst players in the league at GA60 are Joe Colborne (7.14), Kris Russel (6.06), and Sean Monahan (5.84).

Jooris, Wideman, and Hudler also all show up in the top-20 worst GA60 skaters, which I am, again, going to pin on our catastrophic goaltending.

(Again, these are 5v5 stats, not all-situations).

[Mathgod]

That's what I was looking for. Cheers.

[harmony]

unsustainable.

[ernie]

Advanced stats (or any stats) aren't useless. They often get misused and misunderstood but they aren't useless.

In the original post I wouldn't put a lot of stock in a 5 game stretch of what looks to be good numbers. It's just not a big enough sample size.

Now, if there is a 20+ game stretch where everything is looking good and the results aren't coming, you have to think to yourself "hmmm, I wonder why things don't seem to be lining up?". Instead, what many will say is "Advanced stats are crap!".

No they are data. And data will lead you to explanations. So...the results aren't adding up. The team in the 20+ game case should be seeing more points in the standings. Why aren't they? Well perhaps the goaltending is under performing and undoing the good the advanced stats show. For the Flames through 13 games that is certainly part of the issue and it's obvious. But you may also want to look at turnovers, prime scoring chances etc. Have those been disproportionate?

The flip side is what happened to the Leafs when everyone was pointing out the bubble was going to burst at some point because of the underlying numbers were simply not good. Instead of ignoring those numbers Leaf fans should have been looking at the reasons why the team may have been over performing. In that case, if I recall correctly, Bernier was playing stupid good with a 0.950 SV% and they had a traditionally unsustainable shooting percentage. Those aspects were going to remain and make hockey history or more likely regress at some point. They regressed.

Basically, advanced stats are useful as data and should be used to prompt thoughts as to why things may not be completely lining up with the general established trends (over a suitable sample size of course). A team can "beat" the advanced stats by, say, having Carey Price in net. Or they have potent special teams. They can even beat them for a long time for various other reasons.

They are not the end all and be all.

I think when they get brought up on message boards you get a majority of people who just dismiss them out of hand or don't dig into reasons why the results may not line up and simply crow "It'll end as the advanced stats say so!". You need to use the stats to go a step further and look for explanations as to why things deviate from what ARE established norms/trends. And above all remember that trends don't mean everything lines up nicely on the line. Outliers exist. No stat has perfect correlations.

[saillias]

It's almost hard to believe, but Brodie-Engelland would probably be our most reliable pairing that drives possession the most right now, if Hartley were to reunite them.

[CorsiHockeyLeague]

Originally Posted by Cali Panthers Fan

I hate this reasoning. If your record indicates that you're a top 10 team, then you are one until you aren't. Consequently, if you were supposed to be a top 10 team, but aren't performing like one, then you aren't. Simple. It's easy to determine who's doing well and who isn't: look at the damn standings.

And I hate this reasoning because it assumes that literally everything that happens on the ice is entirely within the control of the players on it and that results are a perfect indicator of what happened during the games. This early in the season, there will be outliers because random chance has a greater effect in a smaller sample. This is why you don't ask ten guys in Oregon who they support for president and assume you've figured out who the front runner is. This is not just true of hockey (though a game involving ice and a bouncy rubber object certainly seems well suited to random bounces). It's everything.

This is common sense.

[Resolute 14]

Originally Posted by ernie

I think when they get brought up on message boards you get a majority of people who just dismiss them out of hand or don't dig into reasons why the results may not line up and simply crow "It'll end as the advanced stats say so!". You need to use the stats to go a step further and look for explanations as to why things deviate from what ARE established norms/trends. And above all remember that trends don't mean everything lines up nicely on the line. Outliers exist. No stat has perfect correlations.

Agreed. The problem with analytics is not the stats, but with many of the people who push them.

Last year, the Flames had a horrible CF% and a ridiculously high PDO. Therefore we were just a bad team that got lucky, and we would regress. We didn't. It happens. There's always a team that bucks the trend.

But several of those same people are looking at the Flames' current woes as vindication for their arguments of last year. Except that we actually have a league average CF% (49.9) and a horrible PDO. So if those people were interested in being consistent, they would be arguing just as loudly that we are actually a better team getting bad luck.

The fact that they generally aren't tells you that the problem is less the numbers, and more the bias of those pushing them.

[HotHotHeat]

Kris Russell is leading the league in blocked shots while being a league worst -15 this
season.

Awesome.

[CorsiHockeyLeague]

Originally Posted by Resolute 14

But several of those same people are looking at the Flames' current woes as vindication for their arguments of last year. Except that we actually have a league average CF% (49.9) and a horrible PDO. So if those people were interested in being consistent, they would be arguing just as loudly that we are actually a better team getting bad luck.

The fact that they generally aren't tells you that the problem is less the numbers, and more the bias of those pushing them.

Aren't they? Seems to me that most of the focus has been on the goaltending, with some portion of the explanation being that the team shooting percentage has come down to Earth so far as compared with last year (and is perhaps a bit lower than you'd expect). I'd be interested in comparing shot locations. I haven't personally attempted to look too closely at causes for the exact reason I mentioned earlier; too much noise in the data at this stage.

[ernie]

Well the other problem is that a team can buck the trend for sometime and not have time to fall completely back down to earth or come back down at different rates etc.

So did the Flames not come back down to earth and continued to buck the trend, did the underlying stats change through the year such that they were following the trend or did we need to stretch into the next season to see the regression?

I haven't looked at how the Flames stats progressed through the year last season. If I recall from the infamous "eye test" the Flames were bucking the trend on the back of some strong goaltending and some elevated sh% but as those regressed, the team began to play better to maybe balance out that regression. I'd imagine the advanced stats might say exactly that. Not sure.

[GranteedEV]

Since this thread is about statistics, I thought I'd point out some forwards' statistics:

1) Joe Colborne (-10)
5 vs 5
+8.08 Unblocked Shot Attempts Against Per 60 Relative To Team

SH
-7.44 Unblocked Shot Attempts Against Per 60 Relative To Team (Best on Team)

2) Michael Frolik (-2)

5 vs 5
+7.55 Unblocked Shot Attempts Against Per 60 Relative To Team (Second Worst on Team, Last season he was -4.50 and the best on his team)

SH
-7.64 Unblocked Shot Attempts Against Per 60 Relative To Team (Best on Team)

3) Mikael Backlund (-8)

5 vs 5
+3.14 Unblocked Shot Attempts Against Per 60 Relative To Team (Third Worst On Team, just using for reference. Last season he was -2.81 or 3rd best on team while facing the best competition)

SH
-4.62 Unblocked Shot Attempts Against Per 60 Relative To Team (4th best / 3rd Worst on team

4) David Jones (-9)

5 vs 5
+3.03 Unblocked Shot Attempts Against Per 60 Relative To Team (4th Worst on Team)

5) Sean Monahan (-7)

5 vs 5
+2.38 Unblocked Shot Attempts Against Per 60 Relative to Team (5th Worst on Team)

SH
+15.42 Unblocked Shot Attempts Against Per 60 Relative to Team (Worst on team by far)

First of all, sample size is small. Regardless:

1) Monahan has been getting absolutely shelled on the PK and struggling with 5 on 5 matchups too.
2) Frolik has been getting shelled 5 on 5 but has played well on the PK.
3) Colborne has been getting shelled 5 on 5 but has played well on the PK.
4) Backlund and Jones have been getting shelled 5 on 5. Backlund has been okay on the PK but can be better.

So what is going on? These are four veterans plus Monahan who's supposed to be a great defensive center who have been getting shelled defensively relative to the rest of the team.

There's appears to be some correlation with the poor unblocked shot attempts stat and the +/- of four these players (not Frolik, though).

Is the reason our defense and goaltending has been struggling so much because these key forwards haven't been doing their job defensively? Joe Colborne being a -10 especially stands out because he did it only 8 games. Is he a liability considering he has the worst shot suppression stats on the team while being highly sheltered in an offensive role?

[Resolute 14]

Originally Posted by CorsiHockeyLeague

Aren't they? Seems to me that most of the focus has been on the goaltending, with some portion of the explanation being that the team shooting percentage has come down to Earth so far as compared with last year (and is perhaps a bit lower than you'd expect). I'd be interested in comparing shot locations. I haven't personally attempted to look too closely at causes for the exact reason I mentioned earlier; too much noise in the data at this stage.

I was speaking more to attitude than numbers - as you say, the data is fluctuating pretty wildly right now due to sample size. But there have been plenty of comments online from analytics fans looking at the Flames' win-loss record and bragging with comments like "who could have guessed this was coming?" as if all of their doom and gloom and "the win-loss column is wrong" attitude from last year was justified.

[SanFranFlamesFan]

Originally Posted by Resolute 14

Agreed. The problem with analytics is not the stats, but with many of the people who push them.

Last year, the Flames had a horrible CF% and a ridiculously high PDO. Therefore we were just a bad team that got lucky, and we would regress. We didn't. It happens. There's always a team that bucks the trend.

But several of those same people are looking at the Flames' current woes as vindication for their arguments of last year. Except that we actually have a league average CF% (49.9) and a horrible PDO. So if those people were interested in being consistent, they would be arguing just as loudly that we are actually a better team getting bad luck.

The fact that they generally aren't tells you that the problem is less the numbers, and more the bias of those pushing them.

Here is something that I don't understand about PDO. It's considered "luck" and will normalize around 100. But ~92% of it is made up of Goaltending Save % (EV) which is considered a skill, and the only reliable way to evaluate goalies. How can something that is considered luck be made up of 90+% skill?

So people are saying we are unlucky because we have a godawful PDO. But that's because we have godawful goaltending. We aren't unlucky, our goaltending just SUCKS.

I get that it's so bad that it is historically unlucky and should get better... but still, are people going to say Montreal is lucky because they have Carey Price and his .936 SV%?

[Enoch Root]

Originally Posted by CorsiHockeyLeague

And I hate this reasoning because it assumes that literally everything that happens on the ice is entirely within the control of the players on it and that results are a perfect indicator of what happened during the games. This early in the season, there will be outliers because random chance has a greater effect in a smaller sample. This is why you don't ask ten guys in Oregon who they support for president and assume you've figured out who the front runner is. This is not just true of hockey (though a game involving ice and a bouncy rubber object certainly seems well suited to random bounces). It's everything.

This is common sense.

You've missed Cali's point.

He was originally talking about the quality of teams that the Flames have played so far. And those teams have had great records so far rhis season.

His point was that, for the first part of the season, if they have the best records, they are the best teams.

That does not mean that their records can be expected to continue. It does not mean that they will be the best teams over 82 games. He never made that claim.

But if you run into a team that is 8-2 and playing really well, then you ran into a very good team - regardless of what happens to them for the remainder of the season.

Conversely, if you got to play Anaheim twice already, you were fortunate, regardless of the fact that they will likely be cup favourites when it is all said and done.

[Cali Panthers Fan]

Originally Posted by CorsiHockeyLeague

And I hate this reasoning because it assumes that literally everything that happens on the ice is entirely within the control of the players on it and that results are a perfect indicator of what happened during the games. This early in the season, there will be outliers because random chance has a greater effect in a smaller sample. This is why you don't ask ten guys in Oregon who they support for president and assume you've figured out who the front runner is. This is not just true of hockey (though a game involving ice and a bouncy rubber object certainly seems well suited to random bounces). It's everything.

This is common sense.

So I don't disagree with part of your thinking, but you asserted that there were certain teams that were definitely in the top 10 in the league, and others that were definitely not. I just felt like that was a presumptuous claim that maybe you were using past data to justify. This season is different. That's all.


Enoch responded well on my behalf, but my main point was that you can pretty much only use recent history to determine who's a good team and who's a bad team. Over the long haul, all teams tend to even out except for the top 3-5 and bottom 3-5. That's what statistical analysis usually leads us to. Outliers, and a whole bunch of stuff in the middle.

Also, in regards to the bolded: If what happens on the ice isn't in the player's control, who's control is it in? If you're saying random chance, well then there's enough random chances in a single game to go either way because there are a lot of events. Even if it's heavily skewed one way, it will balance even over the course of 5-10 games. That's a decent enough sample size to rule out random chance of individual bounces.

[Enoch Root]

I agree that the primary problem facing advanced stats is the people wielding them.

I would also add another problem: most fans have very little real experience with statistical analysis and don't really understand the weakness of these stats.

Yes, Corsi etc. are the best predictors currently available, but being the best doesn't necessarily make them good.

The thing is, most stats will correlate more or less with the good teams having better stats. No surprise there. But the problem with hockey is that far too many factors and too much noise is getting bundled into the information. It is impossible to isolate the variables to get more useful information.

The bottom line is that the actual, usable information - the real ability of these stats to convey meaningful correlations with the information that we are actually trying to ascertain, is far too weak to be able to make the conclusions that people are making.

But they make them anyway.

And then, just often enough, a Toronto example will come along, seemingly proving the stats as meaningful, and it solidifies the confidence of the users far beyond where their confidence should be.

And then there is the fact that when people think that they are right, when there isn't enough information to be right OR wrong, they are annoying.