Not to worry; this won't last says Myrtle

[Tinordi]

Originally Posted by Robbob

with the addition of the new tracking system I hope they can extrapolate on the quality of shots. In the first Oilers game sure they got 40 shots on us but only around 10-15 came from quality areas. The facts are NHL goaltenders will stop pucks all day driving up their save % if they face those shots from the outside.

In a perfect world there would be some sort of stat or metric for shots/saves within a defined area.

My read of the shot quality arguments are that they tend to come out in the wash at least over a season long sample. There's very little statistical difference between total shots and shot quality.

[Street Pharmacist]

Originally Posted by Tinordi

My read of the shot quality arguments are that they tend to come out in the wash at least over a season long sample. There's very little statistical difference between total shots and shot quality.

It's been looked at a few times I believe and your read was upheld.

While in small sample sizes shot quality may vary, by the time a reasonable sample is there it's all equal

[EldrickOnIce]

Originally Posted by calumniate

The advanced stats just aren't advanced enough (at least in the article).

One measure I generally find telling is overall goal differential. A team ranked high with a low goal differential is always the one I'll pick to come back to earth at some point.

We're a +9, Chicago is below us in the standings with a +15, Vancouver above with only a +1. Therefore I predict these two teams to flip. Pretty advanced eh! Haha

And most likely accurate. Goal differential is the best predictive stat.
However, in a statistically small sample size, an unusually shooting percentage combined with very high save percentage will skew this.
That said, it's already been pointed out the Flames haven't relied on unstable PDO in the last few weeks. As sample size increases, those numbers will regress more to the norm. Which doesn't mean the wins stop coming. But they will without the other underlying numbers continuing to improve, as they have been.

[Phil Russell]

My feelings on advanced stats is that they are probably being overemphasized as predictors of performance--there are too many intangibles in hockey team success to by able to model performance based on something like possession stats.

But, the question of whether advanced stats do model performance can, ironically, be tested statistically. The question is do we have a statistically significant sample yet to do this (enough years of advanced stats and team performance data to determine if the former predicts the later)?

Please, someone who likes stats, take this on.

[Burner]

Originally Posted by Resolute 14

If you think that, you need to read a lot more of James Mirtle's work.

He was the one guy who told the world last year that Toronto wasn't going to make the playoffs. And he did so using the same arguments he is using here.

admitted poor attempt at Toronto is no good.

However to me the biggest difference is that Toronto was an established team while we are in a rebuild. Its not a far out assumption to suggest that some of our rookies will get more comfortable and produce more as the season goes on

[saillias]

I don't understand why PDO exists. Possession I get, shot attempts I get, those are quantifiable things. But PDO seems to me, someone got stoned and thought it would be brilliant to add a team's save percentage and shooting percentage.

[Cleveland Steam Whistle]

I don't know about the advanced stats, but the shooting and save percentage thoughts make sense I guess.

Don't know what else to say though other than I've watched almost all the games, and I can't remember the last time we won a game (maybe Chicago when we clearly stole it) where I actually thought the boys were outplayed and not full value for the win. Everyone is doing their part, and maybe that isn't sustainable, but it's not like I leave our games thinking, wow I don't know how we stole that one, or holly did we get lucky, or wow our goalie saved our bacon.

I generally leave the game thinking we got the result we deserved, which makes me nervous as my expectations rise.

[GioforPM]

Originally Posted by Street Pharmacist

Lots of goals aren't scored on 5 bell opportunities. Rebounds, tips, etc are the main offense for most teams.

The reason Gio and Brodie are awesome defensively is because they allow less shots towards the net than the they get. Not because they make them take worse shots

You are correct, of course, about where goals come from. However, I was including good tips and rebound saves as 5 bell opportunities. I do think that Gio and Brodie, while allowing less shots to reach the net, do force outside shots with speed and positioning. They reduce shots as well by being quick to the loose pucks and controlling them exceptionally well. They rarely eat the puck along the boards.

[Tinordi]

Originally Posted by New Era

This is pretty simple stuff, even for the Stats 101 set. If statistics are not gathered the same way, using the same criteria for evaluating an action, and maintaining the same level of integrity in judging events, the data then becomes unreliable. I see all sorts of differences in what is considered a shot in the same game let alone different buildings and different data collectors.

When the league can't train its on and off ice personnel to consistently identify what a particular penalty is, and enforce that with any consistency, how can you honestly sit there and smuggly say that off ice officials are also not using similar subjectivity to collect stats? There is no consistency in data collection. Just closely watch one game and compare the stats you collect versus that presented during the broadcast and in the box score afterwards. A great example was last night's game between Edmonton and Phoenix. The Coyotes spent a good minute in the offensive zone and had several close in shots on goal than were never registered on the shot clock. On the other side of the ice a dump in ended up on net was judged a shot on goal. Sure makes those Corsi stats look good for the Oilers! Also makes the advanced stats look bogus. And that is just one game in one building in the league. The same thing happens all over the league, affecting every stat. That is why regular stats are questionable, but when you start making inferences adding multiple stats together to make another statistical category, well the data errors are compounded and the data becomes useless.

To believe that the statistics are meaningful means you have to believe the data collection is accurate and consistent, which has been a problem for as long as hockey has been around, let alone now where teams are hiring these advanced stats guys to dream up new and improved ways to prove their teams are better than they really are. When guys like Mudcrutch, who have made some of the dumbest claims in the history of the game (and has the stats to back himself up) are hired by NHL teams, well, the system is wide open for abuse. I think last nights game was another example of this abuse in action and is a shining example of why these advanced stats are garbage.

You know I'm trying to be nice here but the effect you're describing is essentially meaningless. People have looked into the 'rink effect' and found that it may impact team corsi by a hundredth of a percent over a season.

This gets back to my original point that you have alot of criticisms of 'advanced stats' without a commensurate understanding.

[Street Pharmacist]

Originally Posted by Phil Russell

My feelings on advanced stats is that they are probably being overemphasized as predictors of performance--there are too many intangibles in hockey team success to by able to model performance based on something like possession stats.

But, the question of whether advanced stats do model performance can, ironically, be tested statistically. The question is do we have a statistically significant sample yet to do this (enough years of advanced stats and team performance data to determine if the former predicts the later)?

Please, someone who likes stats, take this on.

The sample size is large enough. War-on-ice.com has over 10 years of data. I would suggest going to that site and looking at different stats and comparing them to what happened. You can even make larger than one year sample sizes. You can then see for yourself if the stat matched what happened. We can't measure future because it hasn't happened.


One very important thing that has only barely been looked at statistically, is advanced stats and playoff success. So far, the only work I've seen was from fivethirtyeight.com showing that playoff success was most strongly correlated with goalie save %. The kicker? Save % for a goalie is only very rarely repeatable suggesting a hot goalie is the most important thing yet almost impossible to predict or procure.

[Robbob]

Originally Posted by Tinordi

My read of the shot quality arguments are that they tend to come out in the wash at least over a season long sample. There's very little statistical difference between total shots and shot quality.

Would it not be beneficial if there was a system where you could properly define what is a valid metric though? A shot from one teams own blue line as they go for a line change isn't a valid data point IMO.

[Jay Random]

Originally Posted by Tinordi

People have looked into the 'rink effect' and found that it may impact team corsi by a hundredth of a percent over a season.

Based on what data? Have they been doing a controlled experiment where they hired their own scorekeepers and gathered their own data by uniform standards in every arena, to compare with the data actually used by the league? That would be valid but difficult.

Or have they merely done calisthenics with numbers and come to the conclusion that the 'rink effect' is too small to measure due to signal noise? But the claim being made is that the rink effect is a significant contributor to the noisiness of the signal. That would require a different methodology to measure.

Now, if they are indeed using a different methodology, I would genuinely like to hear about it. But it puzzles me what they would be using for data.

[Tinordi]

Originally Posted by Robbob

Would it not be beneficial if there was a system where you could properly define what is a valid metric though? A shot from one teams own blue line as they go for a line change isn't a valid data point IMO.

Who knows, the big problem with shot quality is that it gets highly subjective. It would introduce a much larger problem than the one you're trying to correct.

My feeling is that yes some shots are meaningless, but really, think about it, how many are actually meaningless? How many shots on goal a game are to get a line change? Generously 1 a game, on a sample of 60 to 70 shots.

[CliffFletcher]

Originally Posted by transplant99

Why on earth would you think this?


Does Myrtle have more eyeballs than most people? More TV's?

His full-time job is hockey analyst. I assume most people on this forum have jobs that keep them busy 40+ hours a week, and maybe even leisure activities outside watching hockey.

If that's not the case, then you fall into the 10 per cent. Congratulations. You're devoting as much time and energy into analyzing hockey as full-time journalists. And not getting paid for it. Give yourself a pat on the back.

[burn_this_city]

Originally Posted by Jay Random

Based on what data? Have they been doing a controlled experiment where they hired their own scorekeepers and gathered their own data by uniform standards in every arena, to compare with the data actually used by the league? That would be valid but difficult.

Or have they merely done calisthenics with numbers and come to the conclusion that the 'rink effect' is too small to measure due to signal noise? But the claim being made is that the rink effect is a significant contributor to the noisiness of the signal. That would require a different methodology to measure.

Now, if they are indeed using a different methodology, I would genuinely like to hear about it. But it puzzles me what they would be using for data.

Did you read Nate Silver's book, Signal and the Noise? Its a good read for anyone that wants to delve into statistical analysis.

[Resolute 14]

Originally Posted by saillias

I don't understand why PDO exists. Possession I get, shot attempts I get, those are quantifiable things. But PDO seems to me, someone got stoned and thought it would be brilliant to add a team's save percentage and shooting percentage.

One of baseball's currently popular stats is OPS - on base percentage plus slugging percentage - and it has gained acceptance equal to batting average in the real world. I wouldn't be surprised if PDO wasn't created with OPS in mind - accounting for the fact that OPS is an individual stat and PDO is team.

The thing about PDO is that it tends to closely match the NHL standings. Most teams end up + or - three spots difference between their PDO rank and their overall rank. The significant exceptions last year were the Leafs (6th in PDO, 23rd overall), Washington (8th PDO, 17th overall), San Jose (16th PDO, 5th overall)

[Resolute 14]

Originally Posted by Street Pharmacist

The sample size is large enough. War-on-ice.com has over 10 years of data. I would suggest going to that site and looking at different stats and comparing them to what happened. You can even make larger than one year sample sizes. You can then see for yourself if the stat matched what happened. We can't measure future because it hasn't happened.


One very important thing that has only barely been looked at statistically, is advanced stats and playoff success. So far, the only work I've seen was from fivethirtyeight.com showing that playoff success was most strongly correlated with goalie save %. The kicker? Save % for a goalie is only very rarely repeatable suggesting a hot goalie is the most important thing yet almost impossible to predict or procure.

Case in point: Marc Andre Fleury.

Playoff success will always be a major wildcard, I think. For half the teams, the sample size will be as small as 4-7 games. All it takes is for the goalie to be off a little or for a skater to have an ungodly stretch to cause a team to sink or swim.

[Tinordi]

Originally Posted by Jay Random

Based on what data? Have they been doing a controlled experiment where they hired their own scorekeepers and gathered their own data by uniform standards in every arena, to compare with the data actually used by the league? That would be valid but difficult.

Or have they merely done calisthenics with numbers and come to the conclusion that the 'rink effect' is too small to measure due to signal noise? But the claim being made is that the rink effect is a significant contributor to the noisiness of the signal. That would require a different methodology to measure.

Now, if they are indeed using a different methodology, I would genuinely like to hear about it. But it puzzles me what they would be using for data.

The analysis I read was a analyst looked at the average shots to goals ratio of every stadium compared to each rink specifically, they should all roughly equalize to a league-wide average ratio. If a rink had higher shots to goal ratio than the average you know that they were over counting shots.

http://www.hockeyprospectus.com/rink...g-shot-counts/

The finding was that yes while some arenas over or under count shots, it was an additional 0.5 shots for every 100 shots taken by the home team. In other words insignificant.

[Jay Random]

Originally Posted by Resolute 14

The thing about PDO is that it tends to closely match the NHL standings. Most teams end up + or - three spots difference between their PDO rank and their overall rank. The significant exceptions last year were the Leafs (6th in PDO, 23rd overall), Washington (8th PDO, 17th overall), San Jose (16th PDO, 5th overall)

This is why I really don't understand those people who claim that PDO is largely an indicator of luck. The people who are saying the Flames are due for a fall based on PDO are basically saying, 'This team isn't good enough to have a PDO that high, and the proof of it is how high their PDO is.' This is circular reasoning with epicycles and eccentrics. It begins with the assumption that the PDO should regress to 100 in all cases; whereas the fact is that it regresses to a value indicative of the team's overall quality.

Moreover, there seems to be an assumption that the one team you want to criticize is the only one whose PDO will regress. Just from eyeballing the data offered up at various points in the season (and from a little common-sense knowledge of basic statistics), it would seem that the standard deviation of PDO over a small number of games, league-wide, will naturally be larger than the standard deviation over a whole season. That being so, everyone's PDO can be expected to regress somewhat towards the league mean as the number of games increases. But that can happen even if there is no change at all in the relative rankings.

What I mean: A team could have a PDO of 103 at this point of the season, let us say, and be 8th in the league by that measure. By the end of the year, the team could have a PDO of 101.5 and still be ranked 8th, because the grouping is that much tighter due to the larger sample size. To say that the team can't sustain a PDO of 103 is quite correct; to infer from this that the team doesn't deserve to be 8th, and is doomed to fall down the standings, is unjustified.

[Erick Estrada]

Originally Posted by Tinordi

I think one of the reasons there's so much antipathy to "advanced stats" is that they refute long held tropes, biases and stories we tell ourselves about team X or player Y. We don't like that, we like the story of "heart" and a player being "clutch." It creates a causal and hence relatable nature to the game. Watching the game and drawing conclusions is available to anyone who wants to do it whereas those who want and are able to evaluate data is a much smaller segment of the population. That's seen as an attack by many, that somehow there knkowledge of the game isn't as valuable as new knowledge, of course there would be almost a visceral pushback.

But that's the precise value of the stats, they provide a way to 'ground-truth' your biases and assumptions. Are they perfect, no but should that preclude them from being used? Are they any less perfect that Joe fan sitting on the couch complaining about how terrible Phaneuf is?

Nope. It's that in a lot of cases advanced stats tell us nothing we didn't already know. Anyone that watched the Leafs or Avs play last year knew they weren't going to be able to sustain long term success playing the way they were. Lots of posters here had predicted the Leafs would fall off as they did. There aren't a lot of Flames fans here that haven't said more than once that they expect the Flames will eventually hit the skids this season. It's not hard to look at the assembled talent on the roster then evaluate from their goals for, goals against, shots for, shots against, and goaltender save percentage and surmise if their play will carry long term. Myrtle's article here about the Flames is more for people out east that don't watch them or even care to take a look at their record and stats than it is for the fans of the team that already know full well that they are playing over their heads. While some of the advanced stats make for good evaluation tools a lot of it like this article is akin to a yellow book for dummies.