PDO and Unsustainable Stats

[Resolute 14]

Originally Posted by edslunch

Hence my point B. If regress to the mean doesn't mean regress to the mean then use different words. Toward the mean gives a much better picture.

If luck doesn't literally mean luck then stop saying it and find a better way to express it.

Note these comments are not directed specifically at you, but almost every advanced stat discussion involves these exact terms.

No, I use "to the mean" as well, even if it is no less of a misnomer than how we define "winning percentage" in hockey.

And Jay - While that comment is fair, reality does, of course, argue that good teams will inevitably be above 100 and bad teams will be below. (or more accurately, below the median, wherever it falls for a given season.) Poor interpretation - possibly even by a stat's creator himself - doesn't invalidate the stat. It merely means that people need to look at what the data really is saying.

And more importantly, they need to look at more than one stat. i.e.: 5 on 5, our PDO is third in the league at 102.1. But our goal differential is only +3. PDO alone might say we are getting "lucky". Goal differential points us to shot differential and reveals we are -103 in shots overall. So if we get a little "unlucky" and our PDO regresses toward the mean, but we even out that shot differential, then we can still maintain our position in the standings.

I have stated this before, but a lot of predictions using advanced stats are implicitly framed as "if all else remains equal". If our PDO does fall and all else remains equal, we are going to start losing a lot more games. That is indisputable, as it is really just a fancy way of saying "if we give up more goals and score less, we will lose more." But other factors will change too.

[edslunch]

As a simplistic example to score a cheap point, if a team has high SH% then they will take fewer second shots, hence their Corsi will tend to be lower. if SH% drops off who's to say they won't just start banging in more rebounds? In other words all else can't remain equal.

[You Need a Thneed]

PDO doesn't describe luck much at all, perhaps a bit in the short term. Over longer periods of time, good teams will have a higher PDO, and bad teams will have a lower PDO.

Also, that the league wide PDO must be exactly 100 is false. You would have to remove empty nets goals from shooting percentages for this to be true. An empty net goal creates a bump up in shooting percentage, without a corresponding bump down in the other teams' save percentage.

It's perfectly sustainable for a good team to have a high PDO. That's why they are good, because their goaltending is above average, and their scorers are above average.

[belsarius]

I find the whole concept as a statistic than can be analyzed absurd. Shooting % and save % for one team has no impact on each other when comparing to every other team. Because they come close to 1.000 when added doesn't mean they correlate to each other. PDO himself was only using it as a comparison from player to player on the Oilers to determine why people got traded or extended.

All shots are not created equal, and all players are not created equal so using it to compare teams to each other makes no sense to me.

Luck to me is determined by the quality of shots and their result. A lot of high quality shots that get saved is lucky. Irbe's shaft save in 1994 against Calgary was a luck save, consider it good luck for SJ and bad luck for CGY. Salo's five hole non-save from center ice is another consideration of luck. Until they use some metric to classify quality of shots I won't take PDO for the electronic bits it is written on as a way to classify luck against other teams. And good luck qualifying shots, how does a slap shot from Chara from the point compare to a tied up weak wrister from the slot..

I enjoy using it to see how players on the team are performing, shot % while they are on the ice compared to save %, because that shows more of a system metric and quality of player. And compare that to the team PDO for how well the player is doing.

[Enoch Root]

This has been one of the better (and least annoying) discussions on the topic, so thanks to everyone for that.

One quick note on the regression: expecting regression to 100 would be silly, but a useful look at the numbers would suggest a regression away from extreme numbers. It is reasonable to conclude that 103 is high and difficult to maintain. Expecting it to drop into the 101s or low 102s is probably a pretty reasonable expectation.

[Enoch Root]

With all of the talk on sustainability lately, a really important fact is being overlooked.

The Flames have played to a .667 P% so far.

Whether or not they can sustain that is irrelevant because they don't have to.

In order to make the playoffs, they will simply have to play the rest of the year at a .536 P% (assuming 95 Pts).

They can regress significantly for the remainder of the season and they will still make the playoffs. So who cares?!?

[ricardodw]

Has a shot on goal or save ever been taken away upon video review? The slap pass deflection was going to go wide? It was actually a pass across the crease that the goalie stopped. Do you count 2 or 3 shots on goal in a crease scrum.

For example Monahan's OT winner he get credit for a 14 ft SOG and then a 11ft wrist shot for the goal. The goal was scored from just outside the crease so anyone doing effective shooting area analysis has obviously incorrect data.

The whole advanced stats is based on suspect stats.

Trying to bring baseball type analysis to hockey. In baseball there are arguments and reversals on much simpler events.

[Enoch Root]

In baseball, there is no RGI.

[PeteMoss]

blarg... wrong data.

[troutman]

[Street Pharmacist]

Originally Posted by Enoch Root

This has been one of the better (and least annoying) discussions on the topic, so thanks to everyone for that.

One quick note on the regression: expecting regression to 100 would be silly, but a useful look at the numbers would suggest a regression away from extreme numbers. It is reasonable to conclude that 103 is high and difficult to maintain. Expecting it to drop into the 101s or low 102s is probably a pretty reasonable expectation.

Agreed. That's what Resolute said too and how most knowledgeable "advanced stats" use it.

I wish we'd call them "stats" and use them the same. The stats are the same, the analysis and worrying of the stats in discussion of the game is what's changed.
Corsi is not advanced. Shots towards the net. It's less advanced than shots on goal, but somehow that is old so not as scary. PDO takes two very simple terms and adds them together. Again, not advanced.
The discussion is when they go awry. You start seeing columns where they claim one team is better than another strictly based on corsi. Corsi only tells you who puts more shots on net. While it does correlate with success, so does social teams success. Would I hold PP% and say x team is better than y team because they have a better PP? No. You need a deeper analysis than that.

Having said all that, unless the Flames improve a bit, they will regress a bit. Jooris isn't going to keep up this place unless he gets more shots. Giordano isn't likely going to finish at a PPG. The flames aren't going to win 18 games when trailing after 2.
Who knows, maybe Gaudreau goes on a tear, maybe Glencross starts scoring. Maybe someone else steps up. Point is, the winning is not sustainable If nothing else changes.
I have faith that the Flames can make the playoffs for 2 reasons: 1) Their play is slowly improving as the season progresses, and 2) They've built up enough points it doesn't need to sustain to make the playoffs

[Flames Fan, Ph.D.]

Originally Posted by Resolute 14

I have stated this before, but a lot of predictions using advanced stats are implicitly framed as "if all else remains equal". If our PDO does fall and all else remains equal, we are going to start losing a lot more games. That is indisputable, as it is really just a fancy way of saying "if we give up more goals and score less, we will lose more." But other factors will change too.

Just a couple of points:

1) I disagree with the use of the word prediction in relation to these numbers. They're not meaningfully predictive because the magnitudes of the correlations are poor and they do not provide any indication of time frame for the prediction. Without a time frame, they're simply depictions of past data with no indication of how the data will bin within a prospectively defined future time. We stereotypically mock weathermen for their predictions, but these stats are significantly less reliable than the 7 day forecast you hear on your local news.

2) The "all else remains equal" is an assumption that the opposition acts according to a prespecified model or law. As you say, other factors will change as well, and that can include the opposition not acting rationally. Perhaps if people could make an integrated model with all of these factors, then we'll get somewhere. Until then, these individual measures are horribly predictive and (for someone like me) only anecdotally interesting.

I suspect many teams are monitoring these stats not because they're explicitly using them as the foundation of their decisions, but rather that they're afraid of missing out on an edge that another team may have. They look like effective tools for confirmation bias.

[formulate]

Originally Posted by Street Pharmacist

Agreed. That's what Resolute said too and how most knowledgeable "advanced stats" use it.

I wish we'd call them "stats" and use them the same. The stats are the same, the analysis and worrying of the stats in discussion of the game is what's changed.
Corsi is not advanced. Shots towards the net. It's less advanced than shots on goal, but somehow that is old so not as scary. PDO takes two very simple terms and adds them together. Again, not advanced.
The discussion is when they go awry. You start seeing columns where they claim one team is better than another strictly based on corsi. Corsi only tells you who puts more shots on net. While it does correlate with success, so does social teams success. Would I hold PP% and say x team is better than y team because they have a better PP? No. You need a deeper analysis than that.

Having said all that, unless the Flames improve a bit, they will regress a bit. Jooris isn't going to keep up this place unless he gets more shots. Giordano isn't likely going to finish at a PPG. The flames aren't going to win 18 games when talking after 2.
Who knows, maybe Gaudreau goes on a tear, maybe Glencross starts scoring. Maybe someone else steps up. Point is, the winning is not sustainable If nothing else changes.
I have faith that the Flames can make the playoffs for 2 reasons: 1) Their play is slowly improving as the season progresses, and 2) They've built up enough points it doesn't need to sustain to make the playoffs

Exactly. The problem was the original movement came from overzealous blogger-types who thought they had some kind of monumental breakthrough. They took good (surrogate) possession numbers as the be-all-end-all in hockey statistics. That of course pissed off a bunch of people who became the strong opposition.

Your point on PP% is exactly what I've been saying. I think there is value in the stats; taken as a small part of the bigger picture, not the entire picture. It's logical to think that if a team spends more time shooting than being shot at, they should have more success. Of course, this doesn't take into account shot distance or quality, which is why shot attempt stats can't be looked at as the be-all-end-all.

I truly wish that Corsi and Fenwick stats were slowly brought in, just as another metric to determine what makes a good team. Not whisked in on a horse-drawn carriage to the sound of trumpets with a town crier proclaiming the greatness of these newly found statistics.

Though at this point I hope the Flames possession numbers get worse and their shooting percentage gets better to see the heads of guys like Yost, Drance, and Lambert explode.

[Resolute 14]

I wasn't saying the stats are making the predictions. But rather, that people are making predictions using those stats as their basis.

[PeteMoss]

One thing to add - and this applies to the Leafs as well (who as much as the advanced stats like to crow about their collapse last year, they've pretty much been better than their stats for last two seasons besides a month collapse).

The Flames advanced stats suck mainly because the 4th line and 3rd defense pair aren't very good.

Those players play the vast majority of their minutes against other 4th lines. Getting whipped by poor offensive players doesn't hurt as much as getting whipped by 1st liners.

Take a game against Anaheim for example - The Flames have Gio/Brodie and some combo of non-Bollig/Bouma forwards out there against Perry/Getzlaf so possession is even or in the Flames favour.

Then the Ducks 4th line goes out there with Tim Jackman/Nate Thompson and co and they dominate the Flames. Those types of guys have 3-5% shooting percentages. Getting whipped by them isn't going to hurt much. Obviously you'd prefer to not have it happen, but if you are going to get whipped by a line, you'd want it to be Jackman/Thompson and not Getzlaf and Perry.

[EldrickOnIce]

Originally Posted by PeteMoss

One thing to add - and this applies to the Leafs as well (who as much as the advanced stats like to crow about their collapse last year, they've pretty much been better than their stats for last two seasons besides a month collapse).

The Flames advanced stats suck mainly because the 4th line and 3rd defense pair aren't very good.

Those players play the vast majority of their minutes against other 4th lines. Getting whipped by poor offensive players doesn't hurt as much as getting whipped by 1st liners.

Take a game against Anaheim for example - The Flames have Gio/Brodie and some combo of non-Bollig/Bouma forwards out there against Perry/Getzlaf so possession is even or in the Flames favour.

Then the Ducks 4th line goes out there with Tim Jackman/Nate Thompson and co and they dominate the Flames. Those types of guys have 3-5% shooting percentages. Getting whipped by them isn't going to hurt much. Obviously you'd prefer to not have it happen, but if you are going to get whipped by a line, you'd want it to be Jackman/Thompson and not Getzlaf and Perry.

This is the first time I have heard this viewpoint. And it makes such good sense. I expect there would be a way to break that down, if someone cared to lol.
Elite 'possession' teams tend to have a strong 4th line (Kings and Bruins last year).
Interesting

[H2SO4(aq)]

Originally Posted by sureLoss

http://blogs.thescore.com/nhl/2013/01/21/pdo-explained/

If you are too lazy to click the link: The short of it is PDO was the username of the person who first brought the stat to people's attention. It is a stat that tries to determine "luck" by simply adding save% and shooting %. Too high of a PDO and the implication is that the team has too much good luck and too low of a PDO means that a team has bad luck. The theory is that too high or too low PDO will eventually regress to a mean value (i.e. a lucky team's luck will run out and regress or an unlucky team will eventually play better when more pucks start going in and/or a goaltender starts putting up some lucky saves).

That's what some "advanced" stats people believe anyways.

Are you kidding me? its an alleged "luck" calculator? And reporters and media types are allowed to pretend they're stats experts and present "credible" data to back up their arguments based upon this?

what in the hell?

[AC]

Originally Posted by edslunch

As a simplistic example to score a cheap point, if a team has high SH% then they will take fewer second shots, hence their Corsi will tend to be lower. if SH% drops off who's to say they won't just start banging in more rebounds? In other words all else can't remain equal.

This is the most interesting counter-point I've heard yet. A worse shooting percentage is guaranteed to result in more rebounds, which result in more shots, and better Corsi numbers.

I agree with the sentiment by most here that the advanced stats are too much in their infancy to be used as a true measure of much right now, though they show promise. You can see why Treliving has stressed it a few times now that the key is filtering out the noise and finding true, meaningful data.

[Street Pharmacist]

Originally Posted by AC

This is the most interesting counter-point I've heard yet. A worse shooting percentage is guaranteed to result in more rebounds, which result in more shots, and better Corsi numbers.

I'm not sure it makes much difference. The flames are on average worse by 12 shot attempts per game, but only better by about 10-20 goals total of you tone down our shooting percentage to normal. At most that means that about 0.7 wrist shots go in instead of causing rebounds. To explain the corsi then, that would mean that each goal extra because of "wrist shot skill" would prevent about 20 or so rebounds. I don't buy it

[Jay Random]

Originally Posted by Enoch Root

It is reasonable to conclude that 103 is high and difficult to maintain. Expecting it to drop into the 101s or low 102s is probably a pretty reasonable expectation.

Reasonable, but completely uninformative.

PDO does not tell you anything that the separate components, shooting percentage and save percentage, don’t tell you. In fact, PDO gives less information, because it is one number instead of two. A team could have an unsustainably high shooting percentage and very poor goaltending, in which case it would have a PDO of 100 but still be due for a fall in the standings.

In the Flames’ case, they have roughly average goaltending (as measured by SV%) and a very high shooting percentage. The shooting percentage may or may not fall down to earth; because part of it may have been produced by their deliberate tactical choice to pass the puck more often instead of taking low-percentage shots. This may result in better percentages on the shots they do take. Even if not, the fact that they deliberately refrain from taking low-percentage shots will increase their average by omission.

Then consider that the team has had a huge amount of turnover in the past two years, and many of the offensive contributors are rookie or sophomore players. This means that we really have no baseline for what kind of shooting percentage we ought to expect. Is it unrealistic to suppose that the Flames have a lot of good shooters on their team? They didn’t have a couple of years ago – but most of those players aren’t on the roster anymore. Without a baseline, there is no possible way of measuring the deviation that might be attributed to luck.

To make my position perfectly clear:

I have nothing against advanced statistics. I like advanced statistics. But PDO is not an advanced statistic. It is an artefact produced by adding two unrelated statistics together, and it is junk.