[Itse]

Originally Posted by Cecil Terwilliger

Well an NHL team should hire you because clearly you have access to stats that the rest of us don’t. Either that or you still aren’t willing to recognize that things like wins and goal diff aren’t predictive, they are the results. They provide no useful data from game to game. This is about so much more than saying you think the correlation between goal diff and points is a positive one.

Proof of the predictive power of standings and goals has been posted on this board dozens of times by now. Just because you don't bother to read the facts doesn't change the fact that they exist.

But here, let's go again.

Goal-based Metrics Better than Shot-Based Metrics at predicting hockey success

NHL teams know better than you. The fact that since you don’t know you dismiss the entire concept of advanced stats makes this an effort in futility.

Wait, I thought we can't know how the Flames use stats? Make up your mind?

Me not following posts on CP that discuss advanced stats is irrelevant to their usefulness to NHL teams because you don’t know what they are.

Ok, so essentially your argument is that there might be advanced stats nobody knows about that are actually good, and therefore you shouldn't dismiss it? This is just absolutely ridiculous.

But okay, let's limit the argument to advanced stats we know exist and are used, shot-based metrics, and agree that there might exist stats we don't know about that provide useful additional information.

(I actually assume there are, but this isn't what is commonly meant by "advanced stats" and thus I find it a disingenuous point.)

BTW, while that SJ journal article isn't supposed to be about "advanced stats are useless", this is not what the numbers really show.

Sure, they have some predictive power, but they are just not that great, and there's never been shown any proof that you can combine them with primary stats in a meaningful way the way he suggests.

Here's a few quotes:

Despite the recent trend towards shot-based metrics for evaluating team and individual success, I found that comparable goal-based metrics consistently outperformed shot-based metrics at predicting team success and individual player contributions to that team success. Linear models showed that the best single predictor of team success was the amount of goals a team allows, while the best overall model predicted team success using goal differential. Of all single parameter and composite parameter models, those incorporating goals invariably outperformed those using shots. The “shots for” model was not even as highly ranked as the “faceoff wins” model. When applied to data from 2015-16, which was withheld from model building, the top goal-based model “goal differential” correctly predicted future winning %, while the comparable shot-based model “shot differential” did not.

Given the poor value of shot-based metrics at predicting team success, it was then not unexpected that shot-based metrics were also poor measures of individual player contributions to team success.

Also, the argument that "since teams are using them they must be useful" is ridiculous even if assume that you somehow know they are actually making significant decisions based on them. (We don't know how many teams use them and how, so your argument that somehow 31 teams know something I don't is just an appeal to authority who's stand you don't actually know.)

Here's the thing; most statistics are not actually reliable enough to base single actions on. Like, you can't hire a man over a woman based on gender-based statistics because the variation between individuals is so big. Even if statistically men were better at something.

Even if there's strong statistical evidence that men are stronger than women, if you have actual strength tests, adding gender to that equation doesn't actually improve your decision making process.

Despite this, people and organizations CONSTANTLY do things like this, to the point where you have to go out of your way to hide information from them to get them to ignore it. Like hide gender information from applications.

It's also so much easier to defend your decision making if you say you based them on some stat rather than intuition, or more stats rather than less. Even if that means including stats that don't add value but rather just confuse the issue.

In hockey there's just no evidence that shot-based metrics are a good tool for acquiring players.
Of course there's the problem that this is almost impossible to prove either way, because there's just so much randomness and so many variables and the number of players moved each year is just too small. IMO a good statistician with understanding of basic psychology should say that most likely it's just better to mostly ignore the advanced stats.

The human psychological desire to believe in what looks like evidence is essentially how we got the science replication crisis. If scientists specifically trained to not get fooled by statistical evidence constantly get fooled by it on a massive scale, I don't see why it wouldn't be likely that you can easily fool most NHL GM's.