[Enoch Root]

Originally Posted by CorsiHockeyLeague

Nope, even after you adjust for opponents it was still better. I think Tanev was just done in Vancouver.

What does that even mean? He came to CGY and proved that he isn't done - did he suddenly become undone?

It is far more plausible that either A) the model is failing, or B) there was something else going on that the model didn't account for. Perhaps Tanev was playing injured during the season where his stats sucked (for instance)

Yes. Or at least, the guy who created the one we're talking about says he's tested it, and I don't know why he'd lie. Dom L from the Athletic runs his model thousands of times and then evaluates its performance against the results and against other models at the end of the year to determine whether or not it'd make money if you used it as a betting tool.

Pretty safe to assume he isn't lieing. But the question is: what was tested? And how was it tested? One of my contentions is that these stats take team events, and then apply the results to individuals. If the testing focuses on team success, it will probably prove to be quite reliable.

But we wan to draw conclusions about individual player performance from the model - how can we even test that? ow do we assess whether it accurately tells us that Player A is terrible defensively? Again, individual performance - especially defensively - is highly dependent on situation. That being the case, how do we extract situation effects from the data? How can that be tested? And then, do we see consistency in the results when we look at that player in a new situation? (and again, we have to then extract the new situational effects from the new data - ow the hell can we then compare that to the prior data? And how can the model test for that?

The fundamental problem is that we have to use team data, and then try to account for it. But we have no way of knowing (or by extension, testing) if our efforts to do that have been effective.

Actually, the model we're talking about has an R2 of .888, although that's at the team level. It's obviously going to be less accurate at the individual player level, but that doesn't mean it doesn't have any utility. It just means that if you use it as a predictor of next season, it's going to be close in many players' cases, a bit off on some others, and wildly incorrect on a few.

Same issue, team stats vs individual results. We have a high correlation at the team level - well yeah, because it is team data. We can assume that there will be some transference of applicability at the individual level, but we can't know how much. And when we see how much individual player's stats can vary, when they move from team to team, or up and down the lineup, it suggests not very much.