Quote:
Originally Posted by Enoch Root
But I can say this much with complete accuracy and justification: the burden of proof is on the model. And knowing A) that there is really not very much data when it comes to hockey, and B) the data that we have is almost impossible to completely isolate (which is what we really need), the likelihood of obtaining truly valid results for individual player performance that we can accurately and fairly compare against other players' performance is quite low.
|
Well, maybe I'm expressing this incorrectly. I guess what I mean is that the burden has been met, once the person who's created the model has explained how the model has been developed, how it has been tested, and how it has performed. They've done that. So now it would be up to whoever is attempting to discredit the model and claim that it isn't actually useful in the ways that the developer says it is to explain why they're wrong, and why the degree of accuracy they claim to have achieved is a mirage. As you say, you have some experience here, which would seem to suggest that you're qualified to do just that.
I obviously can't explain why you, as one person with some knowledge and expertise on statistical models, have come to the conclusion that the data available for hockey is inadequate to yield results that can be fairly used to compare one player to another, while another person with knowledge and expertise has come to the opposite conclusion. You'd have to explain where the difference of opinion lies.
I certainly see the challenge you're highlighting when talking about reliably assigning outcomes (data) to individual players. I think most hockey analytics people would agree that that's the main source of error in their predictions. But it seems to me that there are enough data points to narrow the error bars to an extent where the model output is reliable enough to be useful and worthwhile. Again, the bar to clear here is nowhere ear perfection.
Quote:
|
And then there is the added problem that, with hockey stats, the vast majority of the users of the data aren't in a position to properly filter and interpret the results.
|
Well, that's definitely true, though it isn't the fault of the person doing the analysis. If you want to tell me I'm misinterpreting their results and how I'm doing that I'm all ears.