Corsi: have you ever built statistical models? Real, largely-scoped, used by the public, models?
Because I have. And most people have no idea how difficult it actually is. I take that back - they are easy to create - but how difficult it is to actually create useful models that do what you want them to do.
Any model will spit out results. And, assuming your inputs and variables are relevant to the subject, the model will spit out results that appear to be doing what the model is intended to do.
However, proving that the results are valid is the hard part.
I am not going to go into an essay on how to stress-test statistical results - there is an internet for that. But I can say this much with complete accuracy and justification: the burden of proof is on the model. And knowing A) that there is really not very much data when it comes to hockey, and B) the data that we have is almost impossible to completely isolate (which is what we really need), the likelihood of obtaining truly valid results for individual player performance that we can accurately and fairly compare against other players' performance is quite low.
And then there is the added problem that, with hockey stats, the vast majority of the users of the data aren't in a position to properly filter and interpret the results.
|