[Jay Random]

Originally Posted by Enoch Root

Except that people use part of the season (for example, the first 29 games) to represent the season, and as such it is a sample.

If people do that, that's on them and not on the data. The question is how large a population of data you need for it to have predictive value; which is not the same as the question of statistical significance, though obviously some of the same problems arise in dealing with it.

Also, if each data set is in fact its own population, then extracting meaningful information from it, with respect to providing insight into the future, is also, by definition, not valid. Either it is a subset of the universe of data, and thus valid for analysis, or it is its own data set, and thus of little value.

It is the set of all data that exist at the present time for the conditions being observed. The possibility of bias due to an insufficiently random sample does not figure into it. In any case, we were talking about data concerning Hamilton playing as a Calgary Flame, so the statistical universe for that situation is closed and complete. There is not, and probably never will be, an unfinished season in which Hamilton plays for the Flames.

This also doesn't solve or address the problem of non-isolated variables, and the amount of noise in the data.

Yes, now we're down to the meat and potatoes. Hockey statistics are notoriously non-orthogonal, and there is hardly such a thing as an isolated variable in the whole shooting match.

In the case of Hamilton, the best analytic tool I know of offhand is WOWY data, which at least removes the noise introduced by his playing with different teammates. It's been a while since I looked at those data, but as best I can recall, there was no smoking gun: Hamilton did not perform strikingly better with one particular teammate than without him, which pretty much rules out the hypothesis that he was being carried by (for instance) Giordano.

In fact, the basic counting data seem to suggest that Hamilton was not particularly bad defensively. (That is, not noticeably worse than the team as a whole; which is not much of an endorsement.) If the eye test suggests that he was, that may be down to the problem of ungraceful failure. To paraphrase what one of Winston Churchill's friends said about him: ‘Hamilton was usually right, but when he was wrong, my God.’