Q&A With Athletic Calgary's Kent Wilson on Flame's Analytics

[Metro Gnome]

Originally Posted by dammage79

I've never been a fan of advanced stats. But I tend to think of them like Treliving does. They paint a small part of the picture. If they didn't, teams wouldn't use them.

So yes, they do have a function in the league but relying on them to tell your story is too binary of a logic process.

The game is far too fluid to rely on the picture they paint alone.

using them as a predictor is fool hardy as well. If player X was successful with his mates but when he gets paid as a UFA to perform like he did with those linemates and falters greatly on the new team with new players, then the stats were wrong, due to human dynamics that skew the results.

I note in the article that the ideal combination for player and team eval is a mix between real-world experience, math, and empiricism (being able to design/conduct useful experiments). Chris agreed with that.

As for your particular scenario, there are efforts to correct for the influence of linemates in various stats and models being created now, but it's definitely a big challenge. Essentially the value of the process can be evaluated when we see if a stat or model is improving our success/predictive rate.

That said, the are factors outside of models that may not be captured but may have to be considered. For example, injuries, personal factors, etc.

[Caged Great]

Originally Posted by pepper24

Where did Kent Wilson play junior and/or pro hockey? Don't see him on Elite Prospects.

You don't need to play hockey at a high level to understand the game, even though me and Kent disagree on most things.

I certainly never did. It doesn't matter. A guy like Ken Hitchcock never played the game but did alright for himself.

[Metro Gnome]

Originally Posted by evman150

What is his opinion on media stat guys like Yost and Cullen?

Both of those guys tend to do pretty good work I find. Although "analytics guys" is considered something of a monolith, the truth is there tends to be disagreement within the community about a lot of stuff. I don't necessarily grade stats writers by how much I agree with them, but how much I can follow their reasoning and if they are at least making sound arguments.

[Pellanor]

One of the biggest issues with advance stats seems to be how difficult it is to gather good data, this is why you see shots being used as a measurement for possession. What have teams been doing to gather better data, and do you see the league making any efforts to provide better data for both the teams and the fans?

[Calgary4LIfe]

Treliving has stated on a number of occasions that the Flames have their own data that they look at, and IIRC, CORSI wasn't one of them. I don't know how that has changed with the new hiring, but that is what I remember from the Flames.

As for advanced metrics as a whole, I do enjoy them. I prefer the eye-test myself, but I also do enjoy the advanced metrics. However, I can't speak for everyone here, but I have to roll my eyes every time I see advanced metrics proving a bias by using an incomplete set of metrics. This is where people who are deep into advanced metrics dig themselves into a big hole with the non-advanced metrics crowd I feel. It happens too often at times.

At the end of the day, as a fan, I am 90% about using the eye test since I enjoy watching players make good plays, and screaming when players make terrible plays. I don't need advanced metrics to help me enjoy watching a hockey game, but it provides more 'fun' to analyze players as a season progresses and helps me to focus on certain things within a game a bit better.

[Metro Gnome]

Originally Posted by Bingo

Given what happened in Montreal 18 months ago (Subban trade), I wonder what kind of voice analytics departments have around the NHL.

If I remember correctly the guy disagreed with the trade, they made it anyway, he talked against it and was let go.

I wonder what the fight against traditional hockey views looks like from the inside. My only look at it was the movie Moneyball.

One of my big questions to Chris was how to submit counter-intuitive results to decision makers. There's potentially huge value to getting one of those calls correct, but if it flies in the face of common knowledge or wisdom, it can easily be ignored or even get you fired (it's too late by the time you are proven right in that instance).

Chris talked about a couple of things in this regard: the need to have support and open-mindedness in an org (he said there's both in CGY currently), and the need, again, to include and collaborate with other stakeholders like the coaches and scouts. He also said that there should likely be an 80/20 split between your findings - meaning roughly 80% of your analysis should roughly agree with scouts and such, and 20% probably shouldn't. If you are in total agreement with scouts, then your analysis isn't adding anything. And if it's total contrary all the time, then no one will believe you.

He also talked about the need to buttress recommendations with a very clear background/narrative when presenting them. Essentially he said you have to do all of these things to build up credibility with everyone else in the org, or else you risk being siloed/ignored.

[dustygoon]

If a team has its own analytics, is that different way of looking at same data or are they gathering different data (like quality of shots, time with puck in possession, board battles won). What technology exists or will soon exist to do better than shot attempts (high shot rate by Evander Kane from the boards is less valuable than from slot for example).

[Metro Gnome]

Originally Posted by pepper24

Where did Kent Wilson play junior and/or pro hockey? Don't see him on Elite Prospects.

I played hockey for almost 10 years, coached for a couple. My brother was part of Alberta Selects team that went to Europe, my cousin played for the Calgary Canucks. My father ran hockey schools and was the head of Bow Valley area hockey program. Ironically, I have a deeper hockey background than Snow himself.

Of interest on this topic - Snow was initially hired by Risebrough BECAUSE he was a hockey outsider. Risebrough wanted someone who was curious and intelligent but had "no institutional knowledge" because he saw big changes coming to the league thanks to the salary cap (and rise of analytics in baseball).

[Metro Gnome]

Originally Posted by Pellanor

One of the biggest issues with advance stats seems to be how difficult it is to gather good data, this is why you see shots being used as a measurement for possession. What have teams been doing to gather better data, and do you see the league making any efforts to provide better data for both the teams and the fans?

Quality data is definitely a challenge. I know teams have often taken to personally gathering their own data by hand counting events during games and then going back and evaluating their data with game video. Flames have PUCKS, which allows them to call up just about any event or series of events during a season.

From there, I believe teams are building internal tools and databases like some of the public tools you see out there. This is primarily why David Johnson was hired this summer.

[corporatejay]

Ken, one question I've always had with Corsi/Fenwick is has anyone actually evaluated how close of a proxy they are to possession? Obviously they don't measure ACTUAL possession (which I agree helps drive the play) but has someone gone through and made sure that what we are trying measure is accurate measured by a stat like corsi?

Further, I think advanced stats are excellent in determining your overall game plan since it helps identify strengths and weaknesses in your lineup that you can try and cover up with style of play.


As an aside, I love the athletic, while I find your work very analytic, it acts as a nice compliment to Duha's work which is more traditional. Good pairing for this market.

[mikephoen]

Does Chris Snow have a degree in statistics or any math degree at all? One of my biggest problems in the past with the analytics community is that most of them seemed to be lawyers or from disciplines other than math. Are there any analytics guys that have a masters or doctorate in statistics or applied maths?

[Metro Gnome]

Originally Posted by Calgary4LIfe

Treliving has stated on a number of occasions that the Flames have their own data that they look at, and IIRC, CORSI wasn't one of them. I don't know how that has changed with the new hiring, but that is what I remember from the Flames.

As for advanced metrics as a whole, I do enjoy them. I prefer the eye-test myself, but I also do enjoy the advanced metrics. However, I can't speak for everyone here, but I have to roll my eyes every time I see advanced metrics proving a bias by using an incomplete set of metrics. This is where people who are deep into advanced metrics dig themselves into a big hole with the non-advanced metrics crowd I feel. It happens too often at times.

At the end of the day, as a fan, I am 90% about using the eye test since I enjoy watching players make good plays, and screaming when players make terrible plays. I don't need advanced metrics to help me enjoy watching a hockey game, but it provides more 'fun' to analyze players as a season progresses and helps me to focus on certain things within a game a bit better.

Like anything, stats have to be considered in context and the analysis/weighting of factors is often up to the analyst. Sometimes you see things like "corsi says..." execpt corsi doesn't "say" anything per se. It's simple a statement of fact and the implications of it are up to the analyst to determine (or prove).

As for the Flames right now, I frequently hear GG and Treliving say things that suggest they are looking at corsi/possession theory.

[Metro Gnome]

Originally Posted by dustygoon

If a team has its own analytics, is that different way of looking at same data or are they gathering different data (like quality of shots, time with puck in possession, board battles won). What technology exists or will soon exist to do better than shot attempts (high shot rate by Evander Kane from the boards is less valuable than from slot for example).

Mentioned earlier, but we're moving towards expected goals as a better model/stat than possession/corsi. This combines various factors including the nature and position of the shot.

Right now, this is based on just so-so data (NHL gamesheets) at least in the public domain. When/if the NHL even moves to a system that captures player position and puck position on the ice automatically, it will improve the quality of these models.

[StrykerSteve]

Kent, I’m admittedly not the biggest fan of your Athletic articles so far. You tend to lose me when you go too deep into analytics I simply don’t understand. That’s not necessarily your fault, the same could be said of anyone writing about the subject. What I’d love to see more from you, is layering it with more eye test to support the figures you’re trying to detail.

I can read a ED article and not care how long it is, because it tells a story I become invested in. Maybe try wrapping these fancy stats in a more conventional story to make it easier for a guy like me to follow. I’m a fan of what the Athletic is trying to accomplish and am subscribed for the next year, look forward to seeing your style develop.

Just my 0.02.

[Metro Gnome]

Originally Posted by mikephoen

Does Chris Snow have a degree in statistics or any math degree at all? One of my biggest problems in the past with the analytics community is that most of them seemed to be lawyers or from disciplines other than math. Are there any analytics guys that have a masters or doctorate in statistics or applied maths?

He does not. His background is journalism. He was a Boston Red Sox beat writer when he was approached by Doug Risebrough - in part because he had covered the analytics "revolution" as it happened in Boston.

I have recommended in public and during consultations with teams in the past that analytics departments should have a mix of skills, including data science/computer science, math/stats, and in-depth hockey knowledge.

[Metro Gnome]

Originally Posted by corporatejay

Ken, one question I've always had with Corsi/Fenwick is has anyone actually evaluated how close of a proxy they are to possession? Obviously they don't measure ACTUAL possession (which I agree helps drive the play) but has someone gone through and made sure that what we are trying measure is accurate measured by a stat like corsi?

Further, I think advanced stats are excellent in determining your overall game plan since it helps identify strengths and weaknesses in your lineup that you can try and cover up with style of play.


As an aside, I love the athletic, while I find your work very analytic, it acts as a nice compliment to Duha's work which is more traditional. Good pairing for this market.

Yes, in the past actual possession has been correlated with corsi. It was a high correlation, but not at all perfect. When either of those correlated with scoring and winning, corsi came out as more predictive.

[Calgary4LIfe]

Originally Posted by Metro Gnome

Like anything, stats have to be considered in context and the analysis/weighting of factors is often up to the analyst. Sometimes you see things like "corsi says..." execpt corsi doesn't "say" anything per se. It's simple a statement of fact and the implications of it are up to the analyst to determine (or prove).

As for the Flames right now, I frequently hear GG and Treliving say things that suggest they are looking at corsi/possession theory.

I have definitely heard that they are looking at possession as being critical (and that was a point that Treliving raised when Hartley was dismissed), but I do remember his speaking about CORSI and stating that they don't use CORSI in their metrics, and instead utilize something that is more accurate. Is this a recent change, or was Treliving just being coy? I wish I could find his quote, but I am pretty sure it came from a Fan960 interview a couple of seasons back. Not questioning you, just asking for some clarification.

[Boreal]

Are you familiar with the modified areal unit problem of geographic statistical problem solving? It is essentially being aware of the statistical spatial bias of whatever you are trying to evaluate. What you measure changes based on how you measure it.

https://en.m.wikipedia.org/wiki/Modi...l_unit_problem

I'm interested in your thoughts on how this bias is managed when evaluating hockey with spatial metrics.

[Boreal]

Just to sum up the Modified Areal Unit Problem. It seems to me that this is what the Oilers are doing with Kris Russel & their "own internal stats".

[Metro Gnome]

Originally Posted by StrykerSteve

Kent, I’m admittedly not the biggest fan of your Athletic articles so far. You tend to lose me when you go too deep into analytics I simply don’t understand. That’s not necessarily your fault, the same could be said of anyone writing about the subject. What I’d love to see more from you, is layering it with more eye test to support the figures you’re trying to detail.

I can read a ED article and not care how long it is, because it tells a story I become invested in. Maybe try wrapping these fancy stats in a more conventional story to make it easier for a guy like me to follow. I’m a fan of what the Athletic is trying to accomplish and am subscribed for the next year, look forward to seeing your style develop.

Just my 0.02.

Thanks for your candor Steve, I understand. One of the reasons The Athletic went with myself and EH is that we cover the game in very different ways. I have my audience and niche, which I seek to do as well as possible. Unfortunately, it's impossible for one person to be all things to all people.

That said, as I build the CGY Athletic team I plan to add people with a variety of skills. I'd like someone who can do in-depth X's and O's, someone who can do player profiles/human interest, as well as the analytics I do and high-level interviews and profiles or EH.