[CP Story] D. Ryan, the Trade and the WAR Model

[Bingo]

Originally Posted by SuperMatt18

I have a hard time with any model that says Stajan is better than Gaudreau. And I don’t mind Manny’s work, just think it needs a bit more development.

For forwards especially it relies too much on shot data and doesn’t seem to favour playmakers as much.

And I wonder on the Hamonic thing.

When I look at three years total and then sort by the shooting parameter to the negative you see almost nothing but defensemen.

Krug, Keith, Ceci, Hutton, Vatenen, Paralyko, Gudas, Severson, Bieksa, Klefbom, Roustalainen, Yandle, Orpik etc

Only a handful of forwards in the top 25 worst.

[tkflames]

Great write up Bingo!!! I usually worry that when you apply team stats to individual players that there tends to be a disconnect in actual meaning. This does not entirely appear to be the case here and I like that you have addressed the short comings of these specific statistics in the example of Hamonic.

I am pro the Derek Ryan signing at the current rates and would suggest that the Flames do nothing if the prices are $2.5M/yr x 3 years for Jay friggen Beagle.

I would be very interested to see how the WAR stats stack up on a per line and per player basis across the NHL. It would be interesting to see entire lineups not as names, but strictly as WAR/OWAR/DWAR values. I believe while there is definitely a chemistry component, I like the direction that this type of statistic is taking compared to some of the more traditional "analytic" stats.

[bubbsy]

Nice write up. In the end, the last statement sums up my feeling about the team:

"The Flames have taken some interesting steps forward this summer in making the team more solid defensively, and bringing better balance to their lineup."

We went into last year with folks applauding this team based on the supposed defence.

At what point will treliving invest on the other side of the team? Is a Ferland/lindholm swap and adding this Ryan guy really the fix for the lack of offensive and a pp?

[evman150]

I love WAR in baseball. I think it likely requires more tweaking here (like a poster said above, Stajan ahead of Gaudreau doesn't jibe), but I really enjoy the analysis.

[Enoch Root]

Originally Posted by Bingo

And I wonder on the Hamonic thing.

When I look at three years total and then sort by the shooting parameter to the negative you see almost nothing but defensemen.

Krug, Keith, Ceci, Hutton, Vatenen, Paralyko, Gudas, Severson, Bieksa, Klefbom, Roustalainen, Yandle, Orpik etc

Only a handful of forwards in the top 25 worst.

Clear flaw in the valuation.

Also, not taking quality of competition into consideration is a bit of a deal breaker.

Or (if I were inclined to look for stats that supported my view)... Hamilton bad defensively? Huh.

[Enoch Root]

Originally Posted by evman150

I love WAR in baseball. I think it likely requires more tweaking here (like a poster said above, Stajan ahead of Gaudreau doesn't jibe), but I really enjoy the analysis.

In baseball, all situations are isolated. And they are repeated over and over.

In hockey, it is 5 players against 5 other players, in constant motion. Attempting to isolate variables is impossible - there is far too much noise (from the 9 other players) to ever provide useful information.

Also (and this is the real kicker)... one of the foundational principles of statistics is that, given larger sample sizes, idiosyncrasies get averaged out. In hockey that doesn't happen. Coaches use players in certain situations, and against certain levels of competition. Also, quality of line-mates does not simply get corrected with more data points. These factors don't get 'averaged out' with larger sample sizes.

It isn't comparing apples to apples, like baseball is.

[SebC]

Where do you even find these WAR stats? My Google-fu is failing me.

[FireGilbert]

Good analysis. WAR seems like a improved indicator over Corsi however it has many flaws not including quality of competition, line mates, and situation. Stajan and Brouwer as the top defensive players jump out as a big flaw. They play against other 4th liners with the same offensive talent they have so obviously this stat will rate them highly.

I agree with others WAR is more suited to baseball where there are repeated one on one match ups. If you replace your 3rd baseman and right fielder you have some idea if your team will be better or worse. With the recent Flames trade for example it is not a simple replacement and will have ripple effects throughout the lineup. Lines will change, players will switch sides, and new chemistries will be found. It is very difficult to predict.

[Calgary4LIfe]

I am still very leery of Ryan personally. I get a feeling he is an undersized player who just found himself with a great opportunity alongside Skinner. I get vibes of Nigel Dawes or Mason Raymond - tweener types that under the right circumstances can put up points, but offers little in the way of defensive acumen (according to Canes' fans), physicalty, terrible on the boards...



I wouldn't hate it if he is signed, but I am leery. I would love to have further info as to his QofC and so on, to separate the potential 'noise' coming from Canes' fans for realities vs perceptions based on biases.


He could very well be one of the more shrewd FA signings this off-season, or he could be the next whipping boy. I have no idea, and it is always difficult to tell what a players' weaknesses are from a highlight package. He does seem shifty. WAR says he is good - what do the other metrics state?

[SebC]

Without diving too deep, since I haven't found the data, if Ferland's value is -0.2 then you can improve 0.2 simply by letting his contract expire and replacing him with a replacement-level player.


Seems like the numbers are off though, they suggest a team with 12x Monahan would only finish 15 wins better than a team with 12x Lazar. All the magnitudes seem too small and replacement level seems too high.

[Dajazz]

Better than Corsi. Doesn't make sense, at all, in some situations (like Backlund and shutdown defense men). Overall, pretty cool, although it naturally favours offensive shooters (I guess it should, scoring a goal is pretty much the whole idea of hockey).

And Ryan is going to get 3-4m for 3-4 years I think. It might be the Flames.

[Samonadreau]

Sure, Ryan would be a 2c on a Montreal, Ottawa or Vancouver.doesnt mean hes a 2 or even 3C on a contending team.

My worry is he will garner at least a 3x3 or something and we will have another 3M 4th line center. But I must admit theres worse ways to spend money on July1. And there are no bargains on July1.

If this is ALL the Flames do on July1 I will be ok with that for sure.

[Metro Gnome]

WAR is an interesting concept and people are coming along with it in hockey, but there are still wide variations between models and I'm not sure I'd trust any solid analysis on any of them.

Manny's WAR, which Bingo uses here, is very strange this season. For instace, there isn't a universe in hockey where 12 point Matt Stajan is worth more wins above replacement than 84-point Johnny Gaudreau, who was in on 40% of the Flames offense. Backlund a near replacement level player because of penalties and low SH% suggests there's some weighting that is way off somewhere in the guts of this thing.

In contrast, Chace McCallum's GAR (Goals Above Replacement) model has Gaudreau at an elite 15.9/60 and Stajan down at 3.9/60 for instance. Gaudreau leads the forwards and the team in GAR according to McCallum, in fact.

The same model has Hamilton with a GAR of 12.98/60, Ferland at 2.69/60, Hanifin at 6.96/60, Lindholm at 4.25/60 and Ryan at 7.19/60. People can play with the numbers and visualizations here.

[Bingo]

Originally Posted by SebC

Without diving too deep, since I haven't found the data, if Ferland's value is -0.2 then you can improve 0.2 simply by letting his contract expire and replacing him with a replacement-level player.


Seems like the numbers are off though, they suggest a team with 12x Monahan would only finish 15 wins better than a team with 12x Lazar. All the magnitudes seem too small and replacement level seems too high.

I think you have to look at it as four centers on a team, you can't have 12 Monahans, I guess you could have four.

Yes you take out Ferland and you improve because in this estimation he's just below average. When you take into consideration his play in his own zone maybe he's on to something.

Gaudreau and Ferland have terrible shot suppression numbers, while a player like Bennett is getting weighed down by his penchant for bad penalties. A Bennett supporter would almost suggest that shouldn't count, but taking penalties certainly hurts your hockey team so it's valid for Bennett and Backlund and Hamilton.

Monahan has an elite skill in shot generation and then doesn't have the warts that drag that number down.

McDavid is off the charts.

The site is Corsica, WAR is under skaters.

[Bingo]

Originally Posted by Metro Gnome

In contrast, Chace McCallum's GAR (Goals Above Replacement) model has Gaudreau at an elite 15.9/60 and Stajan down at 3.9/60 for instance. Gaudreau leads the forwards and the team in GAR according to McCallum, in fact.

The same model has Hamilton with a GAR of 12.98/60, Ferland at 2.69/60, Hanifin at 6.96/60, Lindholm at 4.25/60 and Ryan at 7.19/60. People can play with the numbers and visualizations here.

So 12.98 + 2.69 vs 6.96 + 4.25 has Carolina winning the deal based on last year's stats

But I agree, and as I said in the piece any model is just that, a model ... to be challenged and modified.

What I like about models though is the independent, non subjective take on all hockey players. It gives you a solid start to an even playing field.

It is interesting to see the Tierney GAR model giving an add to Gaudreau for defense while Monahan has a negative value. Not sure that adds up for me.

[Enoch Root]

Originally Posted by Bingo

So 12.98 + 2.69 vs 6.96 + 4.25 has Carolina winning the deal based on last year's stats

But I agree, and as I said in the piece any model is just that, a model ... to be challenged and modified.

What I like about models though is the independent, non subjective take on all hockey players. It gives you a solid start to an even playing field.

It is interesting to see the Tierney GAR model giving an add to Gaudreau for defense while Monahan has a negative value. Not sure that adds up for me.

Absolutely.

However, if the take is not representative (Stajan more valuable than Gaudreau?), then I would argue that it is WORSE than nothing at all. Because people will accept the numbers and make judgments from them.

On an academic level, the ongoing discussions and evolution are great. Unfortunately, end users take the numbers and conclude this player is better than that player, based on a number that has no meaning on its own.

The same problem occurs in my industry. Firms spend incredible amounts of money and resources building Optimizers that determine - scientifically! - the optimal amount that should be deployed into each asset class in an asset allocation model. These optimizers include substantial amounts of data including cross-correlations, etc, etc and are, be ANY account, extremely impressive. However, they are fundamentally flawed because the input data, no matter how many years you can acquire (typically 20 to 40 years), is still merely a sample and not necessarily representative. And (most importantly) tiny changes to inputs have MASSIVE changes to the outputs. As a result, the outputs are garbage. However, end users see numbers coming from a computer and see: "you should be 27.4% US Equity!" and accept it as gospel.

The same issues happen with advanced stats in sports. The problem is that there is only a tiny fraction (as in, infinitesimal) of the amount of research (if you can even call it research) going into these sports models, and little to no actual constructive testing of them. Yet people have the same problem of "these numbers come from an unbiased model, and are tested, and are fair for everyone" and people accept them as being information, when in reality they are no where close to being information yet.


(Note: I know I criticize the 'advanced' stats a lot, but I want to make it clear that it is directed only at the stats and the way end users interpret them (not the efforts to develop them). Bingo, your articles are awesome and the efforts that you put into creating them, and bringing other ideas to the conversations etc, should be commended. I for one, greatly appreciate them and think they bring a great deal to the site to discuss. But I am still going to attack these stats until such time as I think they warrant a different view.)

[Bingo]

I totally agree.

And I think if you're a hockey executive you don't take anything like this as a gospel, but you do start asking yourself some questions.

Gaudreau is the big case in my mind.

Elite offensive player, but does he give back in other areas that in some way is hurting his ability to help his team win games to the level he should be?

So by area ...

Shot Suppression - Gaudreau is 2nd on the team behind Chris Stewart (so essentially first). This stands out when Monahan is considerably less.

Shot Differential - Gaudreau doesn't shoot the puck enough clearly as he brings a negative differential to the table despite being an elite offensive player.

Quality of shots - here is where Stajan explodes because he's positive in generating quality (small), but elite in making sure the other team doesn't get quality chances. It skews his numbers way up and is somewhere to focus. Next closest guy on the team is Sam Bennett by the way. In quality shot generating Gaudreau is 2nd and Bennett is 3rd.

Shooting - not sure what this means, the Flames don't have a single guy above zero (replacement). They missed the net a lot, that may factor in.

Gaudreau could tighten up and shoot more and he'd be a bigger factor, but the quality shot suppression area of the model does look a bit off with Stajan boosted.

[Enoch Root]

Or maybe Gaudreau just needs a different guy on RW. It's a team game and roles are important (just another reason why I think stats like this are meaningless - your line mates matter. A LOT).

And again, if I find myself comparing Gaudreau and Stajan - in any way whatsoever - my first thought is that the input parameters are off.

[GioforPM]

Originally Posted by Enoch Root

Or maybe Gaudreau just needs a different guy on RW. It's a team game and roles are important (just another reason why I think stats like this are meaningless - your line mates matter. A LOT).

And again, if I find myself comparing Gaudreau and Stajan - in any way whatsoever - my first thought is that the input parameters are off.

Stajan is better at faceoffs.

[868904]

I don't think it's a shock to anyone that Gaudreau is weak defensively. You don't need advanced stats to know that, just watch the game. It's not even about effort either as he back checks hard most times. It's really a size issue, he just doesn't have the strength to help effectively.

I think his defensive shortcomings are why Peters is so keen to put Lindholm with him. The hope is putting a good defending on that line will balance the defensive deficiencies and also hoping that Lindholm will benefit from some good offensive looks.