FanPost

Comparing CSS and ISS rankings

The NHL's Central Scouting Service provides the best-known and most often cited rankings of draft-eligible prospects, but the International Scouting Service and other organizations/draft pundits provide alternatives in the imprecise art and science of draft rankings.

Direct comparison of the rankings is difficult because the CSS always publishes separate lists for North American skaters and European skaters.  But beyond agreeing that Ryan Nugent-Hopkins is the top North American skater and that Adam Larsson is the top European skater, the CSS and ISS lists differ significantly.  The ISS ranks Sean Couturier #3 overall behind those two, while the CSS ranks Gabriel Landeskog, Jonathan Huberdeau, Dougie Hamilton, and Nathan Beaulieu ahead of him.  And though the ISS also ranks Landeskog and Hamilton highly, they rank Huberdeau at 8th overall and Beaulieu at 13th overall.

So I set out to find answers for the following questions:

(1) How predictive are the CSS and ISS rankings for the actual draft order at the top of the draft (the top 10 picks)?  By inference, how much do NHL GMs and scouts agree with these rankings?

(2) How good are the two at actually predicting which players will have the most NHL success?

I'll analyze the ISS top 10 and the CSS top 10 in North America and the CSS top 3 in Europe each year from 2004-2008.  Also, out of curiosity, I'm including the draft list that TSN's Bob McKenzie publishes every year.  McKenzie doesn't claim to prognosticate the players' talent - he states it represents the opinions of a sample of scouts he has connections with.  He also doesn't really consider it to be a mock draft as he doesn't take into consideration the "needs" of individual organizations, but it would be a mock draft if every team drafted the "best player available".

First, 2008:

2008 CSS-NA CSS-Int ISS Actual Draft TSN (McKenzie)
1 Steven Stamkos C Nikita Filatov W Steven Stamkos C Steven Stamkos C Steven Stamkos C
2 Zach Bogosian D Kirill Petrov W Zach Bogosian D Drew Doughty D Drew Doughty D
3 Drew Doughty D Mattias Tedenby W Nikita Filatov W Zach Bogosian D Zach Bogosian D
4 Tyler Myers D Drew Doughty D Alex Pietrangelo D Alex Pietrangelo D
5 Luke Schenn D Alex Pietrangelo D Luke Schenn D Nikita Filatov W
6 Alex Pietrangelo D Luke Schenn D Nikita Filatov W Luke Schenn D
7 Kyle Beach C Mikkel Boedker W Colin Wilson C Mikkel Boedker W
8 Zach Boychuk C Colin Wilson C Mikkel Boedker W Colin Wilson C
9 Cody Hodgson C Cody Hodgson C Josh Bailey C Cody Hodgson C
10 Colin Wilson C Josh Bailey C Cody Hodgson C Kyle Beach W

I start with 2008 as the most recent draft I'll analyze.  First, props to McKenzie for doing a pretty good job of predicting the draft order.  He got the top 4 right and got 5-10 mostly right minus some minor changes in position.  The top 10 on the ISS list were the top 10 drafted, with the only big positional change being Nikita Filatov (who dropped due to uncertainty at the time over the lack of a NHL-KHL transfer agreement).  The CSS was significantly less predictive of draft order in 2008 - Myers, Beach, and Boychuk all saw players like Wilson and Hodgson go before them.

Now, which ranking got it more right?  It's tough to say exactly where the CSS thought Filatov deserved to be drafted, but regardless both the CSS and ISS appear to have swung and missed on him thus far in his career.  In a famously D-heavy draft, none of the top D prospects have disappointed thus far in their careers - Doughty and Schenn had immediate NHL success while Bogosian and Pietrangelo have been brought along more slowly, but both still have bright futures.  Obviously everyone got Stamkos right.  The rest of the top-10 in either list has been a rather unremarkable bunch so far - they've all made the NHL but have yet to prove that they're impact players.

The tiebreaker has to go to the CSS for making the call on Tyler Myers, however.  The ISS missed the boat on that one.

(1) Draft order prediction: ISS win (McKenzie was better)

(2) NHL success prognostication: CSS win

Next, in 2007:

2007 CSS-NA CSS-Int ISS Actual Draft TSN (McKenzie)
1 Kyle Turris C Alexei Cherepanov W Patrick Kane W Patrick Kane W Patrick Kane W
2 Patrick Kane W Mikael Backlund C James vanRiemsdyk W James vanRiemsdyk W James vanRiemsdyk W
3 James vanRiemsdyk W Lars Eller C Kyle Turris C Kyle Turris C Kyle Turris C
4 Keaton Ellerby D Alexei Cherepanov W Thomas Hickey D Karl Alzner D
5 Karl Alzner D Jakub Voracek W Karl Alzner D Alexei Cherepanov W
6 Sam Gagner C Karl Alzner D Sam Gagner C Jakub Voracek W
7 Jakub Voracek W Sam Gagner C Jakub Voracek W Sam Gagner C
8 Angelo Esposito C Maxim Mayorov W Zach Hamill C Angelo Esposito C
9 Zach Hamill C Keaton Ellerby D Logan Couture C Brandon Sutter C
10 David Perron W Mikael Backlund C Keaton Ellerby D Keaton Ellerby D

McKenzie does OK this time.  He predicts the top 3 in order, and gets Alzner, Gagner, and Voracek pretty close.  No one saw Thomas Hickey coming at #4, and McKenzie struggles a bit in picks 8-10.

The ISS, too, picks the top 3 in order.  Cherepanov fell much farther than the ISS or McKenzie anticipated, but the ISS gets the draft order for Alzner, Gagner, and Voracek pretty close.  The ISS and CSS, like McKenzie, miss on picks 8-10.  The CSS, however, ranks Keaton Ellerby much higher than the NHL teams evidently did and fails to predict 1-3 in the correct order.

Now, in a "weak" draft year, how did the CSS and ISS do?  Well, the CSS struck out with Kyle Turris at #1, and the ISS has been right thus far in predicting Patrick Kane as the best among the most-hyped players in this draft and JvR is starting to look good as the second-best.  CSS also missed with Keaton Ellerby, Angelo Esposito, and Zach Hamill.  Did the ISS miss with Cherepanov?  Tough to say - he died at the age of 19 while playing in the KHL.  We'll never know, but it does appear the ISS missed by putting Maxim Mayorov so high - and badly so.  NHL teams didn't think enough of him to pick him before the fourth round!  Nonetheless, I'll give the ISS the win in a year where the talent wasn't so prominent.

(1) Draft order prediction: ISS win (tie with McKenzie)

(2) NHL success prognostication: ISS win

In 2006:

2006 CSS-NA CSS-Int ISS Actual Draft TSN (McKenzie)
1 Erik Johnson D Niklas Backstrom C Erik Johnson D Erik Johnson D Erik Johnson D
2 Jordan Staal C Michael Frolik W Jonathan Toews C Jordan Staal C Jonathan Toews C
3 Jonathan Toews C Jiri Tlusty W Phil Kessel C Jonathan Toews C Phil Kessel C
4 Phil Kessel C Jordan Staal C Niklas Backstrom C Jordan Staal C
5 Derick Brassard C Peter Mueller C Phil Kessel C Niklas Backstrom C
6 Peter Mueller C Michael Frolik W Derick Brassard C Derick Brassard C
7 Bryan Little C Niklas Backstrom C Kyle Okposo W Peter Mueller C
8 Chris Stewart W James Sheppard C Peter Mueller C Kyle Okposo W
9 James Sheppard C Bryan Little C James Sheppard C Jiri Tlusty W
10 Cory Emmerton C Kyle Okposo W Michael Frolik W Bryan Little C

McKenzie gets the top 5 right, but in slightly different order as Kessel dropped and Staal rose.  He also gets picks 6-8 right, in slightly different order.  He misses on picks 9 and 10.  The ISS doesn't do as well - they rank Backstrom lower and Frolik higher than scouts evidently did.  The CSS, however, impressively picks the top 5 North American skaters in the exact order they were drafted.  Where did they think Backstrom and Frolik would fall?  Tough to say, but I give CSS the win for predicting the order, though they didn't see Kyle Okposo going so high.

Both CSS and ISS put Erik Johnson at #1.  They were both high on Staal, Toews, and Kessel - and rightly so.  All of the ISS top-10 became pretty good NHL players except James Sheppard, who ISS thought of more highly than CSS.  CSS deserves minor props for putting Chris Stewart high, though who knows whether he would have definitely been in their top 10 of NA and Euro skaters combined.  ISS made the mistake of ranking Frolik above Backstrom, however, and that is enough to seal the deal on another CSS win.

(1) Draft order prediction: CSS win (McKenzie better than ISS)

(2) NHL success prognostication: CSS win

2005, the Crosby sweepstakes:

2005 CSS-NA CSS-Int ISS Actual Draft TSN (McKenzie)
1 Sidney Crosby C Anze Kopitar C Sidney Crosby C Sidney Crosby C Sidney Crosby C
2 Benoit Pouliot W Martin Hanzal C Benoit Pouliot W Bobby Ryan W Jack Johnson D
3 Bobby Ryan W Jakub Vojta D Jack Johnson D Jack Johnson D Benoit Pouliot W
4 Jack Johnson D Gilbert Brule C Benoit Pouliot W Bobby Ryan W
5 Gilbert Brule C Bobby Ryan W Carey Price G Anze Kopitar C
6 Luc Bourdon D Anze Kopitar C Gilbert Brule C Gilbert Brule C
7 Kenndal Mcardle W Marc Staal D Jack Skille W Carey Price G
8 Ryan Parent D Ryan O'Marra C Devin Setoguchi W Marc Staal D
9 Marc Staal D Marek Zagrapan C Brian Lee D Jack Skille W
10 Devin Setoguchi W Carey Price G Luc Bourdon D Ryan O'Marra C

A little bit of a monkey wrench thrown into the proceedings as a goalie (Carey Price) makes his first appearance in the top 10.  We'll just have to ignore him.

McKenzie does OK here.  Everyone from here to Mars could see Crosby coming at #1, and McKenzie gets picks 2-7 mostly right, though Kopitar and Staal dropped.  Again, he struggles with picks 8-10.  The ISS also does OK after Crosby in terms of predicting draft order at the top, but does pretty bad in picks 7-10, with none of the actual draft picks anywhere to be found in the ISS top 10.  The CSS does a little better - the order is off but Bourdon and Setoguchi were indeed top-10 picks.

Which list has predicted NHL success better this draft year?  That's a tough one.  Both lists had high expectations for Benoit Pouliot that weren't met, as well as for Gilbert Brule.  The top 5 is pretty much a wash, though perhaps CSS deserves a little credit for ranking Bobby Ryan that highly.  We'll never know about Bourdon, another prospect who died tragically young, though it looks like they missed on Kenndal Mcardle.  ISS did well ranking Marc Staal high, though they had misses of their own in Ryan O'Marra and Marek Zagrapan.  So who wins?  I'd call it a tie, but the division into North American and European and skaters and goalies feels like a big-time cop-out so my verdict is that ISS gets the win by a hair.

(1) Draft order prediction: CSS win (McKenzie better than ISS)

(2) NHL success prognostication: ISS win (for guts)

2004 (A Euro-heavy year at the top):

2004 CSS-NA CSS-Int ISS Actual Draft TSN (McKenzie)
1 Andrew Ladd W Alexander Ovechkin W Alexander Ovechkin W Alexander Ovechkin W Alexander Ovechkin W
2 Cam Barker D Evgeni Malkin C Evgeni Malkin C Evgeni Malkin C Evgeni Malkin C
3 Alexandre Picard W Rostislav Olesz C Lauri Tukonen W Cam Barker D Cam Barker D
4 Kyle Chipchura C Andrej Meszaros D Andrew Ladd W Al Montoya G
5 Wojtek Wolski W Robbie Schremp C Blake Wheeler W Lauri Tukonen C
6 Boris Valabik D Cam Barker D Al Montoya G Marek Schwarz G
7 Drew Stafford W Al Montoya G Rostislav Olesz C Andrew Ladd W
8 Dave Bolland C Rostislav Olesz C Alexandre Picard W A.J. Thelen D
9 Mike Green D Andrew Ladd W Ladislav Smid D Ladislav Smid D
10 Robbie Schremp C Wojtek Wolski W Boris Valabik D Rostislav Olesz C

Ah, the year of Ovie and Malkin.  Again, we have a goalie picked in the top-10 (Al Montoya).  McKenzie does well by predicting the top 3 in order, but he misses with Tukonen, Schwarz, and Thelen.  The ISS, too, bit on the Tukonen hype and ranked Meszaros and Schremp more highly than they actually went.  The CSS had issues of their own - they ranked Chipchura far higher than anyone else did, and Picard and Wolski were also much higher than where they were actually drafted.  All around, an epic fail for all the pundits.

What of the future NHL performance of these prospects?  This is tough. The top 3 European prospects all went in the top 7 picks of the draft, so should I ignore 8-10 on the CSS's NA skaters, which includes some pretty good players in Bolland and Green?  Ultimately, neither list did very well.  I give CSS the win because Tukonen was a big swing and miss for the ISS (though the ISS did well to predict Meszaros, who turned out pretty good).

 

Conclusion

Bob McKenzie has done pretty well at predicting the top 7 picks of the draft.  He doesn't do that great with picks 8-10, but that's understandable.  It will be interesting to see the list he comes up with in June, as that may be a guide for some teams to consider trading up/down based on what other teams' opinions of the prospects are.

It seems that teams agree with the ISS more than the CSS in recent years - and the ISS is certainly a better guide for us in this regard because they publish one consolidated list instead of four.

Both the CSS and ISS have gotten better at prognosticating NHL success as time has gone on, and it seems the teams have too.  Even though the players drafted more recently are younger, a higher percentage of top-10 picks in 2006-08 have made a significant impact in the NHL than the ones drafted in 2004-05.  Perhaps that is more a reflection of the opportunities they get in the cap era, but it bodes well for us, even in a weak draft year.

The CSS and ISS have both whiffed on a few hyped prospects.  Ironically, the ISS has whiffed more on European prospects, notably Mayorov, Tukonen, Zagrapan, and the Frolik/Backstrom mixup.  The CSS contributed to the Turris and Ellerby hype and had some misses in 2004 and 2005.  The CSS seems to have done a better job with some less-hyped players, however, like Tyler Myers, Chris Stewart, and a few others.  Overall, I'd say the CSS has done a slightly better job of predicting NHL success, but marginally so.

Considerations for the upcoming draft

2010 CSS-NA CSS-Int ISS
1 Ryan Nugent-Hopkins C Adam Larsson D Ryan Nugent-Hopkins C
2 Gabriel Landeskog W Mika Zibanejad C Adam Larsson D
3 Jonathan Huberdeau C Jonas Brodin D Sean Couturier C
4 Dougie Hamilton D Gabriel Landeskog W
5 Nathan Beaulieu D Dougie Hamilton D
6 Sean Couturier C Ryan Murphy D
7 Sven Baertschi W Ryan Strome C
8 Ryan Strome C Jonathan Huberdeau C
9 Ryan Murphy D Mika Zibanejad C
10 Duncan Siemens D Joel Armia W

Back to the beginning, when I talked about this year's rankings.  Nugent-Hopkins, Larsson, Couturier, Landeskog, Hamilton, Strome, Huberdeau, and Murphy are all considered top-10 prospects by both the CSS and ISS.  That bodes well for their likelihood of making it to the NHL.  The two lists, however, probably disagree more this year in picks 2-7 than in any year since 2004.

Are Jonathan Huberdeau and Nathan Beaulieu this year's Tyler Myers, underestimated by the ISS, or are they more like Keaton Ellerby, overestimated by the CSS?  Is Sven Baertschi this year's Chris Stewart or is he more like Kenndal Mcardle?  Is Landeskog this year's Kyle Turris, a player on whom the CSS has a man-crush and about whom the ISS is a little more reserved?  Or is Couturier this year's Lauri Tukonen, thought of highly by the ISS but not so much by the CSS?

All FanPosts and FanShots are the respective work of the author and not representative of the writers or other users of In Lou We Trust.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join In Lou We Trust

You must be a member of In Lou We Trust to participate.

We have our own Community Guidelines at In Lou We Trust. You should read them.

Join In Lou We Trust

You must be a member of In Lou We Trust to participate.

We have our own Community Guidelines at In Lou We Trust. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9355_tracker