Draft Rater: Prospects first to worst
Comment Email Print Share
Insider
Hollinger By John Hollinger
ESPN Insider


The truth about analytical methods is that once in a while you'll get a result that flies in the face of conventional wisdom. When that happens, it means one of two things: 1. The analytics saw something that everybody else couldn't see. Or 2. Everybody else saw something that the analytics couldn't see.

And in the case of two particular players in this year's NBA draft, it will be very interesting to find out the answer.

The draft is Thursday, June 25, and now that we know who's in and who's out, it's time to unveil this year's Draft Rater -- a statistical projection of the top NBA prospects coming out of the college ranks.

To review for the uninitiated, the Draft Rater looks at a player's college production in a variety of metrics and a few other salient facts (such as his height, birth date and years of college experience), and from that projects what a player's player efficiency rating will be when he reaches his peak.

The basic idea is to use the NBA's past to predict its future. The Draft Rater looks back at prospects from past drafts and then, using regression analysis, identifies which attributes determined pro success and which didn't. My database of college players goes back to 2002, which is still a bit limited, but the rater gets smarter each year because it has more information with which to work -- not only an extra year of drafts but also an extra year of pro seasons from every prospect.

This year, several subtle changes helped reduce the error rate when back-tested on previous drafts. First, I ran a separate regression for each of the three position categories -- point guards, wings and bigs -- something that wasn't really feasible when I started doing this. But now that the pool of prospects is large enough, this method has produced greater accuracy.

Second, instead of tying the projection to a player's third-year PER, I used a more general descriptor of what his peak value was, allowing me to minimize the impact of fluke seasons and better adjust for some players who entered the league young and didn't max out until their fourth or fifth season. (Some of these players will perform much better than projected, but keep in mind that it's all relative. For more on why the projections seem low, see this explanation.)

Using those changes, I was able to reduce the standard error in the projections from last year's 4.0 to this year's 2.8. This means nothing to 98 percent of you, but the number geeks in the crowd will recognize that this is still quite large -- as you might expect when you're trying to project what a 19-year-old will do when he's 25. Nonetheless, it represents a significant improvement from last year.

The one area where the method still appears to struggle is with one-and-done freshmen, and this speaks to a more general problem: Information is the key to making this thing work, and the more information we have, the better. For players who leave after their first year, the picture is often incomplete, whether we're using a statistical model or traditional scouting.

I bring this up because last year, in particular, was a rough one for the projection system. First, it was an unusual rookie class in general because nearly every player taken in the first round was at least somewhat productive; generally, a draft will have 10 to 12 impactful players and the rest will be filler, but this past season blew that standard away.

Moreover, a number of those players played only one college season, and although the rater had an accurate view of a few (such as Kevin Love and Michael Beasley), it missed the boat on some who performed extremely well (including Derrick Rose to an extent, and O.J. Mayo, Anthony Randolph and Eric Gordon). Gordon is perhaps easier to understand because he was playing hurt at Indiana and his primary skill (shooting) didn't show through statistically, but that doesn't excuse the others.

One important thing to point out is that the Draft Rater is rating pro potential, which is sometimes different from pro performance, depending on the professionalism and work ethic of the player involved. In other words, the fact that Michael Sweetney and Shawne Williams rated very highly in previous seasons isn't necessarily a damnation of the system. Rather, their off-the-court habits are the type of thing every general manager has to take into account when evaluating players and something that is usually invisible when looking at their college performance.

That said, before last season, the Draft Rater had performed extremely well.

From 2002 to 2007, 15 players were (a) among the first 10 collegians drafted and (b) excluded from the top 12 by the Draft Rater. In other words, these were the college players the Draft Rater thought were drafted too high. Of those 15, not one has played in an All-Star Game. The only two who have started a significant number of games in the long term have been Kirk Hinrich (who was 13th in the Draft Rater in 2003) and Charlie Villanueva.

Who were the other top-10 picks not favored by the Draft Rater? Spencer Hawes, Acie Law, Fred Jones, Melvin Ely, Marcus Haislip, Jarvis Hayes, Rafael Araujo, Ike Diogu, Channing Frye, Randy Foye, J.J. Redick and Patrick O'Bryant.

In other words, when the Draft Rater has suggested avoiding a player, that has turned out to be good advice.

The Draft Rater also has spotted some of the biggest steals in recent years:

• Carlos Boozer was the 26th collegian taken in 2002; Draft Rater had him second.

• Josh Howard was 17th in 2003; Draft Rater had him fifth.

• Danny Granger was the 13th collegian in 2005; Draft Rater had him third.

• Rajon Rondo was the 16th collegian taken in 2006, but Draft Rater had him second.

• Rodney Stuckey was the 14th collegian chosen in 2007; Draft Rater had him fifth.

• And last year, two players the Draft Rater had rated much higher than others did, Mario Chalmers and George Hill, had productive rookie seasons.

So, most of the time, when the Draft Rater puts a player in the top five, there's a good reason.

All of which leads us to 2009 and whom the Draft Rater likes and doesn't like.

This year, the Draft Rater is closer to the general draft consensus than usual, with two glaring exceptions that I referenced above.

Let's get to them:

The pleasant surprise: Ty Lawson

Two players are neck and neck for the top spot in this year's Draft Rater. You could easily guess that one of them is Blake Griffin, but most folks never would have guessed that the other is Lawson.

Lawson, who is coming off an electric performance in leading North Carolina to the championship, grades out highly for several reasons: Although he's short for a point guard, his shooting numbers (47.1 percent on 3-pointers), strong assist rate and microscopic turnover ratio (9.1, first among point guard prospects) all point to him as an NBA keeper.

The Draft Rater puts Lawson slightly ahead of Griffin for first, but this doesn't mean a team should take Lawson first. The standard error in the projections for point guards is higher than it is for big men, which means random noise could be putting Lawson ahead just as easily as on-the-court performance. If the consensus is that Griffin is the better player, I don't think Lawson's statistical record alone is strong enough evidence to refute it. Additionally, we've heard questions about Lawson's work ethic and injuries.

But the rating is emphatic enough for me to say Lawson should be at the top of the college point guard ladder, ahead of Jonny Flynn, Jrue Holiday, Jeff Teague & Co. (If you're wondering about Ricky Rubio, I'll have more on him next week.)

The unpleasant surprise: DeMar DeRozan

I'd be hard-pressed to name a potential high lottery pick throughout the years about whom the Draft Rater has been less excited. I rated 90 prospects for this draft, and DeRozan ranked 54th among them. Two of his USC teammates -- Daniel Hackett and Taj Gibson -- outranked him, as did assorted other nonentities such as Kevin Rogers, Chinemelu Elonu and Ben Woodside. I'll wait here while you Google them.

Why? Because there really isn't anything in DeRozan's statistical profile that makes you think "NBA star." He rarely took or made 3-pointers, and he had a strongly negative pure point rating, which are two powerful indicators for a wing player. His numbers in other areas were unimpressive, too. In particular, he was a bad free throw shooter, which indicates that his outside shot might never be a strong suit.

Some scouts I have talked to have compared DeRozan to Rudy Gay in terms of being an NBA athlete but having a questionable motor. But that comparison falls flat, according to the Draft Rater: Gay was the top-rated player in his draft class, while DeRozan is nowhere close. And although he's supposed to be a great athlete, he didn't show it on the court often enough: His rebound, block and steal totals were all very ordinary.

As I mentioned above, one-and-done players sometimes fool the system -- they're the youngest, least experienced guys in the pool, and thus, a major factor is how much they improve post-draft rather than just how good they are pre-draft.

Nonetheless, I would back away from DeRozan if the 12 relatively safe guys at the top of the Draft Rater are still on the board.

Speaking of which, let's take a look at the collegians for 2009.

Rankings: The top 12

Top 12 rated collegians for 2009

Player 	        School  	Draft Rater
1. Ty Lawson 	        North Carolina 	16.34
2. Blake Griffin 	Oklahoma 	16.21
3. Tyreke Evans 	Memphis 	15.02
4. Austin Daye 	        Gonzaga 	14.24
5. Stephen Curry 	Davidson 	14.18
6. Nick Calathes 	Florida 	13.66
7. DeJuan Blair 	Pittsburgh 	13.56
8. Danny Green 	        North Carolina 	13.28
9. Jonny Flynn  	Syracuse 	12.99
10. James Harden 	Arizona State 	12.97
11. Hasheem Thabeet 	Connecticut 	12.90
12. Earl Clark 	        Louisville 	12.88

For starters, let's talk about two of the players who play multiple positions -- this matters now that we're rating players in part based on position.

Stephen Curry graded out at 14.18 as a wing but only 12.86 as a point guard. Either way, it puts him in the top dozen players, but by this rating, he's a much better prospect if he's able to defend against wings.

The difference for Earl Clark was less dramatic, but he rated slightly better as a wing than as a big man (12.14), which would have dropped him from 12th to 15th.

A couple of other names on here are likely to raise eyebrows:

Austin Daye might not have had a great season, but the Draft Rater looks favorably upon a 6-foot-11 small forward who can shoot (assuming he can play the 3 in the NBA). His numbers were strongest in the categories that project best to the pros, including 42.9 percent shooting percentage on 3s and 2.1 blocks per game, which is why he moves all the way up to No. 4 on this list.

Nick Calathes is under contract in Greece but still will be draft-eligible, and he rates higher than the hot point guards most teams are discussing in the top 15. Although he has been knocked for his athleticism, he had high rates of rebounds and steals and a strong 2-point shooting percentage. Teams in luxury tax trouble should look particularly hard at him because he can be stashed in Europe for a year or so.

Danny Green is the other surprise on this list. He's rated highly every year I've rated him, so seeing his name doesn't shock me anymore, but he has received little attention nationally. Still, he's a great shooter who can defend, and he rates as the third-best wing after Daye and Tyreke Evans.

Rankings: 13 to 25

Collegians: No. 13 through 25
Player 	School 	Draft Rater
13. Jrue Holiday 	UCLA 	        12.73
14. Jeff Teague 	Wake Forest 	12.50
15. Gerald Henderson 	Duke 	        12.17
16. Paul Delaney 	UAB      	11.85
17. Aaron Jackson 	Duquesne 	11.83
18. Darren Collison 	UCLA 	        11.80
19. Terrence Williams 	Louisville 	11.80
20. Leo Lyons 	        Missouri 	11.53
21. Eric Maynor 	VCU     	11.35
22. John Bryant 	Santa Clara 	11.30
23. DeMarre Carroll 	Missouri 	11.18
24. Tyler Hansbrough 	North Carolina 	11.11
25. Wayne Ellington 	North Carolina 	11.04

This part of the list is an interesting mishmash of potential sleepers and potential busts. In general, players in this range have some kind of NBA career but always can count on getting some quality time with the family during All-Star Weekend.

We're awash in point guards in this draft, and six of the top nine names in this section play the position. The lesson is this: If you're in the market for a point guard, one will fall to you, and they're more or less the same after the first couple.

Down at No. 13, Holiday is a bit of a surprise -- given that he's projected to go higher -- but he has the two characteristics that produce the greatest error rate in the Draft Rater: He's a point guard and has played only one year. In other words, his real value might be much higher or much lower, and because the consensus is much higher, it wouldn't bother me to use a top-eight pick on him.

Delaney and Jackson are second-round sleepers at the point, but because projections for point guards are a bit more volatile, perhaps they shouldn't really be this high. The other "who's he?" on the list, Bryant, is a 6-11, 275-pound center from Santa Clara who could have a fine 10-year career as a third center in the Greg Kite/Aaron Gray mold.

Rankings: Potential disappointments

Collegians: Other notables
Player School Draft Rater

26. Jordan Hill 	Arizona 	10.97
28. B.J. Mullens 	Ohio State 	10.81
30. James Johnson 	Wake Forest 	10.63
31. Chase Budinger 	Arizona 	10.51
45. Derrick Brown 	Xavier  	9.55
48. DaJuan Summers 	Georgetown 	9.38
51. Jodie Meeks 	Kentucky 	9.35
52. Sam Young 	        Pitt    	9.34
54. DeMar DeRozan 	USC 	        9.26
62. Toney Douglas 	Florida State 	8.56
68. Patrick Mills 	Saint Mary's 	8.36
83. Jack McClinton 	Miami 	        6.64

And here's where we get to the players the Draft Rater is down on.

Several potential first-round picks don't pass muster here, with short, shoot-first combo guards in particular bearing the brunt of the Draft Rater's wrath -- Jack McClinton, Patrick Mills and Toney Douglas were the three lowest-rated "name" prospects, and Jodie Meeks didn't fare a whole lot better.

The other big surprise down here is Jordan Hill, who could go as high as No. 4 but rates 26th in the Draft Rater. Hill had solid rebounding and scoring numbers, but his percentages weren't off the charts, and his poor assist and turnover numbers were a red flag. Although one might think that ballhandling categories wouldn't matter for a power forward, apparently they do -- pure point rating (a measure of how a player passes and handles the ball) is a pretty strong success indicator for frontcourt players, and only four prospects rated worse than Hill.

One of those players was Mullens, who was the absolute worst at -2.85. Everyone concedes he's a project, so perhaps it's not such a big surprise to see him down this low. But the Draft Rater is saying that maybe even the middle of the first round is too high to take the risk on him.

Pitt's Sam Young also graded out extremely poorly. He had the worst pure point rating of any wing player, and the other thing that hurt him is that he's one of the oldest prospects in the pool. How old? He's 19 days older than six-year vet Darko Milicic and a full half-decade older than Holiday.

John Hollinger writes for ESPN Insider.

Nice article - thanks for posting that. Interesting: Jordan Hill ranked low, Curry ranks better as a wing player, Mullen ranks low. I'm sure this system has it's flaws but it's probably no worse then the average NBA Scout. It'll be interesting to see how accurate it's ratings turn out over time.

hollinger adds nothing.

draft sucks. all this analysis is for moot.

NBA Draft 2009
Originally Published: June 18, 2009
Draft Rater History: 2002-08 results
Hollinger By John Hollinger
ESPN Insider

To give you some context as to how the Draft Rater has performed in past drafts, I've included its projections for each draft going back to 2002.

Each chart shows the top 12 college players picked by the Draft Rater, as well as where those players were actually selected among collegians on draft night. As the charts show, the Draft Rater has generally held up very well; in fact, in most years it has produced better results than the actual draft.

As I pointed out in the story on this year's class, the Draft Rater rates "pro potential," which isn't always the same things as "pro performance" -- sometimes you'll have a Michael Sweetney eat his way out of the league or a Michael Redd work his way into the league by becoming a deadly jump shooter. Unfortunately, there's no stat for off-court habits.

Despite that limitation, the Draft Rater had performed extremely well prior to 2008, when it seems to have been tripped up by several one-and-done freshmen (the rankings have tended to be least reliable for that subset of players).

As mentioned in the larger story, from 2002 to 2007 there were 15 players who were taken among the first 10 collegians but whom the Draft Rater excluded from its top 12. None of them has played in an All-Star Game, and the only two long-term starters in the group have been Kirk Hinrich (who was 13th) and Charlie Villanueva.

Additionally, the Draft Rater has come up with some big draft steals: Carlos Boozer in 2002, Josh Howard in 2003, Danny Granger in 2005, Rajon Rondo in 2006, Rodney Stuckey in 2007 and Mario Chalmers and George Hill in 2008, among others.

A couple points we must make before we get started. First, the projected PERs for nearly everyone will seem a little low. This happens for two reasons.

First, the players who become superstars are almost always outliers not only in terms of ability but also in how much they improve in their first few seasons, so it's insane to project superstardom as the most likely outcome for any individual player.

Second, it's the nature of the beast: Using regression analysis has a lot of value in terms of ranking the players, but the one drawback is that the estimated PERs tend to clump around the 10-11 range much more than they do in real life.

With that said, let's take a look at the Draft Rater's top 12 players from each season since 2002 and throw in a few comments:

2002

Player 	Projected PER 	Order Picked Among Collegians
Drew Gooden 	        15.18 	3
Carlos Boozer 	        14.75 	26
Mike Dunleavy 	        13.94 	2
Jay Williams 	        13.57 	1
Chris Wilcox 	        13.28 	5
Caron Butler 	        13.17 	6
Steve Logan 	        12.91 	22
Curtis Borchardt 	12.75 	12
Casey Jacobsen 	        12.73 	16
Tayshaun Prince 	12.59 	17
John Salmons 	        12.46 	19
Jared Jeffries 	        12.34 	7
* Top 10 drafted collegians not in Rater's top 12: Dajuan Wagner, Melvin Ely, Marcus Haislip, Fred Jones

2002 was a strong season for the Draft Rater, as it correctly recommended avoiding four players (Wagner, Ely, Haislip and Jones) selected among the top 10 collegians and made only one bad recommendation (Logan) in their stead. Boozer, Salmons and Prince all ended up as great value picks, with Boozer probably the best recommendation in the Rater's history -- the 26th collegian taken, he rated second here and is an All-Star.

2003

Player 	Projected PER 	Order Picked Among Collegians
Dwyane Wade 	        16.64 	3
Carmelo Anthony 	16.51 	1
T.J. Ford 	        14.06 	6
Michael Sweetney 	13.46 	7
Josh Howard 	        13.45 	17
Chris Bosh 	        13.08 	2
Nick Collison 	        13.03 	9
David West 	        12.85 	14
Luke Walton 	        12.70 	19
Reece Gaines 	        12.62 	12
Marcus Banks 	        12.33 	10
Chris Kaman 	        12.20 	4
* Top 10 drafted collegians not in Rater's top 12: Kirk Hinrich, Jarvis Hayes

Draft Rater got the top two right but pushed down Bosh, Kaman and Hinrich a bit too far. On the other hand, it correctly identified David West and Josh Howard as strong prospects, and Luke Walton as a second-round steal. It also correctly avoided Hayes.

2004

Player 	Projected PER 	Order Picked Among Collegians
Luol Deng 	15.45 	5
Andre Iguodala 	13.73 	7
Josh Childress 	13.15 	4
Emeka Okafor 	13.08 	1
Kirk Snyder 	13.05 	10
Delonte West 	12.83 	12
Luke Jackson 	12.69 	8
Ben Gordon 	12.33 	2
Devin Harris 	12.27 	3
Jameer Nelson 	12.18 	11
Kris Humphries 	11.67 	9
Chris Duhon 	11.54 	21
* Top 10 drafted collegians not in Rater's top 12: Rafael Araujo

2004 is the most similar to the real draft -- 11 of the top 12 college selections were the same. Only two of the recommendations were bad ones -- both Snyder and Jackson were busts, and the Rater had each a bit higher than they went in real life. On the other hand, most teams would be happier with Deng or Iguodala rather than Okafor, and the Rater found a second-round steal in Chris Duhon. Kevin Martin, a late first-round steal in real life, was 13th here.

2005

Player 	Projected PER 	Order Picked Among Collegians
Chris Paul 	        15.66 	4
Andrew Bogut 	        14.89 	1
Danny Granger    	14.58 	13
Sean May 	        14.33 	9
Marvin Williams 	13.94 	2
Jarrett Jack 	        13.91 	17
Raymond Felton  	13.59 	5
Francisco Garcia 	13.38 	18
Rashad McCants  	13.31 	10
Julius Hodge 	        12.57 	15
Nate Robinson 	        12.12 	16
Deron Williams  	11.74 	3
* Top 10 drafted collegians not in Rater's top 12: Charlie Villanueva, Ike Diogu, Channing Frye

The Rater judged Deron Williams and Charlie Villanueva too harshly, but more than made up for it by pushing Chris Paul all the way to the top and Danny Granger to the No. 3 position. May came highly recommended and has played well when he's been on the court but has been undone by conditioning and knee issues. Jack, Garcia and Robinson were strong sleeper recommendations; Julius Hodge, not so much.

2006

Player 	Projected PER 	Order Picked Among Collegians
Rudy Gay 	        15.19 	7
Rajon Rondo 	        14.65 	16
Shawne Williams 	14.64 	13
Brandon Roy 	        13.88 	5
Tyrus Thomas 	        13.44 	3
LaMarcus Aldridge 	13.01 	1
Adam Morrison   	12.82 	2
Hilton Armstrong 	12.41 	9
Ronnie Brewer    	12.24 	10
Kyle Lowry 	        12.13 	19
Shelden Williams 	12.03 	4
Marcus Williams 	12.00 	17
* Top 10 drafted collegians not in Rater's top 12: Randy Foye, Patrick O'Bryant, J.J. Redick

This is arguably the Rater's best draft: It nailed five of its top six picks with only the Shawne Williams placement at No. 3 derailing it. That's a big improvement on what really happened on draft day, when Adam Morrison, Shelden Williams, Randy Foye, Patrick O'Bryant and J.J. Redick were five of the top 10 collegians taken. Only Morrison got into Draft Rater's top 10, and he was seventh rather than second.

2007

Player 	Projected PER 	Order Picked Among Collegians
Kevin Durant     	19.01 	2
Mike Conley 	        16.58 	4
Greg Oden 	        14.88 	1
Joakim Noah 	        13.97 	8
Rodney Stuckey 	        13.97 	14
Javaris Crittenton 	13.72 	17
Thaddeus Young 	        13.68 	11
Brandan Wright  	13.52 	7
Al Horford 	        13.45 	3
Jeff Green 	        13.19 	5
Julian Wright    	13.11 	12
Corey Brewer 	        12.73 	6
* Top 10 drafted collegians not in Rater's top 12: Spencer Hawes, Acie Law

Moving Durant to the top spot and promoting Stuckey and Young were good calls, partly offsetting the demotion of Horford to No. 9 and the promotion of Crittenton to No. 6. Though highly anticipated, this draft has turned out to be a bit short on star talent thus far; we'll see in a couple of years how good this projection turned out to be.

2008

Player 	Projected PER 	Order Picked Among Collegians
Kevin Love 	        14.62 	5
Michael Beasley 	14.32 	2
Joe Alexander 	        13.48 	8
Mario Chalmers 	        13.01 	29
George Hill 	        12.77 	23
Chris Douglas-Roberts 	12.46 	33
Derrick Rose 	        12.36 	1
Roy Hibbert 	        12.30 	17
Darrell Arthur 	        12.17 	24
Ryan Anderson 	        12.14 	20
Marreese Speights 	12.11 	16
Darnell Jackson 	12.11 	42
* Top 10 drafted collegians not in Rater's top 12: O.J. Mayo, Russell Westbrook, D.J. Augustin, Eric Gordon, Brook Lopez

2008 saw, by far, the Draft Rater's most radical departure from the actual draft, and also seems likely to go down as the Rater's worst overall season. Love led all rookies in PER and Beasley was fourth among '08 collegians, so the top two picks are solid. After that, it's a mess. One-and-done freshmen Derrick Rose, O.J. Mayo, Anthony Randolph and Eric Gordon all ranked too low, as did solid rookies Brook Lopez and Russell Westbrook.

Of the players moved up in their place, Chalmers, Hill, Douglas-Roberts and Speights seemed like good recommendations; Speights finished second only to Love in PER from this draft class. On the other hand, Alexander, Arthur, Anderson and Jackson still have much to prove.

John Hollinger writes for ESPN Insider. To e-mail him, click here.

I agree about Evans. He should be the second pick in the draft based on everything I have seen an read. If the Knicks move up they need to get Evans, Harden or Thabeet.