Examining the MLS Draft: Part 3(b) - A Few Responses

DIY Options / Revisiting the Nowak Question / What I really believe

OPEN SOURCES: We'll potentially revisit the draft data a few more times between now and January's 2007 SuperDraft, but I wanted to respond to some comments and notes that have been left here, and reemphasize the single most important finding I think we can make. First, over at Black Panther, they're looking at the success of the MLS Combine in getting players selected. They also expressed some interest in the DCenters making our data public. Well, I'm happy to do so. The easiest way seems to be to put the main data set I was manipulating up on Google Spreadsheets, which I've done. Now, I've had to strip out the forumulas I was using and put in the raw values, but it should service. If you want to run your own draft analysis, or check our work for errors, or do something else, go ahead and get the spreadsheet here. I also recommend that you head over to Climbing the Ladder and grab the MLS Lineup database, which was invaluable in making this a project for an evening rather than a weeks worth of time.

REVISITING THE NOWAK ERA: Commenter Bootsy Collins found a nice little methodology flaw in our evaluation of the Nowak era which caused us to reexamine the data.

He wrote:

To illustrate what I'm getting at, imagine that Nowak/Kaspar's nine picks had all been fourth round picks. I know they weren't, but imagine that they had been. In that case, I'd bet that comparing to the league average, 22% would have been pretty good. In contrast, if they were all first round picks, 22% would doubtless have been awful. The question is, how does the pick distribution we had (2004: 1/3/4, 2005: 2/4, 2006: 1/3/4/4) compare to the league as a whole?

That's a damn fine question, and one I should have thought of before undertaking the Nowak Era analysis. Since he brought it up, it was worth investigating in more detail. Given the time frame and distribution among rounds of Nowak/Kasper's picks, how well did they do compared to the MLS average? Well, by looking at MLS as a whole from 2004 onward, we can determine the percentage liklihood, by round and year, that a player made at least 10 GPs. We can then apply any draft pick distribution and determine the expected yield of players making at least 10GPs. That's what the chart below does. The top section shows the MLS success rates by round and year. The middle section shows DC United's draft pick distribution. The last section shows the expected yield by round and year, and an overall expected yield:

We know that DC United's 2 successful picks (Gros and Adu) is a bit short of the 3 to 4 that the MLS Distribution would say we should have, even given that United's distribution of picks skewed to later rounds. But, in comparison, how does it look? The chart below shows the expected yields, actual yields, and differences for all teams from 2004 onward:

So based on this, am I ready to reaffirm that Nowak/Kasper have drafted poorly? Not really. First, I think if you look at these numbers, you see what small values we're really talking about. Rod Dyachenko, if he makes on appearence for Toronto under Mo, will click over into the success column. When one player can cause so much variation in the data, it makes it difficult to draw concrete conclusions. So while the numbers aren't encouraging, I can't say with any confidence that Nowak and Kasper have done worse than the rest of the league. I can say it doesn't look good, especially with the 2005 class, but I can't say it looks unreasonably bad. The second thing to note is that better teams (other than LA, make you own conclusions) seem to be clustered toward the bottom, perhaps reflecting the difficulty of draft picks breaking into more talented line-ups. Now, all that being said, it is ridiculous that Real Salt Lake is so low on this list. As an expansion team, and not a terribly good team after their first year, it seems like they should be getting more mileage out of every draft pick than they are.

THE BIG POINT: If I can urge you to take away one thing, it is the analysis of draft round distribution information. I think we have enough data to say, with confidence, that Round 1 draft picks are worth at least twice as much as Round 2 draft picks, which are twice as much as Round 3 draft picks. Round 3 and 4 draft picks are fairly equal in their value, and neither represents a decent shot at landing a consistently appearing player. That's the take-away. Round 1 means something, Round 2 might mean something, and Rounds 3-4 mean you'll need to get lucky.

As always, I welcome your responses and critiques. I would love someday to learn whether MLS teams have done similar analyses (I'm sure they have) and what their findings were compared to ours.