Every baseball organization spends thousands of man hours scouring the countryside for what they hope will be the next impact player for them at the big league level. And yet they are also the first ones to note that even they don't have it all figured out.
So when it comes down to deciphering who had a successful draft or not, while some publications will hand out grades the day or two after the draft and fan bases alike will argue that their teams missed the boat on a potentially great prospect, people within the game itself say that is nearly impossible to tell so quickly. Just ask the manager of amateur scouting for the Texas Rangers, Bobby Crook.
"I think they can be as accurate as the timeframe allows," Crook said in politically correct fashion. "Everything changes from one year to the next. It's a timeframe issue. The NFL draft has a lot of knee-jerking go around about what happened there, and that's a lot easier because those guys are going to go up and make an immediate impact. In our draft, if you take a high school kid, it's going to be several years."
Former Mets Scouting Director, Russ Bove, now a Major League scout with the Mets who also helps out on the amateur and International sides as well, absolutely agrees.
"It takes years. You're looking at a minimum of three years, especially if you're taking high school guys. If you take a high school guy and then four years late he's an everyday player, you've done well.
"You might take a high school guy and his second year out he's struggling in A-ball, everybody might be saying it's a (bad) draft. And in New York, a year later they're saying it's a (bad) draft."
"I don't think you can touch it for four years," Yankees scouting director Damon Oppenheimer added. "There's just too many things that happen. Some college guys will get to High-A or Double-A immediately and perform and that's great, but other guys will take a while longer.
"The Denard Spans of the world, you'd love to have that guy, but it took him a little bit longer than some other guys. I don't think you can touch it for four years. And I'm not talking about the year that you're drafting them either as one of those years."
The general consensus among the scouting community is that no draft can accurately be assessed for a period of at least three to five years and that's just to start. As Crook points out, grading a particular draft is an ongoing process even after a few years.
"I think there are a few ways to look at it," Crook said. "After three years, you know which guys have a chance to make the big leagues, and then on top of that have an impact at the big league level, and then you know the guys who probably don't. After five or six years, you can nail it down even further.
"I think the next step after that is 10 years. By then, you have a gauge what a guy is at the big league level or what these players have become. After five years, you look at it and you say, ‘we have three really good prospects, one guy that's already up and one guy that has been up and down a little bit in Triple-A', that's a pretty decent draft.
"But if after five years you have only one prospect who makes it and then five years after that he's been in the big leagues for seven years, that becomes better. I think there's different levels of evaluating it."
And yet even though everybody involved in the game of baseball completely understands just how long it takes to accurately portray the merits of a particular draft, those observing the game from the outside, whether it be fans or sportswriters, insist they have the prerequisite knowledge to hand out grades within 24 hours of the final selection.
"It has no value to me," Oppenheimer said. "It can't have value based on how fragile things are that we deal with. It takes a long time to get to the big leagues. I've said it a million times, we're the only sport that has this unique draft setup. In the NBA and the NFL, they go right to the level they're picking them for so we're not the same."
"Nobody can evaluate a draft that quickly, no way," Bove said emphatically. "If you went to 30 teams the day after the draft, every one of them thinks they had a great draft.
"Who's to say that the sixth round pick – as a typical scout, we all have our success stories – in Milwaukee we took Ronnie Belliard in the seventh round. Four years later, he's an everyday big league player, and he's had a hell of a career. But [prior] to that point, they probably said ‘you had a (bad) draft'. All of the sudden that seventh or eighth rounder surfaces as an everyday guy and he was a high school draft."
"It's hard enough to do it in the NFL Draft and these kids are going right into the NFL," Jack Bowen, former Mets Scouting Director and current national supervisor of amateur scouting for the Pittsburgh Pirates, said. "You can't do it [in baseball]. Anybody who gives out grades that soon in the baseball draft is trying to make a buck or get somebody to read something positive or negative about that draft."
And it's that kind of whimsical arm-chair analysis by baseball outsiders that leads to some hasty and inconclusive "conclusions", and in some extreme cases, even costs front office personnel their jobs.
"In 2005, myself and Damon Oppenheimer took a bunch of (junk) in New York because we didn't take [Craig] Hansen, the reliever from St. John's," said Bove, who only served as the Mets scouting director for one year, despite being responsible for ultimately giving the Mets three big leaguers [Mike Pelfrey, Jonathan Niese, and Bobby Parnell] even though they didn't have a second or third round pick that year. "Now, where are all of those guys [who criticized us]? It takes time to evaluate a draft."
There are also other mitigating factors that help shape the context of what constitutes a good draft and some of those aren't always public knowledge either.
"Is it a strong one? Is a weak one? Is it deep? Is it mostly college or mostly high school [players]? Like this year we're picking number two," Bowen said. "Now, it might be a good year to have the second pick. We'll have to wait and see in a few years. It may not be a good year for the second pick.
"I remember the '04 draft when we [the Mets] were picking third and we wanted Justin Verlander, and he went #2 to Detroit and we were left picking three. Our next pick was Stephen Drew, but we didn't get the ‘Ok' to take Drew. And two [front office] guys we had to deal with that year steered us to Philip Humber, who we liked, but he was plan C at that time for us. So there's a lot of variables when you're trying to rate a draft and how well you did."
Inside baseball there is no debate – it takes years to accurately assess how good or bad a draft was for any team. But while the disparity between actual baseball people and onlookers is huge as to when a grade can be given out, so is discrepancy on what criteria is used for determining what makes a successful draft.
"I don't think there's any hard-fast numbers," Yankees senior vice-president Mark Newman, who is in charge of scouting and player development, said. "If we get two solid players, two good Major League players out of a draft, that's a good draft. But everything is relative.
"It's relative to where you pick and it's relative to how many picks you have. While we can't trade draft picks, we do in effect trade draft picks when we sign free agents. And so there have been certain years for the Yankees organization that we haven't had a first round pick, or this year we pick 32nd, and that's like not having a first pick, so when you put things in context – how many picks you have and how high they are, or how many picks you're missing – internally we have to assess the draft within that context."
The general sentiment of securing at least two everyday players in a draft appears to be the main consensus among the scouting community to tab that draft class as a success.
"A lot is determined with how many picks you have, where you pick in the draft, the quality of the draft, etc," Bowen added. "I don't know if there's any set number that you can put on it just because each year is different. And each year you're picking in a different spot.
"If you can get four or five guys to the big leagues out of a draft, that's pretty good. If a couple of those can have an impact, I think that's successful. You're always trying to get as many to the big leagues as you can and hopefully a couple of them have an impact, being realistic about it."
Somehow unrealistic pundits have come to expect teams to hit on nearly all of their top 10 picks, especially in the first three rounds. While that is the ideal scenario, the hard-fast reality is it hardly ever works out in such a neat fashion.
"Years ago when I worked for Milwaukee when I was a cross-checker, one of the old-timers, his theory was a draft was successful if you got three big leaguers," Bove added. "Three big leaguers – are they a long reliever, a utility infielder, and an extra outfielder? To me that's not a good draft.
"We've kind of clarified it to if you can get two contributing big league players like an everyday position player, somebody in your rotation, or an impact reliever, and then an extra guy, you've had a real good draft. Let's say you had a draft and you got your closer, you got another bullpen piece, and maybe you got a fourth starter, to me that's a pretty successful draft."
With the combination of the number of picks involved, the intricacies of player development, and the sheer time needed to allow players to develop, the baseball draft is unlike any other event in professional sports.
Analyzing a baseball draft is arguably one of the most subjective case studies around. There are two consensus sentiments among most baseball people – the draft can't be accurately graded for at least three years and securing at least two everyday players, either homegrown or via trading a prospect, is the industry standard for a successful draft.