Turns out the believers won this one big.
New research shows that stars matter – a lot. A sports consulting firm analyzed the percentage of players from 2002 and 2003 BCS recruiting classes who made First or Second Team All-Conference during their college careers, using Rivals ratings. They found that 34 percent of five-stars, 20 percent of four-stars, 12 percent of three-stars and 7 percent of two-stars made All-Conference during their careers. All those differences are highly statistically significant – they're real differences, not just some numeric fluke. (The firm also found that your rating was a good indicator of your chances of making All-Conference no matter your position, except for kickers, who were underrated by Rivals, and presumably all recruiting services, relative to the number of All-Conference kickers.)
The data allow us to precisely quantify the effect of an additional star: each additional star means a player is exactly 70 percent more likely to make All-Conference at some point during his career. (7*1.7 = 11.9, rounds to 12; 12 * 1.7 = 20.4, rounds to 20; and 20*1.7 = 34.) These numbers fit unbelievably well and stand as a pretty ringing endorsement to the predictive value of recruiting.
Still, the doubters are absolutely right: for all their successes, recruiting services are still very much taking a shot in the dark. Consider that a good 65 percent of "can't miss" five-stars end up busting. (The authors argue that a five-star not making All-Conference qualifies as a bust, and I agree.)
In sum, then, these numbers nicely confirm everything we've thought:
1. Recruiting rankings do matter
2. While higher-ranked players have a better shot, whether or not a player makes it big is highly random, and depends upon an awful lot of unpredictable factors, such as luck, work ethic, coaching, injuries. So recruiting is a numbers game, and signing quality depth matters.
In future "Inside the Numbers", I'd like to use the findings from this report to calculate the strength of various recruiting classes, comparing Stanford's 2009 class to the rest of the Pac-10, Stanford's 2010 class or an average BCS class, for example.
[To do so, we need to see if we can find a better indicator of a player's value than his All-Conference status. His number of starts is initially tempting, but wouldn't work because a start at Washington State isn't as impressive as starting at USC. If WSU signs a majority of two-stars, then necessarily they'll have a bunch of two-star starters, and vice versa with USC, which distorts the numbers. Actual statistics aren't feasible – USC's quarterbacks would have far worse stats at Washington State, for one, plus there's the question of what to do with positions that don't accumulate stats, like the OL. NFL Draft status is tempting too, but what about players like Tim Tebow, who are undeniable stars in college but don't project as well at the next level? Thus, All-Conference status is probably the best measure of "success" we can use.
As different sites grade players differently, there's also the question of whether using an average of recruiting services' rankings would result in more accurate results. It probably would marginally, as would adjusting for things like commitment date (earlier commits might not get rated as highly) and region (players from Texas will be seen by more scouts, and likely benefit in the ratings, than players from New Hampshire, for example), but for the sake of simplicity, we'll simply use Scout's ratings moving forward.]
These findings allow us to make our All-Conference-based recruiting ratings. Each school will receive seven points per two-star, 12 points per three-star, 20 points per four-star and 34 per five-star recruit. Dividing a school's total score by 100 predicts the number of players in the class who will make First or Second-Team All-Conference at some point during their playing careers.
Please refer people to this research whenever the debate upon recruiting rankings comes up, and see the other "Inside the Numbers", where we've released All-Conference-based recruiting ratings of interest to Stanford fans.
Do you have a "premium" subscription to The Bootleg? If not, then you are seriously missing out on all the top Cardinal coverage we provide daily on our award-winning website. Sign up today for the biggest, baddest and best in Stanford sports coverage with TheBootleg.com (sign-up)! At The Bootleg, "WE WRITE!"