Cowboys Q&A: "Ask BTB" - Is The Third Round A Crapshoot? Part I: Measuring Success

NEW YORK, NY - APRIL 26: NFL Commissioner Roger Goodell stands on stage as he announces a draft selection during the 2012 NFL Draft at Radio City Music Hall on April 26, 2012 in New York City. (Photo by Al Bello/Getty Images)

Welcome back to another edition of "Ask BTB," in which we give our loyal readers the opportunity to ask the front-page writers a nagging question to which we will offer our best response. A reminder at this juncture that there are several ways for you to submit your questions. First, you can send an email directly KDP10For10@gmail.com. This allows you to make you questions as detailed as possible. If you have a quick, short-winded question, feel free to Tweet KD @BloggingTheBoys (in addition to all his fine posts, KD is the voice of our Twitter presence). Make sure you use the hashtag #AskBTB so he'll know to send it down the front-page pipeline. And: remember to include your BTB handle in your correspondence!

As KD suggested when he launched the "Ask BTB" series, some questions will be answered by multiple writers each offering a short paragraph or two in response. Others merit a singular response, often because they require some substantial research. This post falls under the latter category. BTB reader Eli Landy writes in:

A while ago, several writers quantified the success rate of draft picks over the past ten years, and found that the success rate -- the rate of draft picks actually making and staying on the team -- of third round picks was something like 13%, and that the rate decreased for later round selections. Has anyone looked at the success rate of draft picks over that time frame specifically for the Steelers, Patriots, Ravens, Eagles and Packers? I suspect that their success rates are considerably higher, and would be a much better indicator of the success of draft picks by successful teams. If their success rate is higher, it would also show that the draft may not be a total crapshoot if the team knows what it's doing.

Thanks for the question, Eli; as a dedicated draftnik, this one is right down my alley. As it turns out, the research was so extensive that it will merit two posts. Today, we'll consider various ways to assess the third round; on Monday, we'll take a hard look at the third round selections from the various teams you named. In fact, I'm going to add the Colts and Giants to your list; to my mind, they both qualify as one of the "successful teams."

How will Rabble tackle this problem? Make the jump to find out...

First off, I think it's important to be clear about our definition for success. I'm going to offer up several ways of determining the success of a given draft or a sequence of drafts by a given team. Here goes:

Pro Bowls: One way to determine top-end success is to look at Pro Bowl nods. Since 2000, 20 players drafted in the third round have been invited to the Pro Bowl. Below, I've listed all the invitees by year, with the player's draft position and (in italics) the year(s) in which he was a Pro Bowler.

2000:
Laveranues Coles (78): 2003
2001:
Derrick Burgess (63): 2005, '06
Adrian Wilson (64): 2006, '08', '09, '10, '11
Steve Smith (74): 2001, '05, '06, '08, '11
2002:
Brian Westbrook (91): 2004, '07
Chris Hope (94): 2008
2003:
Lance Briggs (68): 2005, '06, '07, '08, '09, '10, '11
Jason Witten (69): 2004, '05, '06, '07, '08, '09, '10
2004:
Darnell Dockett (64): 2007, '09, '10
Nate Kaeding (65): 2006, '09
Nick Hardwick (66): 2006
Randy Starks (71): 2010
Chris Cooley (81): 2007, '08
Matt Schaub (90): 2009
2005:
Frank Gore (65): 2006, '09, '11
Justin Tuck (74): 2008, '10
2006:
None
2007:
Marshall Yanda (86): 2011
2008:
Jamaal Charles (73): 2010
2009:
Mike Wallace (84): 2011
2010:
Jimmy Graham (95): 2011
2011:
None

In 442 third-round selections since 2000, there have been 20 Pro Bowlers, with an accumulated total of 49 nominations, so 4.5%, or approximately one in 22 picks become Pro Bowl players. But surely, there must be a lower-end (and thus more reliable) indicator of success? Let's look at them:

Staying on the roster: In his question, Eli defines success as "actually making and staying on the team." With this as our benchmark, let's use the 2008 draft as a test case to establish a norm against which we can test the drafts of the aforementioned teams (you can review it here). I've chosen 2008, since the players from that draft can potentially have been in the league four years, so we can clearly tell who remained with their original teams and who failed to do so. Of the 36 third-rounders in 2008 (including end-of-round supplemental picks), just over half (19) remained with the team all four possible years. So, by Eli's definition, the success rate in 2008 was much higher than 50%.

Games played and starts: "staying on the roster" doesn't account for distinctions between four-year starters and guys who hung around the bottom of the 53. Let's look at games played and starts, as they might give us a clearer sense of who's actually played - and thus been what we might deem as truly successful. Since the 2008 draft, all NFL teams have played 64 regular season games. Among 2008 third-rounders, there has been a wide disparity in terms of games played, from Miami's Kendall Langford (64) to the Patriots' (and two other teams') Kevin O'Connell (2). The average number of games played for these 36 players is 41.5, or almost 65% of all possible games. The average number of starts is roughly half that of games played: 20.5, or 32% of all possible games. That said, only 9 guys (25%) were full time starters for more than one of their four years with the team. In the 2008 draft, at least, the success rate seems to be higher than the 13% that Eli cites.

Massey and Thaler: One of the reasons I've used the above measures of success is that they are precisely those used by two powerful thinkers, Cade Massey and Richard Thaler, to analyze the ways teams behave vis a vis the draft. In the buildup to the 2012 draft, you may recall, I referenced a lecture (video here) Massey delivered in which his central question was "are some teams better than other teams at picking players?" To answer this, Massey used precisely the tools we have employed, starts and Pro Bowls, to establish what the league-wide expectations should be for each draft pick, from the first selection to Mr. Irrelevant.

Comparing every pick relative to the first pick in the draft, Massey develops a Lowess curve for the performance of players drafted between 1991 and 2004 (you can find the slide show accompanying the lecture here). Looking at this, we can see that, in terms of starts, third rounders tend to range from approximately 32 to 48 percent as successful as the draft's first pick. In terms of Pro Bowls, they range from 9 to 17 percent as successful. Looking at these and other factors, Massey and Thaler develop their own curve, which indicates that third rounders are between roughly 40% (pick 33) to 28 percent (pick 65) as successful as the first overall selection. Granted, the first overall selection is not a success one hundred percent of the time. Nevertheless, it appears that third rounders tend to be successful at a rate higher than the 13% with which we started.

But Eli asked about the teams considered the best at drafting talent. Well, Massey has an answer for him. Going into his project, Massey notes, he believed that the better teams should draft better than the above established expectations over the long term, and poorer drafting teams should draft lower than these expectations. Running what is known as an "intraclass correlation coefficient" for sixteen years worth of drafts, however, Massey reached a stunning conclusion: there is in fact no difference in drafting success from team to team. As he puts it, the statistics are "saying is that there are...literally zero differences across teams in player-picking ability." In short, Massey suggests that drafting success is much more a matter of luck than skill.

Can we buy what Massey is selling? In part II, we'll look at the third round selections for the Giants, Eagles, Packers, Steelers, Patriots, Ravens and Colts, and compare them to the numbers we have assembled above to answer more directly Eli's question: is the draft "a total crapshoot"?

X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Blogging The Boys

You must be a member of Blogging The Boys to participate.

We have our own Community Guidelines at Blogging The Boys. You should read them.

Join Blogging The Boys

You must be a member of Blogging The Boys to participate.

We have our own Community Guidelines at Blogging The Boys. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker