Every year, the MIT Sloan Sports Analytics Conference features a bevy of panels looking at the use of deep statistical analysis in a wide variety of sports. Last year, the conference featured a discussion on the use of analytics in football, with an impressive panel: 49ers President Paraag Marathe; former Patriots and Chiefs GM Scott Pioli; Kevin Demoff, the Rams' Executive Vice President of Football Operations & Chief Operating Officer; and Football Outsiders' Aaron Schatz.
Their discussion offered a multitude of golden nuggets about the way NFL front offices approach free agency, the draft, and gamedays, as well as the processes they institute to achieve success. Coincidentally, as I was listening to and transcribing their back-and-forth, I stumbled across an interview with Eagles GM Howie Roseman, in which he addresses many of the same issues. Seeing many points of resonance in these conversations, I decided to use their words to articulate or to support a set of organizational behaviors that suggest on-field success, and then offer the Cowboys a grade for each of these behaviors.
This became a four-part series. In part one, I looked at overall organizational philosophy; in the second installment, I examined the way the team has managed the cap; the third report card looked at their behavior in free agency; today's final post will examine the ways the team had managed (or mismanaged) the draft.
IV: The Draft
Accumulate Lottery Tickets: In my personal favorite of all the posts I've written for BTB, I cited the work of Cade Massey, whose research focuses on the psychology of overconfidence, particularly as it applies to making decisions in uncertain environments. In a stunning presentation at the 2012 Sloan Conference, he proposed that success in the draft is not a matter of skill, but of luck. And not just a bit of luck, but entirely luck. His numbers clearly stated "that there are no differences across teams...literally zero differences across teams in player-picking ability."
Certainly some teams draft better than others. This is not because they are more skilled, Massey argues, but because they do a better job managing chance. How do they do this? He offers three key behaviors: 1) acknowledge the role of chance; 2) minimize the cost of choosing (i.e., don't trade up); 3) maximize the number of chances. The last of these is perhaps the most important. Since drafting 20-year-olds to play professional football is essentially like playing the lottery, the best teams will proceed accordingly. Its a simple formula: to have a better chance to win the lottery, a rational player will buy more tickets.
It looks like the front office guys attending the 2013 conference were also in attendance in 2012; they all seem to have incorporated Massey's lessons. Pioli defines draft picks as multiple forms of "currency" and remarks that, as such, "just having lots of them is good." This is because:
we're all going to blow draft picks, so you better have as many shots at it, as many opportunities as possible, because even if you bat 50%, the more picks you have, the more chances you have of being right.
As a result, Schatz declares, "almost all the most successful GMs and the most successful teams are going to be guys who trade down a lot and amass picks and get more shots at" the lottery. More specifically, the most successful franchises acquire as many tickets as possible to the "premium" lottery (rounds 1-3). Look at the 49ers the last two years: in 2013, they began the draft with a first, two seconds and two third rounders; in May, they'll be working with a first, two second rounders and two thirds. That's two consecutive drafts with a lot of tickets to a high-yield lottery.
Cowboys' Grade: C
In the last six drafts, (2008-14) Dallas has had a total of 46 picks, an average of almost eight per year. That's neither bad nor good; its a touch below the NFL average of 47.6. More noteworthy, however, is that only sixteen of those have been in the first three "premium" rounds (the NFL average is 18.3) Compare Dallas' six first rounders, four seconds and six thirds to the premium picks spent by other teams that cultivated them more aggressively: Cincinnati, with 7 firsts, nine seconds and five thirds; Kansas City, with 7, 6 and 11; St. Louis, with 7, 7, and 7; Buffalo with 7, 9 and 5; and New England, with a whopping 5, 13, and 10. In 2008 and 2010, the Cowboys traded away second rounders while the Patriots, Chiefs and Bengals were all accumulating crucial depth. My grade reflects the fact that, insofar as accumulating lottery tickets is concerned, Dallas is squarely in the middle of the pack. While the team hasn't sacrificed these valuable picks as much under Garrett, the Cowboys will need to find ways to accumulate picks - and do so consistently - if their grade is to improve.
Trade Down or Back, But Not Up: Again, I'll lead with Cade Massey. He and a colleague, Richard Thaler, published an article entitled "The Loser's Curse" in which they conclude that, because the cost of players falls off far more quickly than does their average performance at the spots they are drafted, teams are better off trading down to late in the first or even into the second round, where they can get more and cheaper players who, taken together, will contribute more performance. Yes, you'll say, but the players the teams trading back can acquire will be of a lesser quality that the one they pass up.
Not so fast. Research has shown that teams trading down in the first round (or even out of it) have obtained, according to career AV totals, the best player just over half the time (about 51%). In other words, teams trading back not only get a better player just over half the time, but get an additional pick or picks to boot. Thus it should not surprise that, when it comes to all the picks/ players in the trade, the evidence becomes more compelling. Again using AV as the measure of value, teams trading down have acquired 64.4 percent of the total player value in the trade. That's a very significant margin.
Of course, trading down/ back is easier said than done. As I have written previously, trading back - especially trading current year picks for picks in future drafts - requires a high degree of organizational stability and job security on the part of both GM and coaching staff. Teams are willing to trade the known present for an unknown future because the decision-makers have confidence that they'll be around to reap the rewards of those future drafts. With that confidence, they can take advantage of the panicking coaches and GMs who are feel that they need to mortgage the future to make a splash right now. As the Rams' Demoff points out:
Trading down [in the draft] is great in theory, but it doesn't work when you have a coach who's in the next-to-last year of their contract, or when teams are starting to look at things and you feel like you're in your window...We took the second pick in the draft last year [i.e., 2012] and turned it into five different picks. Which was fine, because we had a new head coach and a new GM, and they had multi-year contracts and they could sit there and look at that.
I promise you, in three years if we were sitting there with the second pick again and they were in year four of five years, they might want to draft a player, because they feel that the need is very different.
In spite of this hard reality, the most successful teams find a way to acquire more picks and, by extension, greater value.
Cowboys' Grade: D
Despite the fact that the Cowboys traded back last year, I still see this as one of the organizations' greatest failings. A simple look at 2013's trade shows why: in a draft with a great deal of quality in the second through fourth rounds, they acquired greater value. According to the Harvard Trade Value Chart, which is what the Cowboys are most likely using, Dallas had a total of 708.4 points before the trade and 780.6 points after the deal. The gain, 72.2 points, is roughly equivalent to a an early fifth round selection. In 2010, however, thanks to two small trades up, they lost more than 40 points (a mid seventh rounder); in 2012, the Mo Claiborne trade up cost them value equivalent to a mid- to late-fourth rounder (82.3 points). They also lost value in 2008 by trading up to nab Mike Jenkins.
Too many times in recent memory, the Cowboys have lost value by trading up rather than acquiring value by trading back. Although the trades up in 2010 netted Dez Bryant and Sean Lee, they came at the expense of a third round selection. A convenient cherry picking exercise shows that both NaVorro Bowman and Jimmy Graham were taken in the next five picks. The Mo Claiborne deal is beginning to smell like something in the back of the refrigerator; not only, as the above research suggests would be the case, has he proven no better than the player he was "traded for" (Michael Brockers) but the players who were taken with or immediately following the Cowboys sacrificed second rounder include Alshon Jeffery, Mychal Kendricks, Bobby Wagner.
The end result of these trades up is decreased depth. In the midst of the last two years' spates of linebacker injuries, it sure would have been nice to turn to Bowman, Kendricks or Wagner - or to turn to Bruce Carter when one of them went down - wouldn't it?
Pursue "BPA" Whenever Possible: In the NFL, rosters are protean entities, in a constant state of change. As a result, the assessments a front office makes one offseason may well not apply a year or two down the line. A painful test case is offered by the Cowboys 2009 draft. In the second round, the Cowboys came on the clock with only one player to whom they had given a first-round grade, Marion Barber, Felix Jones and Tashard Choice. But they didn't really: Barber lasted two more years; Choice only a bit longer, and Felix's last effective season was 2009. Had the Cowboys believed more strongly in the long term, and in their evaluations, one of the NFL's top running backs would be wearing the star.. Instead of taking him, they traded out of the pick, because they already had what they perceived to be an embarrassment of riches at the position, with
The Eagles were the beneficiary, which may help to explain Roseman's position vis a vis BPA:
If we have a significant gap [between] the best player on the board is and the next best player, it's not really not going to matter the next best position because we don't know where we're going to be two, three, four years from now..."
If you're sitting there at 22 and you have a top ten talent sitting right in front of you but it may not be a position that you look at our depth chart right now and you feel good about, I promise you in two years you're going to be feeling good about it, and you're going to be feeling good about the process.
And that's what the draft's gotta be. Its gotta be taking the best players, accumulate really good players, and then you look to free agency to...free you up in the draft to do that sort of stuff [i.e., pursue a purer BPA].
In short, using the draft to fill holes causes teams to veer away from the board that they have spent millions of dollars and thousands of man hours carefully constructing. The best teams do as Roseman suggests above (and Jason Garrett does in numerous pressers): use free agency to fill obvious roster holes, and draft the best possible players. With the astonishing speed with which the NFL's personnel landscape shifts, its the only rational way to proceed.
Cowboys' Grade: A-/ B+
Since Jason Garrett took over, on of the most consistent message we've heard from the Cowboys' braintrust is that they want to use free agency to fill glaring roster holes so that they can have a "purer" draft. And, indeed, their behavior has matched their publicly stated intent. If they continue to operate this way, I see no reason why this grade can't be an "A" in 18 months' time.
Overall Grade: C
While the Cowboys' general approach (using FA to draft BPA) is quite sound - and is how the most successful teams handle the draft - they will never become a thoroughly successful drafting team until they begin to value picks more highly, to view them, as Pioli, as all-important "currency." Too often they have wasted picks in splashy trades to get splashy players, and too infrequently have they made trades to acquire greater value. The exception can be found in 2004 and 2007, when they finagled trades for a future number one choice. Those were great moves; both contributed to strong drafts the following year. But the team's inability to value (and thus to acquire) second- and third-round selections just as highly is troubling, since the players found in those rounds, not the superstar first-rounders, form the backbone of a roster.