On Wednesday, picks 9-16 of the annual BTB Writer's Mock Draft were released, only to reveal that the Cowboys, for whom I was picking, had traded down from the fourteenth pick. I expected to be the recipient of BTB's collective vitriol for making such a move; however, reactions seemed to be more or less favorable, largely since, one pick before the Cowboys were to go on the clock, the Arizona Cardinals had done the unthinkable, drafting Cowboys Nation's object of affection, Stanford guard David DeCastro.
Certainly this influenced my decision to trade back ten spots in the draft, as did the fact that, before making the trade, I surveyed the landscape and saw a lot of players with similar grades available, many of them at positions of need: Dre Kirkpatrick, Michael Brockers, Courtney Upshaw, Stephon Gilmore, Mark Barron, Peter Konz, and Fletcher Cox were all potential targets. With seven targets and ten picks, I figured at least one of them would still be there.
I made the trade, however, largely because I had listened to Case Massey. Massey is a professor at Wharton, the business school at Penn. His area of specialty is the psychology of overconfidence, particularly as it applies to making decisions in uncertain environments. One of his favorite test cases is the NFL Draft; with Richard Thaler, he has published an article, "The Loser's Curse," in which they conclude that, because the cost of players falls off far more quickly than the average performance in the draft, teams are better off trading down to late in the first or even into the second round, where they can get more players cheaper who, taken together, will contribute more performance. For Massey and Thaler, the worst slot in the draft is the first pick (this is before the new CBA was signed). Clearly, Massey has thought about the draft intensively and developed sophisticated statistical models to analyze it.
See what else Massey has to say about the draft after the jump...
So it was with great interest that I watched a lecture Massey delivered in early March at the MIT Sloan Sports Analytics Conference, entitled "Flipping Coins in the War Room" (a video of his talk can be found here; the accompanying slide show is here). In it, he builds upon the ideas established in "The Loser's Curse" to make some startling points about the draft. His central question is: are some teams better than other teams at picking players? To answer this, Massey establishes what the league-wide expectations are for each draft pick, from the first selection to Mr. Irrelevant, over a sixteen-year period. He assumes that the better teams should draft better than these expectations over the long term, and poorer drafting teams should draft lower than these expectations.
I'm not going to delve into the statistical models he uses (hit the above links to check out the nuances of his argument), but will offer a cursory summary. He presents a statistical spectrum, with pure luck one one side and pure skill on the other, wondering where on this spectrum the ability to draft might fall. Running what is known as an "intraclass correlation coefficient" for sixteen years worth of drafts, Massey concludes that success in the draft is not a matter of skill, but of luck. And not just a bit of luck, but entirely luck. As he surmises, the statistics are "saying is that there are no differences across teams...literally zero differences across teams in player-picking ability."
So why do we believe that there is? Why do we assume Indianapolis drafts better than Cleveland? To answer this, Massey turns to research in psychology that shows people under-appreciate the role of chance, leading to what the behavioral economist Matthew Rabin terms "fictitious variation." Part of this has to do with the statistical noise created by small samples. When there are differences between two teams' drafts over a small sample, we can choose to attribute the discrepancy to differences in skill or differences in luck. By not appreciating the role of luck, Massey argues, we tend to attribute distinction in draft success to differences in skill. Using Rubin's theory, Massey postulates that, even though there is far less draft-picking skill than people believe, they still invest in a narrative of skill--and use small samples to substantiate their beliefs.
Massey's larger point here is that differences in draft ability are 100 percent driven by chance; all discrepancies in draft success are a result of luck. Let me repeat: on a skill-based to chance-based continuum, the ability to draft players falls at the far end of the chance-based side of the spectrum. Certainly some teams draft better than others. This is not because they are more skilled, Massey argues, but because they do a better job managing chance.
A rational being, he notes, will go about the decision-making process differently when solving math problems (pure skill) than when entering the lottery (pure luck). He proposes that there is a "logic of change," a distinct set of proscriptions for operating in an environment with high uncertainty, and that successful teams follow this logic. In his lecture, Massey lays out the logic of chance's key tenets. I want to share a couple of these with you:
Acknowledge the role of chance: Experts--and NFL scouts and GMs should be considered experts--struggle to admit that they don't know, and thus are unwilling to acknowledge the role of luck in the process in which they are invested. By coming to terms with this disturbing reality, they can make better decisions.
Minimize the cost of choosing: Massey asks: how often is a player drafted at a given position better than the next guy drafted at the same position (there's an average of seven slots between them)? The answer is 52%, or slightly above a coin flip, which should make us skeptical about teams' ability to identify which of those two players are better and to pick that player. So, the key is: don't pay a high cost (i.e., to trade up in the draft) for that 52% return. Is it a coincidence that teams like the Redskins continually trade up and those like the Patriots continually trade down? No; one organization manages change far better than the other.
Maximize your number of chances: Since the draft is essentially like playing the lottery, the best teams will proceed accordingly. Its a simple formula: to have a better chance to win the lottery, a rational player will buy more tickets. Indeed, there is a strong correlation between draft success and the teams that consistently acquire more draft picks. Look at the 90s Cowboys, who built arguably the most talented team the league has ever seen, hitting on on a bevy of draft picks. They also had a LOT of misses; the key was that they maximized their opportunities. In 1991, for example, they had eighteen (!) picks, so they could afford to swing and miss on James Richards, Mike Sullivan, and Damon Mays.
The takeaway from Massey's lecture is that there are differences in managing chance. Teams like the Patriots and Eagles regularly accumulate a lot of draft picks, and New England often trades out of the end of the first first round into the second (where they can find players with roughly the same grades). Both teams amass multiple second and third rounders. If the draft is a chance process, they are exercising skill in managing chance by maximizing their opportunities to flip the draft coin.
Massey's lecture sealed the deal for Ol' Rabble, who was drafting for the Cowboys. After making the trade with Pittsburgh, Dallas held four of the first 82 selections in the draft (81 after the Saints second-rounder was taken away by Don Goodellini's mafia), and didn't experienced much of a drop-off in talent between 14 and 24. If you give Massey's work any credence--and I think we must--then the Cowboys will be wise to adopt a similar strategy in this and future Aprils.