The Dallas Cowboys have an analytics problem

Dallas Cowboys head coach Jason Garrett is officially on the hot seat. Gifted a talented roster and an easy schedule, the team he’s responsible for has failed to meet expectations. The team currently sits at 6-5 and were it not for a woefully poor NFC East would be on the outside looking in at the playoffs.

This is not what fans, the media, or (most importantly) what Jerry Jones envisioned back in August. Garrett has come under extreme scrutiny because four of the five Cowboys’ losses have been by less than four points. In each of those losses, Garrett has made multiple decisions that seemed to fly in the face of what statistics tells us is the correct decision.

Monday, Garrett was specifically asked how the Cowboys’ incorporate game day analytics in their decision-making. The first instance came on Garrett’s weekly Monday appearance on 105.3 The Fan:

The question, paraphrased, to Garrett was “do you have win probability stats available to you during the game”. Garrett’s verbatim answer:

Yeah, we don’t use those stats within the game.

Predictably, this led to much furor and teeth-gnashing on Youtube and Twitter. A few examples:

It’s not the coach, it’s the entire organization

Jason Garrett is getting most of the heat for this situation but Jerry Jones confirmed Tuesday that the organization’s disdain for game day analytics reaches up to the owner’s box. Appearing on 105.3 The Fan Jones was asked his thoughts on the topic:

“I happen to agree with him. The analytics aspect of it is a history of how these decisions have worked out. That’s probably good to know. But the momentum of the game, the situation of the game, the, if you will in the other night’s game, the conditions of the game. All of that are more important in my mind than the success history of a similar decision. You’re dealing with averages, you’re dealing with almost theory. And you’re certainly dealing with a result but it doesn’t take into account really the kinds of times when you’ve gone against every odds and made it work. I’ve had my biggest success when I’m sure analytics would have said make the other decision the other way.”

The NFL is slow to change

In some ways this is entirely predictable. Long-lived institutions like the NFL are rigid and slow to adapt to change. Heck, the “modern” NFL offenses that are such the rage nowadays were being run in high school and college games since the 90’s. Any suggestions of giving such schemes a try in the NFL were generally dismissed out of hand by, well, pretty much every owner, GM and coaching staff until very recently.

So it’s not surprising that not every team is on-board with utilizing analytics. In fact, the adoption of such ideas has been so slow that coaching staffs that do employ them get feature articles in the New York Times and The Athletic. Those teams are the exceptions, the outliers.

It’s simple math

On the other hand, it’s utterly infuriating.

This is simple, basic stuff when we boil it down, and I’m not talking about the information the analytics yield. I’m talking about the thinking Jones and Garrett employ to make the statements above. We’ll see they are utterly nonsensical and void of logic.

Game day analytics aren’t complex; they’re simply math calculations that are easy to understand. They use historical outcomes from past NFL games to calculate how likely a team is to win based upon:

• Score
• Location on field (yard line)
• Down
• Distance to first down
• Time remaining

You can read here to learn more. How would this be used during an NFL game? Well, imagine you’re facing 3rd-and-7 from the opponent’s 11-yard line with six minutes remaining and you’re down 7 points. The Win Probability Added calculations would tell you:

• Successfully kicking a field goal would change your likelihood of winning by W
• Failing a field goal attempt would change your likelihood of winning by X
• Successfully converting a first down would change your likelihood of winning by Y
• Failing a fourth-down attempt would change your likelihood of winning by Z

Basically, if one option gave you a significantly better outcome than the other you should strongly consider that option.

It’s all pretty simple. These Win Probability factors show us that, historically, NFL coaches have been kicking field goals in many instances where they should be going for it on fourth down. Now, these calculations take nothing about what’s actually happening on the field into consideration:

• Field conditions
• Weather conditions
• Quality/caliber of players/teams involved
• How one unit (offense or defense) has been performing

That’s why no one would ever suggest simply using Win Probability calculations to determine whether to kick or go for it. But refusing to include the information means your decision is less informed than it could be. Who would willingly choose to be less informed when making a decision?

It’s illogical

And that’s where the Cowboys’ approach is indefensible. It’s really simple: teams who utilize these calculations are making better informed decisions than teams that don’t. Why would anyone, any team, any organization willfully remain ignorant of readily available data that could make your decisions better informed?

No NFL team would choose not to study game-film on an opponent; it would be ridiculous to go into a game uninformed on the other team’s schemes and strategies. But the Cowboys are essentially doing the same thing regarding win probability odds by refusing to utilize the information they provide.

The window of opportunity exists right now

Bill James is widely credited with inventing “SABRmetrics” for baseball in the mid-seventies. He figured out that something he called “defense-independent pitching” (DIPS) was better at measuring pitching performance (and predicting future performance) than traditional statistics like earned run average. It would be a quarter century before a Major League Baseball GM strongly embraced such ideas when making front-office decisions. It would take another decade before use of such ideas became widespread throughout MLB.

What that meant was, for a time, some teams had an advantage over their opponents. This is called the “window of opportunity” - a time when simply being more informed gives one organization an advantage over an uninformed opponent.

That’s where we are with game day analytics in the NFL. A window of opportunity currently exists and it’s readily available for any team willing to make the investment. Make no mistake, sometime in the near future (five years? ten years?) every single NFL team will be doing what the Baltimore Ravens are doing right now:

• Employ a professional statistician (perhaps more than one).
• Integrate win probability calculations with game day strategies.
• Utilize win probability calculations when making in-game decisions.

There will be a time very soon when Jerry Jones’ and Jason Garrett’s thoughts on analytics will be considered old, out-dated, misinformed. In the meantime, the Dallas Cowboys are currently on the wrong side of that window of opportunity. While some teams are using them to take advantage of their opponents the Cowboys are the team being taken advantage of.