## 1&10: Defense under the EPA microscope

One area where statistics excel is looking the big picture.

It’s hard to get a sense of where relative strengths and weaknesses are simply by watching games. Lot’s of people can tell you that the Dallas pass defense struggled but it’s impossible to tell by eye exactly when and where they struggled.

Now that the season is over and we have play by play for the full season and we can drill down in the numbers.

One of the best tools we have for this is Expected Points Added (EPA). For those who aren’t familiar with EPA here is a brief explanation.

EPA analysis is based on the idea that for every set of down, distance, and field position there is an Expected Point Value. Here’s the definition from Advanced NFL Stats.

Expected Points (EP/[EPV]) – The value of the current down, distance, and field position situation in terms of future expected net point advantage. In other words, it is the net point value a team can expect given a particular combination of down, distance, and field position. First and goal at the one represents an EP near 6, while 3rd and 20 at a team’s own one yard line represents an EP of about -2. EP differs from Win Probability (WP) in that it does not take into account the game score and time remaining.

For example, a 1 & 10 at the 50 yard line is worth 2.04 expected points. A 1 & 10 at the 40 yard line (40 yards from the end zone) is worth 2.66 points.

EPA then is the change of the EPV. Here’s the definition from Advanced NFL Stats.

Expected Points Added (EPA) – The difference between the Expected Points (EP) at the start of a play and the EP at the end of they play. EPA is the measure of a play’s impact on the score of the game. An individual player’s EPA is the sum of the EPA of the plays in which that player was directly involved. Being directly involved is defined as an offensive player who ran, threw, or kicked the ball, was targeted by a pass, or flagged for a penalty.

Continuing with our example, if Dallas starts with a 1&10 at the 50, and then completes a 10 yard pass to Miles Austin for a 1&10 at the 40, the EPA of that play is +0.62 (i.e. the difference between the ending EPV of 2.66 and the starting EPV of 2.04). If instead Dallas calls a rush that gains 4 yards, leaving them with a 2nd & 6 from the 46, the EPA of that play is +0.01 (i.e. the difference between the ending EPV of 2.05 and the starting EPV of 2.04).

For this analysis I’ll be looking at aggregate EPA numbers for the defense. Keep in mind that an average defense would produce and EPA of zero. Assuming that the team is average in all the other facets of the game (special teams, offense , turnovers, etc) that team would allow the average pts/game in the NFL. Keep in mind that simply by virtue of the regular 10-16 possessions every team has throughout a game (kickoffs, punts, etc) we expect the average amount of points to be scored (e.g. if we assume 13 possessions / game, starting field position 67 yards from the end zone, an EPV of 1&10 / 67 yards from end zone of 1.05, then the baseline points is 13 x 1.05 = 13.65 pts/g … median pts allowed was actually 21.7 pts/g in 2010). What EPA is capturing is the overperformance / underperformance relative to that baseline. So if a defense allows a positive EPA of 48 pts for the season, that would be 3pts/game worse than average.

Dallas Defense

The table below details the performance of the Dallas defense EPA. As you can see the Dallas defense allowed 66 EPA or about 4 pts/game. Keep in mind that the average defense allowed about 22 pts/game. Therefore, this explains the difference between 22pts / game and 26 pts / game.

 Team EPA+ Rank PIT -80.71 1 GB -60.8 2 CHI -54.99 3 SD -53.42 4 NYG -45.57 5 BAL -22.5 6 SF 4.25 7 STL 6.26 8 MIA 9.22 9 NYJ 13.67 10 MIN 14.02 11 TB 21.46 12 OAK 24.04 13 NO 24.4 14 KC 24.51 15 NE 36.65 16 ATL 37.62 17 CAR 43.06 18 TEN 47.16 19 CIN 55.23 20 CLE 57.85 21 PHI 58.16 22 WAS 61.62 23 DET 62.64 24 DAL 66.47 25 IND 75.47 26 ARI 81.58 27 SEA 100.12 28 BUF 108.74 29 DEN 137.27 30 JAC 146.31 31 HOU 176.07 32

Now drilling down, the table below details the performance of the Dallas defense by down, distance, and type of play.

 Dallas Cowboys 1st 2nd 3rd Pass Rush Pass Rush Pass Rush Penalty Total EPA 60.6 (6.0) 26.5 (19.3) (27.0) 0.2 28.1 63.0 % of Total 96.2% (9.5)% 42.0% (30.7)% (42.9)% 0.3% 44.6% 100.0%

 NFL 1st 2nd 3rd Pass Rush Pass Rush Pass Rush Penalty Total EPA 20.6 (6.5) 6.0 (11.0) (6.7) (2.7) 27.7 27.4 % of Total 74.9% (23.6)% 21.9% (40.0)% (24.4)% (9.9)% 101.0% 100.0%

For the NFL data we see what we already knew: that passing on 1st down has a relatively higher payoff than running on first down. But we see that Dallas allowed 60 EPA on 1st down, 40 points over the NFL average. In other word, bad pass defense on 1st down cost Dallas ~2.5 pts/g over the NFL average in 2010.

Here’s what I consider notable:

-Dallas was significantly worse than the average NFL team on 1st down pass defense.

-Dallas was worse than the average NFL team on 2nd down pass defense.

-Dallas was better than the average NFL team on 3rd down pass defense.

-The rush defense was ok, in line with average NFL performance.

Here’s another way to look at it (sorry … this may make you sick … a horrible walk down memory lane). Here are the 20 worst plays by the Dallas defense in 2010.

 EPA Down Distance Description 6.55 1 10 (11:43) M.Vick pass short left to D.Jackson for 91 yards, TOUCHDOWN. PENALTY on PHI-D.Jackson, Unsportsmanlike Conduct, 15 yards, enforced between downs. 5.68 2 4 (12:11) J.Skelton pass deep left to A.Roberts for 74 yards, TOUCHDOWN. Penalty on DAL-M.Jenkins, Defensive Pass Interference, declined. 4.98 3 15 (6:03) (Shotgun) J.Cutler pass deep middle to J.Knox to DAL 20 for 59 yards (M.Jenkins). Pass complete on fly pattern. 4.79 3 11 (5:18) (Shotgun) Sh.Hill pass deep left to N.Burleson to DAL 12 for 58 yards (G.Sensabaugh). Pass complete on a "Go" pattern. 4.6 3 10 (2:25) (Shotgun) D.Brees pass deep right to R.Meachem to DAL 12 for 55 yards (T.Newman). Pass complete on "GO" pattern. 4.36 1 20 (13:55) D.Brees pass deep middle to D.Henderson to DAL 7 for 57 yards (A.Ball). Penalty on DAL-A.Ball, Defensive Holding, declined. Pass complete off play action. 3.78 1 10 (14:53) M.Vick pass deep left to D.Jackson to DAL 13 for 60 yards (A.Ball). Pass complete on fly pattern after Vick roll out. 3.76 1 10 (9:51) D.Garrard pass deep right to M.Lewis for 42 yards, TOUCHDOWN. 3.7 4 1 (12:29) L.McCoy up the middle to DAL 1 for 3 yards (B.James). 3.58 1 10 (1:17) J.Cutler pass short middle to G.Olsen for 39 yards, TOUCHDOWN. 3.55 1 10 (7:39) L.McCoy up the middle pushed ob at DAL 21 for 56 yards (M.Jenkins). 3.36 1 10 (:36) (Shotgun) P.Manning pass deep left to R.Wayne to DAL 1 for 40 yards (M.Jenkins). 3.34 1 10 (5:29) E.Manning pass deep left to K.Boss for 35 yards, TOUCHDOWN. 3.33 3 5 (12:49) V.Young pass deep right to N.Washington for 24 yards, TOUCHDOWN. Penalty on DAL-O.Scandrick, Defensive Pass Interference, declined. 3.3 3 12 (12:22) (Shotgun) D.Garrard pass short right to M.Thomas for 15 yards, TOUCHDOWN. 3.27 1 10 (14:02) (No Huddle, Shotgun) P.Manning pass deep left to R.Wayne for 34 yards, TOUCHDOWN. 3.17 1 10 (9:45) K.Kolb pass deep right to C.Hall to DAL 10 for 48 yards (T.Newman). 3.13 1 10 (2:12) V.Young pass deep middle to K.Britt for 80 yards, TOUCHDOWN. Dallas challenged the runner was down by contact ruling, and the play was REVERSED. V.Young pass deep middle to K.Britt to DAL 28 for 52 yards (M.Jenkins). 3.09 1 10 (7:14) M.Vick pass short left to D.Jackson pushed ob at DAL 2 for 37 yards (A.Ball). Pass complete on curl. 3.05 2 3 (5:57) J.Cutler pass short left to D.Hester to DAL 3 for 38 yards (J.Ratliff). 78.37

As you can see, 11 of the 20 most costly plays from an EPA perspective were pass plays on 1st & 10. Only 2 of the 20 most costly plays were runs.

All the usual caveats apply. We’re looking at a subsample of all plays. I haven’t tested to see whether this pattern is consistent from year-to-year. It’s entirely possible that we're just witnessing random noise. Next year Dallas may be great on 1st down passes and bad on 3rd down passes. We have to be careful about drawing conclusion that aren’t warranted by the evidence. For example, the best play by the Dallas defense in 2010 was Bryan McCann’s 101 yard pick-six interception, worth -11.23 EPA. That single play had a major impact on the 3rd down EPA totals.We also know that defensive pass efficiency (and EPA is an efficiency measure) is inconsistent so we can't read too much into this (hence Part II, which will look at success rate by down, distance, and type of play).

Here’s one hypothesis though. Teams can take advantage of the situational run stuffers Dallas uses. When Dallas has Olshansky, and Spears/Hatcher in the game, there’s just not enough pass rush. Olshansky, Spears, and Spencer are too easy to block. Spencer is great if he’s matched up against a RB, against an OL not so much. Teams know if they handle Ware and Ratliff they’re going to have time to allow pass plays to develop. Hence, teams that were willing for forego a 3 yard gain on 1st down in order to take a shot downfield were able to take advantage of Dallas’s defense.

It’s interesting to note that Pittsburgh substitutes less than the average team. Their defensive players are versatile and can play every down. On the other hand, the Jets do substitute and they have an excellent defense (although it’d be interesting to look at their splits). As I said above, I don’t want to draw unwarranted conclusions. Maybe a topic for further research.

Anyway, it’s worth keeping in mind the strengths and weaknesses as we consider where Dallas needs to improve.

p.s. if anyone is interested in EPV analysis, I’ve already built all the logic into a spreadsheet. You’re welcome to it. Just send an email.

Another user-created commentary provided by a BTB reader.