Points Per Pass Metric, Final 2008 Results (All QBs)
Last year I tracked the efficiency of all NFL quarterbacks each week using my passing offense metric. This year, graduate school soaked up all my spare time, but now that I'm on break between semesters, I've calculated the 2008 values of all NFL QBs. See below an explanation of the statistical details: It's the same text I've posted last year.

Background on my Point Per Pass Metric:
I've been using a passing offense metric for many years now to gauge how well a QB is doing. Inspired by baseball SABRmetrician Bill James, I performed linear regressions of NFL statistics and concluded from a big picture point of view, only two statistics mattered: Yards gained from scrimmage, and turnovers. The best fit to matching points scored came when assuming the following coefficients:
Ten yards gained from scrimmage = +1.0 points; and
One turnover lost = -6.0 points.
Simply put, teams that score points rack up yardage and don't commit turnovers. Note how powerful turnovers are: A team can, for example, gain 55 yards of offense, but if they turn it over the next play, they actually ended up hurting their team.
With that in mind, I created the points per pass metric, which calculates the effectiveness of quarterbacks. I consider it a better measure of quarterback performance than the official NFL rankings, since it weighs the value of fumbles, and has direct ties to points scored. The above two coefficients are used to determine, on average, how many points a quarterback (and his offensive teammates, since everyone else contributes to his success or failure) creates with each passing attempt. It takes the simple yards per attempt metric we're all familiar with, adjusts it for sacks, and further adjusts it for turnovers. One interception equates to minus six points; one fumble equates to minus three points. I treat fumbles as half a turnover, since there's essentially a 50/50 chance of recovering or losing a fumble. The results will probably be unsurprising for the most part, but sometimes they can appear odd. One of the things that may cause these oddities are fumbles. Most people, when looking over a QB's stat line, don't think about fumbles; they just look at yardage, attempts, touchdowns, interceptions, and maybe sacks. But fumbles are very important, since they are essentially half a turnover.
My Research into Romo, and an Assessment of His Future:
Of course, while we may have an interest in how the rest of the league did, mainly we care about Tony Romo. And based on my research, there's good news and bad news.
First, the good news. After the 2006 season, I checked to see how many quarterbacks at the age of 26 had put up values similar to Romo. Although at the time my database was only partially complete, I was pleasantly surprised at the company he was with. Since then, my database has grown considerably, although still far from complete. Eventually I'll have every season by every quarterback in league history, but for now the database is about 60% complete.
Anyway, based on the available data, Romo most closely matched the following quarterbacks in my database: Ken Anderson, Brett Favre, Scott Mitchell, Carson Palmer, Dan Marino, Joe Montana and Johnny Unitas. With the exception of Mitchell and Palmer, we're talking about some of the best QBs in league history. Note that the comparisons with Unitas and Anderson are probably overstated, as they played in an era prior to liberalized passing rules. When my database is complete, I'll be able to normalize the values from the prior era to make more meaningful comparisons.
So that's the good news. Mainly what I wanted to establish after the 2006 season was whether history would indicate Romo was a flash in the pan or someone who would be a franchise-caliber quarterback. Based on my research then, I concluded Tony was not a one-hit wonder, but would be a good passer to lead this team into the future.
Now for the bad news. After reaching that initial conclusion, I began to look into how those similar quarterbacks did in their next year, at age 27. These were great quarterbacks, and with another year of experience under their belts, they got better, right? Unfortunately, no, most of them regressed. While Romo came up with a sparkling 0.51 value in his first season as a starter, in 2007 his efficiency actually declined to 0.48. So before this year, I did a more thorough check in my database to see how Romo would perform at the age of 28. The results were not encouraging: On balance, this group of quarterbacks declined even further at age 28, and that's exactly what happened to Romo this year. His value dropped to 0.45, which is barely above average compared to the rest of the league starting quarterbacks.
There's more bad news. Taking that same group, I followed the trajectory of their careers, and discovered the decline continued until the age of 30. That is, their median values declined each year until they were 30 years old, when they bottomed out at a value of 0.36. A passer in this group has a trajectory radically different from the typical quarterback in the database, who starts out slow, reaches his peak at age 30, then declines until he is out of the league. This group came out of the gates hot, then cooled off with every year, until they reached the bottom when most of their peers had reached their peak. So based on my research, we will likely have two more years of Romo raising our collective blood pressure. And if this research is correct--and the pattern this group of quarterbacks follow is absolutely eerie in its similarity--it's going to get worse before it gets better. This does not speak favorably for Dallas winning a Super Bowl before 2011, assuming Romo remains our starting quarterback.
Okay, that's the end of the bad news. The good news is there is a light at the end of the tunnel. The group began to rebound at age 31, when most quarterbacks start to decline, and at ages 32 and 33 turned in years which approached their former greatness of age 26 (note that Palmer is excluded from this group, as he is 29 this season). The group had median values of 0.48 and 0.46 respectively for those two years, then begin to decline again. Those values are plenty good enough for a quarterback to win a Super Bowl, and in fact was done by Montana in 1988.
Many of you may well be skeptical of this analysis, particularly since it is not delivering news that is reassuring to us Cowboys fans. One may believe that every quarterback and the circumstances of a season is completely unique, and that I am assuming too much in lumping together this group of quarterbacks and following their careers. This would seem to be a reasonable objection, except for the fact that, as I mentioned before, the group moved along with eerie similarity. You would be able to find the occasional reversal of decline in one year of one quarterback, but the pattern has a strong trend to it. And that pattern is decline until the quarterback reaches 30 years old, followed by a three year rebound. It may be that Romo's decline in performance is largely attributable to the play of the offensive line this year, as several have advocated. But if that were the case, why does this whole group follow such a strong pattern? Did all the other quarterbacks in this group coincidentally experience a drop off in pass protection at age 28? Not likely. The pass protection breakdown seems to be secondary to a larger problem, more like a long-term learning process these quarterbacks go through after experiencing the initial flush of success. For example, Romo had the highest rate of fumbles per sack of any of the regular starting quarterbacks this year at 65%, although there were a few other quarterbacks that were close. Is that the fault of his offensive line? No, it's due to poor ball protection.
it will be interesting to watch how Romo's career progresses (or regresses) over the next couple of years. And I should point out that I am not saying with certainty he will get worse the next two years. I am saying that his career fits well into the mold of other great historical quarterbacks, who experienced an inexplicable decline phase at the traditional peak of their careers; and that I consider it very likely he will decline the next two seasons.
One more thought about our star QB before I move on to my comments about the rest of the league. One, despite the apparent increase in impulse throws this year, his percentage of interceptions per attempt actually declined for the second straight season. It was 3.9% in 2006, dropped slightly to 3.7% last year, and to 3.1% this season.
Other notes:
- The league average value this season for quarterbacks was 0.408, the highest value in my database, and my annual league database goes back to 1991. The league-wide benefits of passing have been rising for three straight years. That makes Romo's decline all the more noteworthy.
- Because his career did not get rolling until the age of 28, he did not fit the comparison group to Romo, but the career of Kurt Warner has the same general trajectory of the other similar quarterbacks. After his incredible 1999-2001 seasons, he looked like he was washed up by 2003. This year he posted the third best season of his career, and the fourtth straight strong season. He's 37, and will probably not repeat this season's performance again, but the rehabilitation of his career is complete.
- Matt Ryan had an incredible season as a rookie. I could find only six other quarterbacks in my database who attempted at least 160 passes their rookie season who posted a passing value of greater than 0.40. They were Charlie Batch (1998), Aaron Brooks (2000), Charlie Conerly (1948), Joe Flacco (2008), Jeff Garcia (1999), Dan Marino (1983) and Ben Roethlisberger (2004). However, Conerly and Garcia were relatively old for rookies, being 27 and 29, respectively.
- After posting the second-best season of his career last year, Brett Favre fell back down to earth. His yards per attempt plunged, and also threw 22 interceptions. Not pretty.
- So how smart do the Jets feel now, after trading away the best QB this season for one of the worst? I've always been a Chad Pennington fan, he protects the ball very well and plays a smart game. A perfect quarterback for Sparano's system. And this season, he narrowly missed having his best season ever, in 2002.
- Speaking of Garcia, he followed up the best season of his career last season with another outstanding year. Along with Favre and Warner, Garcia is extending the notion of how long a quarterback can play at a championship level.
- Peyton Manning bounced back from his somewhat disappointing last season. His value this year of 0.56 still puts him among the best quarterbacks in the league. His third MVP award is well deserved. Now 32, he is starting to show signs of decline from his incredible 2004-2006 peak, but I have little doubt he will be the greatest quarterback in league history when he eventually retires.
- Despite the grief given to Donvan McNabb, he turned in another excellent overall season. At 32, his best seasons are almost certainly behind him, but as we found out last week he's still a very dangerous quarterback.
- Both Jason Campbell and Eli Manning turned in the best seasons of their careers. Many think Campbell won't make it in the long term, but I've always thought he's a fine prospect, and his performance this year helped bolster my case. His value this year was virtually indistinguishable from Romo's.
- Few people pay attention to Jake Delhomme, but he had another very good season, picking up where he left off prior to his 2007 injury.
- Phillip Rivers turned in the best season of his career, and one of the 25 or so best ever in my database. This despite the deep decline in the Charger's running game, significant problems with the offensive line, and the worst statistical season by Antonio Gates since his rookie season. This is Exhibit B for those who want to lay the blame of Romo's recent performance at the feet of the offensive line, and end the discussion there. I suspect Norv Turner's game calling is helping him here. Rivers is just entering his prime at 27, and has the ability to be a great one.
- Jay Cutler has shown steady improvement each year, and this year he turned into one of the best in the league. I'm curious how he will fare under the new Denver coaching staff. At the age of 25, he's still got plenty of growth potential.
- It was masked by their gaudy team record, but Ben Roethlisberger regressed back to his 2006 form this season. This is the third straight season of mediocrity, after his first two magical seasons.
- For those who recall my discussions from last year, I noticed Drew Brees alternated excellent seasons with not-so-good years, and this was his bounce back season. He turned in his second best season ever, narrowly missing his incredible 2006 season. If the career pattern holds, however, Brees will decline next season.
The 2008 Rankings:
As I did at the end of last season, I ranked the quarterbacks by efficiency in three groups, based on the number of passing attempts:
- Those with at least 160 attempts;
- Those with less than 160, but at least 30 attempts; and
- Those with less than 30 attempts.
Quarterbacks with at least 160 attempts:
Pennington, Mia 0.60
Rivers, SD 0.59
Brees, NO 0.58
Manning, Ind 0.56
Delhomme, Car 0.53
Warner, Ari 0.52
Cutler, Den 0.51
Garcia, TB 0.51
Ryan, Atl 0.51
Schaub, Hou 0.49
McNabb, Phi 0.49
Collins, Ten 0.48
Rogers, GB 0.48
Cassel, NE 0.46
Manning, NYG 0.46
Wallace, Sea 0.45
Romo, Dal 0.45
Campbell, Was 0.44
Edwards, Buf 0.42
Flacco, Bal 0.42
Garrard, Jax 0.41
Orton, Chi 0.39
Hill, SF 0.38
Rosenfels, Hou 0.36
Orlovsky, Det 0.35
Thigpen, KC 0.35
Russell, Oak 0.34
Roethlisberger, Pit 0.33
Bulger, StL 0.32
Favre, NYJ 0.30
Frerotte, Min 0.29
Griese, SF 0.29
Anderson, Cle 0.26
Fitzpatrick, Cin 0.21
Hasselbeck, Sea 0.20
O'Sullivan, SF 0.19
Quarterbacks with between 159 and 30 attempts:
Leftwich, Pit 0.72
Sorgi, Ind 0.59
Quinn, Cle 0.43
Jackson, Min 0.42
C. Palmer, Cin 0.26
Kitna, Det 0.21
Grossman, Chi 0.20
Culpepper, Det 0.14
Green, StL 0.14
Huard, KC 0.10
Young, Ten 0.07
Johnson, Dal 0.05
Losman, Buf -0.06
Dorsey, Cle -0.13
Walter, Oak -0.14
Kolb, Phi -0.28
Quarterbacks with less than 30 attempts:
Smith, Bal 1.62
Gray, KC 0.95
Carr, NYG 0.87
Brady, NE 0.69
Henne, Mia 0.56
Croyle, KC 0.47
Leinart, Ari 0.40
Stanton, Det 0.37
O'Connell, NE 0.30
Dixon, Pit 0.30
Berlin, StL 0.20
Lemon, Jax 0.00
McCown, TB 0.00
Bollinger, Dal -0.06
Simms, Ten -0.10
Frye, Sea -0.22
Ramsey, Den -0.40
Flynn, GB -0.48
J. Palmer, Cin -0.64
Clemens, NYJ -0.68
Gradkowski, Cle -0.69
Henson, Det -1.30
Tuiasosopo, Oak -1.80
Another user-created commentary provided by a BTB reader.
0 recs |
7 comments
Comments
Romo and Seneca Wallace tied for middle of the pack
Nice.
Something tells me that there is a fundamental flaw somewhere in the metric.
I would say you need to dig deeper. Turnovers at the end of halves on desperation heaves shouldn’t count. 10 yard pass plays don’t tell the story either, since teams that run the ball well won’t be forced in long down and disatance situations as much as teams who struggle to run. That is, Collins benefitted greatly from White and Chris Johnson in a way that Romo benfitted from Julius and Barber last year, but not in 2008. Maybe there is a way to tie the performance of each QB’s running attack to the mix?
I think Brees was outstanding this year, especially because he didn’t have a great running game to rely on and had to convert long yardage situations himself. Eli Manning didn’t have to do that. Neither did Matt Ryan.
"Well, we didn't block real good but we made up for it by not tackling."
- John McKay, the first coach of the Tampa Bay Buccaneers
by 5Blings on Jan 2, 2009 1:55 PM CST reply actions 0 recs
Digging Deeper
Interesting thoughts, although I don’t agree with them.
Interceptions on Hail Marys don’t occur frequently enough to affect the outcomes. Not worth digging into.
The tie between the strength of running and passing games is almost certainly overrated. Joe Montana, Roger Staubach, Dan Marino, Tom Brady and many other passers were able to post some the best years in league history in seasons without very good running games. Rivers this year is another example. A great passing attack can dismantle any defense, whether there is a running game present or not. There may be a correlation between the two, although I don’t believe it’s very strong, and difficult to sort out the cause-and-affect related variables. The second-and-long phenomena assumes teams run on first down. Jason Garrett does this constantly, but most successful modern offenses pass to set up the run, not the other way around.
by kindablue on Jan 2, 2009 2:46 PM CST up reply actions 0 recs
Really?
Montana had a very capable running game with Craig and Rathman. He led a very balanced attack. Marino had nothing behind him, but never won anything of consequence either, so I’m not sure he matters any more than Scott Mitchell.
Your pass versus run comments don’t hold water. I just watched a great passing attack GET dismantled in the Cotton Bowl. If you missed it, the last Super Bowl is constantly being aired on NFL Network as more evidence to refute your point. Staubach won Super Bowls with Ring of Honor backs behind him. Brady won Super Bowls with solid run games and then lost the last one when they became pass happy.
All you were able to accomplish in this exercise is an inability to come up with a measurement that would stand up to the criticism that it does not go beyond correlation to find causation.
The point is this; you randomly picked a yardage gain number to harp on in your measurement and then you randomly assigned a point value to turnovers. Your analysis is thusly…random.
Sorry.
"Well, we didn't block real good but we made up for it by not tackling."
- John McKay, the first coach of the Tampa Bay Buccaneers
by 5Blings on Jan 2, 2009 5:26 PM CST up reply actions 0 recs
You Should Be Sorry
Because you clearly were responding to something besides what I said. Of course Staubach and the others had excellent running attacks, in some years. But that wasn’t what I said. I said they were able to post excellent performances in years in which they did not have that support. Obvious examples include Montana in 1981, Brady in 2003, Marino most of his career, and so on. That’s how one at least indirectly measures the impact of the value of a running game, and the offensive success of these QBs when they did not have one suggests the impact is not as great as conventional wisdom would suppose.
It also appears you either didn’t read or understand the background at the top of the post; performing regressions by definition determines how well correlated are two variables. The r-squared values I found were in the 0.7 to 0.8 range, which is a correlation well beyond randomness. So your belief that my work on this subject is random doesn’t apply or make any sense.
by kindablue on Jan 2, 2009 5:51 PM CST up reply actions 0 recs
You obviously spent a lot of time on this
I apologize for poking holes in it and putting you on the defensive.
It’s a wonderful analytic and I dearly hope the NFL embraces it as the new yardstick by which all NFL QB’s are measured.
"Well, we didn't block real good but we made up for it by not tackling."
- John McKay, the first coach of the Tampa Bay Buccaneers
by 5Blings on Jan 2, 2009 6:48 PM CST up reply actions 0 recs
I think that the examples you're pointing out are the exception, not the rule.
Those are 3 of the 5 or 6 best QB’s to ever play the game. I’m sure others have done it as well, but my point is just because it has been done doesn’t make it the norm. Also, most seasons where teams have great passing years and horrible running years are from lack of effort, not lack of ability.
by Mandmeisterx on Jan 5, 2009 2:52 PM CST up reply actions 0 recs
good stuff man...although some how, some way more variables should be introduced..
A true diehard Cowboys fan since 1975.
President - Garrett MUST GO campaign -
by BoyzRback on Jan 2, 2009 2:56 PM CST reply actions 0 recs

by 
















