Taking The Stat-Heads To Task: Looking Back At Week 10 NFL Results

On Tuesday this week, like every Tuesday, we looked at the latest summary of Power Rankings here on BTB.

Some of those rankings are little more than a single person's opinion. But others are based on some quite sophisticated statistical modeling, and were developed to show the true team strength of each individual franchise. And if they do indeed show true team strength, then if Team A is ranked higher than Team B, Team A should win a direct matchup. So in a sense, the stat-based power rankings should provide an easy way to forecast game outcomes, right?

After the break, we'll look at ten stat-based predictions made for last weekend's games to get a feeling for how good these stat-based rankings and predictions are. And we'll match them up against our own BTB pick 'em panel.

The BTB Panel: Before we get started with the stat-based projections for week 10, here are the results of our pick ' em panel as a reference against which to compare the stat-based results further down the post.

Kegbearer walks away with a nice win, correctly predicting 12 out of 16 games. Our consensus pick comes in at a respectable but not spectacular nine correct picks, and nobody picked less than seven correct games.

Stat-based Predictions: The Contestants

Massey-Peabody Power Rankings: These guys generate their predictions by working up from play-by-play data to calculate a performance figure for rushing, passing, scoring and play success. Once they have that data, they generate a projected point spread for each team, and make their picks with that point spread, adjusted for home field advantage. Here's how their point spread projections look for this week, summarized in the nifty table below:

Oddsshark: Oddshark calculate their power rankings from a plethora of offensive and defensive stats and adjust for strentgh of schedule and home field advantage to get their power rankings.

AdvancedNFLstats: Instead of taking ANS's power rankings, I used their game probabilities for this excercise.

Football Outsiders: For FO, I used their overall team efficiency stats

Pythagorean Formula: This little formula is included just for fun, and measures team strength based on points scored and points allowed over the course of the season so far.

Sagarin NFL rankings: These are rankings based on W/L record, points differential and schedule strength. These rankings can be used in a similar fashion to a pointspread, to use them to predict outcomes the home team gets a 3-point bonus.

Teamrankings.com: This site measures the relative strength of all teams using team performance, home/away status, margin of victory and other factors.

WhatIFsports: These guys simulate every possible match-up in the NFL 100 times, and use the winning percentages from those nearly 50,000 games to evaluate every team, with updated rosters and depth charts.

Aikman Ratings: The Aikman Efficiency Ratings use seven statistics in five categories to calculate team strengths: yards per rush and yards per pass; third-down conversions; total first downs; adjusted points (total scoring minus points from returns and safeties); red-zone performance; and turnovers.

Beatgraphs: Beatgraphs derive their ranking by looking at head-to-head matchups. If Team A beat Team B, then Team A is better than Team B. A couple of iterations of this and they have their rankings.

Stat-based Predictions: The Results

Four of the weekend's games were not picked correctly by any of the stat-based panels: The Raiders, Cardinals, Rams and Seahawks all won games they had no business winning - according to the stat-based models. And also according to most pundits in the country, who gave these teams little to no chance.

For any type of statistical model, the more input data you have, the more precise the results of the model will be. After week 10, the models should have enough data to be relatively accurate.

The best predictions for week 10 come courtesy of Massey-Peabody and Oddsshark, both with ten correct picks, followed by ANS and Football Outsiders. But Keg and Tom beat all of the stat-based models, so where does that leave us?

It leaves us with the realization that a lot of things happen in a football game that cannot be predicted and cannot be sufficiently quantified. And that's a good thing.