Grading Mock Drafts And Big Boards

I am running a series of articles where I use a weighting system to come up with a "best guess" average from several mock drafts and big boards, in the hopes of coming up with a more accurate guess than individual mocks do. It is just my contribution to the overall giddy anticipation of the NFL draft, which is approaching with agonizing slowness.

But in the process of working on these, I got to wondering if anyone had ever done anything like this. Or had anyone ever gone back after the draft and looked at how accurate past mocks were?

As it turns out, Fan In Thick and Thin ran a similar exercise last year. He looked at four "big boards", which ranked prospects by how good they were, rather than how the writers actually thought they would be drafted, and used a similar approach to mine to come up with his own best guess as to what order the players should go in. I was very curious to see how close this might have gotten, and maybe to find out if there is some analyst out there that we should pay attention to.

After looking over his information, I am thinking that I might want to adjust my methodology. You know, examine some goat entrails, ingest some funky mushrooms and see what the visions tell me, throw darts. The so-called experts FiTaT looked at were, shall we say, not too good.

But there were also some very interesting lessons to be learned, so take a look at the numbers with me.

Raw data and rude analysis after the jump.

I am focusing this, as I do my own "averages", on the first round. I think the ability to make any kind of accurate prediction declines rapidly as you go deeper into the draft, so I am just looking to see if the first round can be called with at least a little reliability. Unfortunately, FiTaT's data says "probably not".

Here is what he came up with last year. The "Actual" column shows when the player was picked last year, and the Projected column is when the average of the boards FiTaT looked at predicted the player should go, based on their evaluation. The last column is how far off they were. (A minus number is a low projection, where a plus is one that was high.)

Not exactly stunning accuracy. This is an average of the boards from Draftek, National Football Post, Scouts.com, and CBS. I use at least three of these in my own posts. I have looked to see if one of those was much more accurate than the average, and don't really see any difference.

Only two players were pegged exactly here, Von Miller at 2 and our own Tyron Smith at 9. The rest gets pretty random. But right off the bat, there are some significant differences between 2011 and this year.

The biggest is the quarterback situation. Cam Newton was a hotly disputed commodity last year, and many were very critical of Carolina using the number one overall pick on him. The average used here had him not even breaking into the top half of the first round; CBS had him highest, at 13, but none of the boards saw him as worthy of the top ten, much less the first pick in the entire draft. Given Newton's exciting and fairly productive season, that looks in retrospect to perhaps have been an error.

Last year was seen overall as a very weak quarterback class. Look at the four quarterbacks taken in the first round. All were taken well ahead of where the "experts" predicted they should be, showing that the QB position is it's own breed in the NFL.

This is not going to be the case this year, as Andrew Luck and Robert Griffin III are both very highly ranked, and almost certain to be taken 1 and 2, pretty much in line with expectations. They also head a quarterback draft class that is seen as much stronger.

And yet, 2011 was probably typical in the aspect that quarterbacks almost always get overvalued in the draft. So what might that imply about this year?

Well, I think it almost guarantees that Ryan Tannehhill is going to go before Dallas comes on the clock at 14 (barring any trades). But looking at those numbers from last year, and the fact that Cleveland and Miami are just two of the teams still needing a signal caller, and I wonder if Brandon Weeden might become part of the mix, even before Dallas comes up. Based on how CBS currently rates him (42nd overall player, all positions, in the draft), it would not be much more of reach for a team ahead of Dallas to get him than it was for Christian Ponder or Jake Locker to go in the top 12 last year.

There will undoubtedly be some big jumpers this year, and not just at quarterback. It is obvious that there is a wide range of opinion, and scouting is far from an exact science. Teams make what can only be described as a reach (OT James Carpenter was taken 54 spots ahead of the prediction) - or they see something that the outside evaluators may have been completely wrong about. Cam Newton is the prime exhibit last year that maybe the teams can be on to something the writers are not - although it is hard to say at this time that he is a better pick than Von Miller or Patrick Peterson. But he may have been the most valuable for the team.

There are also going to be some sliders. Mark Ingram was the best example last year, and he was certainly a bargain (if you believe in taking running backs in the first round). But the key for each team is how they see the players. If you see a top ten player on your board still hanging around at, say, fourteen, you might get a little excited.

About the only thing that I saw here that I really expected to see was that the predictions seemed to get less accurate as they went along. As a matter of fact, if you pull out all the quarterbacks, the first ten picks were not far off at all, and the first twenty were reasonable. It was after that the variances started to get really big. So there does seem to be some agreement of opinion in the first half to two-thirds of the draft.

Frankly, given the situation with the top two quarterbacks on the board, I think my exercise in averaging the Mocks may be a little more accurate than this was, just because there won't be as much quarterback distortion. But we will know in a little over three weeks.