Wednesday, February 18, 2009

Rating Quarterbacks

by Jason Winter of Defensive Indifference

There’s no really good existing way to rate quarterbacks, from a statistical standpoint. Most people understand the flaws with the widely known statistic, though there are alternatives. Adjusted Yards Per Attempt is one of the simpler ones. To review, that’s:

AYA = (Passing Yards + 10*TD passes - 45*Interceptions)/(Pass Attempts)

It is, essentially, a QB’s “yards” (counting TD passes as 10 yards and interceptions as -45 yards) divided by his attempts. Seems logical enough, and fits with other generally accepted “average” stats (like yards per carry for running backs or yards per reception for wide receivers).

But it’s always bugged me how pretty much every stat ever used to rate quarterbacks only accounts for their passing numbers. Since the days of Fran Tarkenton, carrying forth into the days of Randall Cunningham, and, more recently, with Michael Vick and Vince Young, people have argued that rushing stats should play a role in whatever system exists to rate quarterbacks. It makes sense; how can you say a QB with a 6.5 AYA (or 88.7 passer rating) for the season who runs 40 times for 40 yards is the same, statistically as one with a 6.5 AYA who runs 40 times for 200 yards? Isn’t the second guy better?

But if we get into adding rushing yards to a quarterback’s numbers, why shouldn’t we add other things, as well? Isn’t it better if a QB takes 20 sacks instead of 50, or if he fumbles five times instead of nine (assuming identical attempts)?

Ah, but now that opens up another can of worms. Many people would argue that some sacks – not all, but a fair number – are the fault of the offensive line and not the quarterback. You could make similar arguments regarding fumbles, many of which occur as the result of sacks. Why should those numbers be counted against a quarterback in any “comprehensive” statistic?

My response is: Why should passing yards be counted? Or touchdown passes? If we’re going to credit the QB for a completed pass, and assign him positive stats based on his good play, why shouldn’t we blame him for a bad play, like a sack or fumble? True, the negative play might not have been his fault entirely. But no completed pass is ever completely due to his efforts – the receiver and offensive line likely had a role in it, as well – and nobody would suggest we not include passing yards in our rating of a quarterback. If we can’t determine what percentage of a quarterback’s “good stats” are due solely to his efforts and adjust his numbers downward as a result, there’s no reason we should exclude his “bad” stats just because some part of them wasn’t his fault. Just as it’s clear that some QBs are better at running than others, it’s clear that some QBs are better at not taking sacks or not fumbling than others? Why shouldn’t the QBs who are good at this be rewarded and the ones who are bad at it penalized?

Even if we accept that some guys are “just playing on a bad team” or behind a bad offensive line and want to make adjustments for that, why shouldn’t we make adjustments for players with good offensive lines or receivers? Would Tom Brady have thrown 50 TD passes in 2007 if he’d been playing for San Francisco instead of New England and been throwing to Arnaz Battle and Darrell Jackson instead of Randy Moss and Wes Welker? Of course not.

That comes to the heart of this system, however. It’s not meant to show who the “better” QB is, not from a strict interpretation of the word. Tom Brady would have been the exact same person and same player in San Francisco, but his stats would have been lower, due to having worse teammates and coaching. A quarterback who takes a lot of sacks puts up worse numbers than one who doesn’t, all other things being equal. He may or may not actually be worse, but his stats should suffer, just as a quarterback who has Randy Moss and Wes Welker to throw to should have better numbers than one who didn’t (and we might get a chance to see how Matt Cassel does when he doesn’t have those two to throw to). Rather, this system is meant to show which quarterback had the best statistical year, taking all available statistics into account and not biasing the results with any judgments based on credit or blame. If it’s on the player’s stat line, it counts, whether for good or bad.

We can use AYA as a baseline for our “new” stat, which I like to call Total Yards per Attempt (TYA). We can stick with the basic premise of Yards/Attempts, but we’ll need to add a few things to each side. Our numerator has to include:

Passing yards
Rushing yards *
Passing touchdowns
Rushing touchdowns *
Interceptions (negative)
Sack yardage (negative) *
Fumbles (negative) *

And the denominator includes:

Pass attempts
Rush attempts *
Sacks *

(* indicates new statistic. And yes, I know I could add receiving stats, but those are so rare for QBs that I think we don't lose much by leaving them out.)

Rushing yardage can be added easily enough. Rush TDs can have the same weight as passing TDs, 10 per. Sack yardage can simply be negative yardage. Pass attempts, rush attempts, and sacks present no problems, either.

What to do about fumbles, though? Not every fumble results in a turnover, so they should be weighed less than an interception (which is always a turnover). Yet I don't know of any easy place to find the stats on individual quarterbacks' lost fumbles, and I can't assign a -45 per lost fumble for each quarterback, some of which are recovered by the offensive team.

Looking at the last few years, I find that, overall, fumbles (by any player) are recovered by the defense about 2/3 of the time (usually 65-70% per season). That's a convenient enough result, since 2/3 of 45 is an even 30. Works for me. Thus, fumbles are worth -30 to a quarterback.

(I realize, too, that fumbles by a quarterback are generally worse than an interception, since they often occur on or behind the line of scrimmage, as opposed to downfield, but unless there’s something in The Hidden Game of Football (which I haven’t read) or anyone else has any other data out there to assign better values to fumbles, I’ll go with what I’ve got.)

So, putting it all together, we get the following formula:

TYA = (Passing Yards + Rushing Yards - Sack Yards + 10*TD passes + 10*Rushing TDs - 45*Interceptions - 30*Fumbles)/(Pass Attempts + Rush Attempts + Sacks)

Which appears to take pretty much every QB stat into account.

To close, here’s the leaders in TYA for 2008, along with their rank in passer rating. The average among these quarterbacks was 5.04; figuring the average for the entire league is difficult because it requires individually looking up rushing numbers for every quarterback, as well as excluding players who threw passes and whose rushing numbers would skew the results (like running backs).







































RankQuarterbackTYAPR Rank
1Philip Rivers6.61
2Drew Brees6.524
3Chad Pennington6.392
4Peyton Manning6.195
5Jake Delhomme5.8618
6Kurt Warner5.823
7Matt Ryan5.811
8Jay Cutler5.816
9Jeff Garcia5.559
10Aaron Rodgers5.516
11Matt Schaub5.497
12Donovan McNabb5.4614
13Tony Romo5.238
14Matt Cassel5.1910
15Kerry Collins5.1823
16Seneca Wallace5.1113
17Eli Manning5.115
18Jason Campbell4.919
19Trent Edwards4.8317
20Shaun Hill4.7612
21David Garrard4.6620
22Tyler Thigpen4.5727
23Kyle Orton4.5625
24Dan Orlovsky4.2830
25JaMarcus Russell4.2326
26Joe Flacco4.1722
27Ben Roethlisberger4.0324
28Brett Favre3.9121
29Gus Frerotte3.8328
30Marc Bulger3.8131
31Derek Anderson3.2433
32JT O'Sullivan3.1229
33Ryan Fitzpatrick2.9432

7 comments:

bruddog said...

1. Not to rip on you but a number of other websites/blogs have come up with similar rating schemes.

2. I think pro-football-reference.com came up with a better number for the TD bonus as 10 yards was sort of an arbitrary number thats been thrown around. I believe they came up with something like 18 yards as the TD bonus.

3. There is going to be a lot of noise in QB rushing numbers. Good qbs on winning teams are going to have a number of kneel downs that mess with the average. And I just think with the low sample size for most qbs its hard to get a good picture of their true rushing effectiveness.

Jason said...

1. Point them out to me. I'd be interested in seeing their findings.

2. I know, they use 20 on at least a semi-official basis (though their stat pages still use 10 for stuff like AYA). Like I said, I haven't read The Hidden Game of Football, which is where (I think) the 10 yards for a TD comes from.

3. I did address the issue of kneeldowns in another blog post. If we exclude those, then we need to exclude spikes, as well, since they're as indicative of QB ability as kneeldowns are (and, to some extent, sneaks). Without any actual data to back it up, I'd guess that the number of kneeldowns per QB is about equal to his number of spikes -- one, maybe two, per game.

As for sample size, the fact that Peyton Manning had 20 rushing attempts in 2008, as compared to 569 passes/sacks means that his rushing numbers already account for a very tiny percentage (about 3.4%, to be precise) of his "attempts" in 2008. It's not a perfect system, by any means, but it's better (I think) than ignoring all non-passing stats entirely.

bruddog said...

1. Here are two websites that use very similar rating schemes

www.profootballreference.com

Look at the adjusted net yards per attempt stats. It's the same formula as you have except it doesn't include rushing yards.

http://www.wagesofwins.com/QBNP08ALLFinal.html

QB score is again basically what you have with a slightly different turnover penalty and no TD bonus. They've been doing it for a few years i believe

I've seen something similar at a few other blogs but can't locate them right now.

2. I guess I like keeping the passing and rushing numbers separate because usually qb rushing yards on average aren't nearly as valuable as qb passing yards at least based on the data at www.footballoutsiders.com. Perhaps if they were weighted a bit differently.

Even this own website does something similar except using "air yards" instead of total yards.

http://www.advancednflstats.com/2007/08/qb-wins-added-with-rushing.html

Anonymous said...

I suggest combining the interception and fumbles lost into a Turnover catergory.

Anonymous said...

Even if it's found someplace else, I like what you did with the rating. One suggestion would be to use your rating with others and test their relationship with outcomes such as wins, etc. See how much variance you can account for using your ranking vs the others.

Anonymous said...

I recommend you to read dissertation.

Anonymous said...

Just write my essay

Post a Comment

Note: Only a member of this blog may post a comment.