Sunday, January 23, 2011

Eight Football Stats I Hate and why the professional pundits put so much effort into selling them to us.


by Jim Glass

I may be getting old and cranky, but TV analysts repeatedly, breathlessly throwing out junk stats to me during all the big games is getting to me. And I expect it will only get worse, because there is something about the culture of football that encourages them to do it, makes it profitable for them -- and football is a business.

Here are some of the stats I really dislike and why, followed by a little editorializing on why NFL game producers and their attending media put so much effort into pushing them.

Official NFL passer rating. All football stat mavens know that passer rating is troubled. But consider the details...

* It so overvalues completion percentage that a completion that loses yards in any amount is rated at 79.2. Hitting 10 of 10 and losing 67 yards (or for that matter of 999 yards) is rated the same as hitting 5 of 10 for a gain of 67 yards: 79.2.  If the 5-of-10 QB gains only 66 yards he has the lower of the two ratings in spite of doing 13.3 yards per attempt better.  A quarter of the league's starting QBs had ratings below 79 this year -- they could have improved their ratings by throwing more passes that lost yards.

* Is is so biased towards safe short-throwers that the NFL Hall of Fame's list of top 20 all-time passers by rating includes only QBs who played within the last 16 years.  Joe Montana is the oldest of the 20.  These QBs of course all played well after the rule changes of 1978 that freed the passing game and resulted in creating new short-throwing offenses.

Pete Palmer noted in the 1980s that passer rating reduces to...

" weighted yards per attempt with a bonus of 20 yards for each completion, an additional 80 yards for each touchdown, and a 100-yard penalty for each interception. So two completions for ten yards each are worth the same as one completion for forty yards and one incompletion. A ninety-yard pass play from goal line to the opponent ten is worth the same as a ten yard TD pass..."

Do those weights make sense? Anything plausible must better, right? Just to show it, I put all of PFR.com's team passing data for the last five years on a spreadsheet to correlate winning with all variations of yards-per-attempt: straight, net minus sack yards, net AYA with the much more sensible +20 yds for TDs, -45 yds for interceptions etc. The result was that the version that correlated most strongly with winning was ... NFL passer rating.

Now you see why I really hate this stat. It is so misconceived it shouldn't have survived to take first breath -- yet it will not die and thrives! (More on this another time.)

Red Zone efficiency. At best this it is bad stat -- and does concept of it really even make sense? 

Back after week 11 when I last looked, the #1 and #2 teams by red zone efficiency were Detroit and Buffalo, then a combined 4-17 by won-lost. The full-year numbers for 2009 showed the top five teams by red zone efficiency went a combined 40-40 in won-lost.

I can think of two reasons for these counter-intuitive numbers: (1) Small sample size. Only a small percentage of total plays occur in the red zone,  so the number for it contain a lot of noise; and  (2) Maybe red zone scoring really just doesn't matter as much as it seems?

Imagine you are a coach and the night before the big game the Ghost of Football Future visits you and says: "Tomorrow you will have two 3rd-and-1s, one at the other team's one-yard line and one at mid-field. One will result in a TD and one in a lost fumble -- but you can choose which will be which".

That choice is a no-brainer: you want to score from the 50 and fumble on the one leaving the other team backed up against its goal line -- not score from the one and give the other team the ball on the 50. But what does this say about any special importance of scoring in the red zone compared to scoring from elsewhere?

QB win-loss record. I just saw a major sports web site going on in big headlines about "Tom Brady is 14-4 in the playoffs, a staggering .778..." Of course he's not, the Patriots with him at QB are 14-5 now. Football is a team game, not a "punt, pass and kick" competition minus punting and kicking.

Here's a ballpark estimate of the QB's contribution to team strength: Of the 53 players on a roster say 36 significantly contribute in a game. Credit each of the 22 starters with contributing a "share" of the team's strength, and each of the other 14 a half-share, 29 shares total. Now say the QB is worth four other starters (a lot) so he gets three extra shares. The QB's four shares of 32 is 12.5%. (In contrast, sabermetricians say the pitcher is 37% of baseball.)

Or to be empirically objective, PFR.com did a study on what a starting QB is worth and found the answer to be on average 2.3 points more than a backup QB. With about 22 points scored per game on average, that's about 10.5% of team strength.  Football really is a team game.

QB come-from-behind victories. This is just a much worse version of QB won-loss record. John Elway supposedly won 47 games coming from behind in the fourth quarter.  Forget that this is truly a junk stat counted up by team PR departments, not any single source, and that QBs don't win games, teams win games. At face value, what does it mean? That John Elway actually won 47 close games net for the Broncos by coming from behind?

I counted the Broncos as being +18 over the career of John Elway in one-score (7 pt) games -- and certainly some of these weren't late come-from-behinds. How do they squeeze 47 wins into fewer than 18? By not counting late fall-from-ahead losses, for beginners.

This stat doesn't mean anything at all if it isn't given with the contextual numbers of fall-from-ahead losses, stay-behind losses, stay-ahead wins, total games, and the league norm for QB performance in these situations. How do we know Elway wasn't actually worse than average? Yet fans swoon over it -- and cheered as it was cited during Elway's Hall of Fame induction.

I'll hazard a guess that the single biggest factor in how many fourth quarter come-from-behind wins a QB tallies is how many games he plays.

Offense and defense rankings by yards. "The Packers are fifth in pass defense, giving up only 194 yards per game" . I still remember Herm Edwards' Jets putting up a tremendous defensive performance against the Oakland Raiders -- who took the ball in the first quarter and ran on every play, going the length of the field for a TD, then on getting the ball back started all over gain. The Jets' pass defense gave up zero yards and intimidated the Raiders out of attempting even a single pass in that quarter. If only they could have kept that up!

This year Buffalo went only 4-12, but its pass defense ranked even higher than the Packers': third!  Though the Bills did have the #32 ranked run defense.

Clutch performance history. During the Jets-Colts playoff game, how many times did Chris Collingsworth proclaim, "The Colts have the greatest clutch kicker of all, Adam Vinatieri, who's made more big playoff kicks than anyone and never misses when it counts"?

In the 2004 Super Bowl Vinatieri missed from 31 and 36 before "winning" the game with a clutch last-moment 41-yarder. Sure, he's hit more big playoff field goals than anyone else -- because he's played for the Patriots and Colts for 15 years and been in far more playoff games than any other kicker.

I might add that Adam's famous partner in super-clutchness, Tom Brady, who led the last-minute drive to set up Adam for that Super Bowl-winning field goal, did so a few minutes after throwing a 4th-quarter interception on the Carolina 9-yard line that the Panthers took back for a TD -- a  10-to-14 point swing. And for that matter, Joe Montana, before he led the great last-minute drive that finished in "The Catch" and cemented his reputation as maybe the greatest clucth QB ever, had thrown three interceptions. People only remember the last play.

"This [team/player]'s great record of winning close games shows..." Covered earlier at length.

Time of possession. This stat may be only semi-junk as it can have some positive significance. But the reason usually given for valuing it: "To keep the ball away from Peyton Manning [Tom Brady, Drew Brees, whomever] ..." is near always wrong, or at least mis-expressed.

Last year the Dolphins kept the ball away from Peyton Manning for a solid 45 minutes and lost. During 2006 the entire NFL pretty much succeeded at keeping the ball away from Peyton by rushing an average of 173 yards per game -- the Colts had one of the worst rushing defenses on record.

Colts coach Tony Dungy mocked the "keep the ball away from Manning" strategy saying: If they spend 40 seconds per play getting four yards and we spend 20 seconds per play getting eight yards, who's winning? Against the Colts the Giants ran for 189 yards, Jaguars for 191, Titans for 214, Broncos for 227 ... and all lost. The Jaguars in the rematch ran for 357 .. well, they won -- but that's a whole lot of rushing yards! In nine games in which the other team "won" time of possession the Colts went 6-3 as they won through to the Super Bowl.

"Winning" time of possession can be useful for keeping the defense rested and reducing number of possessions, both of which can be a good idea if the other team's offense is better than yours. And it certainly pays off if you are ahead and running out the clock.

But victory goes to the team that scores the most points per possession --  when behind on the scoreboard  playing "keep away" from the other team to run down the clock is suicide, you run the clock out on yourself.

In this sense "winning" time of possession is a strategy to use in the right circumstance, not a virtue in itself (like gaining the most turnovers). The "time of possession" fallacy is very much like the "run to win" fallacy. Running a lot to win makes perfect sense if you are already winning -- but if you are not winning and it compromises your offense, or you are behind with the clock running out on you, then it is not so good. In fact, to the extent  teams run to control the clock, one may consider the two fallacies to be the same thing.

Driving all the junk

The NFL and its partners spend millions of dollars publicizing the game ñ and these junk statistics. Why haven't they developed better ones? It costs just as much to push junk stats as good ones, so why not go with quality? Why doesn't the Pro Football Hall of Fame have a data base anywhere as good as PFR.com's? Why do independent sites like this one innovate all the good new stat analysis? Why does MLB.com publish a raft of stats out of sabermetrics while NFL.com publishes only the same junky stats it printed in 1970, unimproved?


Football is a business, and we must figure it gives its customers what they want. As an entertainment business it sells the stories, myths and dreams that its audience most enjoys, like Hollywood does. The great majority of football fans just don't care about sound stats ñ and many turn openly hostile to any that contradict the stories and myths they want to believe (and have been sold over and over). The NFL and its partners push good stories and sound stats accordingly.

The football fan base culture is different from others. If MLB and its Hall of Fame ever produced an official all-time rating list of batters with a top 20 including players of only of the last 16 years ñ Ty Cobb, Babe Ruth, Ted Williams, George Brett excluded, but the baseball equivalent of Daunte Culpepper at #9 ñ those responsible for it would be ridiculed into early retirement. Nobody is going to nominate a pitcher for Cooperstown just because he won 47 games after the fifth inning. ("Uh, how many did he lose?" "Gee, I don't know." "Go away.") But for football it works.

Baseball's fans appreciate it as a game: a round ball hitting a round bat, played every day ("Let's play two!"), a championship that's best-of-seven because one win (or two or three) doesn't prove anything. "There is no joy in Mudville, for mighty Casey has struck out", is fun for them because it's true. The Babe could look pretty foolish after a mighty swing when the catcher held up the ball -- but they know there will always be another pitch and another game. In football lore there is no corresponding "... for Mighty Montana has thrown a pick".

Football's fans see it as "war" game, controlled violence, imposing your will on the enemy, victory or defeat-and-failure right now. The ethos is Vince Lombardi's...

"I firmly believe that any man's finest hour, the greatest fulfillment of all that he holds dear, is that moment when he has worked his heart out in a good cause and lies exhausted on the field of battle - victorious."

There's no room in that to consider the effects of chance and probability inherent in sound statistical analysis.  Nobody wants to think of "the hero who has worked his heart out in a good cause, in his final and finest hour, lying exhausted on the field of battle, defeated by random chance".  It is not an acceptable thought.  So it turns out "he didn't want it enough", or something. 



On the other side, when competitors like Tom Brady and Bill Belichick win more than even a top team should they aren't fortunate, they are superheros. A column on NFL.com right now is headlined "the Belichick-Brady combo isn't Superhuman, after all". (Did anyone ever imagine the Torre-Jeter combo as "Superhuman"?)

Editors at NFL.com using the term "Superhuman" is not an accident. They are playing to their market. At the site proclaiming "Brady's .778 winning percentage" to argue that he's better than Joe Montana, amid all the whipped up "Tom is best!", "Joe forever !" comments I put a couple pointing out that Joe threw those three picks before "the catch" and Tom made some big-game errors too. They were deleted by the site administrator -- such is business, focusing the message on what sells.

Of the eight stats above, six are built on the fans' firm belief in clutch play. As long as fans believe in clutchiness,  the NFL mavens will keep producing new stats documenting it, Chris Collingsworth will keep going on about it in games, and game broadcasters' will create new TV technology outlining the red zone in red on my screen to be sure I can't miss the special importance of it.  I will never get a respite from it all.

And I'll watch every game anyway. :-)

21 comments:

Tarr said...

The passer rating link you put up there is a bit out of date and doesn't include pre-merger stats. Here's a more complete listing:

http://www.pro-football-reference.com/leaders/pass_rating_career.htm

The issues with the passer rating stat have really come to the fore in the last decade. To wit:

In 1955, when Otto Graham retired, he was the all-time leader in passer rating.

Twenty-five years later, in 1980, Graham was STILL the all-time leader in passer rating.

Twenty years after that, in 2000, Graham was THIRD in passer rating, behind only Joe Montana and Steve Young.

Today, just ten years later, Otto Graham is SIXTEENTH in career passer rating, behind greats as Daunte Culpepper and Jeff Garcia.

Anonymous said...

Great Job, great article. You just forgot the single most BS: "Defense wins Championships" as said again by Easteregg today (OK, it´s not a "Stat", but it perfectly qualifies for your post).

Thanks, Karl from Germany

James said...

Easterbrook annoys me even more than the TV analysts who spout this nonsense. I can no longer stand to read what he writes.

Ian Simcox said...

The 'defense wins championships' bit annoyed me this week because he uses the NFL's official stats based on total yards which any stats guy knows are rubbish metrics of which the 'best' teams are. On a Yards per play basis (a much better metric), the Superbowl features offenses #7 and #6, versus defenses #1 and #12 (PIT and GB respectively). Two ten offenses versus one top ten defense - hardly proof of 'defense wins championships'.

The 'smart after the event' gets me too. If Rodgers threw a TD rather than the INT to Urlacher, Easterbrook doesn't mention the play call. Of course if McCarthy knew the pass would be intercepted he would call a run.

He's been right about 4th-and-short punts and pointless field goals (coaches more concerned with not being shutout than taking risks to win). But sometimes he's just taking advantage of hindsight to try to demonstrate how much he knows.

Anonymous said...

@ James

I always asked myself why so much BS out of the mouths like Prisco, Jemele Hill, Easteregg, TV, etc. is believed by the Avg.-American-Fan.

Brian Burke should take their jobs... But what is he getting instead? Flak from all comments at the "5th-Down", only because he (rightfully) said ATL played way above their level of talent.

For Brian, the readers of this site and the smart Fan: Please, please let the Packers win the SB. Otherwise we must endure another year of "Defense wins Championships". Oh no!

Because of all that repeated mainstream nonsense; on Sundays i always have to explain Football for 3 hours instead of concentrating on the Game.

I know Americans can do better. In cycling they really understand ("our") sport. Why, oh why, don´t they understand their own Sport Football?

Cheers, Karl from Germany

Anonymous said...

Easteregg "plays" with the readers. He is always "Cherry-Picking". His mind is made up before and then he writes Stats in his column which favours his points (70´s Football, Run more, Passing is evil etc.). Last week was a perfect example: He picked the High-Scoring offenses which failed. I got some very critical e-mails against me from my american Friends. So i had to work, just looked at OTHER high scoring offenses... and boom i won the argument. But it really cost me always time and endless debating. That´s why Easteregg annoys me too.

He´s right about Punting and FG´s. But the more i read his columns, it seems he just picked up the work done by others and print it.

He should do that with Passing-Efficiency, at least once:
GB No. 3 Y/PP-Offense, No. 3 Y/PP-Defense,
PIT No. 5 Y/PP-Offense, No. 2 Y/PP-Defense.
The 2nd and 3rd best combined Passing-Teams meet in the SB (as everybody here knows, SD is No.1).

Karl, Germany

Adam said...

Great stuff. That "# of QB Comebacks" one drives me most nuts of any, and for basically all the reasons you describe. There is literally zero context to it. Number of opportunities is the chief one. Giants for whatever reason had very few close games where they trailed this year, meanwhile Jets had a chance at a late drive to go ahead just about weekly. I'd even settle for passer rating in "close and late" situations, at least its apples to apples.

Another one I saw last week was "Steelers never lost playoff game when winning by 7 in 2nd half", something like that. Left unanswered....how many times have they led playoff game in 2nd half by "x"? How often does any team blow a 2nd half lead of a TD or more? If the Steelers are 12 of 12, and any team holds the lead 90% of the team, I'm not particularly wowed.

Adam D said...

"The result was that the version that correlated most strongly with winning was ... NFL passer rating.

"Now you see why I really hate this stat. It is so misconceived it shouldn't have survived to take first breath -- yet it will not die and thrives! (More on this another time.)"

Um... maybe incompletions really are that bad? I don't understand... why do you hate the stat? Because it correlates well with winning?

Nate said...

"Please, please let the Packers win the SB. Otherwise we must endure another year of "Defense wins Championships". Oh no!"

In this case, if the Steelers win, it may be true. Using the above rankings, the Packers are 6th and the Steelers 7th in offense. That makes the Pack better offensively...but not by much at all. Meanwhile, the Steelers are 1st in defense and the Packers are 12th. That's a much bigger difference. If you have two equally capable offenses, and one team has a much better defense, you have to give the edge to the team with the better defense, no?

Even on a per-play basis, the offenses are pretty equal. Pittsburgh has a 5.6 YPP, Green Bay a 5.7. Again, Green Bay is better, but by a pretty razor-thin margin. Now, defensively, Pittsburgh gives up 4.5 YPP, Green Bay 5.1.

Assuming the Steelers win the Super Bowl...would it really be so crazy to say their defense was the deciding factor?

Anonymous said...

Nate,
yes of course, if PIT wins it´s rightfully said "Defense won the Championship". But i am tired of hearing pundits say "I told you so". Because Teams that won solely because of the strength of their Defense are the exception. But those pundits sell it as a rule.

Karl, Germany

Bigmouth said...

Excellent, excellent job, Jim! Can't wait to read more from you!

Bigmouth said...

According to both DVOA and EPA, Green Bay had the second-best defense in the NFL this year behind only Pittsburgh. I'm afraid that, no matter who wins the Super Bowl, proponents of the "defense wins championships" myth will claim validation.

Anonymous said...

Bigmouth,

i had bought two Books from the "Outsiders". They were full of errors...
And i trust Brian here; The "DVOA" seems to have its flaws. So i go by the "old school" (proven here, proven on my historical Stats), the good old Y/PP:
Offense
GB No.3 (7,1 = 15,5 % better than NFL-Avg.)
PIT NO.5 (6,9 = 11,9 % better)
Defense
GB No.3 (5,4 = 12,3 % better)
PIT No.2 (5,3 = 13,4 % better)

Playoffs so far:
GB 7,7 on Offense (24% better than NFL-Avg.)
5,6 on Defense (8% better)

PIT 5,3 on Offense (14% WORSE than NFL-Avg.)
4,4 on Defense (28% better)

Conclusion: In the Reg.-Season it was close in GB, but their Offense still outperformed their Defense. In the Playoffs they are clearly an "Offense"-Team. And we shall not forget they play most of their Games in the North (Y/PP 1 Yard less then in Domes!!). They are a heck of an Offense Team.

So go Packers...

Karl, Germany

Bigmouth said...
This comment has been removed by the author.
Bigmouth said...

Karl, I specifically mentioned EPA, which is a stat used by this site. And according to EPA, both Pittsburgh and Green Bay had more highly ranked DEFENSES than offenses, hard as that may be to believe in the case of the latter. Still, I'll be cheering for Green Bay, too!

Anonymous said...

Bigmouth,
nothing against EPA. But to be honest, EPA and WPA is real heavy stuff to understand. And as far as i know, Brian uses Y/PP in his model too. And what i see there is that GB has a slightly better Offense-Y/PP (16% above NFL-Avg.)than Defense-Y/PP (12% above NFL-Avg.). In the Playoffs 2 of 3 games were "decided" by GB´s Offense (the Y/PP-Stats show that too; 24% Offense, 8% Defense above NFL-Avg.). So in GB´s case it certainly can´t be said this is a Defense-Team (only). Especially we shall not forget that the SB-PPG historically is up by +10% compared to Regular Seasons. So we can expect a higher scoring Game than the Projections would suggest.
Conclusion: If GB wins, we certainly don´t have a "Defense-Champion". I wish they had somebody like Walsh as HC (no timid Play-Calling), i would feel better. Now we all will see how McCarty tries to run against the PIT-Wall which will keep the Game close. OMG.

Cheers, Karl

Anonymous said...

I've been surfing online more than 3 hours today, yet I never found any interesting article like yours. It is pretty worth enough for me. In my opinion, if all webmasters and bloggers made good content as you did, the net will be much more useful than ever before.
Feel free to visit my webpage ; erinmore mixture

Anonymous said...

Hey there! I've been reading your weblog for a long time now and finally got the courage to go ahead and give you a shout out from Humble Tx! Just wanted to mention keep up the excellent job!
Review my homepage ; amber leaf

Anonymous said...

Hello There. I discovered your blog the usage of msn. This is a really
neatly written article. I'll make sure to bookmark it and come back to learn extra of your useful information. Thanks for the post. I will definitely return.
Here is my web page : mac Baren

Anonymous said...

First off I want to say superb blog! I had
a quick question in which I'd like to ask if you don't mind.
I was interested to know how you center yourself and clear your head
before writing. I've had difficulty clearing my thoughts in getting my ideas out. I truly do take pleasure in writing however it just seems like the first 10 to 15 minutes tend to be lost just trying to figure out how to begin. Any suggestions or hints? Many thanks!
Here is my site :: http://www.yourtobaccosstore.com/

Anonymous said...

Thank you for any other fantastic post. Where else may anybody get that type of info
in such a perfect approach of writing? I've a presentation subsequent week, and I'm at
the look for such info.
my web site: propertyinturkeyforsale.net

Post a Comment