Saturday, January 29, 2011

Does "the law of diminishing returns" apply to passing?

by Jim Glass
Does "the law of diminishing returns" apply to passing? If so, does it mean anything for team strategy?

Does diminishing returns apply to passing? As a QB passes more, should the effectiveness of his passing be expected to decline? Or do mismatches rule and quality win out in NFL contests, so a top-quality QB's performance should be expected to be consistent during an entire game – or even rise as he finds the weaknesses in a defense and maybe even "breaks" it?

Diminishing returns isn't really a law so much as the observation that people logically put a resource to its most productive use first, then to its second most productive, and so on, so the return from its use incrementally declines. This phenomenon is all around us in modern life.

In football the logic is that teams use the pass plays they can execute best for greatest results first. But when teams throw a lot they must move beyond those few plays -- so their AYA will fall to significantly below its level in few-attempt games.  There's lots of anecdotal evidence supporting this idea.

In my file of the last 15 years of playoff games Mark Sanchez has by AYA the 7th best of 326 playoff performances, 13.5 AYA (versus Cincinnati last year) and Trent Dilfer had a very superior run of 9+ AYA through the playoffs leading up to the 2000 season Super Bowl. Sanchez had another top performance of 10+ AYA against the Patriots a couple weeks ago. These are impressive playoff numbers for QBs who rated league-bottom during the regular season - and one thing all these game had in common was few passes attempted. NFL teams pass near 34 times per game on average, but Dilfer in his playoff run averaged only 16 while Sanchez threw 15 times in his 13.5 AYA game and 25 times against the Patriots this year.

Of course, in 16 years you'd expect a few bottom-rated QBs to have some top-rated games, and the fewer attempts tried the greater is the possibility of it happening by random chance, so that could be it. But there are many other examples more difficult to explain by invoking chance.

An historic one dates to the 17-0 season of the 1972 Miami Dolphins. Before the season they paid $100 to pick up 38-year old Earl Morrall off waivers as their backup QB. Then starter Bob Griese went down and Morrall QBed the team through most of the perfect season. He was named All Pro First Team and AFC Player of the Year after compiling an 8.4 AYA, far above the league's next highest of 6.9 put up by Joe Namath. Morrall averaged only 16 attempts per game as the Dolphins ran on more than 70% of their plays ñ yet it doesn't seem likely that random chance would produce one high-number game after another for him to earn him such honors for the full season.

If diminishing returns applies to passing one would expect the QBs with the fewest pass attempts to have the highest AYAs, other things being equal. Is that what's happened here?  To judge, we have to look beyond anecdotes.

League-wide numbers over several years show a modest negative correlation between AYA and number of passes attempted. One might assume that in today's pass-friendly NFL the best pass offenses would pass the most, but the top five teams ranked by offensive AYA in 2010 (Pats, Chargers, Steelers, Packers and Bucs) by pass-run ratio ranked only 26, 17, 27, 14, and 21 (with a similar pattern is seen in reverse for the lowest-rated teams by AYA) quite consistent with diminishing returns.

But in league-wide numbers none of the other things are equal. One obvious point is that these five teams were all winners, and winning teams run more after getting ahead, so their numbers may show high AYA reducing passing rather low passing keeping AYA high. Since league-wide data are unconvincing I'll skip to the career numbers of individual quarterbacks, which are more interesting.

Quarterback career numbers
From the database I took the career game logs of all QBs who started this year and had at least a five-year record (80 games) as a starting QB with 14 or more attempts in each game (then added Palmer and Garrard who just barely missed the cut). I divided their games into quartiles by number of passes attempted, and computed their AYA for their fewest-attempts and most-attempts quartiles.

For example, Peyton Manning had 204 games, thus 51 games in each quartile. In his fewest-attempts quartile of games he averaged 24.5 attempts and had an AYA of 9.3, while his most-attempts quartile he averaged 46.4 attempts and had an AYA of 5.8, a 37% decline. The results for all...

QBFewest Attempts AYAMost Attempts AYADecline
Roethlisberger 9.30 6.59 -29%
P Manning 9.26 5.84 -37%
Rivers 8.81 6.91 -22%
Brady 8.61 7.19 -16%
Garrard 8.10 6.42 -21%
Palmer 7.93 6.44 -19%
Brees 7.81 5.95 -24%
Pennington 7.60 5.59 -26%
Brunell 7.52 6.39 -15%
Favre 7.47 5.96 -20%
McNabb 7.31 6.48 -11%
Hasselbeck 6.63 6.14 -7%
Delhomme 6.53 5.60 -14%
E Manning 6.19 6.42 4%
Kitna 5.98 4.98 -17%

The numbers show a significant general drop in AYA with increasing attempts. For perspective on their size, Peyton Manning's 9.3 AYA in his "fewest attempts" games is higher than the highest regular season AYA for any QB in 2010, while his "most attempts" AYA of 5.8 matched Colt McCoy's this year, 26th in the league.

And the low AYA number for "most attempts" games may understate how low AYA performance actually falls. Since it is an average for the entire game, to the extent that passing effectiveness decreases during the game the final passes must have lower AYA than that. In his passes number 41+, Peyton Manning's career AYA is only about 4.3.

Surprisingly, it is the best QBs who incur the biggest declines in AYA with additional attempts. Of the QBs on the table, the top eight by career AYA have a drop of 24%, the bottom eight one of only 10%. (That the poorer QBs have lower, or no, AYA decline with increasing attempts is supported by looking at other low career AYA QBs who missed the 80-game cut, such as David Carr.) 

Declining AYA with increasing attempts is also most steady with the "pre-eminent" quarterbacks, as seen in their numbers for all four quartiles.

QBLeast Attempts AYA2nd Least AYA2nd Most AYAMost Attempts AYATotal Decline
Roethlisberger 9.30 8.63 7.17 6.59 -29%
P Manning 9.26 8.09 7.62 5.84 -37%
Rivers 8.81 9.19 7.95 6.91 -22%
Brady 8.61 7.43 7.37 7.19 -16%
Brees 7.81 7.76 7.12 5.95 -24%
Favre 7.47 6.97 6.25 5.96 -20%
Average 8.54 8.01 7.25 6.41 -25%

I'm not sure what to make of this pattern. My first guess is that perhaps superior QBs, when they can pick their shots, have a special ability to exploit defenses that they lose when forced into a broader range of passing, while lesser QBs don't have that special ability so they don't lose it. But you may come up with a better idea.

Causation is always a problem when interpreting football stats ñ it can run either way, both ways at once, or in several roundabout ways. For instance...

* Teams certainly throw more when they are losing, and facing a top defense is more likely to cause them to lose ñ so maybe facing tough defenses both forces more attempts and drives AYA down. But that's not inconsistent with diminishing returns. A top defense forcing a QB away from what he does best while throwing more than he'd prefer is a fine example of diminishing returns in action. The two causations can co-exist.

* A QB may come out and throw 4 TDs in his six first passes so his team barely has to pass again. That's luck giving high-AYA, low-pass numbers. But it is also entirely consistent with diminishing returns - the team started with its best passing plays and they worked great, giving it no reason to move on to its next-best plays.

One can think up many different variations of causation, but the fact of diminishing returns with increased passing is hard to avoid in them.

Some may think about all this, "So what? QBs who get behind and have to go chucking in the 4th quarter trying to come back throw a lot of picks. What's news about this?" And that's a fair enough point, the reader can decide if all this has any worthwhile meaning. But from it I draw three conclusions that aren't generally appreciated...

* Good defense improves offensive passing efficiency. The logic is simple: If the game is always no worse than close, and the QB is confident it will stay that way ñ he doesn't have to go chucking to keep it close ñ he can always stay within the realm of what he does best and never be forced to get beyond himself. I can think of no other explanation for the Morrall-Dilfer cases, and others like them going back to Bart Starr. A good running game helps too, but you can't run if you are 17 points down.

Defense and special teams do at least half the job of keeping a team ahead or close ñ so to the extent that being ahead or close helps passing efficiency, they can boost a QB's AYA. QED. (I have other data on this, perhaps for another time.)

* Balanced teams win more games. If you accept the prior point, this must follow. You want to balance your team's investments in offense and defense to equalize their return and thus maximize their value.

Say you are lucky enough to have a superior QB and you believe, credibly, that offense is more important to winning than defense. You may decide to invest your salary cap overwhelmingly in your passing offense to make the most of your strength. That seems logical enough, but diminishing returns shows a flaw in the idea: your eighth unit of payroll invested in your O produces less than your third invested in your D, so such unbalanced investing weakens the team overall.

The result can be what I think of as "Coryell Syndrome" ñ Don Coryell coached teams with historically high-scoring offenses that never won anything because of their all-time worst defenses. (Did his offenses so dominate because he was such a great offensive coach, or because he utterly depleted his defenses to invest more in offense than anyone else?)

That bad defense is not only bad in itself, but by putting the offense in bad positions it takes back some of your QB's AYA obtained by investing so much in the offense, wasting that investment. In contrast, investing to bring the defense up from the bottom also helps place your offense in better positions ñ and so provides a bonus in helpings its AYA.

Have the Polian-Manning Colts suffered from Coryell Syndrome? With the dominant QB of a generation and an offense famously built around him they are 9-10 in the playoffs, and Peyton has lost far more playoff games in which he put up winning-level numbers than has any other QB. Even in 2006 when they won they won they were lucky ñ their defense was ranked 31st by EPA and gave up 173 yards per game rushing, and Peyton put up his worst post-season numbers, though they won anyhow.

The obvious contrast is the Belichick-Brady Patriots. Their defenses over the last 11 years has been much better than the Colts, (average -6 EPA versus +41 for the Colts) and they won the Super Bowl with defenses ranked 2nd, 6th, and 8th by EPA. Brady has won more playoff games when he put up poor numbers than any other QB. By "attempts quartile" Brady's high AYA is not as high as Peyton's ñ but his low is not as low and his drop is much less. Is all this evidence of a more balanced team?

* Not all passing offenses are equally scalable.  A team's passing AYA is a stunning 9.3 (far ahead of #2 team's 7.8) while its average yards per run is a mere 3.5, second-worst in the league. Yet the team has the highest run-to-pass ratio in the league - it runs more and passes less than any other. Is its coach a dummy who somehow doesn't understand that 9+ yards per play is better than 3.5? Should he pass a lot more?

A "yes" answer assumes that with a lot more pass attempts the great AYA will remain stable or fall only slowly.  If the team's QB is one who can make all the throws ñ a Namath, Fouts, Warner, P. Manning ñ decline in AYA due to extra passing may be modest, so that might be right. But if the team's QB has a very limited skill set ñ he's a Morrall or Dilfer ñ you may want to think twice. Remember, declining AYA comes largely from the -45 charge for interceptions, which can be game-turners.

The numbers above are for Lombardi's last great, dominant Packers team in 1966. Bart Starr, its QB, was a 17th round draft pick who was Lombardi's third choice as QB, getting the job only after the first two were injured. He kept the job when Lombardi was impressed by his intelligence, leadership and accuracy passing off of play action behind the run ñ not because of his throwing ability. The Packers also had the #1 defensive AYA, so they had a huge total AYA edge over the rest of the league.

But none of Lombardi's contemporaries thought he was leaving anything on the table by not having Starr hurl the ball like John Unitas ñ and I see no good reason to second guess them.

Concluding opinions
For what these are worth:

* Passing is generally subject to diminishing returns, one way or another, though at rates that vary greatly by team.

* The quality of a team's defense helps (or hurts) its offensive passing efficiency.

* In the NFL teams needs to pass effectively to win consistently ñ but don't need to pass at high volume if they pass very efficiently.

* If a team isn't lucky enough to have one of the few superior passers in the league, it can build an efficient passing game in other ways.

* Although offensive passing is the single most important contributor to winning, it is still possible to over-invest in it.  (Yes, I suspect the Colts have suffered a bit from Coryell Syndrome.)  Balanced teams have the advantage.


Anonymous said...

To be honest i didn´t read it all. But your point is clear. More passing = less efficiency. But you also bring the points which causes this seemingly less efficiency: Teams behind throw more AND they are normally behind against good defenses.
My opinion is: As long a OL is good enough to protect the QB, you can pass all day (NE did it 2007 vs. PIT´s No.1-Pass-D, SL did it in the 99 Play-offs, IND did it vs. DEN twice and GB once).

My suggestion:
Do Stats ONLY when the score is still close, compare Teams who usally come out throwing to built a lead to those who don´t. I think those who have like a 70/30-Pass-Ratio score more Net-Points than those who are balanced like 50/50 Pass/Run.

My point: You can´t stop perfectly thrown and protected passes. Thus you can throw endlessly in a game. The opposing defense either a.) tries to blitz alot or b.) tries to stop the deep pass.
Both are not working: a.) PIT-84 by MIA and ATL-91 by WSH were destroyed when the opposing defense had the "Master-Plan" of Blitzing alot.
b.) Prevent defense does nothing but preventing Teams from winning. That is known by everybody who watches football.
So, no passing all day long can´t be stopped if the OL/QB are superior than the opp. DEF. (as Bruces Studies show; the best Offenses are normally superior to the best defenses. That maybe the reason why SL couln´t be stopped in 99 and 01 even tough they never really tried to run)!
You just simply score more by passing than running, even tough it has a higher variance.

Karl from Germany

Tarr said...

This is a very interesting article and was a great read; thanks for posting it.

I actually find it completely unsurprising that the most drop-off in efficiency between low-attempt and high-attempt games show up in the best QBs. If a great QB has 20 attempts in a game, it probably means that his teams was doing extremely well and they only had to pass when in favorable situations. After all, if a team with a great QB is struggling, they are going to try to throw. By contrast, if a bad QB only has 20 attempts, that could simply mean the team was avoiding selecting plays where they have to use their bad QB. So the negative correlation between attempts and efficiency should naturally be greater for good QBs.

I dispute some of your "implications" points. I agree that a good defense will improve passing efficiency, but only in the sense that this allows you to more frequently avoid the "late and behind" situation where you have to take greater risks. QBs in shootouts often have extremely high efficiency. It's only the _losing_ QB in a blowout who consistently looks bad.

I don't think the "balanced teams win more games" point follows at all, even if I take the previous point as granted. Truthfully, this is simply out of the scope of your analysis. I don't mean that as a criticism of the analysis at all; it just doesn't really follow from what you've done. Improving passing efficiency across the board might be a more resource-efficient way to improve a team than improving the defense, particularly if you already have a strong QB to leverage your investments in that area. That was clearly the choice of the 2007 Patriots, and it sure worked, Super Bowl 42 notwithstanding.

I agree on the scalability point.

DSMok1 said...

I agree with Anonymous above: when playing a lesser team or having a good game, a lead is obtained and the team then runs more to protect the lead, dropping the number of attempts. That's a big-time confounding variable here.

Anonymous said...

I like the vintage Rodgers picture. That must have been taken way back while he was still wearing the small helmet.

anthony said...


I think you're hitting the correct explanation here

"A QB may come out and throw 4 TDs in his six first passes so his team barely has to pass again. That's luck giving high-AYA, low-pass numbers."

The high AYA numbers were in the 9 yard range, right? An AYA of 9 yards is killing the other teams. So I'm not surprised that you would see a combination of high AYA / low passes.

I'm sure there are some diminishing returns to passing ... if you think about how you would try and max out your AYA it would be to have a strong running game that you could use for all 'low potential gain' situations (e.g. 1st & goal at the 3) and then only use the pass to go for big plays. To the extent the offense is using the pass to replace everyday run (short passes, screens, etc) then you would expect that to bring AYA down.

Statalyzer said...

"Diminishing returns isn't really a law so much as the observation that people logically put a resource to its most productive use first, then to its second most productive, and so on, so the return from its use incrementally declines."

It's also the fact you are dealing with an intelligent opposition who adapts to what you're doing and the more you do it the better they ought to get at stopping it.

Anonymous said...

Dear Mr. Glass,
CHFF published a similar article on Aug. 2, 2009, by M.Ward entitled "QBs Who Can Carry a Team". His parameters were 1. Regular Season 2. Thirdy or more pass attempts a game 1940-77 3. Forty or more pass attempts a game 1978-2008. The winning percentage for all such games was 30.7%. Ward also listed some individual QBs who had high winning percentages going back to 1960. I expanded this back to 1950; in the process, I eliminated several QBs (Starr, Tittle, and Wade) who had poor w-l records in such games during the 1950s. What I had left was this: Only 4 old-time QBs had a WP of over 50% in such games---Lamonica 71.4%; Nelsen 63.6% (low totals); Graham 60%; Unitas 56.5%. Only 4 modern QBs had a WP of over 50%---Brady 65.9% (through 2010); Young 55.0%; White 53.9%; McNabb 51.4% (through 2010).
In post-season games (1940-77) (30 or more passes attempted), WP stands at 25.0%. From 1978-2010 (40 or more passes attempted), WP stands at 18.3%.
From what I can tell from post-season games, from 1940-77, the break even point, as far as WP, is approximately 10 pass attempts---more than 10 pass attempts and you lose more often than you win. The magic number from 1978-2010 is 15 pass attempts---which I find sort of surprising.

Post a Comment

Note: Only a member of this blog may post a comment.