Saturday, February 12, 2011

Balanced teams have higher EPA and win more games

by Jim Glass

Should NFL teams invest the most in offense or defense?

Previously, discussing diminishing returns to passing, I noted that marginal return analysis says investment resources - including NFL salary cap money - should be allocated among different investments to produce the largest return at the margin.


In plain English about football, this means that if your team has two all-pro wide receivers and zero competent linebackers, and you have a choice between adding a third all-pro receiver or an all-pro linebacker, you'd better go for the linebacker. Adding your first good linebacker will give more net points to your team than adding a third top receiver.

This is true even if you believe offense is more important to winning games than defense. Whatever their relative importance in the big scheme of things, you are best off making salary cap investments that produce equal returns at the margin, so that your next million cap dollars invested in offense produces the same net return as the next invested in defense. With diminishing returns to additional investments, if you overinvest in offense you will receive fewer net points than from making that last investment in defense – that third good receiver gives you fewer net points than a first good linebacker. Overinvesting in defense is the same mistake in reverse.

Moreover, underinvesting on one side of the ball can undercut, and so waste, part of your high investment in the other. If you invest so much in offense that you have a bad defense as a result, the bad positions it places your offense in will hurt the offense and offset some of the high investment in it, wasting it. If you invest so much in defense that your offense is very poor, it will likewise handicap the defense and waste some of your investment in it. Either way, your team will be worse off than if it invested on both sides of the ball evenly.

Thus says the logic of marginal return analysis, right out of any economics textbook. But is it actually true for football?

Test
In terms of EPA, there is a simple test to make. Marginal analysis says that teams with equal-sized offensive and defensive EPA numbers should have higher total EPA than teams that have lopsided high-low offensive-defensive (or the reverse) EPA numbers. And their higher total EPA should result in a better won-lost record.

To see if this is so, I added team EPA numbers for 2001 to 2010 to a spreadsheet with W-L, points for/against data, etc., for all teams over the same period. For each team it computed offensive EPA compared to the average for the ten years, defensive EPA compared to the average, and total EPA compared to average (adding defensive EPA better than average, a negative number, as a positive one).

I then took the top 120 teams by the total EPA above average number. (Analysis is limited to this group for several reasons, the simplest being that the effect sought stands out most clearly at the top. Also, the 120 teams should be mostly the same 12 teams per year that make the playoffs, and people are generally most interested in what brings success at the top.) For each team, I computed an "EPA balance ratio" – the percentage of total EPA above average consisting of the larger of offensive or defensive EPA above average.

For instance, if a team's offensive and defensive EPA above average are exactly the same the ratio is .500, if its offensive EPA above average is two-thirds of its total EPA above average the ratio is .667, if its defensive EPA is 1.2 times its total EPA above average (its offensive EPA is below average) the ratio is 1.200, and so on.

Results
Here are the numbers for each of: total EPA over average, EPA balance ratio, and W-L pct (by both actual W-L and Pythagorean expectation) and how each relates to the others. (Pythagorean expectation is given because it is notably more accurate than actual w-l in predicting future w-l, so it may be considered a better indicator of true team strength, but the difference is minor in these numbers).

In each table the teams are ranked by a measure and grouped in thirds (top 40, mid 40, and low 40) with the columns showing the average numbers for each group.

               TOTAL EPA>AVERAGE









TeamsEPA>AVEW - LPythagoreanBalance
top 40 158 0.759 0.747 0.827
mid 40 99 0.666 0.676 1.021
low 40 57 0.607 0.603 1.333




               BALANCE RATIO








 BalanceEPA > AVEW - LPythagorean
top 40 0.632 121 0.691 0.696
mid 40 0.938 114 0.698 0.685
low 40 1.599 80 0.637 0.645




               PYTHAGOREAN EXPECTATION








 PythagoreanW - LEPA > AVEBalance
top 40 0.768 0.770 148 0.878
mid 40 0.679 0.673 99 0.929
low 40 0.579 0.588 67 1.375



Every way one looks at things, greater balance between offensive and defensive EPA lines up with higher total EPA and higher W-L percentage.

Here's how the relationship between total EPA and EPA balance looks graphically, with a trend line:

The teams with the most total EPA are spread out low to the right.

Conclusions
In the endless argument about what is more important to winning games, offense or defense, the answer in my mind is: balance.

The only 16-0 team ever, the 2007 Patriots, are famous as arguably the greatest offensive team ever. Yet even that team was more balanced than most. It is the rightmost dot on the chart, and as the eye sees, its .820 balance ratio is well below average. The fact is that of these 120 "best" teams, 54 have a balance ratio over 1.0, meaning they were below league-average strength on one side of the ball or the other. There are a lot of very unbalanced teams out there.

I won't go into what "wins championships". Basically, the sample size of playoff games is too small and the knockout format of the playoffs too artificial for the data here to prove anything about that, I think. (Although this year's Super Bowl teams, the Packers and Steelers, had balance ratios of .524 and .573.)

But my general opinion is that what wins in the regular season wins in the playoffs as well: balance helps and imbalance doesn't. Overinvestment on either side of the ball in the belief that either offense or defense wins championships is counter-productive.

A bit more evidence appeared lately indicating that the Polian-Peyton Manning Colts may have suffered from what I think of as "Coryell Syndrome"- serious overinvestment in a top offense that wins less than expected because the price of it is a too bad defense. In an offensive era with the best QB of a generation and an offense famously built around him, the team has gone only 9-10 in the playoffs.

PFR.com just produced historical playoff drive data saying that in the playoffs Peyton has been handicapped by getting the ball in the worst field position suffered by any QB in history (matched only by Warren Moon) ... and this year Peyton was eliminated from the playoffs for the fourth time while putting up solidly winning-quality QB numbers - in the last 15 years, no other QB has been unfortunate enough to lose even two such games.

In their last nine playoff seasons the Colts have averaged offensive EPA 135 points above average and defensive EPA 10 points below average. Would they have won more of those playoff games if they'd traded off some offense for a better defense that let Peyton get the ball in better field position?

I'll never know ... but have my opinion.

5 comments:

Bryan said...

Nice work, Jim.

What happens if we do this for the bottom 120 teams in EPA compared to the average? I suspect it will look as if balance is also what makes teams terrible. If you were to extend your graph to the left to include the bottom 120, the lowest, most left-hand dot will represent a team that was equally awful in both offense and defense EPA - like Arizona or even Seattle from 2010. I could be wrong though...but I am picturing it to be something of a mirror image of the graph you've created, perhaps slightly more scattered.

If that is what the full graph would look like, it presents a nice visual depicition of your argument that there are diminishing returns when investing on one side of the ball. Picture the bottom left dot - a balanced team, albeit a terrible team, obviously looking to improve - how should they spend their money? If they only spend money on one side of the ball, their data point inches to the right, but also upward as their team becomes less balanced. A team that invests on both sides of the ball equally moves to the right without taking the long path of the parabola. Investing on both sides = quicker path to higher EPA.

Jim Glass said...

Bryan, your sense of things is right on the money.

I am picturing it to be something of a mirror image of the graph you've created, perhaps slightly more scattered

Yup. The truly terrible teams are centered near .500 because they balance having nothing on either side of the ball. But as soon as they get something on one side they kick away, and since the ratio of a little something to near nothing can be pretty high, the dispersion quickly gets wider at the low end than the top.

At the top the very best teams that have made a lot of successful investment are pretty much driven towards .500. Since it is really hard to get an EPA much over 140 points above average on offense or 120 points on defense, a team with total around 200 EPA over average is pretty well forced to be good on both sides of the ball

The top 15 teams by EPA over the ten years all had balance ratios under 1.00 -- were all above-average on both sides of the ball -- with an average .720 ratio. That's pretty good evidence that if the goal is to be "best" then investing on both sides for balance needs to be done.

Investing on both sides = quicker path to higher EPA.

In principle, yes. Of course the problem in the NFL is that teams often have to invest big chunks of money at a time for very uncertain returns (Peyton Manning to Ryan Leaf). So starting from nothing it may be impossible to avoid becoming unbalanced. But if that happens, teams should have a goal of building back towards balance.

I think the key thing is simply the fact that the data indicate "diminishing returns" does apply here and leads to these results, so building for balance should be a strategic goal. Logically that does not have to be so, it is an empirical question.

There is nothing illogical about believing that if a team has a top QB it should invest everything to make the most of him (or if it has a couple all-pro D players it should invest everything in "a D to win championships") *if* there are increasing returns to investment around star talent, or if there is a floor level of play that an offensive or defensive unit won't fall below when neglected so one can increase the strength of the other unit at no cost.

But the facts I have been looking at indicate these things aren't so. In which case it is mistake to for a team to fixate on building on a strength instead of building for balance.

BTW, I've sort of cricized Polian a couple times for over-investing in the Colts offense around Manning, leading to a lack of balance that has cost them in the playoffs. But when I looked at the Colts' payroll I was surprised at how much they've invested on the D side, a lot more than I thought. It may just be that their investments on the D side haven't paid off. To the extent that's true, I apologize to Polian to the extent appropriate, should he be reading this site. :-)

Andy said...

This was an awesome article, i think the graph is sweet and demonstrates your point really well. I admit im still skeptical, still wanting to believe off > def, but I had a few questions,

1. I am still unclear why only the 120 top teams were selected. Why was that?

2. Can you graph the winning percentage (of top 120 or all teams) vs. balance ratio? Is there a trend there?

3. It seems that all i can really draw from this is that if a team was unfortunate enough to earn all their points that season (or point advantage...expected points) on one side of the ball, they probably didn't score that many points overall. Your idea of diminishing returns makes sense (i read your other article), but as of right now i am not sure that you can assume a high EPA mismatch is indicative of a high investment mismatch.
Ill give you that it makes sense that they should correlate, but im not sure how strong that would be. Maybe sheer luck starts to dominate that stat?

4. Like in comment 2, what would happen if you plotted the off/def money spent ratio vs. winning percentage? That to me, seems like a trend that measures what you say you are trying to measure. Am I missing something?

Thanks
Andy

Bryan said...

The fact that balance is important doesn't necessarily indicate that off > def is not true.

Let's say you have 10 units to spend combined on offense and defense. Jim's analysis seems to indicate that a 5-5 split is better than a 10-0 split. This assumes, as you pointed out, that spending money (the 10 units) is the equivalent of adding talent and increasing quality, which may not always be the case.

But just because balance is important doesn't mean offense isn't more important than defense. What if the best attainable balance was 6-4? Just because balance in general is better doesn't mean a 6-4 distribution is equally good regardless of whether more money is invested on offense or defense. Based on the analysis above, we don't know if 6-4 in favor of offense or defense is better. My guess would be offense, but we would need to check the numbers.

Jim Glass said...

"I admit im still skeptical, still wanting to believe off > def"

Andy, thanks for the kind words. The marginal analysis is about how to invest resources to get most net EPA points -- it doesn't say anything about whether O or D is more important to winning games. Bryan is right that the result regarding balance doesn't necessarily mean off > def is not true. A numeric example may help make things clear.

Assume that O is in fact more important to winning than D, and say investing in offense produces 25% more net points than investing in defense, but with diminishing returns so sequential investments of $X of the salary cap in O and D produce these net point returns...

O: 50 pts, 45, 40, 35, 30, 25, 20, 15, 10 ... 1.25, 1.25, etc.

D: 40 pts, 36, 32, 28, 24, 20, 16, 12, 8 ... 1, 1, etc.

When building a team, the GM of team A, knowing O is more important than D, invests entirely in O. OTOH the GM of team B invests to get the most points from each $X. Both get the best possible investment return using their strategies (no bad signings or wasted cap space). After investing $7X...

[] The GM of A has added 245 net points (50, 45 ... 20) of team strength, 100% in O. Balance ratio well > 1.00

[] The GM of B has invested to get O 50 pts, O 45 pts, O 40 pts, D 40 pts, D 36 pts, O 35 pts, D 32 pts = 278 points. That's 33 more than Team A, and 39% of total investment is in D. After the first two investments on O (for 50 and 45 points) further investments are divided near equally between O and D, driving total investment towards a .500 balance ratio.

Morever, Team A's defense is so bad that its O is handicapped by getting the ball with worse than average field position, having to come from behind, etc. Say that costs its O 20 points. Now with the same $7X investment Team B is better by 53 points.

Team A becomes today's version of the Coryell Chargers, best O ever, worst D ever, its D offsetting part of the value of its O. As O is more important than D it has a net winning record, but not so much as to ever win anything in the playoffs.

Team B becomes a solid contender. Superior O, above average D, O not handicapped by its D (if anything getting a small boost from it), objectively stronger by >53 points, a playoff favorite.

I am still unclear why only the 120 top teams were selected

For clarity and ease of analysis and presenting the result. All teams have the same salary cap, so we can define the top teams with highest total EPA as being the most efficient at investing their cap to get EPA. Among those, the top 15 teams by EPA all are in the "balanced" group (balance ratio under 1.0) so one can conclude the balanced teams are the most efficient investors of all -- they have more EPA than anyone else.

In the middle of the pack this doesn't work. All teams are inefficient investors to varying degrees, comparing total EPA one might be +10 balanced +5 each on O and D, another +10 while +80 on D and -10 on O, or whatever. Comparing them tells nothing about the relative efficiency of their investing. One would have to look at dollars invested on O and D and total EPA, adjust for the difference in cap dollars, maybe dead cap space, etc ... a lot more work plowing through a lot more noise, for dubious result. I prefer simple and easy. :-)

Whether O or D is more important to winning is a different issue. I have my opinion, but for another time.

Post a Comment