Saturday, December 11, 2010

Do Good Teams Win Close Games? Part 2: Overtime Games

by Jim Glass

My obsession, er, interest in the role that luck plays in close games got a sudden endorphin boost a couple weeks ago when an "OT game" button appeared on PFR.com's Game Finder utility. Data on all the sudden death OT games ever played was sitting just mouse clicks away. Joy!

No game is closer than a tie going into sudden-death overtime, so how do strong teams perform against weak teams in OT games?

THE TEST

1) Take all the overtime games played during the last ten years, 2000-2009, 155 of them.

2) For each team in each game, note its Simple Rating System point number as per PFR.com. This is just the team's average "points for" minus "points against" difference per game, adjusted by strength of schedule.

3) Use "Pythagorean expectation" to determine the expected outcome of full games between the contesting teams as indicated by each team's SRS rating, and compare it to the actual outcomes between them in OT games.

(Pythagorean expectation is a method of using points for/against ratio to estimate probable future won-lost record. First devised by Bill James for baseball, it is now widely applied to other sports. It is notably more accurate than past W-L performance at predicting future W-L performance.)

The idea that the outcome of overtime games is highly influenced by luck is hardly new. The goal here is to quantify by how much it is so. By comparing the results of teams of different strengths in overtime, and comparing them to the results of the same teams in regular games, in principle it is possible to estimate how much the results produced in overtime reflect team strength and luck respectively.

For instance, if the Pythagorean projection winning percentage for the stronger team in regular games is 70%, the random chance percentage is 50%, and the actual winning percentage in OT is 55%, we might conclude that OT outcomes are 75% chance and 25% the result of the better team's superiority.

RESULTS

In the 155 overtime games...

* The stronger team won 76 (49%).

* The weaker team won 79 (51%).

Well, that pretty much seems like 'nuff said. Everything else can you can deduce for yourself from that. You can stop reading now, if you want.

But since I'd set up a spreadsheet to produce a bunch more of parsed numbers, I'll give them for any who are interested.

In all 155 games total, the stronger team was on average 5.8 points better than the weaker. The Pythagorean projection for this strength difference is 66% wins for the stronger team, 102-53. The actual result of 76-79 was 49%.

Here are the results by relative strength of the contesting teams:

[Given are the favorite by SRS points (stronger team minus weaker), W-L, the average SRS point advantage for the stronger teams in the group, and the Pythagorean expectation win percentage for that point differential.]

favored by : W-L : win % : average SRS difference : Pyth expectation

0 – 2.9 pts: 22-33, .400, 1.49 pts, 54%

3 – 5.9 pts: 18-15, .545, 4.27 pts, 62%

6 – 8.9 pts: 17-17, .500, 7.49 pts, 70%

9 – 11.9 pts: 8-10, .444, 10.57 pts, 77%

12 – 14.9 pts: 5-1, .833, 13.10 pts, 82%

15 – 17.9 pts: 5-2, .714, 15.70 pts, 86%

18+ pts: 1-1, .500, 18.5 pts, 90%

TOTAL: 76-79, .490, 5.8 pts, 66%

Teams that were better by up to 12 points per game in full-length regular season games went only 65-75, 46%, against their weaker opponents in overtime.

These results for the last ten years provide no support whatsoever for the idea that the best teams win overtime games. If we accept that there's no reason for the better teams to lose systematically in overtime, the evidence here is consistent with the idea that these games are determined entirely by random chance, the equivalent of coin flips.

At an over-12 point differential the 11-4 record of the better teams might support the notion that teams that are much better than their opponents have a better chance to win in overtime – but that's a very small sample, not inconsistent with the coin-flip idea either. (For the record, an over 12-pt edge is an over-80% Pythagorean win expectation, and only about 10% of OT games are in that category.)

HOME FIELD ADVANTAGE

Originally I didn't bother looking at home field advantage because...

(1) Whether the home team is the stronger or weaker team seemed likely to balance out by chance, as it pretty much does, and so shouldn't much affect the numbers above, and

(2) Research at this site and elsewhere shows that home field advantage is strongest at the start of the game and fades later in the game – and you can't get later than overtime.

But what the heck. Of the 155 games the home team won 86, or 55%. That's the equivalent of being better by 1.8 points, as per Pythagorean expectation. In addition, as it happened, the home teams as a group were weaker by an average 1.2 SRS points than the visiting teams. So the home teams outperformed expectations by the equivalent of 3 points -- which often is cited as the number for "home field advantage".

Make of that what you will. I find it odd that home field advantage would reassert itself at the very end of the game in overtime while the real strength difference between teams disappears, but perhaps it is so. Or perhaps as 155 isn't a huge sample size this 55% number is just random variation.

CONCLUSION

I'm rather surprised to see zero reflection of team strength in overtime results (at least up to 12 points of differential).

It's logical to think that while chance factors may have a majority effect in determining overtime game results, yet still, the stronger team should score more frequently than the weaker team – and the team that scores more frequently should have an edge in scoring first to win in sudden death. Moreover, this logic is backed up in data from other sports. In baseball's semi-sudden death extra-inning games outcomes are determined mostly by chance, but the better team does have a slight edge which increases as the gap in quality level between teams grows.

Possibly this is the true state of affairs in NFL football too, but the slight advantage of the better team is invisible because of the curse of the NFL statistical analysis: small sample size. In baseball there are many thousands of extra-inning games to examine, in the NFL there are only about 15 OT games a year. So it's not possible to get more than a rough picture.

Anyhow, as a practical matter, as there is no sign of the better team having any edge in winning in overtime that's visible at all in a full decade's worth of games, one can conclude that overtime games between teams of different strength are effectively determined entirely by chance.

As to home field advantage helping a team win in OT, I'm agnostic, you can make up your own mind. But I wouldn't count on it.

For the larger picture I take this as more evidence that close games are determined overwhelmingly by luck. And, thus, that the formula for a championship team is to win games by a lot, not lose games by a lot, and be lucky enough in the close ones. Teams should be built and coached to play to dominate from the start, taking whatever calculated risks are necessary. The alternative strategy of playing safe to stay close and "have a chance to win at the end" with superior execution is a loser. It sacrifices chances to win big and leaves the team depending on luck in close games anyhow – only now in more of them.

5 comments:

Bruce D. said...

Double dose of "luck"?

There's a totally random coin flip to determine who gets the ball first, then the NFL randomness luck is added.

Maybe you've already done this, but I wonder how it would look as fav vs dog broken down by who gets the coin flip.

dom said...

Jim,check out this community post from last year.It concludes that better teams do beat their lesser rivals more often in OT games.

http://community.advancednflstats.com/2009/10/game-of-two-halves.html

I've just checked all OT games from 2000 onwards and the better pre game team,as measured by the Vegas line won OT games 57% of the time.

Jim Glass said...

dom, thanks for the comment, I'll check it out. Until then, three points...

1) Different samples. I thought of going back all the way to 1974 when OT was instituted, but as this is practical analysis I stuck to ten years (considering for instance the way the rules of both play and building teams change over time). That is, if one expects to see something this Sunday it should be visible within the last 10 years.

I didn't say the better team has *no* advantage in OT games, but that any advantage being invisible during the last 10 years, OT games can be deemed "*effectively* determined entirely by chance".

As I noted, the best team should have an advantage, in principle. For the record, the theoretical advantage is given by a "Pythagorean" using an exponent of 1.18, as per here.

2) Different methodologies. Point spreads set before games should have a significantly larger error margin than those calculated after all games are played using full information for the entire season. E.g., point spreads would be calculated after only 2 games, 8 games, etc., and reflect possible systematic errors by wagerers, especially early in the season, while the SRS numbers I used are calculated post-season. There's a lot more uncertainty in ex-ante than ex-post data.

3) Different purpose. I actually believe that the better team does have an advantage in close games -- if I just wanted to confirm theory I wouldn't bother with any of this, I've no need to re-invent the wheel. But the key issue is perspective, and practical consequences. Here's what I mean:

In baseball with its vast data base the better team's edge in close games is well documented, for instance.

Yet still, it is so slight in applied terms that close-game records diverge so far from other-game records (1935 Yankess, 15-29 in one-run games, 74-31 in others; 1974 Padres 31-16, 29-86 in others), and teams with over-50% wins in them regress so consistently (and the reverse) it's caused Bill James to say that for all practical purposes -- game strategy, judging team results, designing a team, wagering with your local bookie -- record in one-run games should be considered determined entirely by luck. There's nothing practical you can do based on it at all, relying on it will only mislead.

Similarly, the NFL coach with the best record all-time in one-score games (60+ games total) is Vince Tobin, 15-5 versus 13-38 in other games. Dick Jauron got a new contract, then a new job for being Coach of the Year in 2001 for going 8-0 in one-score games -- his other nine seasons were all losers. While the Walsh 49ers were 42% in one-score games, the Lombardi Packers were 50% ... and the best teams at winning close games during the past 15 years went only 8-9 in the playoffs, while close-game losers went 78% and won three Super Bowls (as noted earlier).

So my gist is, when zero evidence of advantage for better teams in close games is visible in the combined careers of Walsh and Lombardi (and Tobin), 10 years of OT games and all the rest, then for all practical purposes -- game strategy, team design, predicting performance, betting -- such close games should be considered effectively determined entirely by chance.

Whatever slight advantage may exist as per theory is so tiny nobody can do anything with it at all, so it effectively doesn't exist. Perhaps that's a point of philosophy. :-)

One person's opinion, your mileage may vary.

BTW, you do give rise to a question I haven't seen answered yet: the relative accuracy of point spread predictions v Pythagorean calculations after the fact. That goes on the "to do" list.

Brian Burke said...

Great post. And it has practical significance too. Whatever the true answer really is, knowing what shot your team has in OT would make the difference in coaching decisions at the end of close games. For example,
kick the FG to tie/go for the TD to win
kick the XP to tie/go for 2pt conv to win

dom said...

Jim,re home field advantage.
If you've got a very slightly superior pythag team on the road against a very slightly inferior pythag team.Which team are you counting as the better team?

Post a Comment

Note: Only a member of this blog may post a comment.