Monday, January 17, 2011

Belichick’s Overlooked Blunder


by Bryan Davies

Amidst the expletives and tongue-in-cheek comments thrown around by Jets and Patriots this past week (ok, mostly Jets) was an actual game to be played on Sunday. And, like any football game, there are a number of decisions that coaches are given the opportunity to screw up. With the underdog Jets ahead for nearly the entire game, it seemed that a lot of the tougher decisions needed to be made by Belichick and the Patriots. A fake punt on 4th and 4 from their own 38 with time winding down in the first half, a two-point conversion in the third quarter, a 4th and 13 rather than a 52-yard field goal. Well, we know which ones turned out to be good decisions, but were they objectively good decisions without the benefit of hindsight?
Well the truth is, I’m not going to address any of those situations. I’m sure you’ll find enough analysis, both good and bad, that will address them extensively. In the spirit of being an overly pedantic statistically-minded football fan, I thought I’d address Belichick’s incorrect decision to go for an extra point rather than a two-point conversion following the Brady-Branch touchdown with less than 30 seconds to go in the game. In fact, when down 14 points, it’s almost always a better decision to go for two following a touchdownlate in the game.

To be fair, the following analysis is hardly original. It’s been discussed at length at many sites and in many forums, but like most sports analytics, still hasn’t received enough attention. Here’s an excellent (and more detailed) article with the analysis: http://www.thesportjournal.org/article/playing-percentages-when-trailing-two-touchdowns. And here’s a more detailed analysis describing slightly different circumstances: http://www.footballcommentary.com/coltcomeback.pdf. Heck, even Brian mentioned it in the WP graph thread of the NYJ-NE game.

So here’s a typical scenario: You’re down 14 points with 5 minutes left. You score a touchdown. Awesome. Now you’re down 8. Obviously, you should just kick the extra point and hope to get the ball back down 7, right? Let’s see.
(Note: the analysis is conditional on you scoring another touchdown, and your opponent not scoring any points. Regardless of what actually happens, the analysis is still valid.)

We have two choices following the first touchdown: Kick an extra point, or go for two. The second attempt is conditional on the result of the first, but the first attempt should (almost) always be a two-point conversion, as I will demonstrate.
If we use our baseline 98.5% conversion rate for extra points, and 47.9% conversion rate for (normal) two-point conversion attempts (discussed here: http://www.advancednflstats.com/2010/12/almost-always-go-for-2-point.html), we can figure out which combination gives us our best chance of winning.

Scenario 1: Kick the extra point following the first touchdown.
(.985)*(.985) = .970225

About 97% of the time, we’ll force overtime with two successful kicks. But, we can also force overtime with a missed kick and a successful two-point conversion:
(.015)*(.479) = .007185

This leaves us with a total probability of forcing overtime at .97741. Since overtime is a 50/50 proposition, we have a .488705 probability of winning, or just under 49%

Of course, you can always go for two following the successful extra point, but this only leaves you with a 47.9% chance of winning (insert complaint that the Panthers actually have a 10% chance and the Patriots have a 99% chance).
And of course, we can always miss the second extra-point after making the first, resulting in a loss.

Scenario 2: Attempt a two-point conversion following the first touchdown.

Again, the second attempt will be conditional on the first. If we make the two-point conversion, all we need is an extra point to win. If we miss the two point conversion, we need to go for another one just to tie.
Make the two-point conversion, make extra point: (.479)*(.985) = .471815, win in regulation
Make the two-point conversion, miss extra point (.479)*(.015) = .007185, force overtime
Miss two-point, Make two-point: (.521)*(.479) = .249559, force overtime
Miss two-point, Miss two-point: (.521)*(.521) = .271441, lose in regulation

And now it all starts to come into focus. If we split the overtime propositions in half, we get a total win percentage of (.471815 + .0035925 + .1247795) = .600187, or about 60% of the time, we’ll win. The other 40%, we lose.

Conclusion: By simply attempting the two-point conversion when down 8 points late in a game (and only expecting 1 more possession), a team increases their probability of winning by over 11%...as long as you score one more touchdown and your opponent doesn’t score. But even if you don’t score, and your opponent does, or any possible combination you can think of, you’ve still made the right decision (again, as long as it’s late in the game).

So why don’t coaches do this? Well, for obvious reasons. Most probably don’t know. And if they do know, it’s not surprising risk-aversion dominates decision making. After all, by kicking extra points, you’ll only lose outright in regulation a mere 2 percent of the time. If you attempt the two-point conversion first, you lose in regulation a whopping 27.1% of the time. That means that over a quarter of the time, you’ll most likely be fired unless your job is as secure as Belichick’s.
Getting back to Belichick, even he is too risk averse (or ignorant) to follow the strategy. And considering how prolific the 2010 Pats offense has been, I imagine their two-point conversion success rate would be even higher than league average, making it an even better proposition. So after Branch came down with the touchdown to make it an 8 point game, Belichickshould have gone for two. Strange, I know. It turned out to be a moot point though as the Patriots failed to get the ball back, let alone score another touchdown.

A quick note on why this strategy works: Similar to Brian’s article on why underdogs should take more chances (and increase variance and thus increase WP) against the better teams, teams that are current underdogs should also take more chances late in the game. What are you really doing by going for two after the touchdown to go down 8? You’re increasing the chance that you lose by 2 (two missed two-point conversions)…for an increased chance that you can win by 1. Variance is the friend of the underdog. Sacrifice the increased chance for a blowout for the increased chance of squeaking out the win.

17 comments:

Tarr said...

An important point that doesn't get discussed enough. I like this presentation of the argument, too.

A classic example of this strategy being screwed up a different way was the 1984 Orange Bowl. This was back when there wasn't OT in the bowl games, so a tie at the end of regulation was a tie. Nebraska was down two scores, scored the first, kicked the extra point, then went for two on the SECOND score. People often discuss whether it was worth it to go for 2, as this would have clinched the perfect season and the national title, but people rarely note that, if you plan to go for 2 there, you should go for 2 earlier as well.

James said...

I have a counter point - if the Pats are the favorites (or if nothing else believe themselves to be the favorites), can't they also claim they have a higher than 50% chance of winning in overtime? I'm not quite sure to what degree their advantage affects the small number of overtime drives, but it seems to me this would offset at least some of the benefit.

This strategy seems to help an underdog the most, who as an underdog has a lesser chance of winning in overtime and therefore is benefited more by the wide variance of a single two-point conversion in comparison with potentially multiple drives in overtime.

Bryan said...

Very true that we should expect the Patriots (or any favorite) to have a better chance of winning in overtime. However, it's partially mitigated by the variance in playing a shortened game. An underdog has a better chance of winning an overtime than winning a 4-quarter game against the same team - meaning if the Patriots had a 75% chance of beating the Jets before the game started, they will have considerably less than a 75% chance going into overtime. I also imagine the new overtime rules have made it slightly harder for underdogs to win in overtime now than with the previous rules (the new rules should tend to lengthen the game).

Like any analysis, its implementation should depend on case-specifics. I tried to avoid using absolutes in the article ("it’s almost always a better decision") because, as you point out, the analysis changes for a team like New England.

But remember, even though they have a better than 50/50 prospect of winning in overtime, it's hard to argue they don't also have a better chance of converting more than 47.9% of their two-point conversions with such a prolific offense. Their chances of winning outright in regulation will go up, and their chances of losing outright in regulation will go down.

If you have an average offense that theoretically converts 47.9% of two-point conversions, you have to believe you have above a 60% chance of winning the overtime to justify kicking. If you believe you have an above 60% chance in overtime, chances are you have a good offense and can actually convert more than 47.9%, thus increasing the 60% from the analysis.

Tarr said...

While I completely agree that the Pats were the favorite at the beginning of that game, we're talking about a situation where you're down 8 late in the fourth quarter. It's very difficult to argue that you're a large favorite in OT given the dynamics of that particular game at that point.

I suppose if the scoring had been really flukey, you could give the Pats some extra credit, but the general consensus is that they were outplayed in the game. If anything, I think the particular way that game played out (Jets missing a field goal and getting into opposing territory multiple times without getting points) points toward even greater Jets dominance.

Bryan said...

I hesitate to say the Jets would be favored in overtime, or even 50/50...while it's possible that the dynamics/matchups that we saw develop throughout the game favored the Jets moving forward, it's also possible that we just happened to see one of the 29 times out of 100 that the Jets would win this game (using the probabilities generated by Brian's model...use whatever probability you'd like, but I'm guessing no one is putting it over 50%).

The Patriots were definitely outplayed, but with what confidence can we say they would continue to outplay them in overtime? The Pats even outscored the Jets in the second half 18-14.

There's also the argument that the Patriots typical strategy of quick slants and dump-off passes in the backfield is typically ineffective when losing, because it takes too much time off the clock and doesn't gain yards quick enough. I don't have the numbers, but it seemed that Brady struggled to throw the ball downfield while trying to play catch-up. Time ceases to be an issue in overtime, and the Patriots could have reverted back to their more effective and efficient offense.

denis said...

James said...

" I have a counter point - if the Pats are the favorites, can't they also claim they have a higher than 50% chance of winning in overtime? "

That's a very good point,James.
Pregame favs do do better in OT and their win percentage correlates to their pregame chances.

check out http://community.advancednflstats.com/2010/12/overtime-revisited.html

Conventional OT would have likely seen the Pats in excess of 60% favs and the extended OT version in use for the playoffs may favour the favs even more.

Nice post,Bryan.

D

Jim Glass said...

Here's the theoretical formula for predicting advantage in overtime. The better team has an advantage but you wouldn't want to give up a whole lot to get it. By this formula, assuming the Pats were playing like the best team as advertised, they had a little over a 58% win probability (considering the new rules) in overtime.

But that's a big assumption. As others have noted, before counting on this probability you have to ask, if you are so much better why are you losing? If you've been playing better than the other guy and had some unlucky fluke plays go against you that's one thing. If you are supposed to be better because of your big passing game but by all evidence the other team has solved it while parking your QB on his butt 10 times, that's something else. Nothing is simple. :-)

Kos said...

The only problem I have with this is that it assumes the second TD comes at 0:00. Let's assume you're playing against an average NFL coach, and your second TD comes with 0:40 left. If you kick both XPs and tie the game, the other coach will probably just kneel the ball and go to OT. However, if you convert on the first 2PC, then kick an XP to go up by 1, the opposing coach is now forced to try to get into FG range for the win. In essence, you're forcing him into a position where he HAS TO play for the win instead of doing what most coaches do: playing to not lose (kneeling and going to OT). He ends up playing more optimally than he would in a situation where he's not forced to be aggressive.

Now, maybe this is offset by the times you miss both 2PCs and have time to attempt an onside kick and a FG to win. Maybe that even makes it MORE valuable to go for 2...I'm honestly not sure. However, I just don't think it's as cut-and-dry as you're making it. We can't know that the second TD will come on the last play of the game, so there are other factors that come into play when using this strategy.

MIKE M said...

The problem I have with these mathmatical formula's is they "ASSUME" and "WRONGLY SO" that each team will play at it's average performance situation after situation after situation and game after game after game.

In the real world it "DOES NOT" work that way.

Look at any teams and any stat and you'll find they vary quite a bit situation after situation and game after game.

If a team is playing well above their averages this game that changes the mathmaticl formula quite a bit for this game, and vise vesa if their playing well below their averages.

Bryan said...

Kos - very good point. I was pretty vague and nondescript when discussing the time at which this strategy should be employed. It does assume the last touchdown is scored with 0:00. Scoring with no time is better than scoring with 1 second left, which is better than scoring with 2 seconds left, etc. At some point, we reach a time value in which it doesn't make sense to employ the strategy.

If we use your example of scoring with :40 left (and taking a 1 point lead), it lowers our intitial value of .471815 probability of winning in overtime. A team down 1 point with the ball on their 29 yard line on 1st and 10 with :40 seconds left has a .17 probability of winning according to the WP claculator (however, timeouts will surely be an important case specific variable). Multiplying .471815 by (1-.17), we should actually only expect to win outright in regulation .3916 of the time (assuming the opposition scores with no time left...), and still expect to win about .125 times in overtime, for a total win percentage of almost 52% - still above the 49% if you were to just kick extra points.

Furthermore, we'd have to dock the 49% as well, since we shouldn't expect the touchdown/ep combo that ties the game to be scored with 0:00 left. According to the WP calculator, the team that has the ball on their own 29 with :40 left with 1st and 10 in a tie game wins 56% of the time. So if we multiply the odds of forcing overtime by kicking EPs (.97741) by the odds of winning with the given situation (.44 rather than .50), we are actually reduced to a 43% chance of winning from 49%. So instead of comparing 60% to 49%, we are comparing 52% to 43% for :40 remaining in the game.

So yes, while the analysis does assume the touchdown is scored with no time remaining, the winning percentages for both strategies decreases, and not only in cases when you are winning (i.e. using the two-point conversion strategy). Like you said, we would expect more conservative play from coaches with <1 minute remaining in a tie game, but the fact is they still win 56% of the time with just :40 left...so it's not necessarily a "take a knee and force overtime" situation.

Bryan said...

And to bring it back to the Jets-Patriots game, Belichick still definitely should have gone for 2. There was less than 30 seconds left when the first touchdown was scored. Had a second touchdown been scored, it would have surely been with close to no time left.

Bryan said...

Mike - you've described the variance around a distribution. We know that a team's performance varies from game to game for specific situations. We use the average of these performances as a baseline from which to make a decision. If the 2010 Patriots are going for a two-point conversion against the 2010 Panthers, we probably want to plug in a number higher than 47.9% as a success rate. Tom Brady, Danny Woodhead, and Benjarvus Green-Ellis are all injured and the two-point conversion is against the 2010 Steelers? We probably want to plug in something lower than 47.9%

I would be cautious in relying too much on whether a team is "playing above their average" (or below) for the specific game (accounting for opponent strength), since it could very well likely be too small a sample to really draw any meaningful information from.

Kos said...

Bryan: great stuff, and thanks for the detailed analysis. The only thing that stuck out to me as being weird was the 56% WP at 0:40. I was playing around with the calculator, and I noticed that WP is 53% even at 0:06. I don't question that the WP is indeed 53%, but I would be shocked if that team actually wins in regulation 3% of the time. The only similar example I can think of was the 2010 LSU/Arkansas game where Mallett dumped it short and saw his receiver take it 80 yards for a TD...and that was to end the 1st half, not the game. I can't think of a single example where an NFL team won a game in that scenario with less than 0:10 left.

Is it possible/likely that the data is skewed by other factors? I would think that the 53% team could have that extra 3% because of other reasons (really bad example: the team that just tied it up is more tired on offense, so they're less likely to score in OT.) I don't doubt the numbers, I just think it's likely incorrect to assume that a team with 1st and 10 from the 29 with 0:40 left actually wins in regulation 6% of the time, even if their WP is indeed 56%.

Bryan said...

It very well could be skewed by other data. I'm really not sure. Although I can't think of any examples, I imagine it's possible to throw a deep ball and get a pass interference call. All you need is a field goal to win. And when you think about 53% over 50%, we're talking about winning in regulation only once every 33 times, so it's definitely not something you'd see often. It could also be something like win once in regulation every 25 games, and lose in regulation once every 100 games (pick 6?).

So maybe the WP probabilities are slightly overstated, or maybe it's because of other factors that will affect the game in overtime. I think the WP calculator uses smoothed data curves as well, so while there might not be any examples of a team winning with :06 left, there might be a couple examples with <6 seconds and >6 seconds. I'm really not sure though, I don't know enough about the WP calculator to give a definitive answer. And of course, if there are a few examples of a team winning with ~:06 left from their own 29ish, that would resolve it as well! The Jaguars won a game this year tied with 3 seconds left from the 50 yard line. That's all I can think of off the top of my head.

Mike M said...

Bryan, thanks for the reply. I'm aware of vairance. Here's my point about using these mathmatical forumla's to make a decision.

When teams are losing such as the Pats were doing they generally are playing below their averages, that's why they're losing, as such why then would you "assume" they can convert a 2pt conversion at their average.

That makes no sense.

A average is a compiling of all periods of play including periods when the team is winning big and playing way over their averages.

But in this game the Pats were obviously playing well below their averages, but now the formula "assume" the Pats will convert that 2 pt play at their average, not likely.

Human being playing a sporting event are not like flipping a coin, which would be a 50-50 chance on any flip.

Human beings posses energy, motivation, skill, coaching, intesity, focus and can become complacent and overconfident all things that coins do not posses.

So to say that a team will convert a 2pt play a average of 47.9% game after game is just not the way it works in the real world of athletes playing a game.

Some games teams will convert at 100%, these are games the teams are playing well, and some games teams will convert 0%, these are games teams are not playing well, they are playing well below their averages, such as the Pats were against the Jets.

Bryan said...

When teams are losing such as the Pats were doing they generally are playing below their averages, that's why they're losing, as such why then would you "assume" they can convert a 2pt conversion at their average.

A couple of things about this:

I think you're correct in saying that if a team is playing below their averages (offensively), we should probably expect them to convert a two-point conversion at a below average rate. But I think this is probably true insofar that the reason they are playing below their averages is that they are likely playing an above average defense. Basically...the reason we should expect the Pats to have less success converting a TPC against the Jets than their theoretical average (or empirical average if we have enough data, which we never will) is because the Jets have a great defense, and not necessarily only because the Pats offense was playing below average earlier in the game.

While I think in-game dynamics (maybe the Jets defense REALLY figured out the Pats offense starting with that game) should be given some weight, I think choosing the larger sample size of data will give us a better estimate - in this case, using all 17 games worth of offense we saw from the Pats rather than just the one playoff game. Perhaps we should give the current game more than 1/17 the weight, but definitely not 100%. Giving too much weight to recent performance seems to err on the side of the "hot hand" phenomenon.

And as a nice little anecdote that proves nothing...the Pats scored rather easily on the only TPC attempt of the game, despite the fact that the Jets played solid defense throughout and the Pats were playing below their offensive averages.

Statalyzer said...

" People often discuss whether it was worth it to go for 2, as this would have clinched the perfect season and the national title, but people rarely note that, if you plan to go for 2 there, you should go for 2 earlier as well. "

Not necessarily SHOULD. There are valid reasons to go "all in" on the very last play as well. But going for 2 on the first TD doesn't happen nearly enough. One great example of when it did - the "Game of the Century" in 1969 - Texas 15 Arkansas 14.

Post a Comment