Should Colts have gone for 2

by Ian Simcox
So there I am, browsing my Twitter feed and I see this

Re HOU FG scenario: IND should still have gone for 2. Failed att means being down 15, which still gives 2 more bites at the 2-pt apple.
— Brian Burke (@Adv_NFL_Stats) November 4, 2013

Predictably the replies featured people as sure of the value of kicking the XP as Brian was of going for the 2.

At a 45% success rate on the 2-pts, the WP calculator says Indy have a 0.10WP if they kick the XP, comparing favourably to 0.09WP if they go for it (0.12 on success, 0.07 on failure). So, marginally the XPers have it. I must admit though, I plugged starting the 4th qtr down 10, 11 and 12 on 1st down into the calculator, to simplify the second part of this article.

What really interests me is the uncertainty around these numbers, which is always important on these tight calls. The XPers may have it on the WP calculator, but if those numbers are +/- 0.03WP then it’s impossible to say which is right.


To analyse these binary W-L uncertainties, I like to use my (admittedly basic) understanding of Bayesian probability. To test whether coach should go for 2, kick the XP, or just flip a coin because you can’t be certain, I used PFRs game play finder to find all plays since 1999 where a team led by 10, 11 and 12 at the start of the 4th quarter with the ball between their own goalline and 30 on 1st down (this is where my simplification earlier comes in, more samples here).

From the Colts point of view then, the win-loss numbers are

Score		W	L
Down 10		14	128
Down 11		3	58
Down 12		1	29

Assuming I know nothing about the probability of winning outside of this information (technically, using a uniform distribution on (0, 1) for my prior), Bayes’ rule gives me a posterior probability distribution for the chance of winning in each of the three scenarios. As we have more data for the down 10 case, we have greater certainty of the probability of winning than the other cases, but all of them feature some degree of uncertainty (the 95% CI for down 10 is 0.06-0.16WP, but for down 12 it is 0.01-0.17WP). Taking some liberties by assuming a 45% success rate on the two-point try, it’s a case of running a monte-carlo on the posterior distributions and analysing the results.

With these numbers, going for it results in a 0.08WP for the Colts and kicking the XP a 0.06WP. So the 2-pters have it on the average. However a coach doesn’t have an average, he can only make the call once and I was looking for uncertainty. It turns out that by about 2-to-1, going for the two results in a higher WP.

So what should a coach do? Well 2-to-1 are good odds, but it’s hardly statistically significant. You can’t say with any certainty that he would have been wrong to kick the XP.

Ultimately we arrive at the unsatisfactory conclusion that you can’t say what the right call is. One thing we do know for sure though. Tweeters will continue to disagree about these calls and both sides will be 100% certain they are right.