Playoff QBs Who Carry Their Teams – And Teams That Carry Their QBs

by Jim Glass
Just a few playoff game performances can earn a quarterback fame or blame forever. But how well is it deserved?
How often does a "winning" playoff QB actually play so poorly as to deserve to lose, only to be bailed out by his teammates (or good fortune)? How often do "losing" QBs play well enough to deserve to win, only to be done in by their teammates (and bad luck)?
How well do QB reputations for being great clutch winners or choking losers correspond to their actual performance in the playoffs?
To see, I looked at the last 15 years of playoff games, 1995 thru 2009.

The tests
Fully separating a QB's performance numbers from the effects of his teammates is notoriously difficult, but being interested primarily just in good performances that resulted in losses, and poor performances that ended in victories, a couple of methods suggested themselves.
To begin I took the adjusted yards per pass attempt (AYA) for each playoff game QB performance since 1995 (using the PFR.com data base). AYA is strongly correlated with winning – of the top 50 performances by AYA, 46 were wins; of the top 100, 78 were wins – so I used it as the measure of quality of QB performance. (Well, of passing game performance, but attributing it to the QB.) After eliminating a handful of instances in which no one QB had at least 14 attempts (the NFL's minimum per game for official rating purposes) there were 69 QBs with 326 game performances – 163 winning and 163 losing. From there I tried two approaches.
• Simply identifying each QB performance as winning quality, above average, below average, or losing quality, and comparing actual game results. Spreading each QB's performances among the four "buckets" gives a simple, clear picture of the overall quality of his playoff play, and matching top performances to losses and poor performances to wins is easy. Nobody will call analysis "by the bucket" scientifically precise, but the results are easy to see.
• For each QB: Taking each individual game performance by AYA, calculating the probability of winning that game as indicated by AYA, and summing for all games. This gives an expected W-L record for each QB that can be compared one to the other, and to actual game W-L results.
The numbers resulting from all this are not meant to rate or rank any QB as being any better or worse than any other. The samples sizes are far too small, for one thing. The intention is simply to obtain an objective perspective on how QBs have played in fact, which may be of interest, and perhaps in a few cases contrast with some popular opinions.

The Bucket List
In the 326 QB performances the average AYA in all winning games was 7.61, in all losing games was 4.58, and in total overall was 6.1. So each QB has his games sorted into four buckets...
* Winning Effort, over 7.61 AYA. A QB who played better than the average winning QB can rightfully feel he did his full part to win and if the team lost it was somebody else's fault.
* Plus Effort, 6.1 to 7.61 AYA. An above-average performance doesn't guarantee victory, but it is a positive thing, it helps.
* Minus Effort, below 6.1 to 4.58 AYA. Below-average performance doesn't doom a team by itself, but it doesn't help win, to win somebody else must rise to the occasion.
* Losing Effort, below 4.58 AYA. If a QB plays worse than the average losing QB and still wins, he owes somebody a big thank-you gift.
Tables are limited for space reasons to QBs who played five or more playoff games in the last 15 years, 26 in all (accounting for 234 of the 326 performances, 72%). They include the tail ends of the careers of Troy Aikman, Dan Marino, John Elway, Steve Young and other QBs who retired shortly after 1995.
The first table gives the QB, his number of playoff games, and his number of winning-quality efforts (WE), plus quality efforts (+E),minus quality efforts (-E), and losing-quality efforts (LE). This shows the quality distribution of the games he played.
Then taking the number for each QB in each "bucket", scoring the WE column and 50% of the +E column as "plus points", and subtracting the LE column and 50% of the -E column as "negative points", we can come up with net "Bucket Points" for each QB – giving a rough but reasonable measure of his total performance. The last column is "bucket points per game played".

QB	Games	WE	+E	-E	LE	Bucket Points	BP/G
Warner	13	6	6	0	1	8	0.62
Favre	20	8	6	2	4	6	0.3
Brees	6	4	2	0	0	5	0.83
Delhomme	8	5	1	0	2	3.5	0.44
Elway	8	4	2	1	1	3.5	0.44
Roethlisberger	10	6	0	2	2	3	0.3
P. Manning	18	7	3	4	4	2.5	0.14
Dilfer	6	2	2	1	1	1.5	0.25
Gannon	7	4	0	1	2	1.5	0.21
Aikman	7	3	1	1	2	1	0.14
Brady	18	5	4	6	3	1	0.06
Cunningham	5	2	1	1	1	1	0.2
Hasselbeck	9	3	1	5	0	1	0.11
E. Manning	7	2	2	1	2	0.5	0.07
McNabb	16	5	3	4	4	0.5	0.03
Rivers	7	3	0	3	1	0.5	0.07
Garcia	6	2	1	1	2	0	0
Pennington	6	2	1	0	3	-0.5	-0.08
Young	6	2	0	3	1	-0.5	-0.08
K. Collins	7	2	1	1	3	-1	-0.08
B. Johnson	7	0	3	1	3	-2	-0.29
Plummer	6	1	1	1	3	-2	-0.33
Marino	6	0	2	1	3	-2.5	-0.42
Brunell	9	1	1	3	4	-4	-0.44
Bledsoe	6	0	0	2	4	-5	-0.83
McNair	10	0	2	3	5	-5.5	-0.55

Tales of the Bucket List
In the 326 games there are only 22 in which a QB played at a winning level and lost, and another 25 where the QB played at plus-level but lost. There were 25 games in which a QB played at a losing level but won, and 26 in which one played at minus-level and won. So 98 of the 326 games (30%) had "contrary" results.
No QB lost more than one game in which he played at a winning level – except Peyton Manning.

How Peyton finally won the big game
For years they said that Peyton Manning, despite all his talent, couldn't win the big game. In fact, Manning has taken the fall three times after playing winning-quality games and twice more when he played above-average. Five losses are a half-decade of elimination when his teammates (or fate) let him down. No other QB lost more than one winning-quality game plus one above-average one.
But Peyton can't complain: playing losing-quality games he has two wins, plus another two wins when playing minus-level ... and three of those four fortunate wins came during his Super Bowl run! He played losing-quality games against the Chiefs and Ravens on the way to the Super Bowl, and a below-average game when he got there against the Bears. (The Bears had a very good defense, but you'd expect the top QB in the league to play at least average-level against anybody – especially in the most important game of his life.)
When Peyton got that ring all the pundits said he had finally matured into a champion capable of winning the big game – after he'd put up by far the worst post-season numbers of his career.

There's something about Patriots QBs
Tom Brady and Drew Bledsoe have 23 games combined, 7.1% of the 326 total. (They shared one game with each throwing more than 14 passes, so they have a combined 24 in the tables.) Random chance would give them 3 games in which they'd played "plus" or better but lost and 4 in which they'd played "minus" or worse but won, giving them a net +1.
Yet they won 4 loss-quality games plus another 6 minus-quality games, against only a single loss of a plus-quality game-- giving them a net of +9. Nine such wins in a single-elimination tournament is HUGE. It's 9 of their total 23 games, 9 of their 14 wins. And Bledsoe, next-to-last on the Bucket List, has 2 of the losing-quality-play wins and 2 more minus-quality-play wins – in six games total! There's nothing else like this anywhere in the data. What explains it?
Have great Patriots teams carried these guys in these games? Is it their great clutch play? (Bledsoe's?) Is it Luck? Cheating? You decide.
The only other QBs who won more than one game played at a losing level are: Steve McNair with 3, of which 2 were on the way to the 2000 Super Bowl (against Jacksonville and Buffalo); and Ben Roethlisberger with 2, one of which was his first Super Bowl victory, which "he won" with an AYA of 1.57.

Underappreciated stars
I knew Kurt Warner was a very good QB, but didn't appreciate how good he'd been in the playoffs until after looking at his performance game-by-game. I had no idea that Jake Delhomme had been as good in the playoffs as he's been – even better than Warner until he cratered in his last two games. Talk about falling off a cliff! It seems odd to call Brett Favre, of all people, "underappreciated" – but he has a big plus in Bucket Points over a large sample of 20 games, which makes it more difficult, as it is less likely to be luck.

AYA and expected wins
The next idea was to take each QB's AYA for each game, compute the probability of winning that game as indicated by that AYA, total the probabilities to get an expected win total for each QB, then compare that to the actual win total.
The concept is that if the given level of AYA wins fewer games than expected the rest of the team (or bad luck) was holding back the QB, while if the level of AYA won more games than expected the rest of the team (or good luck) carried him forward.
For context, data from all 326 games indicate a 99+% chance of winning for AYA over 14.0, 75% chance for AYA of 8.95, 50% chance for AYA of 6.1, 25% chance for AYA of 3.1, and less than 1% chance of winning for AYA under -0.4. (Mark Brunell of the Redskins used an AYA of -0.27 to "beat" Tampa by 17-10 in 2005.)
Say a QB in three games has AYAs corresponding to win probabilities of 72%, 63% and 54%. They total to 1.89 expected wins. If his team actually goes 0-3, we can say the QB outperformed the rest of the team (and the effects of luck) by positive 1.89 wins, which is 1.89 expected wins minus 0 team wins. So two of those three losses were on someone else – he tried to carry the team, but couldn't do it, it held him back. But if his team goes 3-0, we can say it (and good fortune) outperformed the QB and "carried" him for an extra 1.11 wins (1.89 - 3 team wins = negative 1.11).
The next table for each QB gives the number of games played, his average AYA in them, actual W-L, expected wins projected from AYA, and expected wins minus actual wins.

QB	Games	AYA	Wins - Losses	Expected Wins	Expected Wins - Wins
Warner	13	8.66	9-4	8.78	-0.22
Brees	6	8.19	4-2	3.92	-0.08
Elway	8	7.68	7-1	5.13	-1.87
Delhomme	8	7.53	5-3	5.03	0.03
Rivers	7	7.48	3-4	4.14	1.14
P. Manning	18	7.33	9-9	9.89	0.89
Roethlisberger	10	7.32	8-2	5.73	-2.27
Favre	20	7.27	11-9	11.94	0.94
Dilfer	6	7.2	5-1	3.54	-1.46
Cunningham	5	6.76	2-3	2.86	0.86
Brady	18	6.63	14-4	9.72	-4.28
Hasselbeck	9	6.54	4-5	4.83	0.83
Gannon	7	6.51	4-3	3.79	-0.21
Pennington	6	6.13	2-4	3.03	1.03
Aikman	7	6.04	4-3	3.6	-0.4
McNabb	16	5.98	9-7	8.01	-0.99
Young	6	5.88	3-3	2.91	-0.09
E. Manning	7	5.79	4-3	3.32	-0.68
Garcia	6	5.59	2-4	2.74	0.74
Collins	7	5.36	3-4	3.33	0.33
Plummer	6	5.31	2-4	2.7	0.7
Brunell	9	4.6	4-5	3.42	-0.58
Johnson	7	4.54	4-3	2.65	-1.35
McNair	10	4.29	5-5	3.52	-1.48
Bledsoe	6	4.17	4-2	2.17	-1.83
Marino	6	4.13	2-4	2.06	0.06

Again, the big outlier in the last column is the Brady-Bledsoe combination.

Problems with the numbers
I think the numbers above do some things fairly well, but there is a big gap in them.
While they catch QBs who played well but lost (tried but failed to carry the team) and those who played poorly but won (failed to sink their team) they don't catch QBs who played great and won – succeeding in carrying the team that played less well than them – or who played poorly and "succeeded" in sinking a team that played better than them into defeat. The numbers don't catch two of the four quadrants.
I fear that in the second table this makes the "ExW-W" number more precise than accurate in some cases. To get a full picture of how QBs have carried and sunk playoff teams, the other two quadrants will have to be filled in.
Thus, I consider that in the second table AYA itself is the simple and best indicator of the QB's contribution to the team.
But another problem is that while AYA correlates excellently with winning, it doesn't necessarily show how much the QB contributed to winning.
For instance, last year Mark Sanchez put up a stellar 13.5 AYA – #7 among all the 326 playoff games of the last 15 years – against the Bengals in the playoffs. But he passed only 15 times, as the Jets rushed for 170 yards and their league-best defense shut down the Bengals offense to win. One thinks that Kurt Warner's "lesser" performance of 10.1 AYA on 45 attempts for 414 yards probably had rather more to do with winning the game in which it occurred (the 2000 Super Bowl).
Similarly, Trent Dilfer put up a great AYA of over 9.0 in the three playoff games leading up to the 2001 Super Bowl . But in those three games he averaged only 16 attempts (and just 7.7 completions) as the Ravens' great defense and Jamal Lewis rushing game controlled the opposition. I suspect that Dilfer's great AYA (and Sanchez's) was a mostly a consequence of winning, not the cause of it. (If anyone is looking for evidence that the combination of a top defense and big running game can set up an efficient passing game with even only a very mediocre QB, you might begin with these examples.)
So to fully flesh out how much different QBs have really carried their teams in the playoffs, and vice versa, there's still a lot of work to do.
But all this is a start.
Happy New Year!