Friday, January 13, 2012

Best (And Worst) Post-Season Coaching Records Since 1950, via a Binomial View

by Jim Glass

Vince Lombardi's post-season coaching W-L record of 9-1, .900, is the best ever. Or maybe not, some say Joe Gibb's record of 17-7 is standard to beat - "only" .708, but over a run of 2.4 times as many games it is much harder to keep a big winning record. How can we compare them?

One way is to compute the binomial probability of a coach attaining a given won-loss record by random chance. For instance, in Herm Edwards' first year coaching my Jets, he went 10-6. I was happy. That .625 winning record was better than Tom Landry's .607 and our former coach Bill Parcells' .570 - better than a Hall of Famer's and sure future Hall of Famer's! It looked like we had a great coach. Except that Parcells earned his .570 over 303 games and Landry earned his .607 over 418, while Herm had earned his .625 over only 16.

The binomial calculation can give the probability of winning a given number of games out of any number played, and so providea a common standard to apply to the W-L performances of different coaches with different W-L percentages over differing numbers of games. It told me that, assuming game outcomes were random with a 50% chance of winning/losing, Herm's record of .625 (or better) had only a 22% probability of occurring by random chance, which was pretty encouraging - but Bill's .570 record had only a 1% chance, and Tom's .607 had only a 0.001% chance. So perhaps it was premature to declare Herm a better coach than Tuna and Tom (as indeed it turned out to be).

Now it's playoff time, so I wondered: what if we applied this common standard when looking at the playoff coaching records of Belichick, McCarthy, Dungy, and all the other coaches of all post-season games back to 1950? And suppose we set the random probability of winning each playoff game not at 50%, but at the actual probability of winning indicated by the relative strength of the competing teams and home field advantage?

Which coaches would we see to have beaten the odds, which not, and who would the surprises be? This is the exercise followed below.

I used the data base to compile a list of all NFL post-season games and coaches from 1950 through 2010, using teams' "Simple Rating System" numbers to gauge their relative strength. The SRS gives a team's strength-of-schedule adjusted average net points per game (points for minus points against) compared to the league average. To this I added 2.5 points to the rating of the home team (when there was one). From the resulting point differential the conventional Pythagorean formula provides a winning probability for each team in each game.

Taking all the games for each coach, the resulting "Unit Pythagorean" rating provides his expected W-L percentage.
Using this percentage instead of 50% as the probability of winning, the binomial calculation gives the probability of the coach's actual W-L record resulting from random chance.

Take Dick Vermeil as an example. His 6-5 winning record with a Super Bowl championship appears very credible on its face. But his teams averaged a full touchdown better than the opposition by SRS, were the higher rated in every single game, and on top of that had home field advantage seven times to only twice for the opponent. All this gives him a .689 expecting winning rate, the highest of any coach with three or more post-season games. Put that into the calculator and it says his actual 6-5, .545 record was better than only a meager 9% of those expected by chance - random chance would beat his playoff record 91% of the time.

At the other extreme, Blanton Collier as the successor to the great Paul Brown disappointed the Browns fans of his day by going only 3-4 in the post-season. But as he had only a .255 expected winning percentage, his .429 winning record was superior to .746 of those expected by chance, the highest binomial rating of any coach with a losing record.

More recently, Ken Whisenhunt has gone 4-2, .667, in the playoffs with only a .281 expected winning rate, giving him a winning record better than .944 of those expected by chance - the highest binomial rating among all the 128 coaches to take a team to the playoffs since 1950. (Lombardi is #2.)

The numbers, and interpreting them
I've limited the list below to the 70 coaches who have coached five or more playoff games since 1950.

Given for each is the number of playoff games coached, his W-L count, actual winning percentage, expected winning percentage, and the binomial calculation of the percentage of random results his record exceeds. Thus, a figure of .612 indicates the coach's record is better than 61.2% of the records produced by chance (given his expected winning percentage).

Personally, I do not consider these numbers to be very persuasive evidence of "how good" a coach is or was. The sample sizes for all are small to very small and vary greatly, and random results in just a few close games can make a very big difference in the binomial result.

But this data does show, objectively, how well each coach's actual record in the playoffs compares to reasonable expectation. Is Coach X's "great" record in the playoffs really as good as it looks? Is Coach Y's "poor" record actually better than it looks? Does Marty Schottenheimer really have the worst playoff coaching record of all time? (No, but close.)

For perspective on (and arguments about) such questions, I think this data is useful.

RankNameGames WonLostWon Lost%Expected WL%Binomial Performance
1Ken Whisenhunt6420.6670.2810.944
2Vince Lombardi10910.9000.6620.901
3Weeb Ewbank5410.8000.4480.871
4Bum Phillips7430.5710.3080.863
5Joe Gibbs241770.7080.5780.862
6Jimmy Johnson13940.6920.5040.860
7Tom Flores11830.7270.5210.857
8Chuck Noll241680.6670.5450.839
9Bill Belichick211560.7140.6010.797
10Tom Coughlin15870.5330.3960.796
11John Fox8530.6250.4220.790
12Bill Walsh141040.7140.5740.783
13Hank Stram8530.6250.4270.782
14Don McCafferty5410.8000.5300.773
15Blanton Collier7340.4290.2550.746
16Rex Ryan6420.6670.4520.741
17Raymond Berry5320.6000.3790.718
18Bill Parcells191180.5790.5000.676
19Jerry Burns6330.5000.3400.667
20Marv Levy191180.5790.5040.663
21Brian Billick8530.6250.5120.610
22Barry Switzer7520.7140.5940.593
23Dan Reeves201190.5500.4990.592
24Jeff Fisher11560.4550.3790.591
25John Madden16970.5630.5040.586
26Mark Holmgren2413110.5420.5030.569
27Mike Shanahan13850.6150.5560.556
28Bill Cowher211290.5710.5340.548
29John Harbaugh7430.5710.4790.546
30Ray Malavasi6330.5000.4020.540
31Mike Tomlin7520.7140.6300.513
32Sam Wyche5320.6000.4950.509
33Mike McCarthy7520.7140.6340.504
34John Robinson10460.4000.3550.500
35Tom Landry3620160.5560.5430.492
36George Seifert151050.6670.6360.482
37Norv Turner8440.5000.4580.458
38Jerry Glanville7340.4290.4030.413
39Andy Reid191090.5260.5260.409
40John Gruden9540.5560.5410.400
41Tony Dungy199100.4740.4790.393
42Sean Payton6420.6670.6310.392
43Dave Wannstedt5230.4000.3780.376
44Ted Marchibroda6240.3330.3270.364
45Pete Carroll5230.4000.3930.349
46Art Shell5230.4000.4270.292
47Herm Edwards6240.3330.3750.274
48Dick Nolan5230.4000.4580.244
49Bud Grant2210120.4550.5140.220
50Steve Mariucci7340.4290.5130.206
51Mike Ditka12660.5000.5860.184
52Bobby Ross8350.3750.4740.181
53Jim Fassel5230.4000.5060.180
54Chuck Knox187110.3890.4740.169
55Don Shula3619170.5280.5950.161
56Red Miller5230.4000.5320.150
57Lovie Smith6330.5000.6310.139
58Mike Martz7340.4290.5870.109
59Don Coryell9360.3330.4850.106
60Dick Vermeil11650.5450.6890.091
61Mike Sherman6240.2500.5310.083
62Wayne Fontes5140.2000.4090.072
63Dennis Green12480.3330.5190.056
64George Allen9270.2220.4320.048
65Paul Brown12480.3330.5410.041
66Sid Gillman6150.1670.4200.038
67Jack Pardee6150.1670.5520.008
68Wade Phillips6150.1670.5600.007
69Marty Schottenheimer185130.2780.5340.007
70Jim Mora, Sr.6060.0000.5970.000


James said...

So should we call the coaches on the lower half "bad", "underperforming (in the playoffs)", or "unlucky"?

Same for the upper half of coaches.

Jim Glass said...

James, I'd say: "mostly lucky / unlucky".

It's hard for me to believe that Paul Brown and George Allen were really very bad post-season coaches, much worse than say Bum Phillips. For that to be true they'd have to have changed the way they coached in the post-season from the way they did during the regular season, to a worse way, and why would they do that? Who reports they did? But with very small sample sizes, just a few random events could've made a big difference in their numbers.

Take Whisenhunt as an example, #1 on the entire list. His first playoff win in 2008 was a close, less-than-one-score win against the Falcons. Games that close are largely luck. If he'd lost that game his playoff record would be 1-2 instead of 4-2, and he'd be way down the list. In fact he wouldn't even be on the "five games or more list". So one turnover could've knocked him from being ranked #1 of all over 60 years to being just another also ran. That's how much effect random luck can have on these numbers, especially on the guys with small sample sizes, only 5, 7, 8 games.

But, OTOH, it is hard to look at Lombardi and say "just pure luck". And a lot of people say Schottenheimer got very conservative in the playoffs to cause his poor results. I don't know if that is true, or just a story. But it is plausible that in the big games of the playoffs some coaches might change their behavior -- maybe their risk aversion would come out -- so playoff-level coaching quality could vary. Maybe, as organizational leaders, some coaches just deal with big-game pressure better than others.

So it is entirely credible that some degree of difference in post-season coaching ability exists and is reflected in the numbers.

But I'd still say the differences in the final numbers are largely, mostly, luck. IMHO, FWIW.

Again, the purpose of this exercise *wasn't* to find evidence that this coach is/was "better" than that coach.

It was to obtain an objective measure of how coaches actually did in the playoffs, and to maybe compare that to popular conceptions of how they did.

Jim A said...

I guess you are comparing how coaches performed in the playoffs relative to how they were expected to perform. But how they were expected to perform in the playoffs is largely determined by how they performed in the regular season (by using SRS and Pythagorean). So some might say this is a measure of "clutch" coaching. It's an interesting exercise, though I don't know how useful it is.

Andrew Foland said...

A slightly different way to look at this is as an ensemble of predictions of playoff performance, given regular-season performance. If playoff performance is systematically identical to regular season performance, then in such an ensemble, the probability value in the far right column should be distributed uniformly between 0 and 1. With the exception of the pileup of four under 0.01, I'd say this does in fact look very uniform in probability density.

That is an indication that over an ensemble of coaches, regular season performance accurately predicts playoff performance, and that the variations are largely those expected solely due to chance. If there were special "playoff coaching ability", there would be significant bowing at 1 and at 0. There's a slight pileup at 0, but not a net bowing, so there's some weak evidence that about 5% of coaches completely choke, but other than that, playoffs are no different from the regular season.

Anonymous said...

There could be something to George Allen's ranking in this list. As a Redskins famously said after the 1972 Super Bowl, Allen was so obsessed with distraction that he became a distraction.


Jerrell said...

Mike Smith bucking to get into the Jim Mora, the elder Club, "Playoffs?"

Post a Comment

Note: Only a member of this blog may post a comment.