Will Tim Tebow be Denver's Dick Jauron? Or: The Meaning of Clutch Victories
by Jim Glass
Is the Tim Tebow story real? I enjoy Tebow, for years have wanted to see somebody try the option in the NFL, am entertained by watching the challenges it throws up for opposing teams, and am rooting for Tebow football to work. But is the Tim Tebow story as heard everywhere from his fans and admirers (and worshipers) real?
Tebow is a type of QB unique in the modern game. But the Tebow story is very old and familiar: the tale of "clutch winning". It's the story of *not* having the stats that go with winning, but winning anyhow. Thus, credit for winning goes to character, leadership, inspiring teammates, making others better, and - most of all - coming through with big plays in the last minutes to win close games in the clutch.
Tebowmania is based on the Broncos pulling out five close, clutch, one-score victories in the last moments (by 7, 4, and 3 points, and in OT twice) after he became the starting QB. These plus one decisive win and one decisive loss make the Broncos 6-1 with him at the helm, after a miserable 1-4 start with him on the bench. What a difference!
No, Tebow doesn't have the "stats" that usually go along with a QB who wins games. His EPA and WPA numbers are both negative, with his EPA-per-play rank 35th and WPA-per-play rank 27th. Other rating systems agree, by DVOA he is 27th. Tebow passes for only about 130 yards per game, 34th among QBs in a league of 32 teams. These normally are the stats of QBs about to get cut for losing games.
But he wins! The clutch winner like Tebow has "intangibles" that can't be measured in stats, but which *win* and so are more important than skills measured in mere stats, because winning is what it is all about. QED.
And who can deny that this guy has these intangibles that give him a special ability to win close games? The record speaks for itself. So the story goes.
Reality
It's a great story - but does an ability to win close games like this in the NFL actually exist?
To see, I looked at a larger record, that of all teams that (like the Broncos with Tebow) have been +5 net or better in one-score games (decided by 7 points or less) during the era of the 16-game season, from 1978 through 2009. I compared each team's record in its season of close-game success and clutch victories to its record in the following season (1979 through 2010). If there really is some ability, some skill, that enables a team to win close games in the NFL, it should carry over from one season to the next at least in some degree.
Altogether, 47 teams were a net +5 or better in close games, 5% of the 945 team of the 32 seasons. So the Broncos with Tebow at the helm are on course to a top 5%-or-better performance in clutch games. If you're a Broncos fan, what's not to be thrilled about?
But in the subsequent year, only the barest majority of the 47 teams were even +1 in close games, just 24 of the 47. 
The 47 "best in the clutch" teams of the last 32 years were +263 net in close games - but only +15 in the subsequent season. Their won-lost record in one-score games fell in one year from 328-65 (83.5%) to 189-174 (52%). The correlation between their numbers of net close wins over the two seasons was an almost invisible 3%.
Comprehensively, for all 945 teams over the full 32 seasons, the correlation in year-to-year net close wins is 8%. In realistic terms: next to nothing.
Well, OK, that's everybody, the masses, but are the greats different? We've all heard that "Great teams, great players, win close games". Just because the masses don't win close games doesn't mean Tebow can't do it as the other greats have done!
But have the greats done it? A quick look at career records in 3-point games of various greats finds:
• Vince Lombardi's Packers: 12-11, 52%, while winning five NFL titles and the first two Super Bowls.
• Bill Walsh's 49ers: 17-22, 44%, winning three Super Bowls.
• Chuck Noll's Steelers: 13-23, 36%, during the Terry Bradshaw years with four Super Bowl titles.
Those greats sure didn't do it. Who actually was the greatest at winning close games?
Well, during the last 32 years it was Dick Jauron and his Bears team, quarterbacked by Jim Miller, that pulled out one "miracle" win after another, just like the Tebow Broncos and even more so! They went +8 in one-score games amid piling up a 13-3 record and earning Jauron the "Coach of the Year" award plus a three-year contract extension.
The next year the Bears went 4-12, and -2 in close games. In Jauron's ten-year coaching career he never had another winning season, going 47-79 in nine other losing years. Miller lasted at QB one more year.
The other story
Another story becomes visible in the above: Close-game outcomes are overwhelmingly determined by luck, random chance.
There is no skill, no ability, to consistently win close games in the NFL. It doesn't exist.
Vince Lombardi and Bart Starr couldn't do it. Bill Walsh and Joe Montana couldn't do it. Chuck Noll and Terry Bradshaw couldn't do it.
So when the fans of Tim Tebow say he does it - "he wins close games, he just does what it takes to win" - they really are making quite an ambitious claim!
Now consider the alternative. If Tebow doesn't win close games any more than Starr, Montana, and Bradshaw did, then what's left of the Tebow story? The entire Tim Tebow tale is "he makes the team win, he just does what it takes". But if he doesn't, the whole story evaporates and there's nothing left - he's just another young project at QB with very bad numbers so far. Starr, Montana, and Bradshaw put up great numbers. That's how they won.
In this story Tebow is winning like Dick Jauron did. The Broncos - all 53 players, not just Tebow - have played five generally weak teams (combined record of 23-37, 38%) evenly ... and won five coin flips.
If three of those coin flips had gone the other way - say, Miami QB Matt Moore hadn't fumbled in OT in field goal range for Denver, San Diego hadn't missed that field goal in OT, and Ponder this week didn't throw a last-minute interception again in Denver field goal range - then nobody is telling the "he just wins" story about Tebow today, because he isn't winning. The support for Tebow Idolatry is that tissue-thin.
One of the bad mistakes you can make in life is confusing good luck with superior ability, then doubling down on your confidence in that superior ability - and losing your house when the luck runs out.
In 2001 Dick Jauron was voted Coach of the Year over runner-up Bill Belichick. He was hailed for being the only other Bears coach ever to match the top season win totals of George Halas and Dick Butkus. He got a raise and a three-year contract extension ñ and after he was fired, on the strength of his "success" with the Bears, he got *another* head coaching job ... until his bosses had paid for nine years of losing records. That cost them plenty.
Broncos fans and Tebowmaniacs are investing a heck of a lot in their belief that Tebow won those five close games. It will be interesting to see how much Elway and Fox are willing to invest.
Posts
19 comments:
An emotional topic such as this one engenders comment and I have two.
The issue of sustainability is one which is continually discussed. Teams records do not flow from year to year. (That's a sat of the pants assessment with no data to back it.) We see teams win a Super Bowl and follow that with a mediocre year. This gets me to think that clutch does in streaks as your data suggest. And random wins go in streaks as you've suggested above. So as I develop this thought I see I am restating your premise.
A more germain point which I'd like to make is that in the NFL you have two ways to win. THe first is to do what everyone else is doing but do it better. THe second is to do something different. Several years ago MIami adopted the Wildcat. That worked for about a season and a half. It was new and defences didn't know how to defend it. Over time it lost it's effectiveness as teams learned from each other and developed ways to stop it. In this year's incarnation, the Broncos under Tebow are doing something the NFL has not seen in years. So it will succeed for a while until the league learns how to defend it.
Analyses fail in situations of novelty. Miami was winning games with the Wildcat but did not have the numbers to support the wins. I think there is an element of this phenomena at play here.
Is Tebow's success due to random luck? Very likely. Does Tebow optimize lucky situations? Perhaps yes. How much longer can we expect Tebow to continue winning this way? Perhaps through the end of next season.
Great analysis Jim.
Nice analysis. The chart is a pretty clear illustration of regression to the mean.
The stats for Walsh/Lombardi/Knoll are very telling. Good teams don't win close games. Good teams don't let games get close.
Regarding, "There is no skill, no ability, to consistently win close games in the NFL. It doesn't exist."
There's a school of thought that says New England under Belichick and Indianapolis in the Polian-Dungy-Manning years made advances in what can best be called "possession management" in a way that did give them edges in close games (they figured out how to squeeze out an extra possession in the first half and second half more than others did...and figured out how to time things so they're productive offenses would be on the field last in close games).
Using the 7-or less paramaters you used above:
Belichick-Brady: 48-18
Manning from '02-09: 52-21
That's counting playoff games. And, some of those were against each other, so "those two vs. the field" were slightly better percentage-wise. (Just figured those by hand using football-reference, so they might be off by a game or two...but not in a way that would change the lopsided results).
There's also a feeling that many teams have basically figured out what they're doing, which has taken away some of the edge over time (Pats just 9-8 the last 3 years, Colts were 5-5 last year...but 7-0 in '09 so who knows?)
I mentioned a couple of years ago over at FO that their formula for converting stats to wins after the fact had underestimated the Colts in every single year of the Dungy-Manning era...and which suggested at least that the team had figured out something regarding getting more out of their stats than others had. Barnwell mentioned in an early Grantland article that this became statistically significant after the 4th time it happened...even though it kept right on happening through the decade.
So, there at least IS a chance that somebody figured something out...and that using huge sample sizes from the past isn't very helpful. Statheads are prone to find out way after the fact when an innovation occurs because the pre-innovation sample size dwarfs the immediate stats after an innovation.
Anyway, wanted to throw this out there. Manning and Brady are counter-examples to the greats of the past who didn't have eye-popping records in close games. And, at the very least,the fact that those guys are on TV all the time winning close games may have influenced media/fan perceptions about the abilities of QB's to influence close games.
Regarding Tebow, would hope it becomes better understood that his contributions have increased since Denver went to the run-heavy attack. BB's numbers show him at +.31 in WPA and +8.1 EPA since that happened. Per-game would be about +.06 in WPA since then...which would put him with Hasselbeck and Rivers in the middle of the pack. Tebow's EP/Play listing on his player box shows no negatives since then: .11, .11, .00, .00, .09.
So, since Fox's offensive innovation, Tebow's more in middle of the pack in terms of impact rather than at the bottom of the league. The horrible game vs. Detroit pre-change provides a big anvil for the full season numbers.
Not linking these two points together yet...suggesting inexperienced Tebow is a master of possession management or anything. But, as separate entities...additional study may be in order. Football evolves. Strategies evolve. Innovations work until antidotes are devised. There may now be strategies which are applied by the smartest teams late in close games.
Quick fix...Manning's record is thru '10 not '09
Good analysis (I've always thought that "clutch" players are usually just good ones, ie Joe Montana or Michael Jordan were good in the final minute of games b/c they were good in the other 59 or 47 minutes as well), but a couple of issues.
1)As often happens on this site, "luck" does not mean what it's claimed to mean. Someone playing better some times and worse some times is not truly "luck".
2)Since Denver is rushing more and passing less than most NFL teams, their games ought to have fewer possessions by each team, and thus more of Denver's games (win or losses) ought to be by close margins.
3)Two things you can't measure - has his leadership motivates his teammates on both sides of the ball? Yes Denver hasn't played great teams, but they are still playing better as a team. And has the new offense's ability to control more clock helped out the D by resting them more? You can *somewhat* measure this last effect, but it's different than point #2. It's not just that opponents have fewer possession and thus score less, but are they are scoring fewer points per possesion? And if so, is it due to more rested defenders which is then in turn due to Tebow and the offense? Difficult to say. It probably is due somewhat to these factors, but maybe not as much as his rabid fans would claim (how's that for a noncommittal answer?)
Ed, thanks for the kind word.
Michael, you're right. Great teams apply whompings, bury the other guys, don't get whomped, and in close games take their chances like everyone else.
Good teams don't win close games. Good teams don't let games get close.
Oh, man, when Herm Edwards was coaching the Jets and kept saying ...
"Our objective is to just keep the game close to the end, not take any chances that could blow up, just be within a score at the end, and win then by doing the right thing. That's how you play to win the game!"
... listening to it was like torture to me.
Jeff, I use 3-points for close games for coach/QB career and franchise performance. Measuring performance in close games one wants to look at close games, and over such multi-year periods there are enough to do it. The higher the point break one uses the more one-sided results become, because among other reasons at 7 points a larger portion of games were never really close, and just got down to seven at the end.
Better teams should win more close games, in principle, and on the largest scale they do, slightly. There's even a Pythagorean for predicting OT game outcomes between teams of different strengths. In baseball with its 162 game season and managers coaching thousands of games in a career, the close game edge has some small visible application. But in football the edge is so slight and games are so few the edge is swamped by other random factors to produce the results noted.
As to Indy and NE devising secret methods of winning close games, well, if one devises a method to beat the stock market or beat a casino one could keep it secret. But play calling in NFL games is definitionally visible public action and public information -- in fact about the most massively public and thoroughally videotaped and analyzed information in the world.
If they have a secret method of winning close games through the plays they call (instead of, say, by paying off refs or stealing the opposing team's signals, as per Belichick's videotaping sideline) then I'm a lot less interested in their secret method of winning than in their secret of how they keep it secret! They have to perform their secret action in plain sight in front of the whole world, so how do they keep others from seeing what they are doing? (*That's* the secret I'd pay big money to get!).
Half of all coaches and QBs have above average records in close games, obviously. The thing is that close game success doesn't correlate with being a good coach/QB who wins other games.
Walsh/Montana and Noll/Bradshaw have losing records in them on dynasty teams, while the best record in them over 20+ decisions belongs to Vince Tobin with Jake Plummer, 71%. But in their non-close games they went 26%. So a coach/QB having a good record in close games doesn't indicate he's in on the secret of winning them, just that he's one of the half of all coaches/QBs who have to have a good record in them.
You used both 7 and 3 in the article. The first one you used was 7, so that's the one that caught my eye.
Not suggesting Indy and NE "have a secret method of winning close games through the plays they call." I think I was very clear in referring to possession management that allowed them to squeeze out an extra possession per half or per game. Pulling that off could increase a team's percentage in close games. It's not the secret of "winning close games," but it's a strategy for improving your odds of winning. You still have to execute.
Yes, game films would make this evident to people eventually...and the strategy would move across the league as more people figured it out. The edge would lessen over time for the initial innovators...and more teams would try to make it work (New Orleans, Green Bay to name a couple of obvious nominees).
Within the past few years Mack Brown of Texas has talked about this a lot...trying to squeeze out an extra possession in the first half at least...so something that may have started much earlier in the decade in the NFL has even trickled down to some college staffs.
If an innovation is created in the late '90's or early 2000's that works...what happened well before then isn't all that relevant. So, what could have been true in the Walsh/Montana and Noll/Bradshaw eras isn't true any more. It USED to be true that there wasn't a skill or ability to consistently win close games. If somebody figured something out, it's like the Fosbury Flop. What was true before the innovation wasn't true after the innovation.
I don't think it's a secret that at least some teams are trying to be smarter about how they run clock...and that part of what separates success from failure in the NFL are abilities in this area (aided by smart QB's who can execute the plan).
Went back to check the guys at the threshold of 3.
Belichick-Brady: 24-10
Manning from '02-10: 27-13
New Orleans is 9-2 over the last three years at 3 or less, but didn't show anything with Brees in that regard from 2008 and earlier.
Green Bay isn't a good nominee at all upon further review. One of the defining characteristics of last season was that they kept losing close games while they were shorthanded...then they looked like champions once they got healthy at the very end. They are 11-6 this year and last in games decided by 7 or less, but 2-4 in games decided by 3 or less. So, perhaps moving forward they'll show a close game tendency while they're healthy...but they didn't really show that last year.
I've wondered off an on if Bill Polian was kind of a patient zero here. I remember when FO went back and did DVOA for one of the Buffalo seasons he was there that his Bills outperformed projected wins the same way his Colts did.
Polian/Kelly from '86 to '93:
35-16 at 7 or less
24-10 at 3 or less
Give Polian some talent and experience at QB, and he has a strategy his teams use in terms of possession management (he's a hands-on GM, so that's not too far-fetched). He and Belichick have been ahead of the curve (Polian way ahead of it). But,the curve is catching up (Manning just 3-5 last year, Brady just 4-4 from '09 to now). At their peaks...Brady won 20 of 25 from '01 to '07, while Manning went 24-8 from '02 to '09.
Jeff Fogle, how were NE and IND's records in those years overall compared to their 3-or-less wins?
The Browns/Steelers played a game on Thursday that never had a score margin higher than 4 for the first 57 minutes. Yet since it ended 14-3, it doesn't even enter into the record here.
That's why using the final score is never going to do any justice to studying close games.
Pro-football reference makes it relatively easy to track down overall records for teams in the Belichick, Dungy, Noll, Walsh, and Lombardi eras for context on:
"If there really is some ability, some skill, that enables a team to win close games in the NFL, it should carry over from one season to the next at least in some degree."
"Close-game outcomes are overwhelmingly determined by luck, random chance."
"There is no skill, no ability, to consistently win close games in the NFL. It doesn't exist."
"Vince Lombardi and Bart Starr couldn't do it. Bill Walsh and Joe Montana couldn't do it. Chuck Noll and Terry Bradshaw couldn't do it."
Was Brady winning 20 of 25 over a seven-year period, and Manning winning 24 of 32 over an eight-year period evidence of a skill that could be carried over from one year to the next? Or, was it heads coming up 80% of the time for one, 75% of the time for the other...basically simultaneously...so that the best two QB's in the game were being extremely fortunate in tandem in an area that is
"overwhelmingly determined by luck, random chance?"
Since 2002, the Pats are 119-37, a W% of 76.3%. In the low scoring games that JF mentioned, they went 24-10, a W% of 70.6%. Non-close games, the Pats won 77.9%
From 2002-2010, the Colts were 109-35, a W% of 75.7%. In the low scoring gamest hat JF mentioned, they went 27-13, a W% of 67.5%. Non-close games, the Colts won 78.8%
The Pats and Colts "lost" 7% to 11% of their W% between non-close games and close games.
I'm not sure this is a sign of those two teams being capable of winning close games. Maybe they were just so capable of winning games period. And some of those games just wound up to be close.
After all, if a team has a 100% winning percentage, their winning percentage in close games will also be 100%.
I think the year-to-year win graph would be easier to follow if you plotted year 1 close game record against year 2 close game record, rather than having a meaningless X axis.
It does make sense that if one team was better at the coaching decisions that come up more in close games (when to kick/punt/go for it, late-game tactical playcalling, timeout usage, etc) that these things would matter more in the close games. Some aspects of game play (e.g. an offense that plays well in the 2-minute drill and a good FG kicker) seem like they would be more important in these cases, too.
Pointing out that great teams lose some edge in these situations doesn't immediately invalidate the argument, because we would assume that a larger proportion of their close games are against other good teams. So I'd like to see some sort of adjustment to account for this
My first thought would be to look at every game (not just close games) over a very large data set. Compare the two team's SRS, or some other broad team stat that's easy to pull out for a large data set, and tease out the expected correlation between SRS (or whatever you use) and winning percentage. Basically, tease out the expected winning percentage of each team for a ton of games.
Then go back again, just looking at the close games, and figure out which teams, if any, consistently outperform their expected winning percentage (now adjusted for opponent's strength!) in the closer games.
Thanks for posting those anon. Let's compare them to the examples from Jim's article:
• Vince Lombardi's Packers: 12-11, 52%, while winning five NFL titles and the first two Super Bowls. (according to pro football reference...Lombardi was 89-29-4 with the Packers, .754 winning pct. Throwing out ties--which it looks like JG did when listing records since a 0 would be less than three--we get 77-18 in non-close games...which is .811. So, Lombardi's packers fell from .811 to .521.)
• Bill Walsh's 49ers: 17-22, 44%, winning three Super Bowls. (Walsh's overall record with SF was 152-92-1, which is .609. That means 135-70 at margins of 4 or more...and .659 falls to .435.)
• Chuck Noll's Steelers: 13-23, 36%, during the Terry Bradshaw years with four Super Bowl titles. (didn't have time to go through and exclude any games Bradshaw missed with injury, and JG specifically said "Bradshaw years" here. Bradshaw's record as listd at pfr was 107-51-0, which is .678. We can deduce the same dramatic drop-off, since JG shows .361 in close games in the Bradshow years. Looks like Bradshaw was up in the high 70's at 4 or more, but in the 30's at 3 or less.)
Stark contrasts between JG's examples and Manning/Brady. Tougher to buy "capable of winning games period" as an overall theme given what happened with those other dynasties.
Somebody's probably done a study somewhere showing the general relationship for how much the percentage drops in closer games. That will likely help us see to what degree the teams in this discussion our outliers on either end, and what the possibilities are for innovation that took some of the luck out of the equation to have been in the mix in more recent seasons...
PS: like Tarr's idea. I was writing my post when he posted those comments...
This provides evidence counter to Brian's thesis that poor teams should look to increase variance. By keeping a game close (lowering variance), an inferior team can reduce the affect of skill and increase affect of luck on outcomes. The probability of beating the Lombardi led packers goes from 19% to nearly a coin flip. When Herm Edwards says he wants to keep games close, he is basically saying his team sucks and the best thing he can do is rely on luck.
Jeff,
Could the "secret" ability of Manning/Brady to eek out those extra possessions have to do with how much no huddle they both run? I'll admit right now that I don't have any the numbers to back up my assertion, but I'm pretty sure they run more no huddle than all of the other teams, which would allow them more control of the clock. I.e. the length of time they wait to snap the ball might not only be based upon reading the defense but also the time in the game if they want to run a quick drive to try and increase their likelihood of getting another possession at the end of the half.
Other QB's in 3-point games:
Big Ben (w/ both coaches including playoffs): 17-10
All including playoffs for Favre
Favre/Holmgren: 9-12
Favre/Rhodes: 2-2
Favre/Sherman: 14-10 [58%]
The most interesting part about this pairing was that they went 1-5 in 3 pt games in Sherman's last year, and after which he got fired. So in the year's prior they were 13-5 [72%].
Favre/McCarthy: 3-2
Favre in his GB years: 28-26
Sorry derbyguy, got tied up for a few days. Great points I think. The ability to run a no-huddle offense well might give you a chance to manipulate drive times...and would certainly allow a proficient team to have a very fast drive on a final possession that would be the equivalent yardage-wise of a normal drive for other teams...
Post a Comment
Note: Only a member of this blog may post a comment.