What portion of “luck” is skill? (Maybe Atlanta IS that good!?)
by Bruce D
There have been some recent discussions at the Advanced NFL Stats main web-site concerning Atlanta and San Diego, with varying theories about how much luck is involved in their current win records. In a nut-shell, its why is San Diego's win record so much lower than their GWP, and why is Atlanta's win record so much better than their GWP. One of the prevailing theories is its due to bad and good luck(turnovers etc.). The other theory is poor or efficient coaching/skill.
Maybe its a combination of both. But what ratio? To try to answer this I looked for consistency with “luck” in each game, for each team this season.
First a little about how luck is tracked, here's an excerpt from a previous post.
“Team luck points = (bad luck points)-(good luck points), so negative numbers are the luckiest
For a more in-depth explanation of what "luck" points are, go to a previous post here.
Luck is tracked to better analyze a team's true ability and to help predict results of upcoming games where some may not know what portion of a team's record and points performance was due to just luck.
Luck points are valued as follows:
Points for(+) the unlucky team, are the same amount of points against(-) the lucky team.
punts blocked=3
interceptions=2.5
fumbles lost=2.5
field goal miss/block=2.5
punt returns for a TD=4.5
ko returns for a TD=4.5”
So now that we know how luck is tracked, and using the 2 teams mentioned as an example, this is what was uncovered.
First for San Diego:
| Team | Week | Luck Points | |
|---|---|---|---|
| sdg | 1 | 7 | |
| sdg | 2 | -4.5 | |
| sdg | 3 | 16.5 | |
| sdg | 4 | -2.5 | |
| sdg | 5 | 13.5 | |
| sdg | 6 | 2.5 | |
| sdg | 7 | 10 | |
| sdg | 8 | 5.5 | |
| sdg | 9 | 2.5 | |
| sdg | 11 | 2.5 | |
| sdg | 12 | -12.5 | |
| sdg | 13 | 5 | |
| sdg | 14 | 5 | |
| sdg | 15 | -2 |
Next for Atlanta:
| Team | Week | Luck Points |
|---|---|---|
| atl | 1 | -2.5 |
| atl | 2 | -5 |
| atl | 3 | -7.5 |
| atl | 4 | 0.5 |
| atl | 5 | -5 |
| atl | 6 | -5 |
| atl | 7 | 2.5 |
| atl | 9 | -0.5 |
| atl | 10 | -5 |
| atl | 11 | -2.5 |
| atl | 12 | -2.5 |
| atl | 13 | -2 |
| atl | 14 | -2.5 |
| atl | 15 | -5 |
This first thing you may notice is that Atlanta's week to week luck seems smooth and consistent, where San Diego's jumps around erratically.
The next step was to measure variance in a statistical manner, so I went to wikipedia here, and ran the variance formula for each team this season. The results are below.
| Team | Luck Points | Variance |
|---|---|---|
| nwe | -60 | 56.1 |
| atl | -42 | 6.2 |
| phi | -37 | 12.9 |
| pit | -33 | 33 |
| tam | -25 | 6.3 |
| chi | -21.5 | 22.9 |
| kan | -19.5 | 10 |
| cle | -17.5 | 26.5 |
| ari | -11.5 | 34.9 |
| bal | -9.5 | 29.7 |
| nyj | -9.5 | 36.9 |
| buf | -9.5 | 12.6 |
| ten | -7.5 | 57.5 |
| ind | -5.5 | 39.1 |
| gnb | -5.5 | 17.8 |
| oak | -0.5 | 31 |
| stl | 1.5 | 21.5 |
| hou | 2 | 27.2 |
| sfo | 4 | 28 |
| dal | 6.5 | 40.4 |
| det | 9.5 | 25.5 |
| nyg | 9.5 | 40.4 |
| nor | 12.5 | 32.5 |
| sea | 13 | 48.2 |
| was | 13 | 34.2 |
| den | 14.5 | 26.7 |
| car | 22 | 16.5 |
| cin | 27 | 34.2 |
| jac | 32 | 27.4 |
| min | 32 | 21.6 |
| mia | 47.5 | 37.3 |
| sdg | 48 | 49.4 |
Now all kinds of things pop out. Atlanta and Tampa Bay have a very low variance of luck. Is it good coaching, is it skill? I'm starting to think a lot of it may be.
San Diego and New England have the lowest and highest amount of luck respectively and they both have high variances of that luck. So should we consider that's its mostly due to luck?
I'm not a trained statistician, so I don't know what a 56.1 vs 6.2 variance really means, but it's easy to see that one is a lot, and the other is small.
If anyone has the time to comment in layman's terms what the variance numbers might represent (percentages?, avg fluctuations etc..?) it would be appreciated. For now I just assume its an indicative value.
In closing, I'd like to say, I now think Atlanta IS that good.
Posts
11 comments:
Bruce-Really interesting. Strong case for ATL.
The square root of the variance is the standard deviation among games, in points. The standard deviation is basically the amount of variation among the games.
So a team with variance of 56.2 has variation of (crudely speaking) +-7.5 luck points from one game to another; a team with variance of 6.2 has variation of +-2.5 luck points from game to game.
I like this. Seeing how lucky a team is with their luck.
A lower variance (or st. dev.) would imply that the luck is roughly the same from week to week. If you get the same amount of luck from week to week, that's not luck.
Andrew,
Including your standard deviation,
Team Luck
points Varianc Standard
deviation
nwe -60 56.1 7.5
atl -42 6.2 2.5
phi -37 12.9 3.6
pit -33 33 5.7
tam -25 6.3 2.5
chi -21.5 22.9 4.8
kan -19.5 10 3.2
cle -17.5 26.5 5.1
ari -11.5 34.9 5.9
bal -9.5 29.7 5.4
nyj -9.5 36.9 6.1
buf -9.5 12.6 3.6
ten -7.5 57.5 7.6
gnb -5.5 17.8 4.2
ind -5.5 39.1 6.2
oak -0.5 31 5.6
stl 1.5 21.5 4.6
hou 2 27.2 5.2
sfo 4 28 5.3
dal 6.5 40.4 6.4
det 9.5 25.5 5
nyg 9.5 40.4 6.4
nor 12.5 32.5 5.7
sea 13 48.2 6.9
was 13 34.2 5.8
den 14.5 26.7 5.2
car 22 16.5 4.1
cin 27 34.2 5.8
min 32 21.6 4.6
jac 32 27.4 5.2
mia 47.5 37.3 6.1
sdg 48 49.4 7
Looks so much easier to understand. Thanks a lot.
crap, didn't format properly.
The problem with this kind of thinking is that you're asking whether there is a significant difference in a variance. To figure that out, you need the variance of the variance! The error on the error. Working that out would require a sample size so large as to be completely infeasible.
A more sane way of addressing this is to look at the model. The question being asked is basically whether the luck factor (things not included in the model that are assumed random) should actually have been included in the model. This gives you a much larger statistical base; you can use all games over all time instead of just this season's Atlanta results. Unless there's some identifiable problem with the model, you have to assume that luck is luck. It might not be, but you could never prove it...
The fractional variance on the variance is essentially Sqrt(2)/Sqrt(N), which I imagine is about 40% for this sample. So I think you'd be able to say 6 and 56 are different with some statistical confidence; 37 and 56 not so much.
How were the luck-point values assigned? ad hoc or based on some EPA-type study?
Andrew,
The points are very ad-hocish after studying Brian's EPA values for various situations.
The data I have doesn't include where or when a fumble happens etc, so its sort of averaged out.
How do you establish it wasn't random chance that produced Atlanta's low variance? 15 games seems a low sample size.
Anon,
There probably is no way to establish its not random.
IF part of luck is skill, this MIGHT suggest Atlanta is more consistent than most at the skill portion.
Post a Comment
Note: Only a member of this blog may post a comment.