Point Total Analysis with Singular Vector Decomposition

by Oberon Faelord

Any football fan can tell you how well a team did in a game. If they scored more points than the other guys, they won. Great.

That's really only the surface though. There are at least three follow-up questions, which are all a lot harder:

1) Who really played better? Remember, football involves luck and circumstance. Did the winning team really "deserve" it? Did they really play better than their opponents?

2) Who will play better next week? Even things that aren't luck (say, causing a fumble) may not be a repeatable skill. Even worse, everything about a team's performance is in the context of their opponent. Just because a team was mediocre, it doesn't mean they won't be great next week (against substantially worse competition).

3) Why did a team win? Defense? Offense? Running game? Pass rush?

Questions 1 and 2 have been heavily researched, but question 3 has not. Why? Because it's really, really hard. This will be the very first step of an attempt to answer that question.

First off, the challenges. Any statistical analysis is only good as its input data, and the input data for this analysis is going to be...bad. Really bad, in truth. The only input is the final scores from a season of games. No play-by-play, no yardages, no information about when a game happened.

Another challenge of the approach in this analysis is that it provides numbers, but no names. For example, Miami had an Offensive Factor 1 of 4.03 in 2006. What's "Offensive Factor 1"? Good question.

So, with those caveats, what is the approach of this analysis? It's based on Singular Vector Decomposition. More specifically, it's based on the gradient descent SVD-alike solver described at http://sifter.org/~simon/Journal/20061211.html . The inputs are the scores for each game. The two sets are offenses and defenses (as opposed to movies and users, as used in that article). As an example, lets look at the 2007 numbers.

The gradient descent solver has two numbers that affect its accuracy. The first is the number of "features" being calculated. A feature is an aspect of an offense or defense. They each apply to both offenses and defenses. Each score is calculated by going through each feature, multiplying the offense's value by the defense's, then adding all these numbers up. One effect of this is that high numbers are good for offenses, while low numbers are good for defenses. One aspect of the algorithm is that each feature is less important than the previous one, so there are diminishing returns in raising the number.

The second important number is how many refinements are made to each feature before it's considered "done". Once again, this has diminishing returns, as the numbers converge over time. For this analysis, 25 features were used, and 2500 refinements were made to each.

So what are the numbers? First, let's look at what adding additional features does. These are the average amount the algorithm is off on each score after each feature is added:

6.867
5.848
4.864
4.029
3.299
2.790
2.341
1.991
1.645
1.334
1.091
0.896
0.781
0.670
0.579
0.489
0.391
0.341
0.305
0.267
0.226
0.205
0.178
0.165
0.161

To put this in perspective, Feature 1 is accurate enough to get each score of each game within a touchdown. By the time the last feature is added, the formula is less than a 5th of a point off, on average. So what does this formula look like? Here are the first (and most important) 5 features (these alone will get within 3.3 points on average):

		1		2		3		4		5
	off	def	off	def	off	def	off	def	off	def
ARI	4.63	5.54	-0.02	0.53	-0.22	-0.79	-0.33	0.31	3.32	-2.14
ATL	3.75	5.26	-0.79	1.74	3.38	-1.56	-4.93	0.74	-0.05	0.82
BAL	3.64	5.28	2.38	-1.05	-0.98	-0.18	-0.82	-4.31	-1.41	0.3
BUF	3.05	4.22	0	-0.95	0.14	-2.82	-0.2	1.04	10.56	-0.7
CAR	3.74	4.48	-5.14	-1.13	0.55	1.35	2.54	1.1	-0.61	-2.25
CHI	4.69	4.75	0.19	5.53	-1.35	-0.81	-0.45	-0.04	1	0.71
CIN	5.34	6.13	6.98	0.19	-2.11	1.27	-0.01	-4.51	0.42	1.36
CLE	5.55	5.75	3.79	0.54	2.85	-0.7	-0.37	-2.62	4.79	-1.52
DAL	6.46	4.64	1.57	-0.86	0.5	-2.62	-0.55	0.15	1.01	0.69
DEN	4.26	5.45	2.16	0.23	-0.42	-6.25	3.2	0.22	1.2	-0.2
DET	5.38	5.84	0.62	-9.94	-1.62	1.9	-7.67	-0.55	0.28	1.78
GB	6.12	4.06	0.3	0.08	2.13	0.28	-1.46	-0.94	1.05	-0.05
HOU	4.63	4.89	-2.47	-0.9	-0.41	2.01	1.34	0.71	0.17	-0.51
IND	6.17	3.38	-1.43	1.18	-0.59	0.53	-0.08	0.67	-1.16	-1.2
JAX	5.92	4.14	-7.89	0.27	0.79	-4.17	0.18	-1.25	0.91	0.22
KC	3.08	3.74	-0.21	0.65	1.54	0.21	-1.53	-0.4	0.33	5.43
MIA	3.83	5.33	-0.02	1.81	-1.51	2.33	-1.56	6.18	-1.87	-0.33
MIN	4.93	4.18	1.04	1.09	2.04	-1.24	4.12	0.8	-1.59	2.41
NE	7.32	3.54	-1.87	0.14	-0.58	0.06	2.21	-9.34	0.35	-0.59
NO	4.87	5.88	0.42	1.65	-2.62	0.9	-7.26	1.45	-0.25	0.99
NYG	5.13	4.28	0.15	8.87	-4.26	2.44	-0.07	1.51	-3.11	0.74
NYJ	3.53	5.04	0.35	0.83	3.96	-2.75	-0.04	-0.99	-0.55	-0.25
OAK	3.95	5.25	-0.16	-1.51	-0.33	0.86	2.11	-0.69	-0.74	0.1
PHI	5.18	4.08	-1.54	0.17	1.4	-0.69	-0.93	0.16	3.48	0.29
PIT	4.75	3.86	-10.29	-0.02	-0.67	2.41	-0.13	4.74	0.69	-0.33
SD	5.5	3.62	-1.97	0	1.17	6.6	0.69	-0.63	-0.01	-3.48
SF	2.46	4.96	0.33	-1.45	-0.25	-0.01	0.99	-0.06	-1.3	-1.37
SS	5.02	4.92	-0.48	0.03	-8.71	3.72	0.16	-2.04	0.34	-0.62
STL	3.45	5.61	7.16	-1.29	1.12	0.08	2.03	0.09	-1.55	0.64
TB	3.91	4.13	0.15	-1.22	-2.07	-0.24	2.74	0.64	-0.26	-2.07
TEN	3.63	3.92	1.45	0.94	0.69	-0.21	3.62	0.62	1.24	11.37
WAS	4.4	4.23	-0.36	-10.47	-1.23	-0.28	0.5	1.98	4.09	0.09

That first column is the team. Each feature is numbered, with the offense/defense numbers pairs off for each team. So Atlanta's Defensive Feature 3 is -1.56, for example.

So is this a real pattern? Absolutely. With 25 features from 2002-2007, the average error per game-score is:

2002: 0.167
2003: 0.155
2004: 0.166
2005: 0.152
2006: 0.164
2007: 0.161

So, what's left do to?

1) Get better inputs. Final game scores are interesting, but who knows what factors affect them. This ties into the other questions in the introduction to this analysis: numbers that better explain what happened, and what *will* happen, would be more useful. However, because individual matchups are taken into account by this algorithm, it's key that each number be per-game, and not adjusted for opponent.

2) Use a better implementation of the algorithm. A number of refinements suggested by the NetFlix researcher (cutting off out-of-range values and non-linear per-feature output being the most obvious).

3) Figure out what these features are. In theory, they should correlate with something real and observable about each team. It's likely that this would be more obvious with better numbers (produced by improvements 1 and 2).

Full inputs, outputs, and code available upon request.