How does weather affect a QB's QBR

By Krishna Narsu

A few years ago when I interned with ESPN, I had the pleasure of meeting the ESPN Analytics team. This was back when Total QBR was first being rolled out. After listening to a presentation the team gave on QBR, I became a fan of the metric. One of the things I was curious about was how weather impacted QBR. Does QBR go up in domes? Does QBR go down when it’s really cold? Dean Oliver, one of the creators of the statistic, was nice enough to send me the QBR data and I obtained the weather data by scraping the NFL gamebooks. I completed the study a few years ago but never thought of posting the results. Now, with the Green Bay-SF game expected to be bone-chillingly cold, I'm putting this post out.

When I conducted the study, I used an ANOVA to test if two samples of different weather data were significantly different. For example, one of the tests was rain
vs. no rain. I looked at a number of different categories: rain, hot/cold, wind, wind chill, domes, etc. Here were the results:

	QBR	Count
Rain	42.19	42
No Rain	50.79	2074
Dome	51.74	326
Outside	49.67	1524
Road Dome	52.12	163
wind less than 20 mph	50.76	2016
wind greater than 20 mph	48.27	74
wind greater than 25 mph	44.53	28
wind greater than 30 mph	43.04	16
temp less than 32°	49.5	140
temp less than 32° home	49.6	70
temp less than 32° road	49.39	70
wind chill less than 32°	49.34	244
wind chill less than 20°	49.31	88
wind chill less than 10°	45.6	38
wind chill less than 0°	54.59	14
temp greater than 80°	52.8	140
temp greater than 80° Home	51.69	70
temp greater than 80° Road	53.9	70

The first column is the weather type, then the average QBR and the Count is the number of games in that type of weather. I’ve highlighted the only types of weather that had statistically significant differences in average QBR and that was rain versus no rain.

What determines the drop in QBR in Rain? I summed up the EPA (Expected Points Added) for INTs, sacks, fumbles, etc. and divided that by the number of games to get an average EPA per game for INTs, sacks, etc. (not sure if there's a better way to look at this). I found that there was a more negative YAC EPA lost, a higher sack EPA lost per game and a higher Fumble EPA lost per game but less EPA lost due to INTs in rain. I would guess that when it’s raining, QBs become more conservative (by trying to lower the amount of INTs they throw by taking less chances) and take sacks instead, which may lead to more fumbles. Also, I would imagine that offenses throw shorter passes in the rain, maybe explaining the more YAC gained for WRs.

A few more comments: When I looked at dome QBR vs. outside QBR, I noticed there was no statistically significant difference but considering a lot of the games played in domes were biased because Matt Ryan and Drew Brees play all their home games in a dome, I decided to look at road dome QBR vs. outside QBR. It turns out that the average QBR for QBs playing on the road in domes was actually higher than it was for QBs playing at home. However, the difference was still not statistically significant. Furthermore, there are no advantages for the QB at home in a dome as road QBs actually had higher QBRs than home QBs.

We also see that playing in hot weather can help QBR and while it wasn’t statistically significant, it was very close (hypothesis test: 80+ degree mean QBR > than control group mean QBR, one-tailed test, p=.18). If we remove dome QBR from our control group and look at just games outside where the temperature was >80 degrees versus <80 degrees, we find that the differences in means are statistically significant at the alpha level of .10 (p=.09). Another interesting aspect of the study was that if you look at QBR as wind increases, we can see a noticeable difference. Unfortunately, when we get up to wind more than 30 mph, we only have 8 games where that took place. None of the categories for wind were statistically significant but for winds >25 mph and 30 mph, it was close (p=.13). While games played in high wind conditions don’t happen very often, I think it’s safe to say that high winds do have an effect on QBR.

Also, I’m sure we all noticed that when the wind chill is less than 0 degrees, the average QBR is almost 55. This is an excellent example of a case where we don’t have a large enough sample size (7 games in total, 14 observations including home and road QBR). Perhaps including the last two years of data would help with this specific case but more likely, we’d need about 10+ years’ worth of data because games where the wind chill is under zero degrees simply don’t happen very often.

However, those of you who are keeping tabs on the weather in the upcoming 49ers-Packers game are probably aware that the game is supposed to have wind chills under zero degrees, with a game time temperature under zero degrees as well. So with that in mind, what kind of QBRs can we expect for Aaron Rodgers and Colin Kaepernick given the last seven games where the wind chill was at or below zero degrees? Below is a list of those games with their weather conditions based on the official NFL gamebooks:

Game	Date	Time Est	Temp	Wind Speed	Wind Chill	Direction	Home QBR	Road QBR	Weather
GB@Chi	12/22/08	8:40	2	9	-13	WSW	13.7	72.99	Cloudy
Mia@KC	12/21/08	1:00	10	20	-12	NNW	67.36	81.68	Sunny, Breezy
Pitt@Cle	12/10/09	8:21	15	25-48	-6	W-SW	59.23	8.59	Cold
Hou@GB	12/7/08	1:02	3	3	-3	SOUTH	70.74	94.65	Mostly Cloudy
Ind@Buf	1/3/10	1:02	12	12	-2	WNW	73.89	14.31	Snow, 4-6 inches possible, Winds gusting to 25-30 MPH
Jax@Cle	1/3/2010	1:03	16	20	-1	WEST	65.04	46.59	Snow Showers
Cin@Cle	12/21/08	1:03	18	26	0	SW	0.7	94.82	Sunny/Cold

Rodgers has actually played in a pair of these games back in 2008. As we see in the table, his QBR was around the low 70s (if the QBR is slightly different from the ESPN game logs, keep in mind QBR has gone through some adjustments over the years since I completed this study). Still, two games certainly aren’t enough to tell us what Rodgers will do. And ultimately, neither are 7 games. However, it is expected to be fairly windy and we do know that high winds have an effect on QBR. Given that, it’s likely both Kaepernick and Rodgers won’t perform up to their usual standards on Sunday.

I will eventually update the study to
include the last two years of data but I don’t anticipate any significant
changes in the results since the sample of data I was originally working with
was more than adequate in terms of sample size (2008-2011).

Feel free to send any questions or
comments about this to me on twitter @knarsu3