Sunday, January 9, 2011

Completion rates aren't the complete story

by Denis O'Regan

In a follow up to my post about interception rates for quarterbacks corrected for air yards per throw, I've taken a similar look at completion rates.

A completion rate taken in isolation can be a very misleading indicator of a player's real skill level. For example Michael Vick's average completion rate for his last season in Atlanta and his first year in Philadelphia was just 58%. By contrast the 2006 version of David Carr had a success rate of over 68% and tied the record for most consecutive completions in a game. Two rates at polar extremes, but whereas Carr's passes on average only travelled just over 5 yards per attempt, Vick's went nearly twice that distance.

To try to introduce air yards per attempt into the equation I firstly regressed air yards against completion rates for ever primary starting QB since 2006. This produced a fairly smeared out scatterplot and although increased pass length per attempt reduced the completion rate, correlation was very low at 0.09.

I therefore used the results I'd obtain from looking at interception rates corrected for distance thrown through the air. I chose the 30 worst QBs at avoiding interceptions from the previous 5 seasons, surmising that an inability to avoid picks indicated an underlying inaccuracy. This time the scatterplot was much tighter and the correlation between average distance thrown in the air per attempt and completion percentage was 0.44

Completition rate for "inaccurate" QBs was

87.66 minus (average air yards per attempt *3.32)

I then inputted the average air yards for all other QBs from those years into this equation and compared their predicted completion percentage with their actual figure. 25 QB's had completion rates which exceeded the rate predicted for them by the regression equation for inaccurate QBs. Almost every elite QB over the last 5 years made this list.

I therefore repeated the regression of air yards per attempt against actual completion percentage for this set of elite passers.

Again the correlation was much improved, relatively strong (0.43) and the equation for elite QBs was
completion%=86.60 minus (ave. air yards per attempt*2.74)

We now have two regression equations which allow us to relate air yards per attempt to completion rates, one for poor QBs and one for elite ones. So we'll revisit the question of Vick verses Carr.

David Carr threw an average of 5.5 air yards per attempt for his 68.3% completion rate. If we plug his air yards into both equations we find the "poor" QB equation predicts he should have thrown completions at 69.4%. The elite equation predicts a rate of 71.5%. Carr's actual completion% is much closer to the one predicted by the "poor" QB equation. From that we can conclude that he was actually an inaccurate QB, despite his very high completion rate. He didn't pass as well as he should have done given the many, short passes he threw.

Vick, by contrast threw on average 10.6 air yards per attempt. The equation for poor passers predicts he would complete at 52.5%, the elite one predicts 57.6%. Vick's actual % was 57.5%. That puts Vick very much in the accurate, elite passer category.

I'm currently looking to see if this approach can be used to predict a player's career path. Joe Flacco(2008) was sandwiched half way between being a elite QB or an inaccurate QB judged on his actual completion % and his predicted numbers from both equations. By 2010 he had moved very much into the former category.


Bruce D. said...


#1, could we not use the "elite" formula in all cases, and anything under is non-elite or bad, depending on the value?

#2, We(NFL analysis fans) need this for all QBs.

Who could provide such a thing?

Thanks for the read.


denis said...

Hi Bruce,neat idea.

the top five QBs who outperformed the "elite" equation by most,cumulatively over the five seasons were,in reverse order Rivers,Roethlisberger,Brady,Rodgers and Manning,P.
Because of his depth of throw Manning would have been expected to complete 62% of his passes to be considered a normal,elite QB.He was good for 66.5%.
So Manning should be considered the most accurate QB overall since 2006.

The bottom four were,last Anderson,then Russell,then Grossman and next Sanchez.An elite QB would have been expected to complete around 62% of the passes Anderson has been asked to throw,he made just over 53%.


Peter Stimpel said...

Just a question, and blame my bad english if it was answered already, or if it is stupid:

The shown formulas don't take into account, if the QB or the Receiver broke the pass, right? Just plain complete/incomplete - without checking the reason. Thanks for confirmation.

Regards from Germany


Post a Comment

Note: Only a member of this blog may post a comment.