Nick Foles Is The Playoff GOAT

The following is a guest post by @sunset_shazz.

Nick Foles is a high-variance quarterback. His performance ricochets from abysmal to sublime with such frequency that he made me re-adjust my chart axis, twice. And yet: including the 2013 loss to the Saints (in which he engineered a comeback from a 13-point deficit and left the field with the lead) his postseason play has been consistently excellent. There have been 93 quarterbacks since the 1970 merger who have played at least 4 playoff games. Of these, Foles ranks 1st in completion percentage and 2nd in Adjusted Net Yards / Attempt (ANY/A).

Obviously, this is not statistically dispositive. Nothing about playoff analysis is. Mark Messier and Reggie Jackson’s playoff performances comprised a mere fraction of their total careers, yet their knack for elevating their game on the biggest stage is what made them memorable. One way to think about the playoffs: there is a tide in the affairs of men, which, taken at the flood, leads on to fortune. As I will show, Foles has taken the tide at the flood in historic fashion.

Note, from the chart above, that the fewer games played, the greater variance in ANY/A between individual players. But what about each player’s game-by-game variance? I measured the standard deviation of each player’s game ANY/A, and scaled this by his mean ANY/A, thus constructing a coefficient of variation.

Of all 93 QBs in the sample, Foles has been the 4th most consistent (i.e. has the 4th lowest variation). Moreover, he has the lowest variation of the 16 QBs who have only played 4 games.

Perhaps Foles has benefitted from playing in a QB-friendly era? I compared each QB’s game ANY/A to the league average for the year in which that game was played. One can then plot mean Relative ANY/A against the coefficient of variation:

Foles has the 5th highest Relative ANY/A in addition to having the 4th lowest variation. One way to think about the above graph is to imagine an “efficient frontier” on the upper left quadrant. When considering similar efficient frontiers in the context of financial economics, Nobel Laureate William F. Sharpe constructed a “Sharpe ratio” which compares a fund manager’s relative return (e.g. versus an index) to the standard deviation of the fund’s return.

I similarly devised a playoff QB Sharpe Ratio, which is each QB’s mean Relative ANY/A divided by the standard deviation of his game ANY/A. Think of it as one number which captures both efficiency and consistency of play. The following table shows the top 10 playoff QB Sharpe Ratios since the merger:

All 10 of these quarterbacks played in a Super Bowl, and all but two of them were champions. Only Bengals starter Ken Anderson and Bills backup Frank Reich did not win the season’s final game. (Reich, of course, will receive a ring as Offensive Coordinator of the 2017 Super Bowl champions.)

By this metric, Foles will have to settle for second place out of 93 playoff QBs. The Raiders’ Ken Stabler, who played in 13 playoff games between the 1971 and 1979 seasons, passed for 3.08 ANY/A above average (3rd) and had the 8th lowest coefficient of variation in the sample. Combining efficiency and consistency, he is the greatest playoff quarterback of all time. Here are the rankings of some other notable QBs, and Eli Manning:

Obviously, I’m not suggesting Foles is better than any of those quarterbacks (except Eli; he’s indisputably better than Eli, it’s not even close). However, in the inherently limited sample that consists of the playoffs, Foles has performed at a historically great level, in terms of both efficiency and consistency. Also, he can catch.

Clearing Up The Coaching Confusion

The following is a guest post by @sunset_shazz.

Should the Carolina Panthers have fired Head Coach Ron Rivera or traded QB Cam Newton the day after they lost the Super Bowl? Scott Kacsmar at FiveThirtyEight argues they should have done either or both. Do read the whole piece; the argument is presented as follows:

• In NFL history, only 4 coaches have won their first Super Bowls after 5 seasons on the job with the same team;
• No team has ever started the same quarterback under the same head coach for more than 5 years and seen that duo win its first championship.

Having examined the history of prior first time Super Bowl winners, FiveThirtyEight infers that these characteristics are conducive to winning championships. The study’s conclusion: “If championship success doesn’t come within five years, things tend to get stale, and someone eventually has to move on from their position of power.”

Can you spot the flaw in this reasoning?

How about if I used the same exact logic, using a more emotionally salient characteristic:

• In NFL history, only 4 minority head coaches have won Super Bowls. Therefore you shouldn’t hire minority head coaches. [1]

Does that framing device make the flaw in reasoning clearer?

FiveThirtyEight’s study suffers from the confusion of the inverse, a statistical fallacy that undergraduates are commonly taught to avoid. One of the best recent treatments of this problem was a brilliant piece by Katherine Hobson on the lab-testing startup Theranos  (also, funnily enough, at FiveThirtyEight). Chapter 8 of Nate Silver’s excellent The Signal and the Noise provides a lucid discussion on this topic, in the context of Bayes’s theorem.

Here is the issue: the fraction of Super Bowl winners that possess a certain characteristic, by itself, tells you nothing about the probability that those who possess that characteristic will win a Super Bowl. A better way to estimate the latter would be to go back and examine the historical success rate of coaches who possess the characteristic you’d like to study.

I compiled every season coached since the 1970 merger, then excluded the seasons after a coach has won his first Super Bowl.  Coaches who were tenured 5 years or fewer with their teams won 24 first Super Bowls in 1009 opportunities, for a success rate of 2.38%. Coaches tenured 6 or more years won 4 first Super Bowls in 176 opportunities, a 2.27% success rate. Using a technique previously used in the Duck Bias study, I applied the cumulative distribution function of the binomial distribution to test whether the success rates were different, to a statistically significant degree. The P value of 0.592 indicates no statistically significant difference.[2]

However, Super Bowl success is a noisy, sparse data set, due to the very small sample size. An alternative measure of coaching success which enjoys the advantage of more data is the frequency with which a coach makes the playoffs. I compiled the playoff rate for every coach in the dataset, and compared this with the base rate of success for that year.[3] The data shows that coaches with longer tenure are actually ­­more likely (47.7%) to make the playoffs than shorter tenured coaches (31.1%) and the base rate (38.0%); both of these differences are statistically significant.

Obviously, this data doesn’t tell you anything about causation. There is likely a survivorship bias / selection effect: those coaches who are kept by their team after 5 years without a championship are likely of higher quality than average, which is probably why their subsequent success rate is higher.

For Coach-QB pairings, 28 Super Bowls were won in 1137 opportunities for the short tenured pairs, a 2.46% success rate. There were only 48 seasons where a Coach-QB pairing lasted more than 5 years without having won a Super Bowl. The zero success rate is, statistically speaking, the effect of randomness, rather than a measured effect. In terms of playoff success rate, once again the longer tenured coaches had a higher success rate, though this effect was not found to be statistically significant.

The data is pretty clear: you shouldn’t fire your coach, or your QB, just because he has not won a Super Bowl after an arbitrary number of years. The only reason short tenured coaches seem to have been historically more successful is they vastly outnumber the long tenured ones. FiveThirtyEight’s model was fooled by the fallacy of the inverse.

But given that we’re in the midst of a coaching carousel accompanied by a Rooney Rule kerfuffle: what about the reductio ad absurdum argument I cheekily proffered above? What does the data say about minority head coaches?

I used Wikipedia’s Rooney Rule page to code every minority head coach in the dataset, presented below. This data comprises every coaching season between 1970 and 2016, including all seasons for coaches who won multiple Super Bowls.

Minority coaches won Super Bowls in 3.31% of their opportunities, which is statistically indistinguishable from the base rate of success of 3.22% (note that minorities have disproportionately coached in more recent years, after the league has expanded, which lowers the base rate of championship success). Interestingly, minority coaches made the playoffs 58 times in 121 opportunities (47.9%) which is 11.6 more times than one would expect given the base rate, a difference that is statistically significant (p=0.02). This is a noteworthy result: historically, the presence of a minority head coach is associated with a 25% greater rate of making the playoffs.

Again, one shouldn’t make causal inference claims from historic data. I’m not arguing that minorities are inherently better coaches. In this situation, there is no survivorship bias. Might there be a selection effect? The late Nobel Laureate Gary Becker argued in 1957 that employment discrimination (racial or otherwise) is inefficient. Not only does the victim of discrimination bear a cost, but so does the discriminating employer (through lower productivity per unit labor cost). Axiomatically, to the extent that some employers exhibit an unfounded bias, an employer who doesn’t discriminate can capture a portion (but not all) of the foregone surplus. Moreover, selecting from a pool of employee candidates who are the victims of racial discrimination will yield supernormal productivity. Becker’s theory of discrimination is one plausible explanation for the effect shown by the data.

What does this mean for NFL teams today? The data is unequivocal that Ron Rivera shouldn’t be fired solely because he hasn’t yet won a Super Bowl. The data also shows that Al Davis, who got many things wrong, got a few things very right. An NFL team should examine the pool of minority head coach candidates very carefully, and should strongly consider hiring from this pool.

Not merely because it’s the right thing to do, but because the data suggests it helps you Just Win, Baby.

[1] To be clear, this is not argued by FiveThirtyEight. I employed this reductio ad absurdum to permit the reader to more easily intuit the confusion of the inverse.

[2] The P-value is the probability that, conditional on the null hypothesis being correct (i.e. no effect), one would observe the data in question by chance. Though subject to recent debate, the conventional standard for social science is to reject P-values greater than 0.05.

[3] The base rate of success is # of playoff teams / # of total teams in the league, both of which have changed over time. I accounted for the league’s expansion of teams in 1976, 1995, 1999 and 2002, as well as the evolution of the playoff format from 8 to 12 teams in 1978 and 1990, and the 16 team playoff that occurred during the strike-shortened 1982 season.

There Are No Shortcuts To The Super Bowl

Last March, Jeff Lurie told reporters he was was tired of waiting for the Eagles to be great. He’d seen sustained success, he’d been to a Super Bowl, and he’d watched his franchise post a solid 20-13 record (including a playoff loss) over the prior two seasons. It wasn’t enough.

“I’ve lived through a lot of division championships, a lot of playoff appearances, a lot of final four appearances, but our goal is we want to deliver a Super Bowl,” he said at the time. “And sometimes maybe I’m influenced by the notion of it’s very difficult to get from good to great, and you’ve got to take some serious looks at yourself when you want to try to make that step. It’s a gamble to go from good to great because you can go from good to mediocre with changes, but I decided it was important enough…”

On Wednesday, after firing Chip Kelly before the final game of the season, Lurie didn’t walk away from his words earlier in the year.

“I said, with Chip’s vision, it was an opportunity that he wanted to lead the way, to try to go from good to great,” he said. “In fact, I remember saying to all of you, there’s dangers in that, in terms of having two 10-6 seasons in a row, and when making significant changes, you can easily achieve mediocrity. I think it would be a shame not to try, but… that is the danger when you take a risk.”

I hope Lurie learned more than that, because his "strategy" was little more than a desperate hope. He handed over all power to Kelly, a moderately successful coach with zero experience running the intricacies of a NFL organization. 

The road to the Super Bowl is not paved with such gambles, with an outsider making a bunch of questionable bets that luckily pay off. You don’t win by cutting talent and building #culture. You don’t win with a rigid set of measurables that dictate player acquisition. You don’t win by ignoring critical positions and spending excessive guaranteed money on less important ones. You don’t win by overpaying for a subpar quarterback coming off two major knee injuries, or turning over half the roster in one offseason. That’s how you end up 7-9 in one of the worst divisions ever.

There are no shortcuts.

Remember Andy Reid’s binder? Reid came to Philadelphia with a detailed multi-year plan of how to build an organization. He arrived in 1999, overhauled the roster, brought in an impressive group of experienced coaches (including 6 eventual head coaches), drafted a quarterback in the first round, and went 5-11. In 2000 the team and young quarterback improved, overachieving to reach the playoffs. By 2001 they were one of the best teams in the conference. In 2002 and 2003 they missed the Super Bowl by inches. In 2004 they came minutes away from the trophy itself.

Lurie said he doesn’t want the middle steps, he just wants the Super Bowl, and he was willing to gamble to get there. But those middle steps are important. That’s how you build a champion—not overnight, but consistently, step by step.

The best teams in the NFL have the best talent. The best teams in the NFL have a top quarterback they drafted and groomed. The best teams in the NFL have smart, experienced coaches who adjust their schemes to the players they have. The best teams in the NFL have a front office structure that empowers multiple voices and balances scouting with analytics and financial understanding.

Going into 2016, the Eagles need to avoid the quick fix, or the allure of competing for the playoffs in year one of a new regime. That won't set the team up for long term success. (Plus, they'll likely be in the running anyway unless the putrid NFC East changes significantly in a year.)

The blueprint needs to be for a Super Bowl contender in 2018. Let's lay out what that looks like...

Front Office: Howie Roseman has a mixed reputation among fans and league sources, but he can succeed in the Joe Banner role. He has experience on the personnel side to pair with a firm grasp of the league's economics. Roseman does need to find a qualified general manager-type to run player personnel, someone less washed-up than Tom Donahoe, with more experience than Ed Marynowitz, who's not obvious idiot Ryan Grigson.

Coaching: A NFL team is a crazy thing to manage, and any head coach needs to be able to bring in the right people and command respect across the organization. He doesn't have to be a brilliant innovator on the cutting edge, but he does need to be flexible enough to adapt his schemes and techniques to get the most out of his players in each situation. The single most important skill set within that is the ability to find and develop a franchise quarterback, which is what makes a candidate like Adam Gase so attractive.

Quarterback: You cannot be a consistent Super Bowl contender until you have a quarterback. As such, you should exhaust every possible avenue to get one, especially the draft. I stand by my recipe for QB hunting laid out four years ago, on the eve of Andy Reid's firing:

Draft a quarterback early and late. Sign somebody in free agency. Trade for a promising backup. Rinse and repeat. You're never going to be able to compete for the Super Bowl until you find your one franchise guy. Might as well cycle through as many potentials as you can until you do. The financial cost of doing so is less than the opportunity cost of sitting pat with one player, [Bradford], who is statistically unlikely to ever become an elite quarterback.

In the Eagles' case, that means avoiding any multi-year guarantee to a still-unproven quantity like Sam Bradford, and perhaps letting him go entirely depending on the cost.

Roster Building: Outside of quarterback, which takes priority over everything else, the offensive line is next on the list of must-haves. Though I don't know many of the names, Jimmy Kempski's mock draft would thrill me based on the selection of two quarterbacks and three offensive linemen. Lane Johnson and Jason Kelce are the only long term building blocks you can count on there. Meanwhile, don't waste resources on the rest of the offensive skill positions, none of which will matter much until the offensive line and quarterback are fixed.

The Eagles defense needs more talent across the board, but it also likely needs a scheme that better takes advantage of the players in house. Kelly wanted a two-gapping 3-4 system, but it would be nice to see what Fletcher Cox and company could do in a one-gapping 4-3 instead.

Overall: Both the organization and its fans need patience. We were spoiled by Chip's quick turnaround, but where did that leave us? Let's plan for sustainability.

Read more: How The NFL Chewed Chip Kelly Up And Spit Him Back Out

Long Road Ahead

The Super Bowl takes place this weekend, and the Eagles will not be participating. Breaking news, everyone.

But in all seriousness, this is the height of the sport, and the Eagles are far, far away from getting back there. Let's just take a moment to realize how far. Look at the 49ers roster and try to think of players on the Eagles who could start for the NFC Champs. It's a short list.

On offense, San Francisco has two quarterbacks who are better than anyone on the Eagles roster. They have an athletic freak of a tight end, and two Pro Bowlers on their mauling offensive line. 2011 Jason Peters could play for them, but not sure you take the 2013 version over Joe Staley. Evan Mathis, the Eagles' best offensive lineman in 2012, is probably a notch below former first round pick Mike Iupati. I would take LeSean McCoy over Frank Gore, certainly. And DeSean Jackson beats old Randy Moss. But that's all.

Moving to the defensive side brings an even worse comparison. The 49ers have two of the best linebackers in the NFL, Patrick Willis and Navarro Bowman. They also have two Pro Bowl pass rushers, Justin Smith and Aldon Smith, and both Pro Bowl safeties, Donte Whitner and Dashon Goldson. I love Fletcher Cox and maybe he can sneak in on the defensive line if we're being generous, but that's all.

That's three players -- make it four with Alex Henery over the dearly departed David Akers. You'll get the same results if you compare the Eagles to other championship teams, even the 2004 squad that made it to the Super Bowl. You wouldn't be able to sneak many 2013 Eagles on to that roster.

All of which is to say: this isn't a quick-fix job, it's a long-term rebuilding process. Set your expectations accordingly.

Photo from Getty.

Super Bowl or Bust!

Alec Nation, in his article “2012 Philadelphia Eagles Mirror 2004 Super Bowl Squad”:

If this Eagles team can harness just a fraction of the swagger that the 2004 squad had, then they will be headed in the right direction. Solid team defense combined with a potent offense will serve this team well. So far things look promising, just the way they did eight years ago.

Alternative title: “Bleacher Report Throws Lots of Sh*t Against the Wall.”

Against the Grain

Tommy Lawlor:

Should Andy Reid embrace Super Bowl talk? I don’t mean go all Rex Ryan and start talking about how we’re the best and definitely will win the SB. I’m talking more about openly using the phrase “Super Bowl” when discussing the season and goals…

Normally I don’t think this would be smart, but since Reid is going to be under big pressure this year…could this actually be smart? Rather than ignore the obvious, embrace it. Reid isn’t on a Super Bowl or bust mandate, but he’s not far off from that if you read between the lines of Jeff Lurie’s PC.

Counter-intuitive, but I like it. Perhaps a slightly better way would be to have Michael Vick and other team leaders spread the “Win one for the Gipper” message, instead of having it come from the top down. The players really seem to genuinely like and respect Reid, so a fiery campaign to prove he’s worth keeping could motivate everyone.

That said, if he pushes win-now mode hard and still finds himself with a losing record down the stretch, Reid could be out before Christmas.

Yes, Michael Vick can be ELIte

Les Bowen:

That last part was the most important; it was what separated the Giants from, say, the 49ers. Eli was sacked 11 times in the playoffs - more than any other quarterback. He still completed 65 percent of his passes, threw for nine touchdowns and was intercepted exactly once. His QB rating was 103.3. He averaged 304.8 passing yards per game.

Good luck copying that “blueprint,” NFL GMs. And Eagles fans, do you really see Michael Vick throwing for nine touchdowns and one interception?

To be fair, Vick had 11 touchdowns and 0 interceptions through his first eight games last season. So yes, I have seen it happen. Not only that, but just a year ago Eli Manning threw for a league-leading 25 interceptions.

Vick has an important offseason ahead of him, but there’s no reason to think he can’t rebound from 2011.

The Giants Won the Super Bowl

… with this touchdown.