The following is a post by @sunset_shazz.
We have previously discussed in these pages the principal-agent problem, and how idiot principals (aka NFL owners) are to blame for suboptimal decision-making by agents (aka GMs and coaches). In an entertaining Twitter thread, some analytics nerds were discussing the use of RPOs and the implications for run/pass decision making. A turning point in the thread:
Josh is citing this piece of descriptive analysis from Brian Burke back in 2014, where Burke showed (from a positive, as opposed to normative standpoint) how coaches tend to maximize the success rate of each individual play, which doesn’t necessarily map to game-level success (“just win, baby”).
I replied with my time-worn take that this suboptimal decision-making is the fault of the principals, who have set up bad incentive structures. Sean Domnick asks an excellent question: why did the coaches choose this particular strategy? It’s not as if the owners incentives are particularly clear in favoring per-play success rate:
The hypothesis below is drawn from social science, and is speculative.
Decision theory is a very young field. My Decision Sciences professor in the nineties came from Xerox PARC and leaned heavily on Kahneman, Slovic and Tversky. But prior to formalized decision theory, Friedrich Hayek and Edmund Burke posited mechanisms by which traditional decision-making can evolve to be “good enough” over time. Conventional wisdom improves through an iterative, blind process. In the decision space, though traditional decision-making may be suboptimal, it can still be good enough to have adaptive success. Joseph Henrich has, using modern data collection methods, formalized how norms and traditions may evolve to be broadly efficient. Bill Walsh did not have access to modern data, software or computing power. But his analysis was good enough, and was dominant enough, to change how coaches stretch the field and emphasize short passes.
Yes, per-play success rate is suboptimal. But it is an evolutionarily stable strategy in that it is dominant, provided that nobody has implemented a better strategy. Moreover, to the extent that norm violators are punished (cf. Henrich) herding behavior is a rational response to the threat of being fired. Thus, a “good enough” strategy such as per-play-success-rate maximization will perpetuate because everybody is doing it.
1) Maximizing success rate, though suboptimal for maximizing wins, has the advantage of being an easy, tractable heuristic that dominates over other, worse, decision rules.
2) Over time, principals and agents will coalesce around this (arbitrary, though minimally successful) decision rule, which becomes self-perpetuating.
Is this equilibrium stable? It is until it isn’t. Just as Roger Bannister showed the world what is possible, it takes only one coach to show that sound evidence-based decision-making will dominate the conventional wisdom. We are at a moment in time when internet nerds are mining data to show the traditionalists there is a better way. My tentative prediction is that NFL coaching heuristics will change for the better, just as they have in the NBA and MLB.