Vince Lombardi

QuantCoach.com

No Monday Morning Quarterbacks

Follow QuantCoach on Twitter

THE CHALKBOARD

Myth Busters: Dear Nate Silver: The NFL Is NOT Resistant To Analytics

Let's get this straight all you SABR-metricians: The NFL embraced objective, data-intense, computer-based analytics before Moneyball author Michael Lewis entered elementary school and still does today.

Yet, at the same time pitchers and catchers were reporting for spring training this year, ESPN The Magazine published an article, "The Search for Intelligent Life," in which Nate Silver claimed the NFL "has failed for more than 20 years to adopt some of the most basic tenets of analytics."

The truth is the NFL adopted the basic tenets of analytics decades before MLB and the NBA even knew of the existence of a device known as a "computer," as is made indisputably clear in Tex Maule's January 29, 1968 Sports Illustrated article on Dallas Cowboys general manager Tex Schramm's pioneering adoption of analytics, "Make No Mistakes About It."

In America's Game, Michael MacCambridge succinctly summarized how Schramm toiled with analaytics.

“At the 1960 Olympics in Squaw Valley, Tex Schramm had been fascinated with the information generated by an IBM computer, which calculated statistics, medal standings, and record times," MacCambridge wrote in America's Game. "Frustrated during his time with the Rams about how long it took to wade through the avalanche of data on college seniors, Schramm was convinced that computer technology could be used to organize the scouting data, and more reliably isolate the most salient information….

"By 1964, Schramm was ready to implement a system that was a synthesis of past and future. It would incorporate the old-school notions of bird-dogging and beating the bushes with the Rams-inspired thoroughness that had become the norm in the league, while adding a third element, a new mind-set that attempted to quantify every opinion."

How far ahead of MLB was Schramm and the NFL in adopting analytics? Consider that in 1964:

* Oakland A's GM Billy Beane was a 2-year-old toddler.
* Moneyball author Michael Lewis was a 4-year-old pre-schooler.
* Bill James was a decade way from penning the first Baseball Abstract.
* Steve Jobs and Steve Wozniak were a dozen years away from inventing the Apple I computer.

Clearly, the NFL adopted the basic tenets of analytics decades before MLB did so.

But how does the NFL compare to the NBA?

********

Houston Rockets general manager and Sloan Sports Conference founder Daryl Morey was still eight years from birth when Schramm sprung analytics on the NFL at the 1964 NFL Draft.

In his ESPN article, Silver lauds NBA teams for increasing their application of an established play--the corner 3-point jump shot--in response to a rule change that improved the play's risk/return ratio. Obviously, given it's distance relative to other 3-point attempts, the corner jump shot is the shortest distance to the basketball goal that returns 3 points, as opposed to 2 points.

But adopting the highest percentage shot at the goal is nothing more than the hoops equivalent of what Bill Walsh started in Cincinnati in the 1970s and perfected in San Francisco in the 1980s in response to new NFL rules that made pass protection easier and made pass coverage more difficult.

"[B]y shortening—and timing—the passing game, Walsh reduced its two biggest risks: interceptions and incompletions," Lewis wrote in his book The Blind Side. "‘Our argument was that the chance of a completion drops dramatically over twelve yards,’ said Walsh. ‘So, we would throw a ten-yard pass. Our formula was that we should get at least half of our passing yardage from the run after the catch.’”

While Walsh may not have utilized traditional mathematics per se to reinvent NFL football, the first quarterback to run the new, new operating system--the Bengals' Virgil Carter--deeply immersed himself in probabilistic football analytics that were decades ahead of their time.

"Carter has set a literal new standard for the shopworn phrase 'student of the game,' for he is a computer analyst whose diligent research into football has led to findings that dispute some of the most sacrosanct coaching theories," Ron Reid wrote in an October 2, 1972 Sports Illustrated article, "Handy Pair of Brainy Bengals."

While working on his master's degree at Northwestern in 1970, Carter coded 53 variables in 8,373 plays from 56 games played during the first half of the 1969 season. It added up to over 440,000 bits of data that Carter then fed into the computer.

"Among other things, the study showed that in some cases field position was more important than ball possession, and that the laws of probability could determine an offensive team's scoring chance from any spot on the field," Reid wrote. "Moreover, probability could dictate an expected value to the offensive team's field position, which could be expressed in points."

NFL coaches eventually copied everything Walsh designed.

NFL coaches discarded Carter's probabilistic analytics.

Clearly, the question is ... Why?

Why did NFL coaches treat Walsh's play designs as super valuable products worthy of infinite copying and discard Carter's analytics as so much byproduct waste?

********

The answer rests in Silver's limited understanding of what he calls "elective choices--options that opponents have little or no choice of preventing."

The greatest play designers in football history became so by embedding elective choices in their plays. For example, in his famous power sweep, Vince Lombardi embedded elective choices known as ... wait for it ... option blocking, which Lombardi developed at West Point as an assistant to Red Blaik.

At the college level, Bill Yeoman's veer and Darrell Royal's and Emory Bellard's wishbone embedded three elective choices in a single play known as ... wait for it ... the triple option (dive, keep and pitch).

Likewise, Walsh embedded three elective choices in his short passing game (primary, alternate and outlet receivers) and the most famous play he ever designed, "The Catch," was called ... wait for it ... "sprint right option."

The elective choices embedded in today's "spread designs," such as the designs Chip Kelly brought with him to Philadelphia from Oregon, are direct descendants of the embedded elective choices in the designs of Lombardi, Yeoman, Walsh, et al.

Silver and others who allege the NFL has been resistant to analytics make no distinction between an embedded or endogenous elective choice--which can be imposed upon the opponent on first, second, third or fouth down from inside the play after the ball has been snapped--and far less powerful exogenous elective choices that can only be made from outside the play before the ball is snapped, e.g., a fourth-down choice.

But the difference between an endogenous embedded elective choice that is delayed until after the ball is snapped and an exogenous elective choice that must be made before the ball is snapped is enormous because the longer the elective choice is delayed the less time and opportunity the opponent has to counter-adapt and prevent the manifestation of the consequences of the choice before the whistle sounds and ends the play.

In other words, the longer a coach waits to impose an elective choice on an opponent, the more powerful that choice will be.

Macro-economist Paul Romer essentially captured this principle in his seminal paper on change, Endogenous Technological Change, when he found "patience is always undervalued."

Paul Romer is of no relation to and should not be confused with macro-economist David Romer, who has written that NFL coaches should almost always eschew patience when making elective choices on fourth down.

Like David Romer, Silver criticizes NFL coaches who refuse to eschew patience on fourth down.

Silver characterizes NFL coaches as collectively "gutless" on fourth down, which is an absurd characterization on its face, not to mention more than a little condescending, rude, and mean-spirited.

Silver speculates without a shred of data that NFL coaches stubbornly refuse to eschew patience on fourth down because the average NFL team has been owned by someone other than the coach for 34 years. Apparently, coaches of NBA teams that are owned by others for only 15 years are not saddled with this debilitating lack of impatience although Silver did not provide any data to support that conclusion either.

"This [NFL] is a culture that fosters extreme risk aversion," Silver wrote. "Going for it on fourth down is risky twice over: in the micro sense of staking more on the result of one play, and in the macro sense of defying custom and tradition."

Are NFL coaches really that risk averse?

********

Let's first talk macro.

It is true on the micro level that going for it on fourth down is risky in that it stakes more marginal risk on an individual play than on first, second, or third down because the offense risks losing possession without losing the ball on a turnover, but on the macro level it is Silver who is clinging to custom and tradition when he insists on judging NFL coaches solely on their exogenous fourth down elective choice-making.

Think about it, before they became legends, both Lombardi and Walsh defied custom and tradition.

Did the NFL punish Lombardi and Walsh for doing so? To the contrary, at the end for their respective Hall-of-Fame careers, the NFL named the Super Bowl trophy after Lombardi and adopted the video of "The Catch" as the video accompanying the NFL's copyright disclaimer.

Did the NFL punish Chip Kelly when the Philadelphia Eagles signed him to a $32.5 million contract before he ever designed a play in the NFL?

As the title of his seminal paper suggests, Paul Romer brought change "inside" the field of macro-economics by constructing mathematical representations in which change is the result of the intentional actions of people, such as research and development, and the new knowledge that results therefrom.

Like the designs of Lombardi, Walsh and Kelly, Paul Romer's model broke with and sought to supplant well-established conventional thinking: Nobel Prize winner Robert Solow's exogenous model of change.

While new knowledge--the engine of change and more efficient production--lay outside Solow's model, it was segregated in such a way that the contribution of knowledge to the growth of productivity could be measured. But, there were plenty of economists who figured that the riddle of change and growth "hadn't been solved by Solow's model with its not-my-table exogenous technology variable doing almost all of the work," David Warsh wrote in Knowledge and the Wealth of Nations.

Enter Paul Romer.

According to Warsh, "[w]hen [Paul] Romer began thinking about what exactly might constitute a new and valuable piece of knowledge worth spending money on, he soon decided it must indeed be something like a new good, a successfully differentiated product."

The Lombardi sweep? Yeoman's triple option? Walsh's short passing game? Kelly's manic spread?

Each of these play designs was at one time "a new and valuable piece of knowledge worth spending money on." And the returns to the teams that stiff-armed custom and tradition and implemented the new knowledge on the field were enormously positive.

********

In contrast, Silver's own analysis confirms an NFL coach's exogenous elective choice on fourth down is of miniscule importance on the macro level.

"I estimate ... that the typical NFL team sacrificed about half a win over the course of the 16-game regular season," Silver wrote. "That might not sound like much. But 0.5 wins over a 16-game football season is equivalent to five wins over a 162-game baseball season."

Here Silver makes a fatal macro mistake: He fails to account for the indisputable fact that a win, like a piece of knowledge (or a turnover), is an indivisible good.

As David Warsh pointed out in Knowledge and the Wealth of Nations, a good is said to be indivisible if there is a minimum size at which it is unavailable.

"You could have half a box of cereal, but not half a bridge over a river," Warsh explained.

Obviously, the minimum size at which a win is available is one. Thus, if NFL fourth down strategy is "costing" teams half of a win per year, it really isn't costing them anything because the average fourth down error rate is below the threshhold of divisibility.

That, according to Silver's accounting, half a loss in football is "equivalent" to 5 full losses in baseball, a game in which the most important decisions such as who will pitch (the right-hander or the left-hander?) are exogenous elective choices, is irrelevant because the so-called NFL loss never reaches the divisibility threshhold and thus probably can be safely ignored at the macro level.

The reason Silver ignores the critical concept of indivisibility is probably because he is unaware of it, but maybe it is because indivisibility tends to wreak havoc on neat and tidy math because mathematically an indivisible good is a nonconvexity.

One of the greatest economists in history, Kenneth Arrow, found this out in the early 1960s. After Solow developed his exogneous model of growth in the late 1950s, Arrow tried to bring knowledge inside economics by using what we call today "the learning curve," a model of knowledge accumulated through experience.

"Then it turned out that the mathematics was too much," Warsh wrote. "Arrow's model didn't have the well-behaved qualities expected of a reliable guide to action. It was unstable. Little changes through it off. So the learning-by-doing model failed to become part of the underground economists' tool kit."

And from the time Arrow's math proved unstable in the late 1960s until Paul Romer started to rethink the economics of change in the late 1980s, the role of knowledge as the engine of change remained quietly outside the field of macro economics.

********

Interestingly, Paul Romer's path to bringing knowledge "inside" macro-economics began at a ski resort on a chair lift. After publishing a paper in 1987 on congestion in chair lift lines, economist Tyler Cowen and others pointed out Romer's results had a lot in common with other club goods, such as highways, which also can exhibit congestion problems.

At the 2010 Sloan Sports Conference, Brian Skinner, a physicist at the University of Minnesota, presented essentially the same concept in the context of basketball in a paper entitled, "The Price of Anarchy in Basketball." In 2013, Professor Skinner applied the same approach to NFL football and Peyton Manning.

In "The Price of Anarchy in Football," Professor Skinner noted "[i]n football, one can think that the goal of the offense is to acquire yards. Every time the ball is snapped, the offense looks to move the ball as far down field as possible. To do so, that offense has a number of distinct [... wait for it ...] options."

More than 50 years ago, in defiance of custom and tradition, Lombardi began devoting all his attention to a subset of these options--embedded endogenous elective choices.

It's not that the NFL coaches retract into the fetal position when they consider defying custom and tradition.

It's that some of the analysts, like Professor Skinner, are finally starting to catch up to the NFL coaches who have been decades ahead of the analysts in defying custom and tradition and practicing growth theory since about the same time Robert Solow invented it.

That NFL coaches have been unaware they have been practicing growth theory is irrelevant.

But it is something that an analyst might want to know.

********

We've talked macro (growth theory; endogenous vs. exogeonous elective choices).

Now, let's talk micro.

"A computer will never make coaching decisions. The idea is ridiculous," none other than Virgil Carter himself said in 1972.

While the idea that a computer could make coaching decisions does not sound so ridiculous today, on the micro level, an NFL coach that listens to a computer and stakes more on the result of one play and loses may get himself fired, particularly when he does it with a team that is better than its opponent.

For example, in Week 3 of the 2009 season, Jim Zorn's Washington Redskins (1-1) travelled to Detroit (0-2) to meet the Lions who had lost 19 games in a row and 25 of their last 26, including an unprecedented all 16 during the previous 2008 season.

On the first possession of the game, Washington stymied the Lions. Redskins quarterback Jason Campbell then led a brisk drive that stalled at the Detroit 1-yard line. Rather than be patient and accept a near certain field goal (3 points), Zorn went for the touchdown (7 points) and failed.

After Washington eventually lost by 5 points, 19-14, those missing 3 points hung around Zorn's neck like a cement block when the Redskins' final drive ended just one 10-yard pass short of field goal range in a desparate series of improvised, chaotic and unsuccessful laterals. The hapless Lions would go on to lose 12 of their next 13 games, the only victory being a wild 38-37 win over the downtrodden Cleveland Browns (5-11).

Washington owner Daniel Snyder responded swiftly.

Just 10 days after Zorn eschewed patience, the Redskins pulled Sherman Lewis out of a bingo hall and a couple of weeks later publicly emasculated Zorn by announcing Lewis would be the new playcaller in charge of elective choices. At the end of a dismal 4-12 season, Zorn mercifully received his pink slip.

This example is telling. It may be uncivil, but accurate, to call a person gutless who fails to do the right thing based on a fear that never materializes. But it is simply incorrect to apply the gutless label when the feared consequences of the action are potentially real.

"Returning to the world periodically in search of relevant examples of the phenomena he was describing was not 'dumbing down,' Romer said," according to Warsh. "'Once you've erected the mathematical machinery and understood how it works, you've got to cut back to the world to see how it works, you've got to see whether you've cut to the essence of it."

********

So what, you say. Daniel Snyder is no Dan Rooney when it comes to football knowledge.

True, but if you closely examine Zorn's decision using the type of analytics that Silver views as the only way to evaluate fourth down decisions, you can see that choosing the field goal was as justifiable a choice as going for the touchdown.

In a blog post entitled "Zorn is My Hero 2" following the debacle in Detroit, Brian Burke, a Silver favorite, calculated that Zorn's decision to eshew patience and go for the first quarter touchdown was worth 4.6 "expected points." A synonym for "expected points" is "pretend points."

Simply, Zorn traded 3 real points for 4.6 pretend points.

Let's try it this way. What would you call a person who traded 3 real apples for 4.6 pretend apples? You probably would call him hungry.

But analysts like Silver and Burke--who called Zorn's lack of patience "a slam-dunk good decision" because according to the data 4.6 is greater than 3--simply ignore the different units of measurement, pretend points as opposed to real points.

As Warsh stated in Knowledge and the Wealth of Nations, there is a big difference between kowledge and data.

"Information consists of facts, not necessarily dependable, that may or may not be gleaned," Warsh wrote. "There's something intrinsically loose about information, even in its plural form, which is data...."

Zorn's fateful 2009 fourth down decision in Detroit is a perfect example.

While 4.6 is undeniably greater than 3, there is one other mathematial problem with Zorn's lack of patience: Even if you ignore the different units of measurement, 4.6 pretend points still would not have been enough for Washington to beat the Lions because the Redskins lost by 5 points. So, even if the Redskins had received 4.6 real points for Zorn's lack of patience, they still would have lost 19-18.6 if Zorn's decision to go for it is isolated.

Furthermore, if Zorn had simply taken the pretend 2.6 points that Burke calculated the field goal attempt to be worth, Washington would have only trailed 19-16.6 and could have won, 19.6-19, if the Redskins could have moved into position for a last play of the game field goal.

So, any reliance on prentend points in the first quarter leaves Zorn needing 3 more real points on the last play of game to win.

To "tighten up" the loosness in Burke's data, you have to subject Zorn's decision to the rigorouos test of isolation.

If one does so, one sees that as Zorn contemplated his fourth down choice in Detroit, he confronted a subjective choice pregnant with multiple equilibria. He could choose to further his expectation interest and go for touchdown. In the alternative, he could choose to further his reliance interest and send in the field goal team.

Neither choice could really be considered better than the other. If Washington had scored the touchdown, it would have received a windfall above the 4.6 value of the expectation interest that would have been enough to secure victory.

On the ther hand, if the Redskins had made the field goal, they would have been in much better position to win the game with a field goal at the end of regulation than they were as a result of the failed fourth down attempt.

Such multiple equilibria are signature characteristics of a monopoly and, as Silver pointed out, NFL coaches possess a monopoly over exogenous elective choice-making because the opponent has no way of preventing the coach from making the choice.

Consequently, there simply was not any objective mathematical formula that could tell Jim Zorn how to make his fourth down decision in Detroit. It was a pick 'em choice.

In 2011, Professor Skinner presented a principled approach to risk-based choice-making at the Sloan Sports Conference entitled "Scoring Strategies for the Underdog: Using Risk as an Ally in Determining Optimal Sports Strategies."

Under Professor Skinner's approach, "an underdog should be willing to sacrifice from its expected final score in order to increase the variance. A favored team should be willing to sacrifice from its expected final score in order to reduce the variance."

In other words, generally, if the opponent is better than your team on a given NFL Sunday, "sell short" by buying risk and go for it on fourth down. If your team is superior to your opponent, then "bet long," send out the kicking team, and patiently wait for your team's superiority to assert itself.

But an NFL coach still might go against Professor Skinner's approach under unusual circumstances. For instance, in the 1967 NFL Championship Game--the famous "Ice Bowl"--Vince Lombardi found himself with the superior team at the Dallas 1-yard line and 0:16 to play, but trailing the Cowboys, 17-14, and without any timeouts. Professor Skinner's approach would counsel to "lengthen the game" and play conservatively for the field goal and overtime.

But Packers' quarterback Bart Starr told Lombardi that he could wedge the ball into the end zone on the frozen field.

Lombardi did not hesitate to go with his gut and told Starr, "Then run it and let's get the hell out of here."

The rest, of course, is history. Green Bay guard Jerry Kramer pried Dallas defensive tackle Jethro Pugh out of the way and Starr collapsed into the end zone for the touchdown and a 21-17 Packers' victory.

It's not that the outcome of Lombardi's gamble was successful and Zorn's was not that makes Lombardi, not Zorn, a legend and a hero. That difference is mostly the result of Lombardi's relentless deployment of endogenouse elective choice-making in the power sweep design as opposed to his single exogenous elective choice in the Ice Bowl.

But the outcome of Lombardi's Ice Bowl elective choice sure didn't hurt Lombardi's legacy either.

********

Today, the NFL clearly continues to embrace the analytic approach pioneered by Dallas' Tex Schramm and to follow Lombardi's lead in the search for designs that contain embedded endogenous elective choices.

Reportedly, at least six NFL teams have purchased Paul Bessire's "Prediction Machine" simulation analytics to help them in the NFL Draft. About a third of the NFL reportedly subcribes to Pro Football Focus' player grading analytics.

This off-season, the NFL general managers have refused to shower big money on the top free agent running backs because analytics have proven that running backs do not exert much influence over wins and losses.

"When it comes to RBs, teams have already learned their Moneyball lesson: Don't overpay for talent at an injury-prone position that doesn't necessarily correspond to wins," Ilan Mochari wrote on Inc.com.

Outside the NFL, at Football Perspective, economics professor Andrew Healy used win probabiliy analytics to identify "the twenty plays that shifted the probability of the eventual Super Bowl winner the most."

Professor Healy's findings are eye-opening. In summary, he found that:

* 35% of the plays were "big" offensive plays, i.e., passes of 25 or more yards (7 of 20 plays)
* 35% of the plays were "big" defensive plays, i.e., turnovers (7 of 20 plays)
* 15% of the plays were "little" defensive plays, i.e., incomplete passes (3 of 20 plays)
* 10% of the plays were "little" offensive plays, i.e., runs, including Starr's TD in the Ice Bowl (2 of 20 plays)
* 5% of the plays were special teams breakdowns (Buffalo kicker Scott Norwood's missed field goal in SB25).

Only 10% of the plays were fourth down plays and only 10% of the plays probably can be categorized as true exogenous elective choices: Starr's TD in the Ice Bowl and John Riggins' fourth-and-1 run on "70 chip" that morphed into a tide-turning 43-yard TD in SB17, the longest TD run in Super Bowl history at the time.

Professor Healy's new contribution to the stock of football knowledge might help an NFL team design its approach to the game better by zeroing in on players and plays capable of making big plays.

Somebody inside an NFL team has probably already read Professor Healy's blog post and is wondering if there is anything that can be learned from the distribution of impact plays Professor Healy has contributed.

********

Nate Silver probably has not read Professor Healy's post and still doesn't know that only 10% of the plays that shifted the probability of the eventual Super Bowl winner the most were exogenous elective choices.

The logo of Silver's fivethirtyeight.com web site depicts a fox as an allusion to a phrase originally attributed to a Greek poet (not "Jimmy the Greek"): “The fox knows many things, but the hedgehog knows one big thing.”

In time, Silver may live up to what that logo is intended to represent.

But right now, he doesn't seem to know enough about NFL football to badger its coaches about their approach to analytics or their elective choices on fourth down.

(Archives Home)

10 Commandments

About QuantCoach

Recipe Equations

WeeklyStatistics

Team Standings

Football Knowledge

YOU MAKE THE CALL:
What game or team would you like the QuantCoach to analyze using QC's coaching statistics? Use Twitter or the headset to send QC a game that you remember or a team that you followed and QC will post an analysis using the QC coaching statistics on the Chalkboard.

Contact QuantCoach

© 2009 QuantCoach.com
QuantCoach.com is not affiliated with the National Football League or any of its members.
The material contained on QuantCoach.com is for entertainment purposes only.