Book Review: The Theory That Would Not Die
Overview
I had put off reading Sharon Bertsch McGrayne's The Theory That Would Not Die for a long time, but I'm glad I finally got to it late last year. Although I had previously read some books with "history of statistics" content, like Bernoulli's Fallacy and The Lady Tasting Tea, this book introduced a large number of new stories I hadn't heard before about the development and application of Bayesian methods as well as its detractors.
While Bernoulli's Fallacy primarily focused on a narrow lineage of academic thought for and against Bayesian methods up through Fisher, and The Lady Tasting Tea ran through all the big names in academic statistics in the 20th century without an overarching theme, McGrayne follows the thread of Bayesian statistics from its early development all the way through to its modern usage. And whereas Bernoulli's Fallacy advocates and explains the Bayesian philosophy, McGrayne is focused on telling the full story with historical context, highlighting key figures, turning points, and historical use cases.
The book's long subtitle presents a nice synopsis: "How Bayes' rule cracked the Enigma code, hunted down Russian submarines & emerged triumphant from two centuries of controversy."
New to me
I learned something new in just about every chapter. I was even embarrassed to realize I didn't know the motivating example of Bayes' original paper, using a Beta distribution to estimate the unknown location of a ball on a square table, using only information about whether other randomly thrown balls had landed to the left or right of it. That really is clever, given the historical context.
There was some overlap between the sections on Laplace and Fisher from the aforementioned books, but many of the other stories I either hadn't seen at all or hadn't seen in as much depth.
For instance, I wasn't aware that Bayesian shrinkage had been imbued into actuarial practice as Credibility theory, unbeknownst to many actuaries. Or that there were proponents at Harvard Business School starting in the late 1950s. There were many such stories that were completely new to me.
World War II
I had long been familiar with Turing's work at Bletchley Park for cracking the Enigma Code with Banburismus using early electro-mechanical computers, known as bombes, which enabled the Allies to navigate the Atlantic safely and evade U-boats. McGrayne dedicates most of a chapter to this, as well as some other applications of Bayes to the war effort. Helping to defeat the Nazis is a nice notch on Bayes' belt.
Nate Silver's Prequel
I hadn't previously heard the story of John Tukey, a well known statistician of Princeton and Bell Labs, joining NBC News to help call elections for 18 years with statistical methods, starting in 1960. Some who helped with the programming claimed they were using empirical Bayes, although Tukey never admitted to using Bayesian methods. The author shares speculation that Tukey's involvement with Cold War era national security, where Bayes' rule was widely used, may have been part of his reluctance to credit the method.
Biggest surprise
The most surprising thing I learned in this book was that the Kalman Filter was not motivated by Bayes rule, and in fact Kalman disliked Bayesian methods. I had previously assumed it was intended as an analytical solution to updating a Bayesian model with conjugate priors. In any case, it does indeed have a Bayesian interpretation, like other filtering methods for localization.
Nitpicks
While McGrayne is not a mathematician or statistician herself, she seemed to have a pretty good handle on the Bayesian framework at a conceptual level. However, there were two things that bothered me. The first is that Bayes' rule is not a theory; it's a theorem that can be proven from the axioms of probability. I think perhaps calling it a "theory" may have been in reference to using it for statistical inference, but it makes for a confusing book cover when "theory" appears to reference to Bayes' rule itself.
Second (and related to the first point), I kept waiting for her to state that Bayes' rule is not at all controversial in probability theory; it's only been debated in its application to estimating parameters in statistical models, where Bayesian practitioners apply probability to model parameters just as they would to any other uncertain quantity of interest. I would also separate out the philosophical disagreements on the interpretation of probability from Bayes' rule itself. Frequentist statisticians don't doubt the validity of Bayes' rule; it's a particular application of it they doubt.
Recommendation
What I like about books like this and Bernoulli's Fallacy is that you can see the path dependence of history so clearly. It's not as if people wake up each day and reevaluate what statistical methods they plan to use. These things carry momentum and inertia with them, and wrong turns can last a long time. "Science progresses one funeral at a time," as it were.
I've only covered a handful of stories told within this book; I was pleasantly surprised at its breadth and selectively chosen depth. While it avoids the technical weeds, The Theory That Would Not Die does the most thorough job surveying the history of thought and application of Bayesian statistics as I've seen. So if that interests you, I recommend it.