The TASC List: Praise for Bayes

Prediction. Projection. Forecast-ing. Looking into the future. Every community depends upon the use of methods to determine how tomorrow will look.

Will it look like today? If not, how different will it be? How certain are we of the difference? How can we improve upon the uncertainty?

We’ve come a long way since the days when town “elders” read the future in tea leaves or animal entrails. Statistical models have replaced best estimates. Simulation has provided an approach to replicating factors, trends, and influences on a controllable stage that mirrors to some extent the real world. Computer analysis has allowed the running of unlimited cases or trials where one or more variables can be tweaked and tested. The search for significant impact or optimal outcomes is made easier.

We forecast weather, predict sporting event results, and project successes and failures on the battlefield. Closer to home, we forecast student enrollments, predict mill rate effects from budget levels, and project numbers of emergency vehicles necessary to provide adequate services for town residents.

And now there emerges a methodology that promises improvement in the way we approach estimation and consider dealing with data and information collections.

A book by Nate Silver may be useful to people who work at trying to find out what will happen next. The Signal and the Noise (Why So Many Predictions Fail — but Some Don’t) was published in 2012 by Penguin Press and demonstrates how we can extract truth and substance from mountains of data and separate clutter and random disturbances from meaningful trends and directions.

This is a powerful work that is both illuminating and fun to read. More importantly, it shows us how to operate more effectively in a world of Big Data in order to improve on our estimates of the future. It’s about information, technology, and scientific progress, competition, free markets, and the evolution of ideas, according to the author, and about human error.

But mostly it’s about prediction, which he says “sits at the intersection of all these things. It is a study of why some predictions succeed and why some fail. My hope is that we might gain a little more insight into planning our futures and become a little less likely to repeat our mistakes.”

Silver introduces the theorem and thinking of Thomas Bayes, an English minister who in the mid-18th Century moved from theology to breaking new ground on probability, uncertainty, and approximation. “In its most basic form [the theorem] is just an algebraic expression with three known variables and one unknown one. But this simple formula can lead to vast predictive insights. Bayes’ theorem is concerned with conditional probability. That is, it tells us the probability that a theory or hypothesis is true if some event has happened.”

Convergence toward the truth, or “the Bayesian Path to Less Wrongness” is increasingly taking hold in scientific communities  today. Bayes is becoming more popular in the realm of statistical methods. It does require an explicit likelihood statement for an event before the approximation process begins. The author demonstrates how and where predictions can fail and the potential of applying Bayes’ Theorem today in such diverse areas as global warming, terrorism and financial markets. He makes it all relevant.

So if you’re interested in how to make better sense of this exploding world of information, and perhaps how to arrive at improved predictions based on both objectivity and subjectivity, this work may appeal. TASC predicts you’ll enjoy a journey with Silver. Take him along as your own summer reading companion. Put it on your own “Task List” to get acquainted with the quaint but insightful Thomas Bayes.

TASC stands for Toward A Stronger Community. Information: