The Signal and the Noise

''The Signal and the Noise: Why Most Predictions Fail – but Some Don't'' (alternatively stylized as ''The Signal and the Noise: Why So Many Predictions Fail – but Some Don't'') is a 2012 book by Nate Silver detailing the art of using probability and statistics as applied to real-world circumstances. The book includes case studies from baseball, elections, climate change, the 2008 financial crash, poker, and weather forecasting.

The book was the recipient of the 2013 Phi Beta Kappa Society book award in science. It has also been translated into several languages.

Synopsis

The book emphasizes Silver's skill, which is the practical art of mathematical model building using probability and statistics. Silver takes a big-picture approach to using statistical tools, combining sources of unique data (e.g., timing a minor league ball player's fastball using a radar gun), with historical data and principles of sound statistical analysis, many of which are violated by many pollsters and pundits who nonetheless have important media roles.

The book includes richly detailed case studies from baseball, elections, climate change, the financial crash, poker, and weather forecasting. These different topics illustrate different statistical principles. For example, weather forecasting is used to introduce the idea of "calibration," or how well weather forecasts fit actual weather outcomes. There is much on the need for improved expressions of uncertainty in all statistical statements, reflecting ranges of probable outcomes and not just single "point estimates" like averages.

Silver would like to see the media move away from vague terminology like "Obama has an edge in Ohio" or "Florida still a toss-up state" to probability statements, like "the probability of Obama winning the electoral college is 83%, while the expected fraction won by him of the popular vote is now 50.1% with an error range of ±2%". Such statements give odds on outcomes, including a 17% chance of Romney winning the electoral college. The shares of the popular vote similarly are ranges including outcomes in which Romney gets the most votes. What is highly probable is that the voting shares are in these ranges, but not whose share is highest; that's another probability question with closer odds. From such information, it's up to the consumer of such statements to use that information as best they can in dealing with an uncertain future in an age of information overload. That last idea frames Silver's entire narrative and motivates his pedagogical mission.

Silver rejects much ideology taught with statistical method in colleges and universities today, specifically the "frequentist" approach of Ronald Fisher, originator of many classical statistical tests and methods. The problem Silver finds is a belief in perfect experimental, survey, or other designs, when data often comes from a variety of sources and idealized modeling assumptions rarely hold true. Often such models reduce complex questions to overly simple "hypothesis tests" using arbitrary "significance levels" to "accept or reject" a single parameter value.

In contrast, the practical statistician first needs a sound understanding of how baseball, poker, elections or other uncertain processes work, what measures are reliable and which not, what scales of aggregation are useful, and then to utilize the statistical tool kit as well as possible. Silver believes in the need for extensive data sets, preferably collected over long periods of time, from which one can then use statistical techniques to incrementally change probabilities up or down relative to prior data. This Bayesian approach is named for the 18th-century minister Thomas Bayes who is credited for first deriving a simple formula for updating probabilities using new data. For Silver, the well-known method needs revitalizing as a broader paradigm for thinking about uncertainty, founded on learning and understanding gained incrementally, rather than through any single set of observations or an ideal model summarized by just a few key parameters. Part of that learning is the informal process of changing assumptions or the modeling approach, in the spirit of a craft whose goal is to devise the best betting odds on well-defined future events and their outcomes.

Release and sales

Published in the United States on September 27, 2012, ''The Signal and The Noise'' reached the ''New York Times'' Best Sellers list as No. 12 for non-fiction hardback books after its first week in print. It dropped to No. 20 in the second week, before rising to No. 13 in the third, and remaining on the non-fiction hardback top 15 list for the following thirteen weeks, with a highest weekly ranking of No. 4. The book's already strong sales soared right after election night, November 6, jumping 800% and becoming the second best seller on Amazon.com.

''The Signal and the Noise'' (print edition) was named Amazon's No. 1 Best NonFiction Book for 2012. It was named by the '' Wall Street Journal'' as one of the ten best books of nonfiction published in 2012.

The book was first released in the United Kingdom in April 2013, with the title ''The Signal and the Noise: The Art and Science of Prediction,'' in hardback under an Allen Lane imprint and in paperback under a Penguin imprint.

It was published in Brazil in a Portuguese translation, ''O sinal e o ruído'', in June 2013. A German edition was released in September 2013 by the publisher Heine Verlag, using a somewhat different main title: ''The Calculation of the Future''. An Italian version, ''Il segnale e il rumore. Arte e scienza della previsione,'' appeared in October 2013. A Finnish translation, ''Signaali ja kohina'', appeared in 2014.

The book was published in simplified Chinese characters by China Citic Press in Beijing as well as in traditional Chinese characters by Suncolor Press in Taipei in Fall 2013. It was published in Japanese in November 2013. A Spanish translation appeared in April 2014, under the title ''La señal y el ruido.''

In 2013, an edition was published in Romanian by Publica press: ''Semnalul și zgomotul: de ce atât de multe predicții dau greș - pe când altele reușesc.'' In May 2014, an edition was published in Czech: ''Signál a šum. Většina předpovědí selže. Některé ne''.

A Korean language edition was published by The Quest in July 2014.

A Polish edition was scheduled for publication in hardcover in 2015 by Helion: ''Sygnal i szum: Sztuka prognozowania w erze technologii''.

''The Signal and the Noise'' was first published in paperback in the United States on February 3, 2015. It contained a new introduction. It reached ''The New York Times'' paperback best seller list, ranked No. 4 in the field of Education in March 2015.

Criticism and commentary

Climate scientist Michael E. Mann criticized the book for analyzing the "hard science" physical phenomena of climate trends with the same approach as used to analyze the social phenomena of voter preferences, which he characterized as "laden with subjective and untestable assumptions".

In praise of the book, Murray Cantor, IBM Distinguished Engineer, wrote

Awards

* In October 2013, ''The Signal and the Noise'' won the 2013 Phi Beta Kappa Society book award in science, which recognizes "outstanding contributions by scientists to the literature of science".

See also

* Game theory
* '' Noise: A Flaw in Human Judgment''

References

External links

Book at Penguin Group (publisher)



Video of Silver discussing the subject

{{DEFAULTSORT:Signal and the Noise, The
2012 non-fiction books
American non-fiction books
Statistics books
Penguin Books books