A review by jasonfurman
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us about Who We Really Are by Seth Stephens-Davidowitz

5.0

A wide-ranging, provocative, entertaining examination of big data that offers an ambitious framework for what big data can and should do--while also evidencing awareness of what it cannot and should not do. The title and promotion center around the "Everybody Lies" theme--with Seth Stephens-Davidowitz describing Google searches as a "digital truth serum" that allows us to see, for example, the extent of racism by following searches for racist jokes. But the book goes much further in identifying and providing examples of three other uses of big data: (1) allowing us to use new types of data, (2) zooming on on very particular places or cases without losing the ability to discern in a smaller sample, and (3) providing an easy way to conduct A/B tests and other experiments.

In the course of this framework, Stephens-Davidowitz provides a constant stream of examples drawn from economics (including his own work), social science, and elsewhere to illustrate the many ways that "big data", much but not all of it Google searches, has helped us learn about what people are really thinking and doing.

But Stephens-Davidowitz also describes the problems with big data--the curse of dimensionality that leads to overfitting models that work really well in sample but terribly outside it (e.g., predicting stocks) and the overemphasis on what is measurable (e.g., moneyball-oriented baseball teams ending up with inefficiently weak fielders because they were focused on measurable batting statistics). And he also has warnings on privacy and more.

Strongly recommended.