Scan barcode
A review by inquiry_from_an_anti_library
Superforecasting: The Art and Science of Prediction by Philip E. Tetlock
adventurous
hopeful
informative
inspiring
fast-paced
5.0
Is This An Overview?
Forecasting is a skill that everyone uses everyday to predict the effects of potential changes. Like any skill, forecasting can be improved. Experts are often sought out for decisions and event interpretations, to forecast what will come about. Although many provided forecasts appear valuable, their quality is often undetermined. The public tends to favor those who make the future appear more certain, even though their overconfidence is a source of lower quality forecasts. On average, experts can provide a better narrative of events, but their forecasts are as good as random guesses.
Forecasting is a skill that everyone uses everyday to predict the effects of potential changes. Like any skill, forecasting can be improved. Experts are often sought out for decisions and event interpretations, to forecast what will come about. Although many provided forecasts appear valuable, their quality is often undetermined. The public tends to favor those who make the future appear more certain, even though their overconfidence is a source of lower quality forecasts. On average, experts can provide a better narrative of events, but their forecasts are as good as random guesses.
Part of the reason for the poor performance of forecasts is that reality is complex and dynamic, making predictions difficult. Society might have more knowledge and computational power, but less confidence in predictability. There might be limits on predictability, but people can become better at making forecasts. To find out how people can make better forecasts, and methods to avoid, many diverse people participated in a forecasting research project.
What made some people better at making forecasts, what made people superforecasters, was based on how they thought about information, how they used information. Not intelligence, not ideology, not numeracy skills. The forecasters were doubtful of their claims, and sought to improve them. Complex problems which seemed impossible to forecast, were reconsidered through a variety of questions seeking to find ways for the event to occur, or not occur. They looked for the base rate, a general probability of an event happening before going to the unique case. Anchoring their views to the outside view, rather than the inside view. They seek to improve their own forecasts by looking for what others think about the event, they look for alternative forecasts. They adapt to new information, update their forecasts to new information, and try to not underreact or overreact to the information.
These methods of thinking, these guidelines might improve decision making, but better to change guidelines than make a terrible forecast. People can become better at forecasting, but teams have better results than an individual superforecaster, as each member can help others to refine ideas, and no individual can do everything. But teams take effort to make them productive, and can create processes that exacerbate bad decisions.
How To Get Better At Forecasting?
To become better at forecasts, people need to practice. There is a lot of tacit knowledge that cannot be learned through how others describe forecasting. Feedback is needed to train in any skill, including forecasting. But the feedback to forecasts, usually lack quality. They do not provide immediate feedback nor provide clear results. Without appropriate feedback, people can become overconfident in their forecasts. People can gain an illusion of control from seemingly favorable random outcomes. Judging forecasts would depend on running many forecasts, such as in weather. But there are forecasts that cannot be rerun, such as history. Need to run experiments to verify claims.
To become better at forecasts, people need to practice. There is a lot of tacit knowledge that cannot be learned through how others describe forecasting. Feedback is needed to train in any skill, including forecasting. But the feedback to forecasts, usually lack quality. They do not provide immediate feedback nor provide clear results. Without appropriate feedback, people can become overconfident in their forecasts. People can gain an illusion of control from seemingly favorable random outcomes. Judging forecasts would depend on running many forecasts, such as in weather. But there are forecasts that cannot be rerun, such as history. Need to run experiments to verify claims.
The language around what people mean by possibilities need to be more specific rather than ambiguous. People can mean drastically different possibilities, which can create a dangerous misunderstanding. Teams can use a chart to numerically define possibility claims, to reduce confusion. Numbers are an opinion, but can be used to reduce confusion. Forecasts also need timelines. Without timelines, forecasts become perpetually in dispute at what they meant.
Caveats?
Forecasting on problems will always have uncertainty. As referenced in the book, no matter the quality of the better decision making, there will be uncertainty and wrong decisions. The process of decision making matters more than the outcome, as there will be more opportunities for better decisions with a better decision making process than a randomly favorable outcome under a worse decision making process.
Forecasting on problems will always have uncertainty. As referenced in the book, no matter the quality of the better decision making, there will be uncertainty and wrong decisions. The process of decision making matters more than the outcome, as there will be more opportunities for better decisions with a better decision making process than a randomly favorable outcome under a worse decision making process.