Take a photo of a barcode or cover
123 reviews for:
Science Fictions: Exposing Fraud, Bias, Negligence and Hype in Science
Stuart Ritchie
4.33 AVERAGE
123 reviews for:
Science Fictions: Exposing Fraud, Bias, Negligence and Hype in Science
Stuart Ritchie
4.33 AVERAGE
informative
medium-paced
I agree with most problems of science, but it's tedios to read. I think there is very little attention given to who does the science and the hierarchies inside institutions. It reads liberal, technocratic solutions by just pass our papers through algorithms. Boring and simplistic.
Although a number of examples in this book of scientific malfeasance were well known to me, seeing all the issues in today's scientific world gathered in one book makes for quite an impactful read. Recommended!
informative
When a measure becomes the target, it ceases to be a good measure.
~Goodhart’s Law (adapted by Marilyn Strathern)
Wonderful tour through the (current) limitations of science, focusing specifically on how we get funded, how we get promoted, how we publish, how publications get hyped, and (the real focus) how statistical analysis can be incomplete or misleading. Written at a perfect level for someone like me, who has a small amount of statistical understanding and works in medicine adjacent to, but not within, the basic sciences. The areas of inquiry are:
Fraud
Negligence
Bias
Hype
Perverse Incentives
I was so impressed by the legions of scientists who are acting in good faith to correct the errors of wayward authors, whether performing replication studies or re-analysis or devising AI-based algorithms to detect shenanigans.
Many of the examples come from social psychology research, but among the authors highest ranked on Retraction Watch are clinical physicians.
Ritchie ends with:
How to Read a Scientific Paper
and
How to Fix Science
In that last, preprints, funding “people” and not projects, smarter fraud-detection, abandoning p-values, and many other recommendations are on offer. Because no data is better than bad data:
“Scientists who knowingly run low-powered research, and the reviewers and editors who wave through tiny studies for publication, are introducing a subtle poison into the scientific literature, weakening the evidence that it needs to progress.”
~Goodhart’s Law (adapted by Marilyn Strathern)
Wonderful tour through the (current) limitations of science, focusing specifically on how we get funded, how we get promoted, how we publish, how publications get hyped, and (the real focus) how statistical analysis can be incomplete or misleading. Written at a perfect level for someone like me, who has a small amount of statistical understanding and works in medicine adjacent to, but not within, the basic sciences. The areas of inquiry are:
Fraud
Negligence
Bias
Hype
Perverse Incentives
I was so impressed by the legions of scientists who are acting in good faith to correct the errors of wayward authors, whether performing replication studies or re-analysis or devising AI-based algorithms to detect shenanigans.
Many of the examples come from social psychology research, but among the authors highest ranked on Retraction Watch are clinical physicians.
Ritchie ends with:
How to Read a Scientific Paper
and
How to Fix Science
In that last, preprints, funding “people” and not projects, smarter fraud-detection, abandoning p-values, and many other recommendations are on offer. Because no data is better than bad data:
“Scientists who knowingly run low-powered research, and the reviewers and editors who wave through tiny studies for publication, are introducing a subtle poison into the scientific literature, weakening the evidence that it needs to progress.”
Excellent eye-opening book about modern science’s “replication crisis” that has resulted from a large percentage of published studies being unable to be replicated, and how various systematic factors can undermine the scientific method and search for truth. A great book for learning more about the various flaws that can creep into science (and any discipline), including fraud, negligence, bias, and hype. Clearly organized with tons of examples and references to learn more - including a guide on how to read a scientific paper, which includes the following top-ten list of questions:
1. Is everything above board? (Trustworthy source)
2. How transparent is it?
3. Is the study well-designed?
4. How big is the sample?
5. How big is the effect?
6. Are the inferences appropriate?
7. Is there bias?
8. How plausible is it, really?
9. Has it been replicated?
10. What do other scientists think about it?
Here’s a few other interesting selections:
“Discovering the serious problems with the way we do science will be disconcerting. How many intriguing results that you've read about in the news and popular science books, or seen in documentaries, discoveries you've been excited enough to share with friends, or that made you rethink how the world works - are based on weak research that can't be replicated? How many times has your doctor prescribed you a drug or other treatment that rests on flawed evidence? How many times have you changed your diet, your purchasing habits, or some other aspect of your lifestyle on the basis of a scientific study, only for the evidence to be completely overturned by a new study a few months later? How many times have politicians made laws or policies that directly impact people's lives, citing science that won't stand up to scrutiny? In each case, the answer
is: a lot more than you'd like to think.”
“In general, though, the effect on psychology has been devastating. This wasn't just a case of fluffy, flashy research like priming and power posing being debunked: a great deal of far more ‘serious' psychological research (like the Stanford Prison Experiment, and much else besides) was also thrown into doubt... Studies that failed to
replicate continued to be routinely cited both by scientists and other writers: entire lines of research, and bestselling popular books, were being built on their foundation. ‘Crisis'
seems to be an apt description.”
“There are countless other examples: almost every case I'll
describe in this book involves a scientific "finding' that, upon closer scrutiny, turned out to be either less solid than it seemed or to be completely untrue. But more worryingly still, these examples are drawn just from the studies that have received that all-important scrutiny. These are just the ones we know about. How many other results, we must ask ourselves, would prove unreplicable if anyone happened to make the attempt?”
“For a scientific finding to be worth taking seriously, it can't
be something that occurred because of random chance, or a glitch in the equipment, or because the scientist was cheating or dissembling. It has to have really happened. And if it did then in principle I should be able to go out and find broadly the same results as yours. In many ways, that's the essence of science, and something that sets it apart from other ways of knowing about the world: if it won't replicate, then it's hard to describe what you've done as scientific at all.”
“In 1942, Merton set out four scientific values, now known as the Mertonian Norms, None of them have snappy names, but all of them are good aspirations for scientists. First, universalism: scientific knowledge is scientific knowledge, no matter who comes up with it… Second, and
relatedly, disinterestedness: scientists aren't in it for the money, for political or ideological reasons, or to enhance their own ego or reputation (or the reputation of their university, country, or anything else)… The third is communality: scientists should share knowledge
with each other… Lastly, there's organised scepticism: nothing is sacred, and a scientific claim should never be accepted at face value.”
“The typical paper starts with an Introduction, where you summarise what's known on the topic and what your study adds. There follows a Method section, where you describe exactly what you did - in enough detail so that anyone could, in theory, run exactly the same experiment again.
You'll then move on to a Results section, where you present the numbers, tables, graphs and statistical analyses that document your findings, and you'll end with a Discussion section where you speculate wildly - er, I mean, provide thoughtful, informed consideration - about what it all means. You'll top the whole thing with an Abstract: a brief statement, usually of around 150 words, that summarises the whole study and its results. The Abstract is always available for anyone to read, even if the full paper is behind the journal's subscription paywall, so you'll want to use it to make your results sound compelling. Papers come in all lengths and sizes, and sometimes mix up the above order, but in general your paper will end up along these lines.”
“There's one field of research that consistently generates more hype, inspires more media interest and suffers more from the deficiencies outlined in this book than any other. It is, of course, nutrition. The media has a ravenous appetite for its supposed findings: The ‘Scary New Science That Shows Milk is Bad For You'; ‘Killer Full English: Bacon Ups Cancer Risk'; ‘New Study Finds Eggs Will Break Your Heart'. Given the sheer volume of coverage, and the number of conflicting assertions about how we should change our diets, little wonder the public are confused about what they should be eating. After years of exaggerated findings the public now lacks confidence and is sceptical of the field's research. Nutritional science, like psychology, has been going through its own replication crisis.”
“Rather like psychology, nutritional epidemiology is hard. An incredibly complex physiological and mental machinery is involved in the way we process food and decide what to eat; observational data are subject to enormous noise and the vagaries of human memory; randomised trials can be tripped up by the complexities of their own administration. Given that context, the sheer amount of media interest in nutritional research is particularly unfortunate. Perhaps the very scientific questions that the public wants to have answered the most - what to eat, how to educate children, how to talk to potential employers, and so on - are the ones where the science is the murkiest, most difficult, and most self-contradictory. All the more reason that scientists in those fields need to take more seriously the task of sensibly communicating their findings to the public.”
“It’s not just that the system fails to deal with all the kinds of malpractice we've discussed. In fact, the way academic research is currently set up incentivises these problems, encouraging researchers to obsess about prestige, fame, funding and reputation at the expense of rigorous, reliable results.”
“Goodhart's Law: ‘when a measure becomes the target, it ceases to be a good measure.’… once you begin to chase the numbers themselves rather than the principles that they stand for - in this case, the principle of finding research that makes a big contribution to our knowledge - you've completely lost your way.”
“To paraphrase the biologist Ottoline Leyser, the point of
breaking ground is to begin to build something; if all you do is groundbreaking, you end up with a lot of holes in the ground but no buildings. How do we reverse the prioritisation of novel results over solid ones? How do we combat publication bias, ensuring that all results get published, no matter whether they're groundbreaking or null?”
“Somewhat scandalously, the majority of science frames exploratory results as though they were confirmatory; as though they were the results of tests planned before the study started.”
“The usual reaction I received when I told my friends about this book was a broader concern regarding trust in science: 'Isn't it irresponsible to write something like that? Won't you encourage a free-for-all, where people use your arguments to justify their disbelief in evolution, or in the safety of vaccines, or in man-made global warming? After all, if mainstream science is so biased, and its results so hyped, why should the average person believe what scientists are telling them?'”
“It's with more science that we can discover where our research has gone wrong and work out how to fix it. The ideals of the scientific process aren’t the problem: the problem is the betrayal of those ideals by the way we do research in practice. If we can only begin to align the practice with the values, we can regain any wavering trust and stand back to marvel at all those wondrous discoveries with a clear conscience.”
“The fundamental lesson is to be humbler about what we do and do not know. At first this might appear to be antithetical to the idea of scientific research, which is surely about uncovering new facts about the world and always adding to our knowledge. But if you think about it for longer, it turns out to be the very essence of science itself.”
1. Is everything above board? (Trustworthy source)
2. How transparent is it?
3. Is the study well-designed?
4. How big is the sample?
5. How big is the effect?
6. Are the inferences appropriate?
7. Is there bias?
8. How plausible is it, really?
9. Has it been replicated?
10. What do other scientists think about it?
Here’s a few other interesting selections:
“Discovering the serious problems with the way we do science will be disconcerting. How many intriguing results that you've read about in the news and popular science books, or seen in documentaries, discoveries you've been excited enough to share with friends, or that made you rethink how the world works - are based on weak research that can't be replicated? How many times has your doctor prescribed you a drug or other treatment that rests on flawed evidence? How many times have you changed your diet, your purchasing habits, or some other aspect of your lifestyle on the basis of a scientific study, only for the evidence to be completely overturned by a new study a few months later? How many times have politicians made laws or policies that directly impact people's lives, citing science that won't stand up to scrutiny? In each case, the answer
is: a lot more than you'd like to think.”
“In general, though, the effect on psychology has been devastating. This wasn't just a case of fluffy, flashy research like priming and power posing being debunked: a great deal of far more ‘serious' psychological research (like the Stanford Prison Experiment, and much else besides) was also thrown into doubt... Studies that failed to
replicate continued to be routinely cited both by scientists and other writers: entire lines of research, and bestselling popular books, were being built on their foundation. ‘Crisis'
seems to be an apt description.”
“There are countless other examples: almost every case I'll
describe in this book involves a scientific "finding' that, upon closer scrutiny, turned out to be either less solid than it seemed or to be completely untrue. But more worryingly still, these examples are drawn just from the studies that have received that all-important scrutiny. These are just the ones we know about. How many other results, we must ask ourselves, would prove unreplicable if anyone happened to make the attempt?”
“For a scientific finding to be worth taking seriously, it can't
be something that occurred because of random chance, or a glitch in the equipment, or because the scientist was cheating or dissembling. It has to have really happened. And if it did then in principle I should be able to go out and find broadly the same results as yours. In many ways, that's the essence of science, and something that sets it apart from other ways of knowing about the world: if it won't replicate, then it's hard to describe what you've done as scientific at all.”
“In 1942, Merton set out four scientific values, now known as the Mertonian Norms, None of them have snappy names, but all of them are good aspirations for scientists. First, universalism: scientific knowledge is scientific knowledge, no matter who comes up with it… Second, and
relatedly, disinterestedness: scientists aren't in it for the money, for political or ideological reasons, or to enhance their own ego or reputation (or the reputation of their university, country, or anything else)… The third is communality: scientists should share knowledge
with each other… Lastly, there's organised scepticism: nothing is sacred, and a scientific claim should never be accepted at face value.”
“The typical paper starts with an Introduction, where you summarise what's known on the topic and what your study adds. There follows a Method section, where you describe exactly what you did - in enough detail so that anyone could, in theory, run exactly the same experiment again.
You'll then move on to a Results section, where you present the numbers, tables, graphs and statistical analyses that document your findings, and you'll end with a Discussion section where you speculate wildly - er, I mean, provide thoughtful, informed consideration - about what it all means. You'll top the whole thing with an Abstract: a brief statement, usually of around 150 words, that summarises the whole study and its results. The Abstract is always available for anyone to read, even if the full paper is behind the journal's subscription paywall, so you'll want to use it to make your results sound compelling. Papers come in all lengths and sizes, and sometimes mix up the above order, but in general your paper will end up along these lines.”
“There's one field of research that consistently generates more hype, inspires more media interest and suffers more from the deficiencies outlined in this book than any other. It is, of course, nutrition. The media has a ravenous appetite for its supposed findings: The ‘Scary New Science That Shows Milk is Bad For You'; ‘Killer Full English: Bacon Ups Cancer Risk'; ‘New Study Finds Eggs Will Break Your Heart'. Given the sheer volume of coverage, and the number of conflicting assertions about how we should change our diets, little wonder the public are confused about what they should be eating. After years of exaggerated findings the public now lacks confidence and is sceptical of the field's research. Nutritional science, like psychology, has been going through its own replication crisis.”
“Rather like psychology, nutritional epidemiology is hard. An incredibly complex physiological and mental machinery is involved in the way we process food and decide what to eat; observational data are subject to enormous noise and the vagaries of human memory; randomised trials can be tripped up by the complexities of their own administration. Given that context, the sheer amount of media interest in nutritional research is particularly unfortunate. Perhaps the very scientific questions that the public wants to have answered the most - what to eat, how to educate children, how to talk to potential employers, and so on - are the ones where the science is the murkiest, most difficult, and most self-contradictory. All the more reason that scientists in those fields need to take more seriously the task of sensibly communicating their findings to the public.”
“It’s not just that the system fails to deal with all the kinds of malpractice we've discussed. In fact, the way academic research is currently set up incentivises these problems, encouraging researchers to obsess about prestige, fame, funding and reputation at the expense of rigorous, reliable results.”
“Goodhart's Law: ‘when a measure becomes the target, it ceases to be a good measure.’… once you begin to chase the numbers themselves rather than the principles that they stand for - in this case, the principle of finding research that makes a big contribution to our knowledge - you've completely lost your way.”
“To paraphrase the biologist Ottoline Leyser, the point of
breaking ground is to begin to build something; if all you do is groundbreaking, you end up with a lot of holes in the ground but no buildings. How do we reverse the prioritisation of novel results over solid ones? How do we combat publication bias, ensuring that all results get published, no matter whether they're groundbreaking or null?”
“Somewhat scandalously, the majority of science frames exploratory results as though they were confirmatory; as though they were the results of tests planned before the study started.”
“The usual reaction I received when I told my friends about this book was a broader concern regarding trust in science: 'Isn't it irresponsible to write something like that? Won't you encourage a free-for-all, where people use your arguments to justify their disbelief in evolution, or in the safety of vaccines, or in man-made global warming? After all, if mainstream science is so biased, and its results so hyped, why should the average person believe what scientists are telling them?'”
“It's with more science that we can discover where our research has gone wrong and work out how to fix it. The ideals of the scientific process aren’t the problem: the problem is the betrayal of those ideals by the way we do research in practice. If we can only begin to align the practice with the values, we can regain any wavering trust and stand back to marvel at all those wondrous discoveries with a clear conscience.”
“The fundamental lesson is to be humbler about what we do and do not know. At first this might appear to be antithetical to the idea of scientific research, which is surely about uncovering new facts about the world and always adding to our knowledge. But if you think about it for longer, it turns out to be the very essence of science itself.”
Anyone working in a STEM field should read this book. Even if you’re into “pop-science” news this worth reading. There is so much nonsense and straight up fraud out there, but it’s not a sign of the end of science. Ritchie shows how being a skeptic is done. He clearly shows that if something is a “peer reviewed” published article, it can still be wrong (and often is). A published article is far from the final step in research, it’s a major one for sure, but is part of an ongoing conversation that continually assesses and critiques it.
He covers everything from the awful paper that falsely claimed autism is caused by vaccines, to sensationalized news headlines, and how every food causes and cures cancer. Highly recommend.
He covers everything from the awful paper that falsely claimed autism is caused by vaccines, to sensationalized news headlines, and how every food causes and cures cancer. Highly recommend.
informative
reflective
slow-paced
challenging
informative
medium-paced
Really interesting book to this lay person. It taught me a lot about how scientific studies work, but also how science isn’t something we should just accept as true, and that scientists need to hold themselves and others to as high standards as possible, because it will benefit everyone to do so.
Please note that this is an informal review and as such citations will not be provided for statements.
Although this book is about the many problems in science, it is entirely pro-science and strives to improve it through constructive criticism and the reduction of harmful behaviors by individuals and institutions. One of the primary criticisms I've seen of it is that it romanticizes the idea of the science and has too high of expectations for how people ought to behave. I understand this, but I don't agree with it. I don't believe it romanticizes science, though it does propose much higher standards than currently exist, and certainly much higher than are followed in practice. There are many unfortunate and relatable reasons why, several of which are described within.
Human limitations are the greatest obstacle to scientific progress, but unfortunately even accepting that doesn't seem to affect behavior much. These limitations are present at every structural level. None of this is new, history is littered with the bodies, figuratively and literally, of those who challenged their prevailing status quo. Their challenges weren't always warranted, but there have been many times when they were.
Example not in the book:
https://en.wikipedia.org/wiki/Ignaz_Semmelweis
A doctor told doctors that they should disinfect their hands because the scientific literature said it greatly reduced mortality rates. The doctors mocked and ridiculed him and eventually he's committed to an insane asylum in 1865, where he was beaten by the guards, and then died from a gangrenous wound. The Semmelweis Reflex, "a metaphor for the reflex-like tendency to reject new evidence or new knowledge because it contradicts established norms, beliefs, or paradigms." is named after him.
https://en.wikipedia.org/wiki/Semmelweis_reflex
The bulk of the book is concerned with psychology, because the author is a psychologist, though many other fields have examples given as well. These problems are not limited to anything specific, they are all-encompassing. The differences are in terms of frequency and severity. There are failures of self-discipline and ethical behavior from individuals and an immense failure of self-regulation by institutions. Not all the blame can be put on them as they are subjected to various perverse incentives and systemic societal pressures as well.
Many example stories are presented of the failures of individuals and institutions. There's overall more a focus on the specific than the general, which probably ought to be expected since psychology is more the study of the individual.
Part I is about how science works and the replication crisis, which is the failure of someone else being able to independently confirm the results of a study. This differs from the reproduction problem, where others cannot even attempt to test their results.
Part II details the faults and flaws, specifically fraud, bias, negligence and hype. Specific stories for each are presented.
A somewhat related story of negligence that comes to mind to me is of this serial killer:
https://en.wikipedia.org/wiki/Michael_Swango
Unfortunately, as noted in this book more than a few times, too much good faith is assumed and too much trust is given. This is a terrible situation, as a lack of trust is extremely corrosive to credibility and expectations, and thus to the smooth functioning of society. Therefore, those who abuse trust and goodwill are one of the greatest dangers to the continuance of civilization. Not the least of which is that allows for the proliferation of conspiracy theories and pseudoscience.
Part III puts forth what author believes to be the causes for these systemic issues and proposes various ways to mitigate them, as resolving them would require a very substantial shift in both individual behavior and global culture.
Here's a related blog post:
https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/
Although this book is about the many problems in science, it is entirely pro-science and strives to improve it through constructive criticism and the reduction of harmful behaviors by individuals and institutions. One of the primary criticisms I've seen of it is that it romanticizes the idea of the science and has too high of expectations for how people ought to behave. I understand this, but I don't agree with it. I don't believe it romanticizes science, though it does propose much higher standards than currently exist, and certainly much higher than are followed in practice. There are many unfortunate and relatable reasons why, several of which are described within.
Human limitations are the greatest obstacle to scientific progress, but unfortunately even accepting that doesn't seem to affect behavior much. These limitations are present at every structural level. None of this is new, history is littered with the bodies, figuratively and literally, of those who challenged their prevailing status quo. Their challenges weren't always warranted, but there have been many times when they were.
Example not in the book:
https://en.wikipedia.org/wiki/Ignaz_Semmelweis
A doctor told doctors that they should disinfect their hands because the scientific literature said it greatly reduced mortality rates. The doctors mocked and ridiculed him and eventually he's committed to an insane asylum in 1865, where he was beaten by the guards, and then died from a gangrenous wound. The Semmelweis Reflex, "a metaphor for the reflex-like tendency to reject new evidence or new knowledge because it contradicts established norms, beliefs, or paradigms." is named after him.
https://en.wikipedia.org/wiki/Semmelweis_reflex
The bulk of the book is concerned with psychology, because the author is a psychologist, though many other fields have examples given as well. These problems are not limited to anything specific, they are all-encompassing. The differences are in terms of frequency and severity. There are failures of self-discipline and ethical behavior from individuals and an immense failure of self-regulation by institutions. Not all the blame can be put on them as they are subjected to various perverse incentives and systemic societal pressures as well.
Many example stories are presented of the failures of individuals and institutions. There's overall more a focus on the specific than the general, which probably ought to be expected since psychology is more the study of the individual.
Part I is about how science works and the replication crisis, which is the failure of someone else being able to independently confirm the results of a study. This differs from the reproduction problem, where others cannot even attempt to test their results.
Part II details the faults and flaws, specifically fraud, bias, negligence and hype. Specific stories for each are presented.
A somewhat related story of negligence that comes to mind to me is of this serial killer:
https://en.wikipedia.org/wiki/Michael_Swango
Unfortunately, as noted in this book more than a few times, too much good faith is assumed and too much trust is given. This is a terrible situation, as a lack of trust is extremely corrosive to credibility and expectations, and thus to the smooth functioning of society. Therefore, those who abuse trust and goodwill are one of the greatest dangers to the continuance of civilization. Not the least of which is that allows for the proliferation of conspiracy theories and pseudoscience.
Part III puts forth what author believes to be the causes for these systemic issues and proposes various ways to mitigate them, as resolving them would require a very substantial shift in both individual behavior and global culture.
Here's a related blog post:
https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/
A must read for anyone who creates or consumes scientific findings. The author provides compelling root causes of the problem with science today then follows it up with equally compelling solutions that seem too obvious and simple to not have been implemented already. Seems to me like all the actors—governments, universities, non-profits, researchers and consumers of science—have all been caught up in an Abilene paradox, "where a group of people collectively decide on a course of action that is counter to the preferences of many or all of the individuals in the group" (source: Wikipedia).
This is a sobering read (particularly due to the fraud that happens on medical trials), but hopefully motivates you, like it has me, to read scientific papers much more closely and never hold them to be fact.
This is a sobering read (particularly due to the fraud that happens on medical trials), but hopefully motivates you, like it has me, to read scientific papers much more closely and never hold them to be fact.