Scan barcode
A review by lkedzie
Unreliable: Bias, Fraud, and the Reproducibility Crisis in Biomedical Research by Csaba Szabo
4.0
Unreliable is the opposite of whatever the true crime genre is.
It is about the reproducibility crisis, but not the crisis in general. It limits its treatment to the author's own specularity, biomedicine. It is not journalistic. It is not a discussion of notable science fraud or that fraud's perpetrators. It is so much Not That that the author summarizes the big events as part of a table and sparingly refers back to it.
It is a savvy choice. The author wants to look at the crisis structurally, which means jettisoning a 'few bad apples' paradigm - you know, the one that focusing on the biggest cases implies - and arguing for a persistent miasma.
Rephrased, Unreliable is a study of the structural causes to the problem of the reproducibility crisis. Individual incidents become unimportant. They are not the real problem.
This might sound like apology and rationalization. It is not.
It does mean that the 'innocent' causes of the reproducibility crisis are discussed at length. While almost the opposite of the book's point, it is a highly memorable chapter. A number of things that researchers have done to prevent reproducibility problems may instead inject them. And this provides something of a sub-theme to the book, namely the ways in which that the practice of science has ossified, where methodology in general, down to the scientific journal itself, came about for useful reasons, but have become contradictory to the process of science.
It is then the later chapters that turn the most dismal. It mirrors the first half of the book. The intentionally malignant acts, like fake papers and sketchy journals, are produced after explaining how the context of even the good ones leads to moral hazard and unintended negative consequences.
The solutions are a letdown. Another sub-theme to the book is how no one gets punished. There are sensible suggestions to address this, foremost being that this sort of verification work has to become its own sort of discipline within science. In general, retractions have to be more forceful and more meaningful, ideally alongside a sort of sabermetrics for scientists that reorients from a ranking system to one that includes other facets of the work.
The insensible solicitations are those that stand out. "This man does not follow U.S. politics," I said to my cats when reading the author's resumption of an older proposal to criminalize research fraud. No Attorney General would permit any immunologist to darken our streets again. And while I think that there is some validity in a surveillance-state approach (think in the context of all labs acting as if a chain of custody was relevant to their work) as it is described here seems creepy.
The question of audience is a problem. While the book specifically avoids being too technical (or per the text it locks that away in the end notes), it does cover a lot of the technical issues of its material, which I suspect that will be off-putting to some members of lay audience.
There are a lot of New Yorker-style cartoons throughout the text. They are primarily unfunny. The author himself, however, is pointedly hiding his own light under a bushel in terms of comedy. The appendices are mostly satire. They are funnier than all the comics.
Unreliable was not what I expected, but in a good way. The book resembles an anthropological study, with a side of memoir, but one with actionable solutions, if imperfect ones. The problem, though, may be similar to the one in Owned, where cures are situated beyond the range of systemic fixes on the scientific-procedural level, as opposed to the economic or political one.
My thanks to the author, Csaba Szabo, for writing the book, and to the publisher, Columbia University Press, for making the ARC available to me.
It is about the reproducibility crisis, but not the crisis in general. It limits its treatment to the author's own specularity, biomedicine. It is not journalistic. It is not a discussion of notable science fraud or that fraud's perpetrators. It is so much Not That that the author summarizes the big events as part of a table and sparingly refers back to it.
It is a savvy choice. The author wants to look at the crisis structurally, which means jettisoning a 'few bad apples' paradigm - you know, the one that focusing on the biggest cases implies - and arguing for a persistent miasma.
Rephrased, Unreliable is a study of the structural causes to the problem of the reproducibility crisis. Individual incidents become unimportant. They are not the real problem.
This might sound like apology and rationalization. It is not.
It does mean that the 'innocent' causes of the reproducibility crisis are discussed at length. While almost the opposite of the book's point, it is a highly memorable chapter. A number of things that researchers have done to prevent reproducibility problems may instead inject them. And this provides something of a sub-theme to the book, namely the ways in which that the practice of science has ossified, where methodology in general, down to the scientific journal itself, came about for useful reasons, but have become contradictory to the process of science.
It is then the later chapters that turn the most dismal. It mirrors the first half of the book. The intentionally malignant acts, like fake papers and sketchy journals, are produced after explaining how the context of even the good ones leads to moral hazard and unintended negative consequences.
The solutions are a letdown. Another sub-theme to the book is how no one gets punished. There are sensible suggestions to address this, foremost being that this sort of verification work has to become its own sort of discipline within science. In general, retractions have to be more forceful and more meaningful, ideally alongside a sort of sabermetrics for scientists that reorients from a ranking system to one that includes other facets of the work.
The insensible solicitations are those that stand out. "This man does not follow U.S. politics," I said to my cats when reading the author's resumption of an older proposal to criminalize research fraud. No Attorney General would permit any immunologist to darken our streets again. And while I think that there is some validity in a surveillance-state approach (think in the context of all labs acting as if a chain of custody was relevant to their work) as it is described here seems creepy.
The question of audience is a problem. While the book specifically avoids being too technical (or per the text it locks that away in the end notes), it does cover a lot of the technical issues of its material, which I suspect that will be off-putting to some members of lay audience.
There are a lot of New Yorker-style cartoons throughout the text. They are primarily unfunny. The author himself, however, is pointedly hiding his own light under a bushel in terms of comedy. The appendices are mostly satire. They are funnier than all the comics.
Unreliable was not what I expected, but in a good way. The book resembles an anthropological study, with a side of memoir, but one with actionable solutions, if imperfect ones. The problem, though, may be similar to the one in Owned, where cures are situated beyond the range of systemic fixes on the scientific-procedural level, as opposed to the economic or political one.
My thanks to the author, Csaba Szabo, for writing the book, and to the publisher, Columbia University Press, for making the ARC available to me.