« EMBL Chemical Biology: Progress in Oncology |
| A Totally New Way to Finance Drug Discovery »
October 1, 2012
Six Rule of Reproducibility
Bruce Booth has a look at some rules suggested by Glenn Begley of Amgen, who's been involved in trying to reproduce published data. He's had enough bad experiences in that line (and he's not alone) that he's advocating these standards for evaluating something new and exciting:
1) Were studies blinded?
2) Were all results shown?
3) Were experiments repeated?
4) Were positive and negative controls shown?
5) Were reagents validated?
6) Were the statistical tests appropriate?
Applying these tests would surely scythe down an awful lot of the literature - but a lot of the stuff that would be weeded out would deserve it. I really wonder, for example, how many n=1 experiments make it into print; I'm sure it's far more than anyone would be comfortable with if we knew the truth. As I've mentioned here before, different fields have different comfort levels with what needs to be done to assure reproducibility, but I think that everyone would agree that complex biology experiments need all the backing up that they can get. The systems are just too complex, and there are too many places were things can go silently wrong.
That "Were all results shown" test is a tough one, too. Imagine a synthetic paper where each reaction has a note next to it, like "4/5", to show the number of times the reaction worked out of the total number of times it was tried. There would be a lot of "2/2", which would be fine, and (in total synthesis papers) some "1/1" stuff in the later stages, which readers could take or leave. But wouldn't it be instructive to see the "1/14"s in print? We never will, though. . .
+ TrackBacks (0) | Category: The Scientific Literature
POST A COMMENT
- RELATED ENTRIES
- How Not to Do It: NMR Magnets
- Allergan Escapes Valeant
- Vytorin Actually Works
- Fatalities at DuPont
- The New York TImes on Drug Discovery
- How Are Things at Princeton?
- Phage-Derived Catalysts
- Our Most Snorted-At Papers This Month. . .