If at first you do succeed: Publish a Replication Report with #RRIDs anyway!

Science is the act of trying and trying again, whether or not we confirm what we think should be happening. 

Begely and Ellis in their 2012 paper from Amgen stated that only about 11% of cancer studies were replicable sending shockwaves through the scientific community for years. However, the authors did not give the scientific community all of the data that showed the replicates.

This week in eLife, the Center for Open Science and a cohort of great ‘re-do-ers’ have just published the first batch of studies that are replicates of influential cancer studies, attempting to confirm or deny what the original study claimed. We at the RRID initiative have noted that the original studies often lacked identifying information in the reagents they used, as is alluded to by some of the replication attempts. These simple omissions make replication much more difficult, something that the ‘re-do-ers’ struggled with.

This is really a monumental step and we will wait for the final publications to determine whether these rigorous and fully transparent attempts also fall in the 11% replication level as claimed by Begely and Ellis, but so far some of the replicates show trends in the same direction reported by the original study authors, though no replication attempt has panned out exactly the same way as the original paper. We certainly need to wait and see for the rest of the reports, but I am personally heartened that the original authors are engaged in the replication, commenting on these reports attempting to understand their own data and the new data. 

The immortal Aristotle was once reported to say “Quality is not an act, it is a habit.” I think that if he were alive today he would be very interested in these developments and would implore us to look at ourselves and if we did not like what we see, he would call on us to change for the better. While none of my papers will likely be the target of this kind of scrutiny, I do hope that the methods and results will stand up in the long term. This is a call to action for all of us, to be more precise and to do better delivering on the promise of science for the patients who deserve our very best attention and very best methodology.