Twitter has been abuzz with Orna Man and Yoav Gilad’s (re)analysis of the data from a recent PNAS paper: “Comparison of the transcriptional landscapes between human and mouse tissues”.
The PNAS paper concluded that the gene expression profiles of different tissues within the same species were more similar than the same tissue across different species. For example, their analysis showed that the gene expression profile of a mouse spleen was more similar to that of a mouse heart than it was to a human spleen. This didn’t seem to mesh with well previous research, so it didn’t make much sense. Well, that is until Man and Gilad decided to probe it a bit further.
Once Man and Gilad got their hands on the data (hooray for data transparency!) they noticed something that quickly raised their eyebrows: the human and mouse tissues were sequenced in separate lanes and on separate runs! In other words, the inter-species comparison they were testing was completely confounded with a well-known batch effect: sequencing run (and lane).
Sequencing lane (a batch effect) was almost completely confounded with species in the PNAS study. From @Y_Gilad
Next, they controlled for the batch effect of sequencing run (and lane). Unsurprisingly they found that gene expression profiles clustered by tissue first, and then by species (see figure below).
After controlling for the batch effects, gene expression clusters by tissue first, not species. From @Y_Gilad
So, what gives? Does this mean that there results of the PNAS paper are completely bogus? Not necessarily. When Man and Gilad removed the batch effects they could have also removed some inter-species effects. Unfortunately, there’s no way to disentangle the two given the poor experimental design. The only way to do so would be to redo the experiment and balance the tissues and species across each sequencing lane (or, even better, just multiplex them all in the same pool and spread it across multiple lanes). We’ll just have to wait and see if someone else, or the authors, decide to generate new data from a more thoughtfully designed experiment.
Retraction watch!
There have been a lot of calls for the authors to retract their original paper given the flawed experimental design. I’m not going to weigh in on this, but I will direct you to a thoughtful blog post from Arjun Raj about dealing with “retraction in the age of computation”.
REFERENCES
Lin S, Lin Y, Nery JR, Urich MA, Breschi A, Davis CA, Dobin A, Zaleski C, Beer MA, Chapman WC, Gingeras TR, Ecker JR & Snyder MP (2014) Comparison of the transcriptional landscapes between human and mouse tissues. Proc. Natl. Acad. Sci. 111, 201413624.
Man O & Gilad Y (2015). I’m calling your experimental design out on Twitter. Twitter.