No. Stop.
At least that is the message from a new publication in Methods in Ecology and Evolution by Pierre Legendre and colleagues (pay-walled, but I found a pdf here).
Mantel tests should simply not be used to test hypotheses that concern the raw data from which dissimilarity matrices can be computed or to control for spatial structures in tests of relationships between two autocorrelated data sets.
The Mantel test (1967) is a non parametric analysis of the relationship between two dissimilarity matrices and was first developed to characterizing the epidemiological structuring of diseases. Decades later, Mantel tests became the most popular component of landscape genetics analyses as a way to straightforwardly integrate spatial relationships (distance matrices) into analyses of genetic data.
After being separated from its original use and adapted across many studies using spatial and genetic data, the Mantel test started to show signs of weakness (reviewed partially in Legendre & Fortin 2010). Fun paper titles such as “Dismantling the Mantel test” were stacking up, showing that this family of analyses often found spatial patterns when there were none. However, this was a point well-debated, as others suggested that the type I error rates from Mantel tests weren’t as big of an issue and could be avoided with careful implementation (Diniz-Filho et al. 2013 is one example).
I, like many, was confused by all this. The statistical validity of Mantel tests was analogous to the nutritional value of milk. Is it good? Is it bad? Does anyone really know?
“Mantel tests were a bad choice”
This new paper from Legendre and colleagues focuses on a different point of contention. Specifically, the word dissimilarity. They argue that distances (genetic distances, euclidean distances) are special types of dissimilarity that are inherently different than a true dissimilarity index (“a function that measures the difference between two vectors”). Because the mantel test is one of the absence of relationship between the dissimilarities of two, true dissimilarity matrices, if one of those isn’t a dissimilarity matrix, you are doing the wrong test.
Another way of putting this would be that by calculating a distance with raw data and then using a Mantel test, you are losing potential information and not implementing a Mantel test in the way it was designed:
This paper has shown that there are more implicit assumptions behind the apparently simple decision to run a Mantel test in the context of spatial analysis than meets the eye.
The authors use a literature review combined with independent simulations to show that Mantel tests are consistently out-performed by distance-based Moran’s Eigenvector Maps (dbMEM), a method that models spatial structure based on the decomposition of a distance matrix. Additionally, the R-squared values produced by dbMEM are probably more biologically-interpretable .
Now, none of this is to say that the Mantel test itself is invalid, just that it has been used for the wrong data in landscape genetic studies. If you are truly working with dissimilarity matrices (which is rare in evolution and ecology), go for it. But anytime spatial relationships come into play, Legendre and colleagues are suggesting new alternatives.
Cited
Diniz-Filho, J. A. F., Soares, T. N., Lima, J. S., Dobrovolski, R., Landeiro, V. L., Telles, M. P. D. C., … & Bini, L. M. (2013). Mantel test in population genetics. Genetics and molecular biology, 36(4), 475-485.
Guillot, G., & Rousset, F. (2013). Dismantling the Mantel tests. Methods in Ecology and Evolution, 4(4), 336-344.
Legendre, P., & Fortin, M. J. (2010). Comparison of the Mantel test and alternative approaches for detecting complex multivariate relationships in the spatial analysis of genetic data. Molecular Ecology Resources, 10(5), 831-844.
Legendre, P., Fortin, M. J., & Borcard, D. (2015). Should the Mantel test be used in spatial analysis?. Methods in Ecology and Evolution. DOI: 10.1111/2041-210X.12425
Mantel, N. (1967) The detection of disease clustering and a generalized regression approach. Cancer Research, 27, 209–220.