In yesterday’s New York Times, Kira Peikoff reported what happened when she took genetic tests for disease risks from three different providers—she got three very different results.
23andMe said my most elevated risks — about double the average — were for psoriasis and rheumatoid arthritis, with my lifetime odds of getting the diseases at 20.2 percent and 8.2 percent. But according to Genetic Testing Laboratories, my lowest risks were for — you guessed it — psoriasis (2 percent) and rheumatoid arthritis (2.6 percent).
For coronary heart disease, 23andMe and G.T.L. agreed that I had a close-to-average risk, at 26 to 29 percent, but Pathway listed my odds as “above average.”
The tone of the article is one of genuine surprise: why would three tests based on reading DNA—the very base code of our biology—come up with such completely different results? But the reaction among the genetics-savvy folks I follow on Twitter was mostly been there, seen that. And, really, Peikoff’s experience isn’t a surprise if you know a bit about the origins of the data used in the genetic tests she took.
To her credit, Peikoff does a good job of explaining the proximate reason for the discrepancies: the different services she used often looked at different genetic markers, or different sets of markers, to come up with disease risk assessments. But why on earth would they be using different markers? It all comes down to the origins of the genetic risk assessments that form the basis of the tests.
A beginner’s guide to association genetics
The bread-and-butter of modern human genetics is the genome-wide association (GWA) study. To do a very simple GWA study, geneticists recruit a sample of people—some who have a particular disease, and some who don’t—and sequence them at many different genetic markers, spaced across the genome. Nowadays, these markers are almost always single-nucleotide polymorphisms, or SNPs—individual “letters” in the genetic code that differ from person to person. Then, they look for SNPs at which everyone with the disease has one version of the SNP, and everyone without the disease has another version. We say that those SNPs show strong association with the disease.
Over the last decade or more, we’ve built up a very large scientific literature of studies like this, identifying markers associated with diseases from diabetes to hypertension and schizophrenia. Genetic testing companies can take a customer’s genotype at SNPs that have been used in lots of different GWA studies, and then dig through that literature to see whether any of the versions of the SNPs carried by the customer have been found to be associated with disease.
And there’s the rub. How testing companies go about digging through that GWA literature, and what they decide to report, will strongly affect how they interpret the raw genetic data they collect from their customers.
Association is not causation
First and foremost, the associations between SNPs and disease found by GWA are just that—associations. Or, to use the synonym that will trigger skepticism in most informed non-scientists, they’re correlations between the presence of the SNP variant and the disease. Geneticists rarely expect that a SNP variant with even very strong association to a disease directly causes that disease; instead, we take strong association to indicate that the SNP may lie in or near the stretch of genetic code that has the direct causal relationship. Generally, we consider it an encouraging sign if multiple GWA studies find associations between a disease and different SNPs in the same general genetic neighborhood—that suggests where we might find an actual causal gene.
Genetics is complex!
Second, many GWA studies of disease are not designed to categorize the people in the study samples in the binary terms I sketched out above—do they or do they not have the disease?—but instead look for associations with risk of disease. One of the broad findings to come out of all the GWA work is that many diseases are affected by multiple parts of the genome, so that each individual genetic variant elevates the risk of disease just a bit. These subtler effects are harder to identify with statistical certainty, and they mean that learning you carry a SNP variant associated with Type II diabetes is not at all the same thing as receiving the results of a blood test that shows your blood sugar regulation is out of whack. To come up with a risk estimate, testing companies may consider the cumulative effects of many disease-associated markers, and it will still be rare that they produce a black-and-white diagnosis.
In her article, Peikoff correctly notes that environmental variation contributes to uncertainty in genetic risk associations—which is obvious if you’ve ever known identical twins who follow different fitness regimens. Modern GWA studies generally aim to use samples of people who have lived in very similar environments, and use large enough samples that the remaining variation can be treated as a statistical “noise,” which weakens the ability to detect associations, but doesn’t positively mislead the test. That said, if Peikoff is living a different lifestyle from the people who participated in a GWA study that found disease risk associated with a particular SNP variant, she might have a very different risk of disease than the estimate from that study.
Ancestry matters
Finally, in part because of their association-not-causation nature, the results of GWA studies can vary in different populations. This is because the frequency of SNP variants differs with geographic origin—two populations that spend many generations without directly interbreeding will develop different patterns of genetic variation as the result of genetic drift, random changes in the frequency of genetic variants over time. Genetic ancestry and paternity analyses take advantage of these genetic differences, and they can have a confounding effect in GWA studies.
As a very simple example, imagine a GWA study based on a sample consisting of two extended families, one with a history of heart disease, and the other with very little heat disease. In that study, any SNP that differed between the two families—and many would do so simply as a result of their different ancestry—would show a strong association with heart disease. For this very reason, nobody actually designs GWA studies this way! Geneticists construct samples very carefully to minimize this effect of relatedness among individuals, and there are statistical methods that can help to control for it as well. Also, it’s now standard practice to conduct GWA studies using at least two independent samples of people—and to only accept SNPs that show up as strongly associated in both samples.
But even with these measures in place, the results of any given GWA study are specific to the population sampled. As a direct illustration of this principle, a study recently published in PLOS Biology took a list of SNPs with disease associations in samples of people with European ancestry, and asked how often the same SNPs had similar associations in samples of African Americans, Asian Americans, Hispanic Americans, Native Americans, and Pacific Islanders. The authors found that while the SNPs they examined generally had similar associations in European and non-European populations, the strength of association often differed between populations. So, if you’re Hispanic, a risk estimate based on a GWA study of Asian Americans may be less accurate than one based on a sample of Hispanic Americans.
Caveat emptor!
Peikoff took tests from 23andMe, the frontrunner in this sort of “direct to consumer” genetic testing, which the FDA recently barred from making health risk assessments; from Genetic Testing Laboratories; and from Pathway Genomics. How these three companies collect and report genetic data differs quite a bit.
The 23andMe test is quite open-ended—it genotypes customers at millions of SNPs, whether they’re known to be associated with disease or not, and generates reports on ancestry and (until the FDA thing) associations with disease based on the whole kit and caboodle. 23andMe also lets customers download all this data for their own examination, and (again, until the FDA thing) sent update reports when new associations were found with the markers they’d already genotyped. (I’ve been meaning to try out the 23andMe service myself for some time, but haven’t got around to it yet—now that it looks like they’re still allowed to sell the raw data, I may finally go ahead.)
The other two companies appear to take a more cautious approach, testing genotypes at markers already known to be associated with lists of specific medical conditions, and very explicitly providing access to professional genetic counseling. The GTL test is only available via a physician, and Pathway includes genetic counseling as part of its purchase price.
As she notes, Peikoff got different genetic test results because the three testing companies based their risk assessments on different sets of genetic markers—and it’s likely that they selected those differing sets of markers based on differing interpretations of the many, many GWA studies looking for genetic markers associated with disease. Whether and how those judgements are made isn’t spelled out on the testing companies’ respective websites, and I wouldn’t expect a layperson to be able to make an informed choice among schemes for interpreting the GWA literature anyway.
It’s also possible that they could have used an estimate of Peikoff’s ancestry to adjust their evaluation of different study results, giving more weight to associations found in people with similar ancestry. The sample report from GTL [PDF] does indeed include some information about ancestry in connection with the risk assessments, but it’s not clear how exactly it’s been taken into account.
As Peikoff notes, we’re still at the very beginning of understanding how best to apply genetic data to medical care, and one hopes that industry and the FDA will begin to develop some concrete standards for dealing with, or at least disclosing, the issues I’ve discussed here. But some of them are still subjects for basic research—particularly questions about how environments modulate specific genetic risks, and how well individual associations translate among different human populations. I’m not sure that getting your genetic profile will ever be as routine and straightforward as a test for elevated cholesterol or strep throat, and it’s going to be a long time before they get anywhere close.
Until we do, I expect we’ll be seeing articles like Peikoff’s at regular intervals.
References
Carlson, C. S., Matise, T. C., North, K. E., Haiman, C. A., Fesinmeyer, M. D., Buyske, S., … Kooperberg, C. L. (2013). Generalization and dilution of association results from European GWAS in populations of non-European ancestry: The PAGE study. PLoS Biology, 11(9):e1001661. doi: 10.1371/journal.pbio.1001661.
Plomin, R., Haworth, C., & Davis, O. 2011. Common disorders are quantitative traits. Nature Reviews Genetics, 10:872-9. doi: 10.1038/nrg2670.
