Handling missing values in SNP matrix

[naglemi]

Thanks for making this tool available. I wasn't sure whether to mark this as a question or issue because it's unclear to me if there is existing support for PCA and GWAS with SNP matrices that contain NAs. I am accustomed to setting thresholds in PLINK and GEMMA for which all SNPs missing in more than X genotypes are excluded, but haven't found a similar feature for bigsnpr in documentation or the demo.

I'm trying to run my data, which I've formatted the same way as the demo data, but there are NAs in my SNP set (from QC filtering, indels, etc) and this leads me to the error below.

Do I need to format the NAs a particular way, or do anything else to build models that include SNPs not found in all genotypes?

Error: You can't have missing values in 'X'.
Traceback:

1. big_univLogReg(G, y01.train = y01, covar.train = covariates, ncores = NCORES)
2. check_args()
3. with(args, eval(parse(text = check[i])))
4. with.default(args, eval(parse(text = check[i])))
5. eval(substitute(expr), data, enclos = parent.frame())
6. eval(substitute(expr), data, enclos = parent.frame())
7. eval(parse(text = check[i]))
8. eval(parse(text = check[i]))
9. assert_noNA(X)
10. stop2("You can't have missing values in '%s'.", deparse(substitute(x)))
11. stop(sprintf(...), call. = FALSE)

[privefl]

Yes, most of the functions in the packages don't handle missing values.
Note that I will add a PCA algorithm that handle missing values soon (in @bedpca branch).
This is not planned for GWAS since I extensively use linear algebra, which does not mix well with missing values.

If you want to use {bigsnpr}, you should therefore filter to no missing values or impute them.
To impute, you can impute using snp_fastImpute() which should be taking a few hours for say 10K samples and 500K variants.
Otherwise, you could impute with the mean (rounded), or the mode, or randomly according to frequencies, or by 0. Although these are not very hard to code, I should probably make them available as functions in {bigsnpr}.

[privefl]

I've just added snp_fastImputeSimple() to do a very fast imputation.

[privefl]

Is this answering your questions?

[naglemi]

Thanks for clarifying. Is imputation mathematically equivalent to excluding genotypes missing a given SNP when building the model for that SNP? Should it produce the same test statistics? I think most of the other GWAS methods we use (particularly GEMMA) exclude those genotypes instead of imputing.

[privefl]

No, I don't think that this is equivalent.
If the imputation is good, then I think you get more power.
Also the best thing to do might be multiple imputation, but requires more computation.

[ghost]

Hi Florian,
Thanks for making the very fast imputation function. Will it be possible that you add "impute by 0" option as well?

[privefl]

Yes, I guess it would be easy.
I just feel like it should be best to impute by any of the other methods.
What do you think?

[ghost]

I just think this is more conservative.

[privefl]

Should be now implemented in latest version, using method = "zero".

[ghost]

Thanks so much Florian!

…

On Wed, Jan 29, 2020 at 9:37 PM Florian Privé ***@***.***> wrote: Should be now implemented in latest version, using method = "zero". — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49?email_source=notifications&email_token=AJWVPHFY5YGITK7ZMGB2C43RAFL5FA5CNFSM4ISQ4MYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKGXHEI#issuecomment-579695505>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJWVPHCDA3Z5GR52Z6LI673RAFL5FANCNFSM4ISQ4MYA> .

[privefl]

@wongck-kevin I just remembered why I didn't implement this before.

In fact, you can do G2 <- G$copy(code = c(0, 1, 2, 0, rep(NA, 252))), and you will access all NAs as 0s when using G2 instead of G.
So, by using this method, you could impute by 1 or 2 as well.

[ghost]

Thanks for the update Florian.

[privefl]

Note that I will soon deprecate 'method = "zero"' in favor of using "$copy()".
This will give a warning only.