K optimization: r2 vs score

[nbedelman]

Hello,

I am running STITCH with a relatively small sample (40 individuals), with ~0.5X per-sample haplotagging (linked-read) sequencing. I recognize, that this sample size is pretty small for imputation, but I'm hoping STITCH will still be at least somewhat effective. I've run the program, varying K and number of generations as suggested, and am now evaluating the output. It seems that the mean score, and number of sites with scores > 0.4 increase as K increases from 2-35, where the values seem to asymptote. However, the r2 values reach their peak around K=14 (r2=0.875), and drop off on either side (K=35, r2=0.73). Number of generations has minimal effect on r2, but runs with fewer generations (10-100) consistently yield more sites with high scores than those with more generations (300-1000). Does this make sense, and would you recommend maximizing score values or r2 values when selecting K?

Thanks!
Nate

[rwdavies]

Hi Nate,

I'm honestly shocked that such large K perform well with N=40 0.5X samples (~20X total coverage).

But anyway, I would go with external r2 as the best measure, in general, for instance what you write about K=14. If as you say for the nGen parameter, there is limited effect on r2, but some on the INFO score, I would go with the option that maximized the INFO score anyway, just in case.

Best,
Robbie

[nbedelman]

perfect, thank you!

[EliseGAY]

Hi @rwdavies
I comment this (old) issue because we are facing similar questions so I would like to share our results so that perhaps you could help us understanding the patterns of our imputations. A bit of context, we have 96 individuals of vivparous lizard (wild population) sequenced with the haplotagging technique which generate linked reads but with a sequencing depth from 0.5 to 1X. We tried to assess the quality of imputation by testing combinations of parameters. We used different gridWindowsSize values (to speed up if possible) and different K. Knowing that the ngen doesn't have much impact I won't present it. We then checked out the info_score, haplotype proportion pattern and the R2.

1. First, we used the grid size to speed up our imputation, but we are not sure at which point it impacts the imputation. We tested various gridsize with a fixed K = 10. The infoscores are much better when the size increases but it has a huge impact on the haplotype frequency patterns. Also the r2 decreases as the gridsize increases :

We are very curious about the impact of the gridsize on the imputation, even if we probably won't use any size at the end.

2. We chose a gridsize = 1Kb in order to test other paramters. We tested several K and assessed the Info_score and the R2. In contrast to Nate's case, the R2 (always with a very huge dispersion) and the info_score increase continuously :

Based on that results, if we chose the pragmatic way we would say that the higher R2 and Infoscores, the better is the imputation. But this would correspond to K = 80. In another hand, we have also been told that the closer our K is to the number of individuals the more the imputation will overfit the data and generate incertainties.
A likely hypothesis is that with our data we are not able to properly assess the quality of imputation ? Or maybe you have some other way of thinking to share with us ?

Thanks a lot !
Have a good day

Elise

[rwdavies]

Sorry I never answered. gridWindowSize is an interesting one. That is supposed to speed things up. I never would have anticipated it would have such a large effect on accuracy! I assume this is because the sample size is very low, and the effective population size is very large, and also likely that the recombination rate is high, so STITCH is having trouble initializing. But having gridWindowSize set effectively very high, as it would be at 50kbp, is forcing it to come to a local solution in 50 kbp chunks, which seems to be helping a fair bit.

K is more standard. INFO scores almost always go up as K goes up. Imputation accuracy, measured externally, goes up to a point but then would start to drop off. I think in general I wouldn't set K more than about 40 without good reason, and with such a small sample size like here of about 100, I don't think I would set K higher than 10.

Hope that's useful, sorry for the slow response!