-
Notifications
You must be signed in to change notification settings - Fork 686
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add umicollapse and benchmark it against umitools dedup #1087
Comments
Hi there, I wrote the module! We are using it in our updated nf-core/clipseq pipeline (not released yet). It is able to handle much larger bams and I have benchmarked that it gives identical results. So hopefully this should be an easy issue. |
Great work and thank you so much for the info! So then this issue comes down to writing a respective subworkflow similar to bam_dedup_stats_samtools_umitools, but using Maybe @drpatelh can give a hint, if he prefers replacing |
Be interested to see what your implementation looks like @CharlotteAnne ! But adding an independent subworkflow either way as you suggested @MatthiasZepper is a good idea. We can then decide how we slip it into the pipeline. |
you can see in the https://github.com/nf-core/clipseq/tree/feat-2-0 branch
of clipseq. we dont used the subworkflow because the samtools stats
clutters up our pipeline output without giving any real useful information
|
actually thats a lie lol at the moment we do
https://github.com/nf-core/clipseq/blob/feat-2-0/subworkflows/local/bam_dedup_samtools_umicollapse.nf
but im gonna refactor becuz of samtools stats clutter
|
Hello! After a conversation with @MatthiasZepper over Slack I decided I would get hands-on on this issue. I've already created a branch on a fork of the repo and added the functionality for the user to select whether they want to deduplicate using I have not made the pull request yet because I have not tested it and benchmarked, but feel free to "read" the changes in the code and give all the feedback you want. I'll keep you posted with the results once I have tested :) |
The added I'm testing only that it currently runs and completes with two samples of the first dataset linked, subsampled to 10k reads (I'm testing it locally 😅 ), but would love to really benchmark this implementation with more real data. Any suggestion or idea on how could we do this? |
@ctuni : Thank you so much for all your work and effort! Greatly appreciated! We have some sequencing data from human XpressRef Universal Total RNA in different input concentrations and kits, which I could run with your pipeline for comparison. It will take a couple of days, but then we have a relatively large dataset with a known ground truth to compare against. I could share e.g. the salmon quant files with you for your own analysis? |
To give you a small update about my progress (or the lack thereof 😬). I fetched @ctuni 's modification to the pipeline based on @CharlotteAnne 's subworkflow, but felt that for evaluation purposes, it would be better to compare the tools on exactly the same BAM files instead of realigning the same reads again. Therefore, I tried to With the same approach, I planned to also get a close comparison of the effects that the Prepare for Salmon process has, since the whole undertaking started in Slack with doubts about that script: #828. Thus, essentially getting 4 quantifications Even 256GB heap doesn't seem to be enough for a single pass run:
Therefore, I still have to play around with the Java Virtual Machine Memory settings for initial heap size, max heap size and stack size and probably set |
Of course, you can help! Much appreciated! The subworkflow with umicollapse is done, but I was holding back with integrating it into the Hoe about assigning yourself to this task and taking over? |
Smrnaseq has it implemented too - if you're looking for insights how it was implemented there, have a look 👍 |
Thank you! I don't have permissions to change assignments. Please do so if you can. I will send a PR shortly. |
Great! Please join the nf-core GitHub organization asap, so you can better interact with the different repos and issues! I think e.g. @drpatelh would be happy to send you an invitation, since you are contributing to the |
Running a little behind on this, but still on my list, somewhere near the top. |
Please don't stress yourself over it. I have been neglecting this for months and without your help, it would probably not make it anywhere soon either. So take your time, it will anyway not be included in the next release 3.15 anymore (for which a feature freeze is essentially in place) and the 3.16 release is still a few weeks ahead! |
Description of feature
umicollapse
is a Java tool and nf-core module so far not used by any pipeline.It was specifically written as a more performant tool, which supposedly produces output equivalent to umi-tools dedup, but with significantly shorter runtimes (minutes instead of hours).
Therefore, it might make sense to switch from
umi-tools dedup
toumicolllapse
, or at least offer it as an alternative route, similar to the coexistence of TrimGalore / FastP in the future.To advance this development, please:
Familiarize yourself with
umicollapse
and its settings.Create a new branch of the pipeline in your fork and add the
umicollapse
module to it, replacing theumi-tools dedup
step. Update the module config accordingly.Benchmark the modified pipeline against version 3.12 when run on data with UMIs, comparing runtimes and quantification results. Possible benchmarking datasets are e.g. GSE207129 or GSE179992 with those parameters
or Experiment 2 of Series GSE171427 when run with
I think, this is a suitable Hackathon task for a more advanced nf-core contributor. Thanks!
The text was updated successfully, but these errors were encountered: