Add merging workflow #15
Conversation
There was a problem hiding this comment.
Interestingly, the runtime seems to be higher with the deduplication included even though the scattered split had 50 intervals vs only 26 from the chromosomes.
This makes sense to me and your benchmarking really proved it. I think we can do #12 next?
| params.gatk_command_mem_diff: float(memory) | ||
| params.parallelize_by_chromosome: bool. | ||
| */ | ||
| process run_MergeSamFiles_Picard { |
There was a problem hiding this comment.
As you are aware, we can implement #12 in a separate PR and that should help reduce the run time for large BAMs.
There was a problem hiding this comment.
That should reduce the runtime yes, but before doing that, I'm planning to add in the remaining processes (intermediate file deletion and the coverage processes) to have an exact copy of what's currently in call-gSNP before modifying the processes
|
Also, let's add @Faizal-Eeman as a reviewer from now on. I want Faizal to familiarize himself with this new pipeline and call-gSNP. @Faizal-Eeman let us know if you have any questions. |
|
Is the recalibration step getting removed from call-gSNP and becoming into this stand-alone pipeline? |
|
Correct, all BAM processing steps will be removed from call-gSNP and put into this pipeline |
Description
Add workflow to merge split BAMs.
If split by chromosome, deduplication is skipped. If split more generally, deduplication process is run. Interestingly, the runtime seems to be higher with the deduplication included even though the scattered split had 50 intervals vs only 26 from the chromosomes.
Testing Results
/hot/software/pipeline/pipeline-recalibrate-BAM/Nextflow/development/input/yaml/single_test_input.yaml/hot/software/pipeline/pipeline-recalibrate-BAM/Nextflow/development/unreleased/yashpatel-add-merging-workflow/single_wgs/single_wgs.config/hot/software/pipeline/pipeline-recalibrate-BAM/Nextflow/development/unreleased/yashpatel-add-merging-workflow/single_wgs/hot/software/pipeline/pipeline-recalibrate-BAM/Nextflow/development/input/yaml/paired_test_input.yaml/hot/software/pipeline/pipeline-recalibrate-BAM/Nextflow/development/unreleased/yashpatel-add-merging-workflow/multi_wgs/multi_wgs.config/hot/software/pipeline/pipeline-recalibrate-BAM/Nextflow/development/unreleased/yashpatel-add-merging-workflow/multi_wgsChecklist
I have read the code review guidelines and the code review best practice on GitHub check-list.
I have reviewed the Nextflow pipeline standards.
The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].
I have set up or verified the branch protection rule following the github standards before opening this pull request.
I have added my name to the contributors listings in the
manifestblock in thenextflow.configas part of this pull request, am listedalready, or do not wish to be listed. (This acknowledgement is optional.)
I have added the changes included in this pull request to the
CHANGELOG.mdunder the next release version or unreleased, and updated the date.I have updated the version number in the
metadata.yamlandmanifestblock of thenextflow.configfile following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)I have tested the pipeline on at least one A-mini sample.