Process CPCG0196-F1 with Spark

[tyamaguchi-ucla]

Updated here.

I reran all testing results and there are no longer duplicates. Here is the multi-library example:

@jarbet Cool. Then, we should be able to process CPCG0196-F1 with Spark if the bug was introduced a while ago.

Originally posted by @tyamaguchi-ucla in #238 (comment)

[jarbet]

Linking issue #229

[jarbet]

@tyamaguchi-ucla: Unfortunately, still running out of scratch space:

Testing Results

• BWA-MEM2

  • sample: CPCG0196-F1

  • input csv: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/input/csv/CPCG0196-F1.csv

  • config: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jarbet-CPCG0196-F1/BWA-MEM2-CPCG0196-F1.config

  • output:

    • /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jarbet-CPCG0196-F1/BWA-MEM2-CPCG0196-F1.log

    Error executing process > 'align_DNA_BWA_MEM2_workflow:run_MarkDuplicatesSpark_GATK'
    Caused by:
    Process align_DNA_BWA_MEM2_workflow:run_MarkDuplicatesSpark_GATK terminated with an error exit status (3)

    22/09/14 22:03:28 ERROR Utils: Aborting task org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device

    • /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jarbet-CPCG0196-F1/align-DNA-8.1.0/CPCG0196-F1/log-align-DNA-8.1.0-20220913T215754Z/nextflow-log/report.html

    run_MarkDuplicatesSpark_GATK:
    

[tyamaguchi-ucla]

Thanks for testing @jarbet. Let me post potential solutions at #229 .