Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add method to dump params object #331

Merged
merged 3 commits into from
Aug 6, 2024
Merged

Add method to dump params object #331

merged 3 commits into from
Aug 6, 2024

Conversation

j2salmingo
Copy link
Contributor

Description

Adds the method to dump the params object to the log directory.

Testing Results

NFTest

input:
sample_id: a_mini_n2_spark
input_csv: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/input/csv/a_mini_n2.csv
reference_fasta_bwa: /hot/ref/tool-specific-input/BWA-MEM2-2.2.1/GRCh38-BI-20160721/index/genome.fa
reference_fasta_index_files_bwa: /hot/ref/tool-specific-input/BWA-MEM2-2.2.1/GRCh38-BI-20160721/index/genome.fa.*
reference_fasta_hisat2: /hot/ref/reference/GRCh38-BI-20160721/Homo_sapiens_assembly38.fasta
reference_fasta_index_files_hisat2: /hot/ref/tool-specific-input/HISAT2-2.2.1/GRCh38-BI-20160721/index/Homo_sapiens_assembly38.*.ht2

output:
work_dir: /scratch/203433
output_dir: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark
output_dir_base_bwa: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark/align-DNA-10.1.0/a_mini_n2_spark
output_dir_base_hisat2: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark/align-DNA-10.1.0/a_mini_n2_spark
log_output_dir_bwa: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark/align-DNA-10.1.0/a_mini_n2_spark/log-align-DNA-10.1.0-20240805T185700Z/
log_output_dir_hisat2: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-use-methods-setup-process-afterscript/a_mini_n2-spark/align-DNA-10.1.0/a_mini_n2_spark/log-align-DNA-10.1.0-20240805T185700Z/

Checklist

  • I have read the code review guidelines and the code review best practice on GitHub check-list.

  • I have reviewed the Nextflow pipeline standards.

  • The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].

  • I have set up the branch protection rule following the github standards before opening this pull request, or the branch protection rule has already been set up.

  • I have added my name to the contributors listings in the manifest block in the nextflow.config as part of this pull request, am listed
    already, or do not wish to be listed. (This acknowledgement is optional.)

  • I have added the changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

  • I have updated the version number in the metadata.yaml and manifest block of the nextflow.config file following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)

  • I have tested the pipeline on at least one A-mini sample with aligner setting to BWA-MEM2, HISAT2, and both. The paths to the test config files and output directories were attached in the Testing Results section.

@j2salmingo j2salmingo requested a review from a team as a code owner August 5, 2024 19:18
@kiarod
Copy link
Contributor

kiarod commented Aug 6, 2024

Looks good! params.json seems to be created properly.

{
    "max_cpus": 32,
    "max_memory": {
        "mega": 62240,
        "kilo": 63733760,
        "giga": 60,
        "bytes": 65263370240
    },
    "min_cpus": 1,
    "min_memory": {
        "mega": 1,
        "kilo": 1024,
        "giga": 0,
        "bytes": 1048576
    },
    "ucla_cds": true,
    "mark_duplicates": true,
    "gatk_command_mem_diff": {
        "mega": 5120,
        "kilo": 5242880,
        "giga": 5,
        "bytes": 5368709120
    },
    "bwa_version": "BWA-MEM2-2.2.1",
    "hisat2_version": "HISAT2-2.2.1",
    "docker_container_registry": "ghcr.io/uclahs-cds",
    "pipeval_version": "4.0.0-rc.2",
    "bwa_and_samtools_version": "2.2.1_samtools-1.17",
    "hisat2_and_samtools_version": "2.2.1_samtools-1.17",
    "picardtools_version": "3.1.1",
    "output_dir": "/hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark",
    "docker_image_bwa_and_samtools": "ghcr.io/uclahs-cds/bwa-mem2:2.2.1_samtools-1.17",
    "docker_image_hisat2_and_samtools": "ghcr.io/uclahs-cds/hisat2:2.2.1_samtools-1.17",
    "docker_image_picardtools": "ghcr.io/uclahs-cds/picard:3.1.1",
    "docker_image_validate": "ghcr.io/uclahs-cds/pipeval:4.0.0-rc.2",
    "docker_image_gatk": "broadinstitute/gatk:4.2.4.1",
    "docker_image_samtools": "ghcr.io/uclahs-cds/samtools:1.17",
    "ucla_cds_analyte": "DNA",
    "sample_id": "a_mini_n2_spark",
    "input_csv": "/hot/software/pipeline/pipeline-align-DNA/Nextflow/development/input/csv/a_mini_n2.csv",
    "dataset_id": "0000000",
    "reference_fasta_bwa": "/hot/ref/tool-specific-input/BWA-MEM2-2.2.1/GRCh38-BI-20160721/index/genome.fa",
    "reference_fasta_hisat2": "/hot/ref/reference/GRCh38-BI-20160721/Homo_sapiens_assembly38.fasta",
    "hisat2_index_prefix": "/hot/ref/tool-specific-input/HISAT2-2.2.1/GRCh38-BI-20160721/index/Homo_sapiens_assembly38",
    "aligner": [
        "BWA-MEM2",
        "HISAT2"
    ],
    "save_intermediate_files": false,
    "cache_intermediate_pipeline_steps": false,
    "enable_spark": true,
    "spark_metrics": false,
    "ucla_cds_registered_dataset_input": false,
    "ucla_cds_registered_dataset_output": false,
    "input": {
        "FASTQ": [
            {
                "read_group_identifier": "HG002.N.seq1",
                "sequencing_center": "BS",
                "library_identifier": "bamsurgeon",
                "platform_technology": "ILLUMINA",
                "platform_unit": "NA",
                "lane": "1",
                "sample": "HG002.N",
                "read1_fastq": "/hot/data/TEST/TEST0000000/TWGSAMIN000001/TWGSAMIN000001-N001-S01-F/DNA/WGS/raw/FASTQ/HG002-N-A-mini-n2/HG002.N-n2_R1.fq.gz",
                "read2_fastq": "/hot/data/TEST/TEST0000000/TWGSAMIN000001/TWGSAMIN000001-N001-S01-F/DNA/WGS/raw/FASTQ/HG002-N-A-mini-n2/HG002.N-n2_R2.fq.gz"
            }
        ]
    },
    "make_duplicates": true,
    "disable_alt_aware": false,
    "ucla_cds_registed_dataset_output": false,
    "output_dir_base": "/hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark/align-DNA-10.1.0/a_mini_n2_spark",
    "log_output_dir": "/hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-param-dump/a_mini_n2-spark/align-DNA-10.1.0/a_mini_n2_spark/log-align-DNA-10.1.0-20240805T185700Z/",
    "work_dir": "/scratch/203433",
    "spark_temp_dir": "/scratch/203433",
    "reference_fasta_index_files_bwa": "/hot/ref/tool-specific-input/BWA-MEM2-2.2.1/GRCh38-BI-20160721/index/genome.fa.*",
    "reference_fasta_index_files_hisat2": "/hot/ref/tool-specific-input/HISAT2-2.2.1/GRCh38-BI-20160721/index/Homo_sapiens_assembly38.*.ht2"
}

@kiarod kiarod merged commit ddd326a into main Aug 6, 2024
7 checks passed
@kiarod kiarod deleted the jsalmingo-add-param-dump branch August 6, 2024 17:21
@kiarod kiarod restored the jsalmingo-add-param-dump branch August 6, 2024 17:21
@kiarod
Copy link
Contributor

kiarod commented Aug 6, 2024

whoops sorry @j2salmingo didn't mean to merge it in myself just went into autopilot

@yashpatel6 yashpatel6 deleted the jsalmingo-add-param-dump branch August 6, 2024 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants