Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jsalmingo-add parameter validation #277

Merged
merged 6 commits into from
Aug 18, 2023
Merged

Conversation

j2salmingo
Copy link
Contributor

@j2salmingo j2salmingo commented Aug 16, 2023

Description

Closes #260

Adds parameter validation

NOTE: The original input csv file contained a UTF-8 BOM which needed to be deleted in order to pass validation. Please see uclahs-cds/pipeline-Nextflow-config#27

Testing Results

  • BWA-MEM2 & HISAT2

    • sample: A-mini-N2
    • input csv: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation/a_mini_n2_no_BOM.csv
    • config: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation/a_mini_n2-all-pass.config
    • output: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation/output
  • BWA-MEM2 & HISAT2

    • sample: A-mini-N2
    • input: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation/a_mini_n2-picard_yaml.yaml
    • config: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation/a_mini_n2-all-pass-yaml.config
    • output: /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation/output

For invalid cases to test validation, the following config files and input_csvs were used, located in the /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/unreleased/jsalmingo-add-parameter-validation directory:

a_mini_n2-picard-invalid-aligner.config
a_mini_n2-picard-invalid-dir.config
a_mini_n2-picard-misspelled-header.config
a_mini_n2-picard-no-fq2.config
a_mini_n2_no_BOM_invalid_dir.csv
a_mini_n2_no_BOM_misspelled_header.csv
a_mini_n2_no_BOM_no_fq2.csv

Checklist

  • I have read the code review guidelines and the code review best practice on GitHub check-list.

  • I have reviewed the Nextflow pipeline standards.

  • The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].

  • I have set up the branch protection rule following the github standards before opening this pull request, or the branch protection rule has already been set up.

  • I have added my name to the contributors listings in the manifest block in the nextflow.config as part of this pull request, am listed
    already, or do not wish to be listed. (This acknowledgement is optional.)

  • I have added the changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

  • I have updated the version number in the metadata.yaml and manifest block of the nextflow.config file following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)

  • I have tested the pipeline on at least one A-mini sample with aligner setting to BWA-MEM2, HISAT2, and both. The paths to the test config files and output directories were attached in the Testing Results section.

@j2salmingo j2salmingo self-assigned this Aug 16, 2023
@j2salmingo j2salmingo requested a review from a team as a code owner August 16, 2023 18:06
@j2salmingo j2salmingo linked an issue Aug 16, 2023 that may be closed by this pull request
@j2salmingo j2salmingo changed the title jsalmingo-add parameter validatio jsalmingo-add parameter validation Aug 16, 2023
@yashpatel6
Copy link
Contributor

The validation seems to be good, a couple of things:

  • For development testing, you'll want to use the directories under /hot/software/pipeline/.../development/unreleased/<branch name> so we have the testing outputs and the tests saved in a common place.
  • For the purposes of this validation, valid inputs are passing, can you also do a run with failing inputs (ex. the input CSV is messed up or missing fields) to confirm that we're catching those cases as expected?

@j2salmingo
Copy link
Contributor Author

I've updated the comment label to use the appropriate directory, and added test cases with invalid config files and invalid input csv cases, as well as testing with YAML input.

Copy link
Contributor

@yashpatel6 yashpatel6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good! Is this closing #239? I don't think any of the changes are relevant to that issue.

Also, if the standard csv has the BOM issue, can you update the original file /hot/software/pipeline/pipeline-align-DNA/Nextflow/development/input/csv/a_mini_n2.csv?

@j2salmingo
Copy link
Contributor Author

Oh, that's my mistake with #239. I confused nextflow-module/validate with nextflow-config. I shall edit and merge.

@j2salmingo j2salmingo merged commit 5c21bc4 into main Aug 18, 2023
1 check passed
@j2salmingo j2salmingo deleted the jsalmingo-parameter-validation branch August 18, 2023 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add parameter validation through schema validation
2 participants