Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple pipeline level nf-tests #1272

Merged
merged 12 commits into from
Apr 9, 2024
Merged

Simple pipeline level nf-tests #1272

merged 12 commits into from
Apr 9, 2024

Conversation

adamrtalbot
Copy link
Contributor

@adamrtalbot adamrtalbot commented Mar 21, 2024

The role of this PR is to open a discussion on how to organise and run pipeline level tests.

I have directly replaced the existing tests with simple tests which change a single parameter and check for succesful completion, no snaps, no hashes etc. That can come later.

In this PR I'd like to discuss test organisation, naming, handling etc. I've tried my best to make it simple and easy to understand while being flexible enough to work efficiently, let me know what you think.

Additions:
- Runs simple tests for workflows
- Directly replaces existing tests in .github/workflows
- No checks for outputs or contents
Copy link

github-actions bot commented Mar 21, 2024

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 76473d2

+| ✅ 169 tests passed       |+
#| ❔   8 tests were ignored |#
!| ❗   7 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: assets/multiqc_config.yml
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • nextflow_config - Config default ignored: params.ribo_database_manifest
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/rnaseq/rnaseq/.github/workflows/awstest.yml
  • multiqc_config - 'assets/multiqc_config.yml' not found

✅ Tests passed:

Run details

  • nf-core/tools version 2.13.1
  • Run at 2024-04-09 11:02:13

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/tests/default/?
why not tests/default/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is to keep it tidy. You have a primary tests/ folder with tests, lib and other stuff (e.g. conf).

tests
├── lib
│   └── UTILS.groovy
├── nextflow.config
└── tests
    ├── default
    │   ├── main.nf.test
    │   └── main.nf.test.snap
    ├── featurecounts
    │   └── main.nf.test
    ├── min_mapped_reads
    │   └── main.nf.test
    ├── remove_ribo_rna
    │   └── main.nf.test
    ├── skip_qc
    │   └── main.nf.test
    └── skip_trimming
        └── main.nf.test

Without it, you would end up with this which I think is messy:

tests
├── default
│   ├── main.nf.test
│   └── main.nf.test.snap
├── featurecounts
│   └── main.nf.test
├── lib
│   └── UTILS.groovy
├── min_mapped_reads
│   └── main.nf.test
├── nextflow.config
├── remove_ribo_rna
│   └── main.nf.test
├── skip_qc
│   └── main.nf.test
└── skip_trimming
    └── main.nf.test

But I could be convinced otherwise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could move the tests/nextflow.config file into tests/config/nextflow.config for consistency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd go with:

tests
├── default.main.nf.test
├── default.main.nf.test.snap
├── featurecounts.main.nf.test
├── lib
│   └── UTILS.groovy
├── min_mapped_reads.main.nf.test
├── nextflow.config
├── remove_ribo_rna.main.nf.test
├── skip_qc.main.nf.test
└── skip_trimming.main.nf.test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh bother automerge was too quick.

@adamrtalbot adamrtalbot added this to the nf-test milestone Apr 9, 2024
@adamrtalbot adamrtalbot merged commit 943e271 into dev Apr 9, 2024
89 checks passed
@maxulysse maxulysse deleted the pipeline_level_tests_simple branch April 9, 2024 11:57
@adamrtalbot
Copy link
Contributor Author

We made it! 🥳

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants