Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add all nf-core modules and CI/CD and all nf-tests to everything #1221

Merged
merged 40 commits into from
Mar 4, 2024
Merged

Add all nf-core modules and CI/CD and all nf-tests to everything #1221

merged 40 commits into from
Mar 4, 2024

Conversation

adamrtalbot
Copy link
Contributor

@adamrtalbot adamrtalbot commented Feb 23, 2024 •
edited
Loading

This PR is in dev while I work out the kinks.

Based on #1220

This PR adds all nf-tests.

Additions:

  • Adds change based CI/CD workflow from fetchngs
  • Require nf-test to pass before merging
  • Add all nf-tests from nf-core/modules

Removes:

  • Removes '-profile test' test and replaces with nf-test version

To do:

  • Add nf-test to all local modules

Broken modules

The following modules from nf-core/modules have broken tests in rnaseq. Fix them there, update them in rnaseq and the tests will start to work. Follow this issue for an up-to-date set of broken modules: #1196

Once those modules are updated, all checks should pass. Once done, this should be able to be merged.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/rnaseq branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 23, 2024 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit b42c8f6

+| ✅ 169 tests passed       |+
#| ❔   8 tests were ignored |#
!| ❗   7 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: assets/multiqc_config.yml
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • nextflow_config - Config default ignored: params.ribo_database_manifest
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/rnaseq/rnaseq/.github/workflows/awstest.yml
  • multiqc_config - 'assets/multiqc_config.yml' not found

✅ Tests passed:

Run details

  • nf-core/tools version 2.13.1
  • Run at 2024-03-04 18:47:47

@maxulysse maxulysse mentioned this pull request Feb 23, 2024
Closed
19 tasks
@drpatelh drpatelh marked this pull request as ready for review February 23, 2024 17:27
@maxulysse
Copy link
Member

maxulysse commented Feb 29, 2024 •
edited
Loading

UNTAR now just failing for TXIMETA_TXIMPORT, but it's coming

@maxulysse
Copy link
Member

ok, we're going to have issues with GFFREAD.

We're setting up ext.args = '--keep-exon-attrs -F -T' in the config for the pipeline.
But that's not working for the module.

@drpatelh
Copy link
Member

drpatelh commented Mar 4, 2024 •
edited by maxulysse
Loading

The scope of this PR should be to finish adding all nf-core/modules and nf-core/subworkflows to the pipeline.

The only remaining tests left to include are:

  • stringtie/stringtie (needs fix to existing test via local nextflow.config)
  • gffread (needs fix to existing test via local nextflow.config)
  • Pipeline level tests (needs fix to existing test by dealing with software versions)
  • umitools/dedup (needs new tests)
  • umitools/dedup (how do we deal with dependencies?)

Once these have all been fixed and merged:

  • Update all nf-core/modules and subworkflows and test locally (nf-test test .)
  • Push a commit to this PR updating all nf-core/modules and subworkflows when tests pass locally
  • Re-enable old tests

@adamrtalbot adamrtalbot changed the title Add nf-test to CICD Add all nf-core modules and CI/CD and all nf-tests to everything Mar 4, 2024
@maxulysse maxulysse mentioned this pull request Mar 4, 2024
Merged
11 tasks
@maxulysse maxulysse merged commit a50212c into nf-core:dev Mar 4, 2024
91 checks passed
@adamrtalbot adamrtalbot deleted the nf-test-cicd branch March 6, 2024 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maxulysse maxulysse maxulysse approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@adamrtalbot @maxulysse @drpatelh @pinin4fjords