Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simple pipeline level nf-tests #1272

Merged
merged 12 commits into from
Apr 9, 2024
Merged

Simple pipeline level nf-tests #1272

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c49f446
Add direct replacements for existing tests
adamrtalbot Mar 21, 2024
4d9acbb
Reorganised into discrete folders
adamrtalbot Mar 21, 2024
08e89eb
More all tests into subdirectory for more organisation
adamrtalbot Mar 21, 2024
967563b
remove redundant tests
adamrtalbot Mar 21, 2024
c2861cc
Remove invalid dependencies from GHA confirm-pass
adamrtalbot Mar 21, 2024
c00917c
Correct pipeline test path
adamrtalbot Mar 21, 2024
622d8c6
Relative path AGAIN
adamrtalbot Mar 21, 2024
fe8673b
Merge branch 'dev' into pipeline_level_tests_simple
adamrtalbot Mar 22, 2024
bcc1a00
Merge branch 'dev' into pipeline_level_tests_simple
adamrtalbot Apr 5, 2024
e01e195
Remove r-base from software versions
adamrtalbot Apr 9, 2024
6e6a254
Move repo tests config to tests/config
adamrtalbot Apr 9, 2024
76473d2
Add test config
adamrtalbot Apr 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 0 additions & 67 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,73 +19,6 @@ concurrency:
group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
cancel-in-progress: true
jobs:
star_salmon:
name: Test STAR Salmon with workflow parameters
if: ${{ (github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/rnaseq')) && !contains(github.event.head_commit.message, '[ci fast]') }}
runs-on: ubuntu-latest
strategy:
matrix:
parameters:
- "--skip_qc"
- "--skip_trimming"
- "--min_mapped_reads 90"
- "--with_umi"
- "--with_umi --skip_trimming"
- "--remove_ribo_rna --skip_qualimap"
- "--bam_csi_index"
- "--save_align_intermeds --save_reference"
- "--featurecounts_group_type false"
- "--trimmer fastp"

steps:
- name: Check out pipeline code
uses: actions/checkout@v4

- name: Setup Nextflow
uses: nf-core/[email protected]

- name: Run pipeline with STAR and various parameters
run: |
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --aligner star_salmon ${{ matrix.parameters }} --outdir ./results

star_rsem:
name: Test STAR RSEM with workflow parameters
if: ${{ (github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/rnaseq')) && !contains(github.event.head_commit.message, '[ci fast]') }}
runs-on: ubuntu-latest
strategy:
matrix:
parameters:
- "--skip_qc"
steps:
- name: Check out pipeline code
uses: actions/checkout@v4

- name: Setup Nextflow
uses: nf-core/[email protected]

- name: Run pipeline with RSEM STAR and various parameters
run: |
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --aligner star_rsem ${{ matrix.parameters }} --outdir ./results

hisat2:
name: Test HISAT2 with workflow parameters
if: ${{ (github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/rnaseq')) && !contains(github.event.head_commit.message, '[ci fast]') }}
runs-on: ubuntu-latest
strategy:
matrix:
parameters:
- "--skip_qc"
steps:
- name: Check out pipeline code
uses: actions/checkout@v4

- name: Setup Nextflow
uses: nf-core/[email protected]

- name: Run pipeline with HISAT2 and various parameters
run: |
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --aligner hisat2 ${{ matrix.parameters }} --outdir ./results

pseudo:
name: Test Pseudoaligners with workflow parameters
if: ${{ (github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/rnaseq')) && !contains(github.event.head_commit.message, '[ci fast]') }}
Expand Down
2 changes: 1 addition & 1 deletion tests/main.nf.test → tests/tests/default/main.nf.test
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/tests/default/?
why not tests/default/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is to keep it tidy. You have a primary tests/ folder with tests, lib and other stuff (e.g. conf).

tests
├── lib
│   └── UTILS.groovy
├── nextflow.config
└── tests
    ├── default
    │   ├── main.nf.test
    │   └── main.nf.test.snap
    ├── featurecounts
    │   └── main.nf.test
    ├── min_mapped_reads
    │   └── main.nf.test
    ├── remove_ribo_rna
    │   └── main.nf.test
    ├── skip_qc
    │   └── main.nf.test
    └── skip_trimming
        └── main.nf.test

Without it, you would end up with this which I think is messy:

tests
├── default
│   ├── main.nf.test
│   └── main.nf.test.snap
├── featurecounts
│   └── main.nf.test
├── lib
│   └── UTILS.groovy
├── min_mapped_reads
│   └── main.nf.test
├── nextflow.config
├── remove_ribo_rna
│   └── main.nf.test
├── skip_qc
│   └── main.nf.test
└── skip_trimming
    └── main.nf.test

But I could be convinced otherwise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could move the tests/nextflow.config file into tests/config/nextflow.config for consistency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd go with:

tests
├── default.main.nf.test
├── default.main.nf.test.snap
├── featurecounts.main.nf.test
├── lib
│   └── UTILS.groovy
├── min_mapped_reads.main.nf.test
├── nextflow.config
├── remove_ribo_rna.main.nf.test
├── skip_qc.main.nf.test
└── skip_trimming.main.nf.test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh bother automerge was too quick.

Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ nextflow_pipeline {
tag "rnaseq"
tag "PIPELINE"

test("Run with profile test") {
test("Default profile test") {

when {
params {
Expand Down
0 tests/main.nf.test.snap → tests/tests/default/main.nf.test.snap
View file
Open in desktop
File renamed without changes.
22 changes: 22 additions & 0 deletions tests/tests/featurecounts/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
nextflow_pipeline {

name "Test pipeline"
script "../main.nf"
tag "rnaseq"
tag "PIPELINE"

test("--featurecounts_group_type false") {

when {
params {
outdir = "$outputDir"
input = 'https://raw.githubusercontent.com/nf-core/test-datasets/7f1614baeb0ddf66e60be78c3d9fa55440465ac8/samplesheet/v3.10/samplesheet_test.csv'
featurecounts_group_type = false
}
}

then {
assert workflow.success
}
}
}
22 changes: 22 additions & 0 deletions tests/tests/min_mapped_reads/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
nextflow_pipeline {

name "Test pipeline"
script "../main.nf"
tag "rnaseq"
tag "PIPELINE"

test("--min_mapped_reads 90") {

when {
params {
outdir = "$outputDir"
input = 'https://raw.githubusercontent.com/nf-core/test-datasets/7f1614baeb0ddf66e60be78c3d9fa55440465ac8/samplesheet/v3.10/samplesheet_test.csv'
min_mapped_reads = 90
}
}

then {
assert workflow.success
}
}
}
22 changes: 22 additions & 0 deletions tests/tests/remove_ribo_rna/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
nextflow_pipeline {

name "Test pipeline"
script "../main.nf"
tag "rnaseq"
tag "PIPELINE"

test("--remove_ribo_rna") {

when {
params {
outdir = "$outputDir"
input = 'https://raw.githubusercontent.com/nf-core/test-datasets/7f1614baeb0ddf66e60be78c3d9fa55440465ac8/samplesheet/v3.10/samplesheet_test.csv'
remove_ribo_rna = true
}
}

then {
assert workflow.success
}
}
}
22 changes: 22 additions & 0 deletions tests/tests/skip_qc/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
nextflow_pipeline {

name "Test pipeline"
script "../main.nf"
tag "rnaseq"
tag "PIPELINE"

test("--skip_qc") {

when {
params {
outdir = "$outputDir"
input = 'https://raw.githubusercontent.com/nf-core/test-datasets/7f1614baeb0ddf66e60be78c3d9fa55440465ac8/samplesheet/v3.10/samplesheet_test.csv'
skip_qc = true
}
}

then {
assert workflow.success
}
}
}
22 changes: 22 additions & 0 deletions tests/tests/skip_trimming/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
nextflow_pipeline {

name "Test pipeline"
script "../main.nf"
tag "rnaseq"
tag "PIPELINE"

test("--skip_trimming") {

when {
params {
outdir = "$outputDir"
input = 'https://raw.githubusercontent.com/nf-core/test-datasets/7f1614baeb0ddf66e60be78c3d9fa55440465ac8/samplesheet/v3.10/samplesheet_test.csv'
skip_qc = true
}
}

then {
assert workflow.success
}
}
}
Loading