Re-create existing pipeline level tests in nf-test format

[adamrtalbot]

Description of feature

The current CI includes many different parameters

• rnaseq/.github/workflows/ci.yml

  Lines 76 to 90 in b89fac3

  	matrix:
  	parameters:
  	- "--skip_qc"
  	- "--skip_trimming"
  	- "--gtf false"
  	- "--star_index false"
  	- "--transcript_fasta false"
  	- "--min_mapped_reads 90"
  	- "--with_umi"
  	- "--with_umi --skip_trimming"
  	- "--remove_ribo_rna --skip_qualimap"
  	- "--bam_csi_index"
  	- "--save_align_intermeds --save_reference"
  	- "--featurecounts_group_type false"
  	- "--trimmer fastp"

• rnaseq/.github/workflows/ci.yml

  Lines 138 to 141 in b89fac3

  	matrix:
  	parameters:
  	- "--skip_qc"
  	- "--rsem_index false"

• rnaseq/.github/workflows/ci.yml

  Lines 189 to 192 in b89fac3

  	matrix:
  	parameters:
  	- "--skip_qc"
  	- "--hisat2_index false"

• rnaseq/.github/workflows/ci.yml

  Lines 240 to 247 in b89fac3

  	matrix:
  	parameters:
  	- "--pseudo_aligner salmon --skip_qc"
  	- "--pseudo_aligner salmon --skip_alignment --skip_pseudo_alignment"
  	- "--pseudo_aligner salmon --salmon_index false --transcript_fasta false"
  	- "--pseudo_aligner kallisto --skip_qc"
  	- "--pseudo_aligner kallisto --skip_alignment --skip_pseudo_alignment"
  	- "--pseudo_aligner kallisto --kallisto_index false --transcript_fasta false"

We should re-create these in nf-test format.

Use this as a chance to review them and decide when, where and how they should be executed. We can use tags and criteria to filter which tests exist. Ideally, we recreate all tests in nf-test format with additional checks for correct outputs etc.

• "--skip_qc"
• "--skip_trimming"
• "--gtf false"
• "--star_index false"
• "--transcript_fasta false"
• "--min_mapped_reads 90"
• "--with_umi"
• "--with_umi --skip_trimming"
• "--remove_ribo_rna --skip_qualimap"
• "--bam_csi_index"
• "--save_align_intermeds --save_reference"
• "--featurecounts_group_type false"
• "--trimmer fastp"
• "--pseudo_aligner salmon --skip_qc"
• "--pseudo_aligner salmon --skip_alignment --skip_pseudo_alignment"
• "--pseudo_aligner salmon --salmon_index false --transcript_fasta false"
• "--pseudo_aligner kallisto --skip_qc"
• "--pseudo_aligner kallisto --skip_alignment --skip_pseudo_alignment"
• "--pseudo_aligner kallisto --kallisto_index false --transcript_fasta false"
• "--skip_qc" (STAR-RSEM)
• "--rsem_index false" (STAR-RSEM)
• "--skip_qc" (HISAT2)
• "--rsem_index false" (HISAT2)

[adamrtalbot]

This follows on from #1194 and #1184. Once they are complete we should see if all these tests are still required.

[adamrtalbot]

Summary of existing tests (by eye):

Skip steps

--skip_qc
--skip_trimming

Run without reference genomes

--gtf false
--star_index false
--transcript_fasta false
--rsem_index false (STAR-RSEM)
--rsem_index false (HISAT2)

Alternative/additional methods

--trimmer fastp
--with_umi
--remove_ribo_rna
--pseudo_aligner salmon
--pseudo_aligner salmon --skip_alignment --skip_pseudo_alignment
--pseudo_aligner salmon --salmon_index false --transcript_fasta false
--pseudo_aligner kallisto --skip_qc
--pseudo_aligner kallisto --skip_alignment --skip_pseudo_alignment
--pseudo_aligner kallisto --kallisto_index false --transcript_fasta false

Save intermediate files

--save_align_intermeds --save_reference
--bam_csi_index

Alternative parameter value

--min_mapped_reads 90
--featurecounts_group_type false

[adamrtalbot]

Proposal

Remove or replace with subworkflow level tests

The following can be made into subworkflow level tests or removed entirely.

• PREPARE_GENOMES More tests for PREPARE_REFERENCE #1256
  • --gtf false
  • --star_index false
  • --transcript_fasta false
  • --rsem_index false (STAR-RSEM)
  • --rsem_index false (HISAT2)
• FASTQ_FASTQC_UMITOOLS_FASTP More tests for FASTQ_FASTQC_UMITOOLS_* modules#5099
  • --with_umi
  • --skip_trimming
• FASTQ_FASTQC_UMITOOLS_TRIMGALORE More tests for FASTQ_FASTQC_UMITOOLS_* modules#5099
  • --skip_umi
  • --skip_trimming

Create pipeline level tests for the following

Global options

• --skip_qc Test if QC is skipped correctly Simple pipeline level nf-tests #1272
• --remove_ribo_rna Test if ribosomal RNA is removed correctly Simple pipeline level nf-tests #1272
• --min_mapped_reads 90 Test if min_mapped_reads pass value is respected Simple pipeline level nf-tests #1272
• --featurecounts_group_type false Test if featurecounts group set as false is handled correctly Simple pipeline level nf-tests #1272

Alternative workflows

• --aligner star_salmon Check if quantification with star_salmon (default) works correctly Add psueudoaligner pipeline level tests to test suite #1279
• --aligner star_rsem Check if quantification with star_rsem works correctly Add nf-test for STAR-RSEM and HISAT2 aligners #1328
• --aligner hisat2 Check if quantification with hisat2 works correctly Add nf-test for STAR-RSEM and HISAT2 aligners #1328
• --pseudo_aligner salmon --skip_alignment --salmon_index false --transcript_fasta false Check if Salmon workflow works Add psueudoaligner pipeline level tests to test suite #1279
• --pseudo_aligner kallisto --skip_alignment --kallisto_index false --transcript_fasta false Check if Kallisto workflow works Add psueudoaligner pipeline level tests to test suite #1279