Skip to content

Dsl2-polyg-polyx-filtering #1177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ilight1542 wants to merge 9 commits into
base: dev
Choose a base branch
from
Open

Dsl2-polyg-polyx-filtering #1177

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6b75295
implemented params for poly-g and poly-x trimming with fastp, adjuste…
ilight1542 Jun 23, 2026
efc160b
fixed help text and descriptions
ilight1542 Jun 23, 2026
9a2c863
linting fix
ilight1542 Jun 24, 2026
d7ba141
adjusted for runtime parsing of meta in ext.args
ilight1542 Jun 24, 2026
938777b
removed trailing whitespace
ilight1542 Jun 24, 2026
91dbf89
set global default params for polyg/polyx trimming
ilight1542 Jun 24, 2026
2abc08c
attempted fix for explicit poly-g trimming for 4-color chem
ilight1542 Jul 3, 2026
9b77958
Merge branch 'dev' into dsl2_polyg_filtering
TCLamnidis Jul 24, 2026
6d483ae
Merge branch 'dev' into dsl2_polyg_filtering
TCLamnidis Jul 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 26 additions & 18 deletions conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,13 +103,17 @@ process {

withName: FASTP_SINGLE {
tag = { "${meta.sample_id}_${meta.library_id}_L${meta.lane}" }
ext.args = [
params.preprocessing_trim5p != 0 ? "--trim_front1 ${params.preprocessing_trim5p}" : "",
params.preprocessing_trim3p != 0 ? "--trim_tail1 ${params.preprocessing_trim3p}" : "",
params.preprocessing_skipadaptertrim ? "--disable_adapter_trimming" : params.preprocessing_adapterlist ? "" : params.preprocessing_adapter1 ? "--adapter_sequence ${params.preprocessing_adapter1}" : "",
"--length_required ${params.preprocessing_minlength}",
params.preprocessing_fastp_complexityfilter ? "--low_complexity_filter --complexity_threshold ${params.preprocessing_fastp_complexityfilter_threshold}" : "",
].join(' ').trim()
ext.args = {
[
params.preprocessing_trim5p != 0 ? "--trim_front1 ${params.preprocessing_trim5p}" : "",
params.preprocessing_trim3p != 0 ? "--trim_tail1 ${params.preprocessing_trim3p}" : "",
params.preprocessing_skipadaptertrim ? "--disable_adapter_trimming" : params.preprocessing_adapterlist ? "" : params.preprocessing_adapter1 ? "--adapter_sequence ${params.preprocessing_adapter1}" : "",
"--length_required ${params.preprocessing_minlength}",
params.preprocessing_fastp_complexityfilter ? "--low_complexity_filter --complexity_threshold ${params.preprocessing_fastp_complexityfilter_threshold}" : "",
params.preprocessing_fastp_disable_polygfilter ? "--disable_trim_poly_g" : "${meta.colour_chemistry}" == 4 ? "--trim_poly_g --poly_g_min_len ${params.preprocessing_fastp_polygfilter_minlength}" : "--disable_trim_poly_g",

@TCLamnidis TCLamnidis Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this turn on polyG filtering only for 4-colour chem, but force it off for 2-colour chem? 🤔 Maybe that's why the tests were failing?

Either way, I think this should be restructured to use a single option for enable/disable, as described in the review.
The min length parameter can then be independent from this logic and always set (except if FastP complains when min length is set while disabling the filter).

params.preprocessing_fastp_polyxfilter_minlength != 0 ? "--trim_poly_x --poly_x_min_len ${params.preprocessing_fastp_polyxfilter_minlength}": "",
].join(' ').trim()
}
ext.prefix = { "${meta.sample_id}_${meta.library_id}_L${meta.lane}" }
publishDir = [
[
Expand All @@ -128,15 +132,19 @@ process {

withName: FASTP_PAIRED {
tag = { "${meta.sample_id}_${meta.library_id}_L${meta.lane}" }
ext.args = [
params.preprocessing_excludeunmerged ? "" : "--include_unmerged",
params.preprocessing_trim5p != 0 ? "--trim_front1 ${params.preprocessing_trim5p} --trim_front2 ${params.preprocessing_trim5p}" : "",
params.preprocessing_trim3p != 0 ? "--trim_tail1 ${params.preprocessing_trim3p} --trim_tail2 ${params.preprocessing_trim3p}" : "",
params.preprocessing_skipadaptertrim ? "--disable_adapter_trimming" : params.preprocessing_adapterlist ? "" : params.preprocessing_adapter1 ? "--adapter_sequence ${params.preprocessing_adapter1}" : "",
params.preprocessing_skipadaptertrim ? "" : params.preprocessing_adapterlist ? "" : params.preprocessing_adapter2 ? "--adapter_sequence_r2 ${params.preprocessing_adapter2}" : "",
"--length_required ${params.preprocessing_minlength}",
params.preprocessing_fastp_complexityfilter ? "--low_complexity_filter --complexity_threshold ${params.preprocessing_fastp_complexityfilter_threshold}" : "",
].join(' ').trim()
ext.args = {
[
params.preprocessing_excludeunmerged ? "" : "--include_unmerged",
params.preprocessing_trim5p != 0 ? "--trim_front1 ${params.preprocessing_trim5p} --trim_front2 ${params.preprocessing_trim5p}" : "",
params.preprocessing_trim3p != 0 ? "--trim_tail1 ${params.preprocessing_trim3p} --trim_tail2 ${params.preprocessing_trim3p}" : "",
params.preprocessing_skipadaptertrim ? "--disable_adapter_trimming" : params.preprocessing_adapterlist ? "" : params.preprocessing_adapter1 ? "--adapter_sequence ${params.preprocessing_adapter1}" : "",
params.preprocessing_skipadaptertrim ? "" : params.preprocessing_adapterlist ? "" : params.preprocessing_adapter2 ? "--adapter_sequence_r2 ${params.preprocessing_adapter2}" : "",
"--length_required ${params.preprocessing_minlength}",
params.preprocessing_fastp_complexityfilter ? "--low_complexity_filter --complexity_threshold ${params.preprocessing_fastp_complexityfilter_threshold}" : "",
params.preprocessing_fastp_disable_polygfilter ? "--disable_trim_poly_g" : "${meta.colour_chemistry}" == 4 ? "--trim_poly_g --poly_g_min_len ${params.preprocessing_fastp_polygfilter_minlength}" : "--disable_trim_poly_g",
params.preprocessing_fastp_polyxfilter_minlength != 0 ? "--trim_poly_x --poly_x_min_len ${params.preprocessing_fastp_polyxfilter_minlength}": "",
].join(' ').trim()
}
ext.prefix = { "${meta.sample_id}_${meta.library_id}_L${meta.lane}" }
publishDir = [
[
Expand Down Expand Up @@ -1099,7 +1107,7 @@ process {
"--no-stats",
"-y ${params.damagecalculation_yaxis}",
params.damagecalculation_mapdamage_downsample == 0 ? "" : "-n ${params.damagecalculation_mapdamage_downsample} --downsample-seed=1",
{ meta.strandedness } == "single" ? '--single-stranded' : '',
"${meta.strandedness}" == "single" ? '--single-stranded' : '',
"-m ${params.damagecalculation_xaxis}",
].join(' ').trim()
}
Expand Down Expand Up @@ -1156,7 +1164,7 @@ process {
"--seq-length=${params.damage_manipulation_rescale_seqlength}",
params.damage_manipulation_rescale_length_3p == 0 ? "" : "--rescale-length-3p=${params.damage_manipulation_rescale_length_3p}",
params.damage_manipulation_rescale_length_5p == 0 ? "" : "--rescale-length-5p=${params.damage_manipulation_rescale_length_5p}",
{ meta.strandedness } == "single" ? '--single-stranded' : '',
"${meta.strandedness}" == "single" ? '--single-stranded' : '',
].join(' ').trim()
}
ext.prefix = { "${meta.sample_id}_${meta.library_id}_${meta.reference}_rescaled" }
Expand Down
2 changes: 1 addition & 1 deletion main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*/

include { EAGER } from './workflows/eager'
include { EAGER } from './workflows/eager'
include { PIPELINE_INITIALISATION } from './subworkflows/local/utils_nfcore_eager_pipeline'
include { PIPELINE_COMPLETION } from './subworkflows/local/utils_nfcore_eager_pipeline'
include { getGenomeAttribute } from './subworkflows/local/utils_nfcore_eager_pipeline'
Expand Down
3 changes: 3 additions & 0 deletions nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ params {
// WARN: slightly different behaviour between fastp & adapterremoval
preprocessing_trim3p = 0
// WARN: slightly different behaviour between fastp & adapterremoval
preprocessing_fastp_disable_polygfilter = false
preprocessing_fastp_polygfilter_minlength = 10
preprocessing_fastp_polyxfilter_minlength = 0
preprocessing_fastp_complexityfilter = false
preprocessing_fastp_complexityfilter_threshold = 10
preprocessing_adapterremoval_preserve5p = false
Expand Down
24 changes: 22 additions & 2 deletions nextflow_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -401,16 +401,36 @@
"preprocessing_fastp_complexityfilter": {
"type": "boolean",
"description": "Specify to turn on sequence complexity filtering of reads.",
"help_text": "Performs a poly-G tail removal step in the beginning of the pipeline using fastp.\n\nThis can be useful for trimming ploy-G tails from short-fragments sequenced on two-colour Illumina chemistry such as NextSeqs or NovaSeqs (where no-fluorescence is read as a G on two-colour chemistry), which can inflate reported GC content values.\n\n> Modifies fastp parameter: `--trim_poly_g`",
"help_text": "Performs a read complexity filtering step using fastp.\n\nThis can be useful for removing highly repetitive and low-complexity reads from downstream analysis.\n\n> Modifies fastp parameter: `--low_complexity_filter`",
"fa_icon": "fas fa-power-off"
},
"preprocessing_fastp_complexityfilter_threshold": {
"type": "integer",
"default": 10,
"description": "Specify the complexity threshold that must be reached or exceeded to retain reads.",
"help_text": "This option can be used to define the minimum length of a poly-G tail to begin low complexity trimming.\n\n> Modifies fastp parameter: `--poly_g_min_len`",
"help_text": "This option can be used to define the complexity of a read to be retained for downstream analysis.\n\n> Modifies fastp parameter: `--complexity_threshold`",
"fa_icon": "fas fa-filter"
},
"preprocessing_fastp_disable_polygfilter": {
"type": "boolean",
"description": "Specify to disable poly-G filtering of 2 colour chemistry reads.",
"fa_icon": "fas fa-power-off",
"help_text": "This option will disable fastp-based poly-G trimming carried out by default on all 2 colour chemistry sequencing data.\n\n> Modifies fastp parameter: `--disable_trim_poly_g`"
},
"preprocessing_fastp_polygfilter_minlength": {
"type": "integer",
"default": 10,
"description": "Specify length of poly-g min for clipping to be performed on 2 colour chemistry datasets.",
"help_text": "This option can be used to define the minimum length of a poly-G tail to begin low complexity trimming on 2 colour chemistry datasets. In short-fragments sequenced on two-colour Illumina chemistry such as NextSeqs or NovaSeqs, where no-fluorescence is read as a G on two-colour chemistry, poly-G tails can inflate reported GC content values and cause mapability issues downstream.\n\n> Modifies fastp parameter: `--poly_g_min_len`",
"fa_icon": "fas fa-ruler-horizontal"
},
"preprocessing_fastp_polyxfilter_minlength": {
"type": "integer",
"default": 0,
"description": "Specify the complexity threshold that must be reached or exceeded to retain reads.",
"help_text": "This option can be used to define the minimum length of a poly-X (A,T,C,N) tail to begin low complexity trimming following poly-G trimming (if performed). This would be carried out on all sequencing data, regardless of the colour chemistry.\n\n> Modifies fastp parameter: `--trim_poly_x` and `--poly_x_min_len`",

@TCLamnidis TCLamnidis Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If setting the parameter to 0 disables polyx filtering, it should be clearly stated.

Suggested change
"help_text": "This option can be used to define the minimum length of a poly-X (A,T,C,N) tail to begin low complexity trimming following poly-G trimming (if performed). This would be carried out on all sequencing data, regardless of the colour chemistry.\n\n> Modifies fastp parameter: `--trim_poly_x` and `--poly_x_min_len`",
"help_text": "This option can be used to define the minimum length of a poly-X (A,T,C,N) tail to begin low complexity trimming following poly-G trimming (if performed). This would be carried out on all sequencing data, regardless of the colour chemistry. A value of 0 means that no poly-X filtering is performed.\n\n> Modifies fastp parameter: `--trim_poly_x` and `--poly_x_min_len`",

@TCLamnidis TCLamnidis Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, the FastP docs have slightly confusing phrasing about this. Does this happen AFTER poly-G or before poly-G trimming?
I think your helptext is correct, but it is worth double-checking.

This setting is useful for trimming the tails having polyX (i.e. polyA) before polyG.

"fa_icon": "fas fa-ruler-horizontal"
},
"preprocessing_adapterremoval_preserve5p": {
"type": "boolean",
"description": "Skip AdapterRemoval quality and N base trimming at 5 prime end.",
Expand Down
4 changes: 2 additions & 2 deletions subworkflows/local/preprocessing.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ workflow PREPROCESSING {

if ( params.preprocessing_tool == "fastp" ) {
ch_processed_reads = PREPROCESSING_FASTP ( reads, adapterlist ).reads
ch_versions = ch_versions.mix( PREPROCESSING_FASTP.out.versions )
ch_multiqc_files = ch_multiqc_files.mix( PREPROCESSING_FASTP.out.mqc )
ch_versions = ch_versions.mix( PREPROCESSING_FASTP.out.versions )
ch_multiqc_files = ch_multiqc_files.mix( PREPROCESSING_FASTP.out.mqc )
} else if ( params.preprocessing_tool == "adapterremoval" ) {
ch_processed_reads = PREPROCESSING_ADAPTERREMOVAL ( reads, adapterlist ).reads
ch_versions = ch_versions.mix( PREPROCESSING_ADAPTERREMOVAL.out.versions )
Expand Down
Loading