Ideas for coding best-practice examples

[ewels]

Pipeline Scripts

• Process basics (input / output / script, channel types, output patterns, tag)
• Use of params (mention need for explicit defaults)
• Paired end inputs (FromFilePairs) and single-end (FromPath)
• Processes discriminating between PE and SE channels
• Process settings (cpus, memory, time etc)
• Clever restarting of failed jobs (errorStrategy, maxRetries etc)
• Publishing (publishDir, different modes, using params.outdir)
• Using the bin directory for bundling scripts
• R Scripts and R package installation

Config files

• Config file params, overwriting script params
• Config file process defaults (all processes)
• Config file process settings (single processes)
• Overwriting these defaults (eg. not loading MultiQC by using user config file)
• Nextflow profiles, loading different config files

Running on UPPMAX

• ClusterOptions (params.project, params.clusteroptions)
• Cluster queue type (process.queue = 'devcore')
• Pipeline checking that project is set if UPPMAX profile
• Preset genome paths in config (params.genome, creating channels from paths, using these channels in a way that they are not consumed)

Running outside UPPMAX

• Docker
• AWS
• Travis (mention memory limit

Bonus features

• Multiple reference index input options (building missing indexes if not supplied)
• Required structure for a pipeline to work with Nextflow GitHub support (main.nf etc)
• Consistency within NGI pipelines (naming NGI-xyz, readme template, versioning)

Common Processes

• FastQC
• TrimGalore! (with individual trimming params and grouped presets)
• MultiQC
• samtools sort / index / stats / flagstat
• Picard MarkDuplicates
• Build generic index / do generic alignment ..?

[ewels]

Oops, this got a bit longer than I intended.. But each bit doesn't need to be very long. Maybe different markdown files with each of these as a header? Then copy in a short code example if it's not very long.

Phil

[Hammarn]

I've tried starting on this several times now but I struggle to get anywhere since a lot of this seems redundant. Most points are covered pretty well by the official Nextflow documentation, example scripts or FAQ.
I agreed that it would be nice to show how we tend to write scripts (with separate config files etc.) but a more generic help documentation seems a bit out of our scope.