Skip to content

Direct execution of named (typed) workflows (module run) - #7379

Open
jorgee wants to merge 7 commits into
masterfrom
module-run-workflows
Open

Direct execution of named (typed) workflows (module run)#7379
jorgee wants to merge 7 commits into
masterfrom
module-run-workflows

Conversation

@jorgee

@jorgee jorgee commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Draft. Ports PR #7208 ("Direct execution for named workflows"). Complements the workflow-modules PR (#7363) but is independent of it — this only touches the script/runtime layer, no module-system changes.

Enable running a single named, statically-typed workflow directly, without writing an explicit entry workflow. This is the execution primitive behind nextflow module run scope/name for workflow modules, and it also applies to a local nextflow run script.nf that defines exactly one named workflow.

What it does

Given a script with a single named, typed workflow:

nextflow.enable.types = true

workflow HELLO_SHEET {
    take:
    greetings: Channel<Map>   // samplesheet
    prefix: String            // scalar

    main:
    ...
    emit:
    result: Channel<String> = ...
}
nextflow run script.nf --greetings sheet.csv --prefix Hi

Nextflow synthesizes an entry workflow that maps --<take> params to the workflow inputs, invokes it, and publishes its emit: channels as outputs.

Changes

  • nf-lang ScriptToGroovyVisitor — lower each workflow take with its declared type (_take_(name, type)) so take types are available at runtime.
  • WorkflowDef / WorkflowParamsDsl — record take types and expose getDeclaredInputTypes(); untyped takes (Object) are omitted.
  • ScriptMeta.hasExecutableWorkflows() — a non-module script with exactly one named workflow.
  • BaseScript.run0() — when no entry is selected, synthesize an entry via WorkflowEntryHandler for a single named workflow, else fall back to the process entry handler; a multi-workflow script still reports "No entry workflow specified".
  • WorkflowEntryHandler (new) — maps --<take> params to inputs using the declared types:
    • Channel<T> → load a samplesheet file (CSV / JSON / YAML) or wrap a collection with channel.fromList;
    • Value<T> → value channel;
    • scalar (String/Integer/Path/…) → coerced value.
      Then invokes the workflow and publishes its emits. Typed workflows only (guarded by isTypingEnabled()).

Not in this PR (follow-ups)

  • Outputs reuse the existing OutputDsl publishing (as the process entry handler does); the ADR's no-output-directory / work-dir-path index-file behaviour is a follow-up.
  • Typed-record samplesheets — rows currently load as Maps; casting each row to a custom record type (Channel<Sample>) is a follow-up (the __Params-class TODO from Direct execution for named workflows #7208).

Testing

  • Unit: WorkflowEntryHandlerTest (samplesheet loading for CSV/JSON/YAML, param→take resolution by type, guards). Full nextflow.script.* and nf-lang suites pass.
  • Manual e2e: ran a typed workflow with a Channel<Map> samplesheet take + a scalar String take via nextflow module run ./mod --greetings sheet.csv --prefix Hi → outputs published; verified the scalar is passed as a value and the CSV becomes a channel.

Co-authored from PR #7208 by @bentshermann.

@netlify

netlify Bot commented Jul 24, 2026

Copy link
Copy Markdown

Deploy Preview for nextflow-docs canceled.

Name Link
🔨 Latest commit a416058
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs/deploys/6a8e034915b396000826394a

Enable running a single named, statically-typed workflow directly. This backs
`nextflow module run scope/name` for workflow modules and `nextflow run
script.nf` for a single-workflow script. Ported from PR #7208.

- nf-lang ScriptToGroovyVisitor: lower each workflow take with its declared
  type (`_take_(name, type)`) so take types are available at runtime.
- WorkflowDef/WorkflowParamsDsl: record take types and expose
  getDeclaredInputTypes(); untyped takes (Object) are omitted.
- ScriptMeta.hasExecutableWorkflows(): a non-module script with exactly one
  named workflow.
- BaseScript.run0(): when no entry is selected, synthesize an entry via
  WorkflowEntryHandler for a single named workflow, else fall back to the
  process entry handler; a multi-workflow script still reports
  "No entry workflow specified".
- WorkflowEntryHandler: map `--<take>` params to workflow inputs using the
  declared types (Channel<T> loads a CSV/JSON/YAML samplesheet or wraps a
  collection; Value<T> becomes a value channel; scalars are coerced), invoke
  the workflow, and publish its emits. Guarded to typed workflows only.

Outputs currently reuse the existing OutputDsl publishing; the no-output-dir /
index-file behaviour and typed-record samplesheet loading are follow-ups.

Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman
bentsherman force-pushed the module-run-workflows branch from 040499f to 0cc07cc Compare August 25, 2026 16:00
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman
bentsherman marked this pull request as ready for review August 25, 2026 21:02
@bentsherman
bentsherman requested a review from a team as a code owner August 25, 2026 21:02
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
| `Duration`, `MemoryUnit`, `VersionNumber` | parsed from the string | parsed if given as a string |
| `Path` | resolved to a path, which must exist | resolved to a path, which must exist |
| `List<E>`, `Set<E>`, `Bag<E>` | not supported | each element converted to `E` |
| `Map<K,V>`, `Record` | not supported | used as-is |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we use dot notation to express maps/records in the case of processes. Maybe we could use it to support CLI maps

final handler = new WorkflowEntryHandler(this, session, meta)
this.entryFlow = handler.createEntryWorkflow()
}
else if( moduleRun && meta.hasExecutableProcesses() ) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is restricting the execution of process executions from module run. I agree but it is causing the failure in the integration test test/process-entry.nf introduced in #6381. From the error message, it looks like you can specify a process as entry point but entryName is only for workflows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants