Apply the POSIX dot rule to each glob pattern component

[robsyme]

Closes #7480

The problem

Output file globs and channel.fromPath walk into hidden directories, and decide hidden-ness from the matched file's own name. A file with an ordinary name below a hidden directory is therefore collected even when the pattern never names that directory.

This surfaced through Fusion, which writes its metadata store to <taskWorkDir>/.fusion/v1/<absolute entry path>/md.json. nf-core/scrnaseq's CELLRANGER_COUNT declares path("**/outs/**"), so those internal files are collected as task outputs and published into the user's results as <outdir>/cellranger/count/.fusion/v1/s3/<bucket>/.... Reported by a user copying published outputs into a data catalogue.

The rule

POSIX Issue 8, §2.14.3:

If a filename begins with a <period> ( '.' ), the <period> shall be explicitly matched by using a <period> as the first character of the pattern or immediately following a <slash> character.

bash and zsh reproduce that wording, and FileOutParam.hidden's javadoc already claims to follow it ("coherently with linux bash rule"). Nextflow implements the first clause and not the second. Note the authorisation is positional: a dotted component authorises a hidden name where it appears, not for the whole pattern.

Given .foo/bar.txt, .secret.txt, pub/bar.txt, visible.txt, bash 3.2, bash 5.3 and zsh 5.9 return:

*.txt      -> visible.txt
*/bar.txt  -> pub/bar.txt
.*/bar.txt -> .foo/bar.txt

Before this PR Nextflow also returned .foo/bar.txt for the second pattern, from both a process output declaration and channel.fromPath. It now returns the three rows above.

The change

The dot-prefixed components of the pattern are collected up-front and compiled to single-component matchers. A hidden name is visited only when it matches one of them. That predicate replaces the tree-wide includeHidden flag at the three places hidden-ness is decided: the sub-tree pruning, the directory match and the file match. Hidden directories are pruned from the walk, which also avoids listing them on an object store.

TaskFileCollector no longer derives hidden from the pattern. Doing so turned a single dotted component into a tree-wide authorisation and defeated the rule for any pattern starting with a dot.

Unchanged: hidden: true, the non-glob syntax (a leading dot in a regex is the any-char meta-character and a slash is not a component separator), and a start folder that is itself hidden, so a work dir such as /scratch/.tmp/ab/cdef stays walkable.

Reproducer

Self-contained, no Fusion, no cloud, no containers.

process COLLECT {
    publishDir 'results', mode: 'copy'

    output:
    path("**/outs/**")

    script:
    """
    mkdir -p sample/outs
    echo "real output" > sample/outs/real.txt

    mkdir -p .fusion/v1/s3/my-bucket/work/ab/cdef0123456789/sample/outs
    echo '{"internal":"metadata"}' > .fusion/v1/s3/my-bucket/work/ab/cdef0123456789/sample/outs/md.json
    """
}

workflow {
    COLLECT()
    COLLECT.out.flatten().view()
}

before:  results/.fusion/v1/s3/my-bucket/work/ab/cdef0123456789/sample/outs/md.json
         results/sample/outs/real.txt
after:   results/sample/outs/real.txt

Notes for reviewers

Two user-visible behaviour changes. A pattern with a glob component above a hidden directory no longer reaches into it, so channel.fromPath('/data/**') stops returning files under .git or .snapshot. And a pattern that leads with a dot no longer switches off hidden filtering for the rest of the pattern, so .config/**/*.txt no longer returns .config/a/.secret/x.txt. Both match bash and zsh, and both are opt-out with hidden: true or by naming the component.

One existing test expectation was inverted. TaskFileCollectorTest > should create the map of path visit options asserted visitOptions('.hidden_file') yields hidden: true. That is the behaviour being removed, so it now expects false, with a comment pointing here.

A known approximation. The authorisation is checked against the name rather than the position of the component within the pattern, because ** matches an arbitrary number of components and cannot be aligned with a path component. A literal component such as .fusion is exact. A glob component such as the .* in **/.*/outs/* over-authorises nested hidden directories relative to bash. Making this exact needs real position alignment, which I did not attempt.

Unrelated, but adjacent. **/.hiddenmid/outs/* does not match a .hiddenmid at the root the way bash does, because Java's **/ requires a leading component. That is #5948, not this change. The test fixtures nest their hidden directories one level down to avoid conflating the two.

S3 was verified specifically, since it is the backend the issue was reported on and S3FileSystem.getPathMatcher() throws, so matching falls back to getDefaultPathMatcher against the path toString(). S3Path.getFileName() returns a bucket-less relative path, so a component matcher sees the plain name. Pinned by tests in FileHelperS3Test, built offline with no credentials, including a dotted bucket case.

Testing

New tests in FileHelperTest (including a file system that refuses getPathMatcher, to exercise the fallback), FileHelperS3Test, and TaskFileCollectorTest at the layer the user sees. Docs updated: the hidden option reference claimed the pattern has to start with a dot and said nothing about directory traversal.

[netlify]

✅ Deploy Preview for nextflow-docs ready!

Name	Link
🔨 Latest commit	91bee00
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs/deploys/6a7f7cb03b7d7e0008ad6a82
😎 Deploy Preview	https://deploy-preview-7481--nextflow-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.