Skip to content

10x chemistry override and read-length check bypass; stop writing unmapped reads (v4.3) - #26

Merged
Claptar merged 5 commits into
mainfrom
dev
Sep 3, 2026
Merged

Claptar merged 5 commits into
mainfrom
dev

Conversation

@Claptar

@Claptar Claptar commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Two new opt-in 10x flags, plus one behaviour change to output:

  1. --wl <chemistry> (10x) — skip barcode-whitelist matching by naming the chemistry directly, using the same ids as infer_10x_run.py.
  2. --skip-length-checks (10x) — downgrade the R1/R2 read-length sanity checks from fatal to warnings.
  3. Unmapped reads are no longer written — --outReadsUnmapped Fastx removed from all platforms.

Commits

d3d9f2c Remove --outReadsUnmapped Fastx option from multiple platform scripts
940b81b Add support for custom chemistry and length checks in 10x platform
f05111f Add chemistry resolution and read metrics functions to platform_10x.sh
9c55460 Add options to skip chemistry detection and read-length checks for 10x platform

Changes in detail

1. --wl <chemistry> — skip 10x chemistry detection (10x only)

When the chemistry is already known, --wl selects the whitelist and CB/UMI
geometry directly and skips the six grep -F -f whitelist scans over the
subsampled reads.

Names deliberately match the chemistry ids emitted by infer_10x_run.py in
nf-reprocessing-public-10x, so a value read out of that script's JSON report can
be passed straight through. Short aliases are accepted for interactive use, and
matching is case-insensitive.

--wl value Alias Whitelist CB UMI
gex_3pv1 v1 737K-april-2014_rc.txt 14 10
gex_3pv2_or_5pv1v2 v2 737K-august-2016.txt 16 10
gex_3pv3_family v3 3M-february-2018.txt 16 12
gex_3pv4_gemx v4 3M-3pgex-may-2023.txt 16 12
gex_5pv3_gemx 5pv3 3M-5pgex-jan-2023.txt 16 12
gex_multiome_arc_v1 multiome, arc gex_737K-arc-v1.txt 16 12
starsolo 10x /data/fastqs SAMPLE1 --species human --wl gex_3pv3_family

Notes:

  • The multiome entry prefers the gex_-prefixed whitelist and falls back to
    737K-arc-v1.txt on older whitelist directories.
  • ATAC chemistry ids are rejected with a clear message — starsolo 10x is
    gene-expression only.
  • Only whitelist matching is skipped. Reads are still subsampled and strand
    specificity is still determined by test alignment, since it cannot be inferred
    from the whitelist. strand.txt reports NA for the per-whitelist match counts
    in this mode.

2. --skip-length-checks — non-fatal read-length validation (10x only)

The three pre-alignment sanity checks (variable-length R1, R1 < 24 bp, R2 < 40 bp)
become warnings instead of aborting. Implemented by swapping the handler
(die → log_warn) rather than skipping the checks, so nothing goes silent.

CB/UMI reconciliation against R1 still applies — a short R1 still trims the UMI
length with a warning. One new hard error was added there: an R1 shorter than the
cell barcode leaves no room for a UMI and would produce an invalid STAR command,
so that dies regardless of the flag. This path was previously unreachable, but
becomes reachable now that a short R1 no longer aborts.

3. Unmapped reads no longer generated

--outReadsUnmapped Fastx removed from all six platform scripts (10x ×2,
smartseq, dropseq, rhapsody, indrops, strt).

lib/common.sh is unchanged: its pbzip2 compression of Unmapped.out.mate1/2
is guarded by [[ -s ... ]], so it simply no longer fires. pbzip2 is retained
as a dependency. Legacy scripts under scripts/legacy/ are untouched.

4. Internal refactor of lib/platform_10x.sh

To let the auto-detect and --wl paths share code:

  • _10x_read_metrics — new; measures R1/R2 length and the R1 length distribution.
    Previously _10x_determine_whitelist did this alongside whitelist matching.
  • _10x_whitelist_geometry — new; the whitelist-file → CB/UMI map, extracted from
    the inline case in _10x_check_read_lengths.
  • _10x_resolve_chemistry — new; chemistry name → whitelist + geometry.
  • _10x_check_read_lengths — now only validates lengths and reconciles CB/UMI
    against R1; takes CB/UMI as arguments instead of deriving them.

Compatibility

  • No breaking CLI changes. Both new flags are opt-in; default behaviour is
    unchanged except that unmapped-read FASTQs are no longer produced.
  • Downstream impact: any process consuming Unmapped.out.mate1.bz2 /
    mate2.bz2 from a run directory will no longer find them. Nothing in this repo
    reads those files.
  • --wl and --skip-length-checks are rejected with an explicit error on
    non-10x platforms, where the whitelist is fixed.

Before merging

  • Bump STARSOLO_VERSION in bin/starsolo from 4.2 to 4.3 (and the
    # Version header comment on line 5) — currently still 4.2 on dev.
  • Confirm no pipeline consumes the unmapped-read output.

Testing

  • bash -n clean on bin/starsolo and all of lib/.
  • _10x_resolve_chemistry: all 6 ids + 8 aliases resolve to existing files in
    /nfs/cellgeni/STAR/whitelists; case-insensitivity confirmed; ATAC ids and
    unknown names produce the intended errors.
  • _10x_check_read_lengths: each of the three checks aborts by default and
    warns-and-continues under --skip-length-checks; the UMI-less R1 case still
    dies with the flag set; normal v3 single-end (16|12) and paired-end
    (True|16|12) results unchanged; short-R1 trimming produces 16|4 as expected.
  • Non-10x guards for both flags fire correctly; --wl v3 --skip-length-checks
    parse together.
  • CI (.github/workflows/test.yml) runs all platforms end-to-end on the test
    reference.

Not run end-to-end against production FASTQs.


Review focus

  • The --wl name table vs. CHEMISTRIES in infer_10x_run_recommended.py — these
    must stay in sync for the nf-reprocessing handoff to work.
  • Whether dropping unmapped-read output is acceptable for all current consumers.
  • The new fatal error in _10x_check_read_lengths for an R1 shorter than the cell
    barcode — the only new failure mode in this PR. Previously unreachable (the
    R1LEN < 24 check caught it first); reachable now that --skip-length-checks
    downgrades that check to a warning. Without the guard, UMILEN goes negative
    and STAR is called with e.g. --soloUMIlen -2.

@Claptar Claptar self-assigned this Sep 3, 2026
Copilot AI lite review requested due to automatic review settings September 3, 2026 14:51
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-09-03T14:53:24.505666Z 12429a2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The 10x auto-detection path still hard-codes the multiome whitelist filename and can abort under set -euo pipefail if only the new gex_-prefixed arc whitelist is present, plus there are a couple of user-facing message/help text issues to correct.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR extends the 10x platform wrapper to optionally bypass chemistry auto-detection (--wl <chemistry>) and downgrade read-length sanity checks to warnings (--skip-length-checks), while also removing unmapped-read FASTQ generation (--outReadsUnmapped Fastx) across platforms and bumping the CLI version to 4.3.

Changes:

  • Add --wl chemistry override (10x only) and refactor 10x whitelist/geometry/metrics logic to share code paths.
  • Add --skip-length-checks (10x only) to log warnings instead of aborting on unexpected R1/R2 lengths (with a new always-fatal UMI-less R1 guard).
  • Remove unmapped-read FASTQ output generation from all platform scripts; bump STARSOLO_VERSION to 4.3.
File summaries
File Description
README.md Documents new 10x-only flags and how to use them.
lib/platform_strt.sh Stops requesting unmapped reads from STAR.
lib/platform_smartseq.sh Stops requesting unmapped reads from STAR.
lib/platform_rhapsody.sh Stops requesting unmapped reads from STAR.
lib/platform_indrops.sh Stops requesting unmapped reads from STAR.
lib/platform_dropseq.sh Stops requesting unmapped reads from STAR.
lib/platform_10x.sh Implements --wl, --skip-length-checks, refactors detection/metrics, and removes unmapped reads output.
bin/starsolo Adds CLI parsing/guards for new flags and bumps version to 4.3.
Review details

Suppressed comments (1)

lib/platform_10x.sh:101

  • Multiome whitelist matching hard-codes 737K-arc-v1.txt. If a whitelist directory only contains the newer gex_737K-arc-v1.txt (which this PR now prefers for --wl), grep -f will error under set -euo pipefail and abort chemistry auto-detection. Make auto-detection choose whichever arc-v1 whitelist exists (and skip the scan if neither is present).
    local NBC1 NBC2 NBC3 NBC4 NBC5 NBCA
    NBC1=$(awk 'NR%4==2' test.R1.fastq | cut -c-14 | grep -F -f "$WL/737K-april-2014_rc.txt" | wc -l)
    NBC2=$(awk 'NR%4==2' test.R1.fastq | cut -c-16 | grep -F -f "$WL/737K-august-2016.txt"   | wc -l)
    NBC3=$(awk 'NR%4==2' test.R1.fastq | cut -c-16 | grep -F -f "$WL/3M-february-2018.txt"   | wc -l)
    NBC4=$(awk 'NR%4==2' test.R1.fastq | cut -c-16 | grep -F -f "$WL/3M-3pgex-may-2023.txt"  | wc -l)
    NBC5=$(awk 'NR%4==2' test.R1.fastq | cut -c-16 | grep -F -f "$WL/3M-5pgex-jan-2023.txt"  | wc -l)
    NBCA=$(awk 'NR%4==2' test.R1.fastq | cut -c-16 | grep -F -f "$WL/737K-arc-v1.txt"        | wc -l)

    local BC=""
    if   (( NBC3 > 50000 )); then BC="$WL/3M-february-2018.txt"
    elif (( NBC2 > 50000 )); then BC="$WL/737K-august-2016.txt"
    elif (( NBCA > 50000 )); then BC="$WL/737K-arc-v1.txt"
    elif (( NBC1 > 50000 )); then BC="$WL/737K-april-2014_rc.txt"
  • Files reviewed: 7/8 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/platform_10x.sh
local rl_info PAIRED CBLEN UMILEN
rl_info=$(_10x_check_read_lengths "$R1DIS" "$R1LEN" "$R2LEN" "$BC" "$WL")
local rl_info PAIRED
[[ $SKIPLEN == "True" ]] && log_warn "Read-length validation disabled (--skip-length-checks)."
Comment thread lib/platform_10x.sh
Comment on lines +363 to +364
--wl <chemistry> Use this chemistry instead of detecting it. Names
chemistry ids:
@Claptar
Claptar merged commit fbcd9ac into main Sep 3, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants