chore(release): promote develop → main - #360
Open
ywatanabe1989 wants to merge 15 commits into
Open
Conversation
Replace the GitHub-hosted release workflow with the proven self-hosted
SIF pipeline (test->build->publish->release inside the reused scitex-ci
ci-cpu.sif via apptainer exec), matching the fleet canary
(openalex-local). The bare Spartan compute node has no Python, so
actions/setup-python and the Docker-based pypa/gh-action-pypi-publish
cannot run there; publish now does MANUAL OIDC trusted publishing inside
the SIF (GitHub OIDC JWT -> PyPI mint-token -> twine upload). Trust
config on PyPI is unchanged.
- KEEP trust-bound filename pypi-publish-and-github-release-on-tag.yml
(PyPI trusted publisher is bound to the workflow filename).
- Add the 4 .github/ci/{exec,run,build,publish}-in-sif.sh helpers.
- Strip the auto sync-main job (develop->main is a deliberate PR).
- runs-on from Actions Variable CI_RUNS_ON; SIF + apptainer paths from
SCITEX_CI_SIF / SCITEX_CI_APPTAINER (same as pytest-matrix).
- --cov=src/scitex; environment url https://pypi.org/p/scitex;
unique-per-run TMPDIR; matplotlib Agg + warm font cache guarded.
- fail-loud: a missing SIF/shim/interpreter is a HARD error.
…d (pinned-in-core) (#340) Complete the mid-release 2.30.2 reconcile to TRUE current-latest PyPI versions, pinned in CORE [project.dependencies] (not extras-only) so `pip install scitex` and `uv pip install scitex[all]` resolve the same stable set. Leaf bumps (current -> latest): - figrecipe 0.28.20 -> 0.29.5 - scitex-audio 0.2.13 -> 0.3.0 - scitex-cv 0.1.5 -> 0.2.0 - scitex-dev 0.18.1 -> 0.20.1 - scitex-io 0.3.2 -> 0.3.3 - scitex-writer 2.17.5 -> 2.22.0 - Add scitex-pd==0.1.6 to core deps AND the [pd] extra (which flows into [all]) so `scitex.pd` works out of the box. - Bump umbrella version 2.30.2 -> 2.30.3 (2.30.2 already published). Conflict-verified: `uv pip compile` of the full resulting pin set exits 0 (732-line lock, no conflicts).
) The self-hosted SIF release pipeline (run-in-sif.sh) drifted out of parity with the hardened ubuntu workflow, breaking the v2.30.3 release at the 3.13 test gate: - It installed only `.[all,dev]`, but the umbrella-aliased PEER standalones (scitex-gen, scitex-dsp, ...) are NOT in [all]. So scitex_gen was never refreshed and the test used the SIF's baked, stale copy (missing `to_even`), failing tests/integration/test_cross_package_imports.py. Reinstall the peers FRESH into the same --target so they shadow the baked copies on PYTHONPATH. - It ran bare `pytest tests/` with no quarantine. Restore the ignores the hardened workflow uses: tests/e2e + the cross-package gate (segfault-prone), and tests/examples/test_09_dev.py (hard-requires scitex_hub, which is umbrella_skip by design and never in the [all] SIF).
figrecipe 0.29.5->0.29.9, scitex-writer 2.22.0->2.23.0, scitex-clew 0.2.17->0.2.18 (PS-170 freshness; strict pin audit green). figrecipe 0.29.9 lets downstream research projects pin figrecipe==0.29.9 consistently alongside scitex==2.30.4.
…eterministic (#343) The v2.30.4 release SIF flaked on test_bundle_submodule_is_a_package: `import scitex.io as sio; sio.bundle` asserts attribute access WITHOUT importing the submodule, so it only passes when a sibling test imported scitex.io.bundle first — nondeterministic under pytest-xdist worker distribution (reproduces back to 2.30.0; not a regression, not the pin bump). Product is fine: `import scitex.io.bundle` is the supported path and works. Import it explicitly so the assertions are deterministic on every worker.
scitex-writer 2.23.0->2.24.1 (post-compile fail-loud gate, incident fix), scitex-dev 0.20.1->0.21.0 (compliance-linter trio), scitex-clew 0.2.18->0.2.22. Strict pin audit green.
…for SIF rebuild (#346) * chore(umbrella): scitex 2.30.6 — re-pin members to current-published for SIF rebuild * chore(umbrella): narrow 2.30.6 to minimal set (clew 0.16.0 / notification 0.2.9 / notebook 0.1.2) Revert the broad fleet-latest bumps (writer/session/dev/container/dataset/ scholar/ssh) that broke the umbrella thin-wrapper MCP-parity test (test_umbrella_exposes_writer_mcp_tools) and segfaulted pytest-matrix. The SIF rebuild only needs the minimal alignment set; broad 'fleet on latest' re-pins require per-member thin-wrapper regen and land as a follow-up.
…gregator (#347) * rescue: pre-stop autosave scitex-python@20260703T223658Z * fix(mcp): bounded per-peer mount so one hung peer can't darken the aggregator The umbrella `scitex serve` aggregator fronts ~33 packages' MCP tools by importing each peer's `_mcp_server` and mounting its FastMCP at module-load time. Previously each peer was resolved SEQUENTIALLY and SYNCHRONOUSLY. If a single peer's import hangs at init (real case: scitex-todo store-wedge stalls 20s+), the whole `scitex._mcp` import never completes and EVERY peer's tools go dark — the failure concentrates 33x vs the per-server split. Fix: resolve each peer's FastMCP concurrently in a bounded DAEMON thread (`_resolve_peers_bounded`). Threads start at once and are joined against one shared deadline, so total load time is ~max(peer) bounded by the per-peer budget — never the sum, and a hung thread can't block interpreter exit. On timeout / exception the peer is SKIPPED with a warning ("... unavailable this session") and the healthy peers still mount. Budget is configurable via `SCITEX_MCP_PEER_TIMEOUT` (default 8s). Mounting stays on the MAIN thread: only the peer IMPORT is slow/hang-prone; `safe_mount` is fast and mutates shared parent-server state, so it is kept single-threaded to avoid concurrent-mutation races. Preserves all prior behavior: `_is_enabled` env gate, `_NAMESPACE_ALIASES`, `_SKIP_CATEGORIES`, `register_peer_extras`, `register_umbrella_tools`, the mounted-count log, and `SCITEX_MCP_USE_<NS>=0` gating. `register_all_tools` gains injectable `iter_registry` / `peer_timeout` params (defaults unchanged) so tests drive the real path without patching module globals. Tests (tests/scitex/test_mcp_bounded_mount.py): real fixture peers written to disk — a 30s-sleep hung peer, an ImportError peer, a sys.exit peer, fast peers. Assert the bounded resolve returns within the budget, the hung peer is skipped + warned, the fast peers still mount, and the env gate holds. * fix(mcp): serialize bounded peer resolve — concurrent imports deadlocked CI collection PR #347's pytest-matrix hung for 1h30m (job timeout, all 3 py versions) with "test session starts" printed and then nothing — i.e. a hang during COLLECTION, before any test ran, where pytest-timeout's per-test SIGALRM cannot fire. Root cause: the first cut resolved every peer's `_mcp_server` import CONCURRENTLY (one daemon thread per peer, ~33 fired at once). Importing a large, interdependent package set from many threads simultaneously races the CPython import machinery and can deadlock via cross-thread circular imports that never occur when modules are imported one at a time. That deadlock strikes at `from scitex import _mcp` import (register_all_tools runs at module load) — which every test module triggers at collection. `develop` passes because its resolve is sequential; the concurrency was the regression. Fix: resolve peers SERIALLY. Each peer still imports in its own bounded daemon thread (`_resolve_one_peer_bounded`, joined for at most `SCITEX_MCP_PEER_TIMEOUT`, default 8s) so a hung/wedged import is abandoned + skipped and can never block interpreter exit — but only ONE import is ever in flight, so imports stay as safe as develop's sequential loop while gaining the per-peer hang bound. Wall cost is the sum of the healthy peers' (fast) import times plus the timeout per wedged peer — a one-time startup cost paid for reliability (the original "~max(peer)" concurrency goal is knowingly traded away; correctness first). Verified locally: real `from scitex import _mcp` completes in ~8s and leaves NO lingering non-daemon threads (so the main-thread timeout signal can always fire); `test_mcp_entrypoint.py` matches the develop baseline (no new failures, no hang). Tests: added `_INFINITE_PEER` (import blocks on `threading.Event().wait()`, never set — the real store-wedge shape, not a bounded sleep) with a fixture asserting the bounded resolve returns within the budget, skips the wedged peer, and still mounts the co-resident fast peer. All hang fixtures are written to tmp dirs and imported ONLY inside the bounded code path at test time — never at module/collection time, so pytest can never auto-collect or import them into a collection hang. 25 tests pass.
… guard (#349) * fix(mcp): skip orochi orchestrator in umbrella aggregator cold-start The umbrella MCP aggregator serially resolves each ecosystem peer's _mcp_server at import time. scitex-orochi is the single-instance agent-communication ORCHESTRATOR — its mcp_server guards sys.exit when a Telegram bot token / telegram agent-role is present, and it is not a per-agent tool provider. Auto-mounting it is both semantically wrong and a cold-start hazard (its heavy import blocks the serial resolution loop). _iter_registry now delegates the skip decision to a new _mount_skip() helper which prefers scitex-dev's is_mcp_mountable() SSoT and falls back to the local archived/category/mcp_mountable-field checks plus a _LOCAL_UNMOUNTABLE guard (scitex-orochi) so the orchestrator is skipped even against an older scitex-dev that predates the SSoT helper. Coordinated two-repo change with scitex-dev _ecosystem._core.is_mcp_mountable / _MCP_UNMOUNTABLE. Adds 4 focused tests to test_mcp_entrypoint.py. * fix(mcp): also skip scitex-types (zero tools, heavy import) in aggregator scitex-types ships no _mcp_server (zero MCP tools) but importing it pulls the heavy scientific stack (numpy/torch/…). Probing it for a non-existent FastMCP is pure cold-start waste. Add it to _LOCAL_UNMOUNTABLE (kept in sync with scitex-dev _core._MCP_UNMOUNTABLE). This rides on top of the #347/#348 bounded per-peer mount: skipped peers are removed from the peer list before resolution, so no bounded daemon thread is even spawned for them. Adds 2 tests (types excluded from _iter_registry / _mount_skip True).
…n 0.3.1 (MCP-aggregator incident close) (#350)
The umbrella aggregator's _mount_skip prefers scitex-dev's is_mcp_mountable SSoT and falls back to the local _LOCAL_UNMOUNTABLE set when an older scitex-dev (without that helper) is installed. scitex-str ships no _mcp server (zero tools) yet importing scitex_str eagerly pulls pandas + numpy via its _search / _plot submodules — pure cold-start waste, exactly like scitex-types. Add it to the fallback set so the skip holds across the coordinated two-repo rollout even before the scitex-dev SSoT bump is installed. Diagnosed in sac's real-SIF re-verify of umbrella 2.30.8. Pairs with scitex-dev _MCP_UNMOUNTABLE bump and scitex-resource lazy-import PR. Tests: assert _LOCAL_UNMOUNTABLE covers orochi/types/str and that _mount_skip returns True for scitex-str via the forced fallback path.
…letion group, categorized help (#352) * fix(cli): repair writer mount crash and mount scholar - scitex writer --help crashed with AttributeError: the registry probe picked up scitex_writer._cli.main (a plain console-script function) instead of the click group. Add the doctrine 5b re-export shim (scitex/cli/writer.py -> scitex_writer._cli.main_group) and make the lazy loader skip non-click-command attributes so a probe can never mount an uncallable object again. - scitex scholar was not mounted: no probe matched the standalone's entry point scitex_scholar._cli_main:cli. Add that probe shape. * refactor(cli): retire duplicate umbrella namespaces via warn-phase aliases notify->notification, verify->clew, events->event, socialia->social. The old names are no longer registered as lazy subcommands; when scitex-dev's click_compat helper is importable (scitex-dev > 0.21.0) they come back as hidden warn-phase deprecated aliases that forward to the canonical command (doctrine 3-phase ladder, removed in v3.0); with an older scitex-dev they are simply excluded. figrecipe/plt is intentionally NOT deduplicated: scitex-plt is a published identity-alias package for figrecipe and doctrine 5b's brand table documents 'scitex plt' as a figrecipe mount. Hidden commands (the aliases) are now skipped in root help output. * refactor(cli): make completion the canonical noun group Doctrine 1b (04_exceptions.md, amended 2026-07-07): a bare completion command is banned; the completion GROUP with verb subcommands is the canon. Changes: - move the group out of main.py into scitex/cli/completion.py (lazily mounted like every other wrapper; main.py shrinks well under the 512-line cap) - bare 'scitex completion' now shows group help instead of silently auto-installing - 'completion install' gains --dry-run: prints the target rc file and the completion script without touching the filesystem (subsumes the old bash/zsh/fish script-dump leaves) - 'completion bash|zsh|fish' become hidden warn-phase deprecated leaves: still print the script, warn on stderr, removed in v3.0 - root help examples updated ('scitex completion --show' never existed) * feat(cli): categorized root help, -V flag, real peer one-liners - root help now renders the doctrine 4a fixed ordered category headers (Core / Data & Sync / Service / Diagnostics / Introspection / Shell; Other is the catch-all and stays empty) — implemented inside LazyGroup.format_commands so no lazy subcommand is imported for help (scitex-dev's CategorizedGroup resolves real command objects, which would defeat the lazy mount and reintroduce the ~45s cold start) - add -V as the short form of --version (doctrine 08 universal flags) - ~25 mounted groups whose one-liner degraded to the bare package name when the peer was not installed (dataset, git, hpc, newb, datetime, ...) now fall back to a curated one-liner sourced from each package's own pyproject description (_INTERNAL_HELP renamed _FALLBACK_HELP) * test(cli): cover slice-5 umbrella CLI standardization - writer/scholar mount smoke (--help exits 0) - retired duplicates hidden from help; warn-phase alias help page when scitex-dev click_compat is importable, unknown-command otherwise - completion noun group: verbs listed, deprecated bash/zsh/fish leaves hidden but still printing the script with a stderr warning, install --dry-run prints plan and never writes - -V short flag; 4a category headers, order, and empty Other - builder unit tests: alias exclusion, scholar probe shape, writer wrapper override, fallback one-liners
Pilot org-level reusable-workflow migration (operator-approved 2026-07-10 incident remediation), following the pattern merged in scitex-ai/scitex-stats#69. Converted (clean mechanical match — job body verified byte-identical to the reusable workflow before conversion): - auto-merge-to-develop.yaml -> caller stub for scitex-ai/.github/.github/workflows/auto-merge-to-develop.yml@main Left AS-IS (do not cleanly match the current reusable-workflow catalog today — converting would silently change behavior or target an unverified runner; see PR description for detail on each): - pytest-matrix-on-ubuntu-py3-11-3-12-3-13.yml: runs on ubuntu-latest (not self-hosted spartan-cpu) with CPU-torch pre-pin, peer-standalone installs, a PS-170 umbrella-pin audit step, and a segfault-retry + JUnit-report pass/fail decoupling scheme with none of that present in the reusable pytest-matrix.yml. - import-smoke-on-ubuntu-py3-12.yml: runs on ubuntu-latest with plain pip/venv and a hardcoded package name, vs. the reusable workflow's self-hosted/uv/derived-package-name approach. - scitex-quality-audit-on-ubuntu-latest.yml: audits the WHOLE ecosystem (clones every scitex-* repo) rather than auditing only the calling repo, which is what the reusable quality-audit.yml does — not the same job semantically. - rtd-sphinx-build-on-ubuntu-latest.yml: also commits the built HTML bundle back into src/scitex/_sphinx_html/ on develop pushes, a repo-specific step with no equivalent in the reusable workflow. - cla.yml: no `push` trigger / no owner-bypass job (structurally simpler than the reusable cla.yml), and uses a differently-named secret (CLA_PERSONAL_ACCESS_TOKEN vs. the reusable workflow's hardcoded GH_PERSONAL_ACCESS_TOKEN) — converting risked silently breaking the CLA-signature push. - newb.yml / newb-docs-quality-on-ubuntu-latest.yml: no reusable workflow exists for this job yet (two near-duplicate files already). - pypi-publish-and-github-release-on-tag.yml: untouched per explicit instruction (PyPI OIDC trusted publishing does not support workflow_call). Branch protection: confirmed via `gh api .../branches/{develop,main}/ protection` that neither branch has protection configured (404 on both) — nothing to update.
chore(ci): migrate to scitex-ai/.github reusable workflows (pilot follow-up)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A develop → main promotion PR, opened by
scitex-devas part of a fleet-wide sweep. Your default branch is behind your owndevelop, which meansmainno longer reflects what has been released.You decide whether to merge it. I do not merge other packages' default branches — opening a PR is reversible and CI-gated; merging is neither. If this is wrong for your repo, close it and nothing is lost.
Why it was opened
Measured across the fleet: 22 repos had
mainbehind their newest tag. That gap is not cosmetic — it means:main.On the "behind" count
If the compare view says
mainis also behinddevelopby a few commits, that is expected and not divergence: merge-based promotion makesmainaccumulate merge commits thatdevelopnever sees. It is the normal steady state for this workflow.Authorisation
Operator ruling, 2026-08-04: PRs against repos
scitex-devdoes not own are explicitly authorized. This sweep was held for six days waiting on exactly that word, and it applies here.Tracked on card
default-branch-behind-published-release-promotion-gap-20260729.