Skip to content

feat: unify fast-build/upstream-sync into cross-platform Python orchestrator and wire just tasks#110

Merged
zapabob merged 2 commits intomainfrom
codex/consolidate-build-tasks-in-just
Mar 20, 2026
Merged

feat: unify fast-build/upstream-sync into cross-platform Python orchestrator and wire just tasks#110
zapabob merged 2 commits intomainfrom
codex/consolidate-build-tasks-in-just

Conversation

@zapabob
Copy link
Owner

@zapabob zapabob commented Mar 20, 2026

Motivation

  • Consolidate platform-specific build/install logic into a single, cross-platform orchestration so just tasks can call a canonical implementation.
  • Make differential build detection and upstream-merge conflict resolution explicit, testable, and overridable (jobs, detection method, profile).
  • Keep Windows convenience wrapper while moving core responsibilities (diff detection, build, install, resolver) into maintainable Python scripts and update docs to reference the real implementations.

Description

  • Added a cross-platform Python orchestrator scripts/fast_build.py that exposes fast-build, fast-build-install, and upstream-sync subcommands, per-target configuration (codex-cli, codex-tui, codex-gui, codex-gui-x, extensions), and change-detection modes md5, mtime, and cargo-metadata (default method md5), and honors CODEX_FAST_BUILD_JOBS/--jobs (default 6).
  • Added scripts/resolve_merge_conflicts.py implementing path-pattern rules and an “upstream-first + reinject unique local lines” strategy, and scripts/upstream_sync.py as a thin wrapper that invokes the upstream merge flow using the orchestrator.
  • Replaced the previous heavy Windows-only script with a thin PowerShell wrapper codex-rs/fast_build.ps1 that forwards task, method, jobs, targets and flags into the shared Python pipeline and supports py -3 when available.
  • Wired just entry points into both repo root justfile and codex-rs/justfile as fast-build, fast-build-install, and upstream-sync to call the shared script (via platform python selection).
  • Introduced .codex-fast-build-cache.json cache path in scripts/fast_build.py and updated fingerprinting to optionally include cargo metadata for Rust targets.
  • Un-ignored the new scripts by adjusting .gitignore and updated documentation and release notes to reference the new implementation names and just tasks (files edited: CHANGELOG.md, CLAUDE.md, codex-rs/README.md, docs/development/*, docs/plan/MERGE_STRATEGY.md).

Testing

  • Ran python3 scripts/fast_build.py list-targets and confirmed target listing succeeded (PASS).
  • Verified CLI help for the new commands with python3 scripts/fast_build.py fast-build --help and python3 scripts/upstream_sync.py --help (PASS).
  • Compiled the added Python files to bytecode with python3 -m py_compile scripts/fast_build.py scripts/resolve_merge_conflicts.py scripts/upstream_sync.py (PASS).
  • Exercised the resolver end-to-end on a synthetic conflict file and confirmed scripts/resolve_merge_conflicts.py resolves markers using the expected strategy (PASS).
  • Performed repository consistency checks including git diff --check on modified docs/justfile entries and validated just wiring by invoking the underlying Python entry points (note: just binary not present in this environment, so just tasks were validated by inspecting and directly running the Python scripts) (PASS with environment note).

Codex Task

@vercel
Copy link

vercel bot commented Mar 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
codex Error Error Mar 20, 2026 2:17pm

@zapabob zapabob merged commit 43cd173 into main Mar 20, 2026
17 of 50 checks passed
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3a0d75ccc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

kind="rust",
cwd=WORKSPACE_ROOT,
build_cmd=["cargo", "build", "--release", "-p", "codex-cli", "--features", "custom-features"],
watch_roots=(WORKSPACE_ROOT / "cli", WORKSPACE_ROOT / "core", WORKSPACE_ROOT / "exec", WORKSPACE_ROOT / "protocol", WORKSPACE_ROOT / "config", WORKSPACE_ROOT / "state", WORKSPACE_ROOT / "mcp-server", WORKSPACE_ROOT / "deep-research", WORKSPACE_ROOT / "utils"),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include all direct workspace deps in watch_roots

These watch_roots cover only a subset of each binary's Rust dependencies. I checked codex-rs/cli/Cargo.toml and codex-rs/tui/Cargo.toml: codex-cli directly depends on crates such as cloud-tasks, login, responses-api-proxy, rmcp-client, and supervisor, while codex-tui depends on backend-client, feedback, file-search, shell-command, and several utils/* crates that are not hashed here. With the new just fast-build --changed-only flow, edits in those crates leave the fingerprint unchanged, so the script skips rebuilding binaries that have actually changed.

Useful? React with 👍 / 👎.

Comment on lines +312 to +316
installable = [name for name in resolve_targets(args.targets) if TARGETS[name].install_map]
if not installable:
logger.info("Selected targets do not produce installable binaries; skipping install.")
return 0
install_targets(installable, logger)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Install only the targets that were actually rebuilt

cmd_fast_build() can legitimately skip every build when --changed-only finds an up-to-date cache, but cmd_fast_build_install() immediately re-resolves the full requested set and copies every installable artifact anyway. In practice, once .codex-fast-build-cache.json exists, running just fast-build-install after cargo clean (or after deleting one artifact) raises Missing build artifact ... even though there were no source changes; if only a non-installable target changed, this path can also redeploy stale CLI/TUI binaries.

Useful? React with 👍 / 👎.

Comment on lines +271 to +274
if "--release" in patched and args.profile != "release":
patched.remove("--release")
patched[2:2] = ["--profile", args.profile]
patched.extend(["-j", str(args.jobs)])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make fast-build-install honor the selected Cargo profile

The new --profile flag rewrites cargo build --release to cargo build --profile <name>, but the install sources are still hard-coded to target/release in the target table above. That means scripts/fast_build.py fast-build-install --profile dev codex-cli either copies an old release binary or fails with Missing build artifact because Cargo wrote target/dev/codex instead, so the advertised profile override is unsafe for install flows.

Useful? React with 👍 / 👎.

@github-actions
Copy link


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions github-actions bot locked and limited conversation to collaborators Mar 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant