Skip to content

Conversation

@zbarsky-openai
Copy link
Contributor

@zbarsky-openai zbarsky-openai commented Jan 8, 2026

This PR configures Codex CLI so it can be built with Bazel in addition to Cargo. The .bazelrc includes configuration so that remote builds can be done using BuildBuddy.

If you are familiar with Bazel, things should work as you expect, e.g., run bazel test //... --keep-going to run all the tests in the repo, but we have also added some new aliases in the justfile for convenience:

  • just bazel-test to run tests locally
  • just bazel-remote-test to run tests remotely (currently, the remote build is for x86_64 Linux regardless of your host platform). Note we are currently seeing the following test failures in the remote build, so we still need to figure out what is happening here:
failures:
    suite::compact::manual_compact_twice_preserves_latest_user_messages
    suite::compact_resume_fork::compact_resume_after_second_compaction_preserves_history
    suite::compact_resume_fork::compact_resume_and_fork_preserve_model_history_view
  • just build-for-release to build release binaries for all platforms/architectures remotely

To setup remote execution:

CI

In terms of CI, this PR introduces .github/workflows/bazel.yml, which uses Bazel to run the tests locally on Mac and Linux GitHub runners (we are working on supporting Windows, but that is not ready yet). Note that the failures we are seeing in just bazel-remote-test do not occur on these GitHub CI jobs, so everything in .github/workflows/bazel.yml is green right now.

The bazel.yml uses extra config in .github/workflows/ci.bazelrc so that macOS CI jobs build remotely on Linux hosts (using the docker://docker.io/mbolin491/codex-bazel Docker image declared in the root BUILD.bazel) using cross-compilation to build the macOS artifacts. Then these artifacts are downloaded locally to GitHub's macOS runner so the tests can be executed natively. This is the relevant config that enables this:

common:macos --config=remote
common:macos --strategy=remote
common:macos --strategy=TestRunner=darwin-sandbox,local

Because of the remote caching benefits we get from BuildBuddy, these new CI jobs can be extremely fast! For example, consider these two jobs that ran all the tests on Linux x86_64:

For now, we will continue to run both the Bazel and Cargo jobs for PRs, but once we add support for Windows and running Clippy, we should be able to cutover to using Bazel exclusively for PRs, which should still speed things up considerably. We will probably continue to run the Cargo jobs post-merge for commits that land on main as a sanity check.

Release builds will also continue to be done by Cargo for now.

Earlier attempt at this PR: #8832
Earlier attempt to add support for Buck2, now abandoned: #8504

@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

bolinfest added a commit that referenced this pull request Jan 8, 2026
…8879)

To support Bazelification in #8875,
this PR introduces a new `find_resource!` macro that we use in place of
our existing logic in tests that looks for resources relative to the
compile-time `CARGO_MANIFEST_DIR` env var.

To make this work, we plan to add the following to all `rust_library()`
and `rust_test()` Bazel rules in the project:

```
rustc_env = {
    "BAZEL_PACKAGE": native.package_name(),
},
```

Our new `find_resource!` macro reads this value via
`option_env!("BAZEL_PACKAGE")` so that the Bazel package _of the code
using `find_resource!`_ is injected into the code expanded from the
macro. (If `find_resource()` were a function, then
`option_env!("BAZEL_PACKAGE")` would always be
`codex-rs/utils/cargo-bin`, which is not what we want.)

Note we only consider the `BAZEL_PACKAGE` value when the `RUNFILES_DIR`
environment variable is set at runtime, indicating that the test is
being run by Bazel. In this case, we have to concatenate the runtime
`RUNFILES_DIR` with the compile-time `BAZEL_PACKAGE` value to build the
path to the resource.

In testing this change, I discovered one funky edge case in
`codex-rs/exec-server/tests/common/lib.rs` where we have to _normalize_
(but not canonicalize!) the result from `find_resource!` because the
path contains a `common/..` component that does not exist on disk when
the test is run under Bazel, so it must be semantically normalized using
the [`path-absolutize`](https://crates.io/crates/path-absolutize) crate
before it is passed to `dotslash fetch`.

Because this new behavior may be non-obvious, this PR also updates
`AGENTS.md` to make humans/Codex aware that this API is preferred.
bolinfest added a commit that referenced this pull request Jan 8, 2026
bolinfest added a commit that referenced this pull request Jan 8, 2026
…8887)

This eliminates our dependency on the `escargot` crate and better
prepares us for Bazel builds: #8875.
@@ -1,6 +1,8 @@
#![allow(clippy::unwrap_used)]

use codex_apply_patch::APPLY_PATCH_TOOL_INSTRUCTIONS;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will go away once #8914 is in.

bolinfest added a commit that referenced this pull request Jan 8, 2026
…ssues (#8932)

I have seen this test flake out sometimes when running the macOS build
using Bazel in CI: #8875. Perhaps
Bazel runs with greater parallelism, inducing a heavier load, causing an
issue?
bolinfest added a commit that referenced this pull request Jan 8, 2026
…t_rule() test (#8931)

Because the path to `git` is used to construct `elicitations_to_accept`,
we need to ensure that we resolve which `git` to use the same way our
Bash process will:


https://github.com/openai/codex/blob/c9c65606852c0cda9d983b4917359a0826a4b7f0/codex-rs/exec-server/tests/suite/accept_elicitation.rs#L59-L69

This fixes an issue when running the test on macOS using Bazel
(#8875) where the login shell chose
`/opt/homebrew/bin/git` whereas the non-login shell chose
`/usr/bin/git`.
@bolinfest bolinfest changed the title [please leave open, collaborating with mbolin] WIP: bazelification feat: add support for building with Bazel Jan 8, 2026
bolinfest added a commit that referenced this pull request Jan 8, 2026
This seems to be necessary to get the Bazel builds on ARM Linux to go
green on #8875.

I don't feel great about timeout-whack-a-mole, but we're still learning
here...
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential curl git python3 ca-certificates \
pkg-config clang musl-tools libssl-dev just && \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't need libssl-dev, it comes from bazel and is built hermetically :)

@zbarsky-openai zbarsky-openai force-pushed the zbarsky/bazel branch 2 times, most recently from f9dc9e8 to aeb3607 Compare January 9, 2026 04:06
pap-openai pushed a commit that referenced this pull request Jan 9, 2026
…8879)

To support Bazelification in #8875,
this PR introduces a new `find_resource!` macro that we use in place of
our existing logic in tests that looks for resources relative to the
compile-time `CARGO_MANIFEST_DIR` env var.

To make this work, we plan to add the following to all `rust_library()`
and `rust_test()` Bazel rules in the project:

```
rustc_env = {
    "BAZEL_PACKAGE": native.package_name(),
},
```

Our new `find_resource!` macro reads this value via
`option_env!("BAZEL_PACKAGE")` so that the Bazel package _of the code
using `find_resource!`_ is injected into the code expanded from the
macro. (If `find_resource()` were a function, then
`option_env!("BAZEL_PACKAGE")` would always be
`codex-rs/utils/cargo-bin`, which is not what we want.)

Note we only consider the `BAZEL_PACKAGE` value when the `RUNFILES_DIR`
environment variable is set at runtime, indicating that the test is
being run by Bazel. In this case, we have to concatenate the runtime
`RUNFILES_DIR` with the compile-time `BAZEL_PACKAGE` value to build the
path to the resource.

In testing this change, I discovered one funky edge case in
`codex-rs/exec-server/tests/common/lib.rs` where we have to _normalize_
(but not canonicalize!) the result from `find_resource!` because the
path contains a `common/..` component that does not exist on disk when
the test is run under Bazel, so it must be semantically normalized using
the [`path-absolutize`](https://crates.io/crates/path-absolutize) crate
before it is passed to `dotslash fetch`.

Because this new behavior may be non-obvious, this PR also updates
`AGENTS.md` to make humans/Codex aware that this API is preferred.
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential curl git python3 ca-certificates \
pkg-config clang musl-tools libssl-dev && \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we try to remove these?

  • pkg-config
  • clang
  • musl-tools
  • libssl-dev
  • (maybe, not sure) build-essential?

sorry to be a pain, but clang in particular is huge and i want to make sure we have no compilers in the base image so they're not used by accident :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentally associate pkg-config with "python stuff," but chat says ends up being needed by some common Python packages that contain C extensions, so I agree that the builder host should not rely on those. Will try removing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in our case it's likely used by rust build scripts to find C libs on the host to link against. But we don't want that, we want the libs provided hermetically through bazel, as otherwise cross-compilation won't work

@bolinfest bolinfest self-requested a review January 9, 2026 18:17
Copy link
Collaborator

@bolinfest bolinfest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is game-changing!

@zbarsky-openai
Copy link
Contributor Author

🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢 🚢

@bolinfest bolinfest merged commit 2a06d64 into main Jan 9, 2026
32 checks passed
@bolinfest bolinfest deleted the zbarsky/bazel branch January 9, 2026 19:09
@github-actions github-actions bot locked and limited conversation to collaborators Jan 9, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants