Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Bazel Build Event Stream support #811

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

aherrmann
Copy link
Contributor

@aherrmann aherrmann commented Nov 15, 2024

This PR adds support to connect Buck2 to a Bazel Build Event Stream server as found among some Bazel remote build execution platforms. It repurposes the existing RemoteEventSink used to connect to Scribe within Meta's internal Buck2 version and connects it to a Bazel Build Event Service over gRPC instead, and it adds a translation layer between Buck2 build events and Bazel build events.

This is a work in progress, only very few build events are translated so far. To test this code you can

  • build Buck2,
  • change into the examples/hello_world directory,
  • configure a BES service by setting the BES_URI environment-variable, and
  • start a build.
$ cargo build --bin=buck2
$ cd examples/hello_world
$ BES_URI=XYZ ../../target/debug/buck2 build :main

For example, with BuildBuddy, if you run bazel run server -- -app.log_level debug locally, set BES_URI=http://localhost:1985.

The Buck2 Trace-ID is used as the BES invocation ID and you can see a Buck2 build on the UI under e.g. http://localhost:8080/invocation/e3123cfa-c350-432b-9784-8b3fe48bed14.

  • BuckEvent: provide a Buck Event Publisher proto
  • Add buck2_event_publisher_proto to Cargo.toml
  • Add Bazel Build Event Protocol/Stream protobuf
  • Implement RemoteEventSink to publish to Bazel BES

sluongng and others added 4 commits November 1, 2024 10:51
Provide a BuckEvent Publisher service which emits BuckEvents to an
external server implementation.

Closes facebook#226
Repurposes the RemoteEventSink used to connect to Scribe within Meta internal Buck2 and connects
it to a Bazel Build Event Service over gRPC instead. Adds a translation layer between Buck2
build events and Bazel build events.
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 15, 2024
@nlopezgi
Copy link

Hi @aherrmann , thanks for putting this together, we are really looking forward to it. I tested it out and had a couple of questions:

  • I see the TLS bit is marked on a TODO as not enabled yet, are you planning on setting this up as part of this CL or a future one?
  • Is there any way that I can get some verbose logs from the service publishing BuildEvents? I did not find a way to see anything from this interaction.

Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants