Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename README.md to -github.com-binance-exchange-binance-official-/ap… #104

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Recepxx34
Copy link

…i-docs.git

# Go Build Tools

[![Go Reference](https://pkg.go.dev/badge/golang.org/x/build.svg)](https://pkg.go.dev/golang.org/x/build)

This repository holds the source for various packages and tools that support Go's build system and the development of the Go programming language.

**Warning:** Packages here are internal to Go's build system and its needs. Some may one day be promoted to another `golang.org/x` repository, or they may be modified arbitrarily or even disappear altogether. In short, code in this repository is not subject to the Go 1 compatibility promise nor the [Release Policy](https://go.dev/doc/devel/release#policy).

## Report Issues / Send Patches

This repository uses Gerrit for code changes. To contribute, see https://go.dev/doc/contribute.

The git repository is https://go.googlesource.com/build.

The main issue tracker for the build repository is located at https://go.dev/issues. Prefix your issue with
"`x/build/DIR: `" in the subject line.

## Overview

The main components of the Go build system are:

* The **coordinator**, in [cmd/coordinator/](https://dev.golang.org/dir/build/cmd/coordinator/), serves https://farmer.golang.org/ and https://build.golang.org/. It runs on GKE and coordinates the whole build system. It finds work to do (both pre-submit "TryBot" work, and post-submit work) and executes builds, allocating machines to run the builds. It is the owner of all machines. It holds the state for which builds passed or failed, and the build logs.

* The Go package in [buildenv/](https://dev.golang.org/dir/build/buildenv/) contains constants for where the dashboard and coordinator run, for prod, staging, and local development.

* The **buildlet**, in [cmd/buildlet/](https://dev.golang.org/dir/build/cmd/buildlet/), is the HTTP server that runs on each worker machine to execute builds on the coordinator's behalf. This runs on every possible GOOS/GOARCH value. The buildlet binaries are stored on Google Cloud Storage and fetched per-build, so we can update the buildlet binary independently of the underlying machine images. The buildlet is the most insecure server possible: it has HTTP handlers to read & write arbitrary content to disk, and to execute any file on disk. It also has an SSH tunnel handler. The buildlet must never be exposed to the Internet. The coordinator provisions buildlets in one of three ways:

  1. by creating VMs on Google Compute Engine (GCE) with custom images configured to fetch & run the buildlet on boot, listening on port 80 in a private network.

  2. by running Linux containers (on either Google Kubernetes Engine or GCE with the Container-Optimized OS image), with the container images configured to fetch & run the buildlet on start, also listening on port 80 in a private network.

  3. by taking buildlets out of a pool of connected, dedicated machines. The buildlet can run in either *listen mode* (as on GCE and GKE) or in *reverse mode*. In reverse mode, the buildlet connects out to https://farmer.golang.org/ and registers itself with the coordinator. The TCP connection is then logically reversed (using [revdial](https://dev.golang.org/dir/build/revdial/) and when the coordinator needs to do a build, it makes HTTP requests to the coordinator over the already-open TCP connection.

  These three pools can be viewed at the coordinator's
  https://farmer.golang.org/#pools.


* The [env/](https://dev.golang.org/dir/build/env/) directory describes build environments. It contains scripts to create VM images, Dockerfiles to create Kubernetes containers, and instructions and tools for dedicated machines.


* **maintner** in [maintner/](https://dev.golang.org/dir/build/maintner) is a library for slurping all of Go's GitHub and Gerrit state into memory. The daemon **maintnerd** in [maintner/maintnerd/](https://dev.golang.org/dir/build/maintner/maintnerd) runs on GKE and serves https://maintner.golang.org/. The daemon watches GitHub and Gerrit and appends to a mutation log whenever it sees new activity. The logs are stored on GCS and served to clients.


* The [godata package](https://godoc.org/golang.org/x/build/maintner/godata) in [maintner/godata/](https://dev.golang.org/dir/build/maintner/godata) provides a trivial API to let anybody write programs against Go's maintner corpus (all of our GitHub and Gerrit history), live up to the second. It takes a few seconds to load into memory and a few hundred MB of RAM after it downloads the mutation log from the network.


* **pubsubhelper** in [cmd/pubsubhelper/](https://dev.golang.org/dir/build/cmd/pubsubhelper/) is a dependency of maintnerd. It runs on GKE, is available at https://pubsubhelper.golang.org/, and runs an HTTP server to receive Webhook updates from GitHub on new activity and an SMTP server to receive new activity emails from Gerrit. It then is a pubsub system for maintnerd to subscribe to.


* The **gitmirror** server in [cmd/gitmirror/](https://dev.golang.org/dir/build/cmd/gitmirror) mirrors Gerrit to GitHub, and also serves a mirror of the Gerrit code to the coordinator for builds, so we don't overwhelm Gerrit and blow our quota.


* The Go **gopherbot** bot logic runs on GKE. The code is in [cmd/gopherbot](https://dev.golang.org/dir/build/cmd/gopherbot). It depends on maintner via the godata package.


* The **developer dashboard** at https://dev.golang.org/ runs on GKE. Its code is in [devapp/](https://dev.golang.org/dir/build/devapp/). It also depends on maintner via the godata package.


* **cmd/retrybuilds**: a Go client program to delete build results from the dashboard


* The **perfdata** server, in [perfdata/appengine](https://dev.golang.org/dir/build/perfdata/appengine) serves https://perfdata.golang.org/. It runs on App Engine and serves the benchmark result storage system.


* The **perf** server, in [perf/appengine](https://dev.golang.org/dir/build/perf/appengine) serves https://perf.golang.org/. It runs on App Engine and serves the benchmark result analysis system. See [its README](perf/README.md) for how to start a local testing instance.


### Adding a Go Builder

If you wish to run a Go builder, please email
[[email protected]](mailto:[email protected]) first. There is documentation at https://golang.org/wiki/DashboardBuilders, but depending on the type of builder, we may want to run it ourselves, after you prepare an environment description (resulting in a VM image) of it. See the env directory.

…i-docs.git

# Go Build Tools

[![Go Reference](https://pkg.go.dev/badge/golang.org/x/build.svg)](https://pkg.go.dev/golang.org/x/build)

This repository holds the source for various packages and tools that support
Go's build system and the development of the Go programming language.

**Warning:** Packages here are internal to Go's build system and its needs.
Some may one day be promoted to another `golang.org/x` repository,
or they may be modified arbitrarily or even disappear altogether.
In short, code in this repository is not subject to the Go 1 compatibility
promise nor the [Release Policy](https://go.dev/doc/devel/release#policy).

## Report Issues / Send Patches

This repository uses Gerrit for code changes. To contribute, see
https://go.dev/doc/contribute.

The git repository is https://go.googlesource.com/build.

The main issue tracker for the build repository is located at
https://go.dev/issues. Prefix your issue with
"`x/build/DIR: `" in the subject line.

## Overview

The main components of the Go build system are:

* The **coordinator**, in
  [cmd/coordinator/](https://dev.golang.org/dir/build/cmd/coordinator/),
  serves https://farmer.golang.org/ and https://build.golang.org/.
  It runs on GKE and coordinates the whole build system. It finds work
  to do (both pre-submit "TryBot" work, and post-submit work) and executes
  builds, allocating machines to run the builds. It is the owner of all machines.
  It holds the state for which builds passed or failed, and the build logs.

* The Go package in [buildenv/](https://dev.golang.org/dir/build/buildenv/)
  contains constants for where the dashboard and coordinator run, for prod,
  staging, and local development.

* The **buildlet**, in
  [cmd/buildlet/](https://dev.golang.org/dir/build/cmd/buildlet/), is the
  HTTP server that runs on each worker machine to execute builds on the
  coordinator's behalf. This runs on every possible GOOS/GOARCH value. The
  buildlet binaries are stored on Google Cloud Storage and fetched
  per-build, so we can update the buildlet binary independently of the
  underlying machine images. The buildlet is the most insecure server
  possible: it has HTTP handlers to read & write arbitrary content to disk,
  and to execute any file on disk. It also has an SSH tunnel handler. The
  buildlet must never be exposed to the Internet. The coordinator provisions
  buildlets in one of three ways:

  1. by creating VMs on Google Compute Engine (GCE) with custom images
  configured to fetch & run the buildlet on boot, listening on port 80 in a
  private network.

  2. by running Linux containers (on either Google Kubernetes Engine
  or GCE with the Container-Optimized OS image), with the container
  images configured to fetch & run the buildlet on start, also
  listening on port 80 in a private network.

  3. by taking buildlets out of a pool of connected, dedicated machines. The
  buildlet can run in either *listen mode* (as on GCE and GKE) or in
  *reverse mode*. In reverse mode, the buildlet connects out to
  https://farmer.golang.org/ and registers itself with the coordinator. The
  TCP connection is then logically reversed (using
  [revdial](https://dev.golang.org/dir/build/revdial/) and when the
  coordinator needs to do a build, it makes HTTP requests to the coordinator
  over the already-open TCP connection.

  These three pools can be viewed at the coordinator's
  https://farmer.golang.org/#pools.


* The [env/](https://dev.golang.org/dir/build/env/) directory describes
  build environments. It contains scripts to create VM images, Dockerfiles
  to create Kubernetes containers, and instructions and tools for dedicated
  machines.


* **maintner** in [maintner/](https://dev.golang.org/dir/build/maintner) is
  a library for slurping all of Go's GitHub and Gerrit state into memory.
  The daemon **maintnerd** in
  [maintner/maintnerd/](https://dev.golang.org/dir/build/maintner/maintnerd)
  runs on GKE and serves https://maintner.golang.org/. The daemon watches
  GitHub and Gerrit and appends to a mutation log whenever it sees new
  activity. The logs are stored on GCS and served to clients.


* The [godata package](https://godoc.org/golang.org/x/build/maintner/godata)
  in [maintner/godata/](https://dev.golang.org/dir/build/maintner/godata)
  provides a trivial API to let anybody write programs against
  Go's maintner corpus (all of our GitHub and Gerrit history), live up
  to the second. It takes a few seconds to load into memory and a few hundred
  MB of RAM after it downloads the mutation log from the network.


* **pubsubhelper** in
  [cmd/pubsubhelper/](https://dev.golang.org/dir/build/cmd/pubsubhelper/) is
  a dependency of maintnerd. It runs on GKE, is available at
  https://pubsubhelper.golang.org/, and runs an HTTP server to receive
  Webhook updates from GitHub on new activity and an SMTP server to receive
  new activity emails from Gerrit. It then is a pubsub system for maintnerd
  to subscribe to.


* The **gitmirror** server in
  [cmd/gitmirror/](https://dev.golang.org/dir/build/cmd/gitmirror) mirrors
  Gerrit to GitHub, and also serves a mirror of the Gerrit code to the
  coordinator for builds, so we don't overwhelm Gerrit and blow our quota.


* The Go **gopherbot** bot logic runs on GKE. The code is in
  [cmd/gopherbot](https://dev.golang.org/dir/build/cmd/gopherbot). It
  depends on maintner via the godata package.


* The **developer dashboard** at https://dev.golang.org/ runs on GKE.
  Its code is in [devapp/](https://dev.golang.org/dir/build/devapp/).
  It also depends on maintner via the godata package.


* **cmd/retrybuilds**: a Go client program to delete build results from the
    dashboard


* The **perfdata** server, in
  [perfdata/appengine](https://dev.golang.org/dir/build/perfdata/appengine)
  serves https://perfdata.golang.org/. It runs on App Engine and
  serves the benchmark result storage system.


* The **perf** server, in
  [perf/appengine](https://dev.golang.org/dir/build/perf/appengine)
  serves https://perf.golang.org/. It runs on App Engine and serves
  the benchmark result analysis system. See [its
  README](perf/README.md) for how to start a local testing instance.


### Adding a Go Builder

If you wish to run a Go builder, please email
[[email protected]](mailto:[email protected]) first. There
is documentation at https://golang.org/wiki/DashboardBuilders, but depending
on the type of builder, we may want to run it ourselves, after you prepare an
environment description (resulting in a VM image) of it. See the env directory.
Copy link

google-cla bot commented Nov 20, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant