Skip to content

Commit 915303f

Browse files
committed
docs: renovate README to reference doco site
The current README is a mish-mash, and needs to be split up into a proper guide.
1 parent 27cb64d commit 915303f

File tree

2 files changed

+21
-217
lines changed

2 files changed

+21
-217
lines changed

README.md

+21-217
Original file line numberDiff line numberDiff line change
@@ -1,239 +1,43 @@
1-
# Chinmina Bridge: Buildkite/Github OIDC token bridge
1+
# Chinmina Bridge
22

3-
[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fjamestelfer%2Fchinmina-bridge.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fjamestelfer%2Fchinmina-bridge?ref=badge_shield)
3+
**Connect Buildkite to GitHub with secure, short-lived tokens.**
44

5+
Chinmina Bridge allows Buildkite agents to securely generate GitHub API tokens
6+
that can be used to perform Git or other GitHub API actions. It is intended to
7+
be an alternative to the use of SSH deploy keys or long-lived Personal Access
8+
Tokens.
59

6-
Allows Buildkite agents to get valid GitHub tokens that can be used to perform
7-
Git or other GitHub API actions. It is intended to be an alternative to the use
8-
of SSH deploy keys or long-lived Personal Access Tokens.
10+
![High level Chinmina diagram](docs/chinmina-high-level.png)
911

1012
The bridge itself is an HTTP endpoint that uses a [GitHub
1113
application][github-app] to create [ephemeral GitHub access
1214
tokens][github-app-tokens]. Requests are authorized with a [Buildkite
1315
OIDC][buildkite-oidc] token, allowing a token to be created just for the
1416
repository associated with an executing pipeline.
1517

16-
The token is created with `contents:read` permissions, and only has access to
17-
the repository associated with the executing pipeline.
18-
19-
Two endpoints are exposed: `/token`, which returns a token and its expiry, and
20-
`/git-credentials`, which returns the token and repository metadata in the [Git
21-
Credentials format][git-credential-helper].
18+
> [!NOTE]
19+
> Find out more about Chinmina Bridge is available in the [documentation][docs].
20+
>
21+
> This has and expanded [introduction][docs-intro], a [getting
22+
> started][docs-started] guide and a detailed [configuration
23+
> reference][docs-config]. This has a more detailed description of the
24+
> implementation, and clear guidance on how to configuration and installation.
2225
2326
[github-app]: https://docs.github.com/en/apps
2427
[github-app-tokens]: https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app
2528
[buildkite-oidc]: https://buildkite.com/docs/agent/v3/cli-oidc
2629
[git-credential-helper]: https://git-scm.com/docs/gitcredentials#_custom_helpers
2730

28-
## Overview
29-
30-
`chinmina-bridge` is used by jobs running on a Buildkite agent to request tokens
31-
from Github. These can be used to communicate with the GitHub API or (via Git)
32-
to enable authenticated Git actions.
33-
34-
Git authentication is facilitated by a [Git credential
35-
helper](https://github.com/jamestelfer/github-app-auth-buildkite-plugin), which
36-
communicates with the bridge and supplies the result to Git in the appropriate
37-
format.
38-
39-
The following sequence illustrates a Git authentication flow facilitated by
40-
`chinmina-bridge`.
41-
42-
```mermaid
43-
sequenceDiagram
44-
box Buildkite Agent
45-
participant Buildkite Job
46-
participant Git
47-
participant Credential Helper
48-
end
49-
box Self hosted
50-
participant Chinmina Bridge
51-
end
52-
Buildkite Job->>+Git: clone
53-
Git ->>+ Credential Helper: get credentials
54-
Credential Helper->>+Buildkite API: Request Buildkite OIDC token
55-
Buildkite API->>-Credential Helper: bk-oidc
56-
Credential Helper->>+Chinmina Bridge: Request GH token (auth bk-oidc)
57-
Chinmina Bridge->>+Buildkite API: Get Pipeline Details
58-
Buildkite API-->>-Chinmina Bridge: pipeline-repository
59-
Chinmina Bridge->>+GitHub: Create Token (auth app JWT)
60-
GitHub-->>-Chinmina Bridge: app-token
61-
Chinmina Bridge->>-Credential Helper: bk-oidc
62-
Credential Helper->>-Git: "x-access-token"/app-token
63-
Git-->>-Buildkite Job: complete
64-
```
65-
66-
## Why?
67-
68-
There are two options generally used to authenticate Buildkite agents to GitHub:
69-
70-
1. Via a PAT (owned by a GitHub user) that is saved in the agent S3 secrets bucket
71-
2. Via a deploy key (registered to a single repository) that is likewise saved to
72-
S3.
73-
74-
As the organization scales however, the overhead of managing them becomes
75-
unwieldy, and it can be quite difficult for an organisation to successfully
76-
manage a rotation scheme.
77-
78-
Unless centralized issuance is practiced as well, both of these schemes can
79-
produce tokens that are tied to a user, leading to unexpected problems when a
80-
user leaves the organization. There is also the potential for key material to be
81-
stored or shared incorrectly, leading to increased possibility of accidental
82-
leakage.
83-
84-
Lastly, all key material is typically stored in an S3 bucket. This is
85-
straightforward to configure and maintain, but creates a significant issue in
86-
the event of an account/bucket breach.
87-
88-
Using a GitHub application to authenticate GitHub actions allows:
89-
90-
1. Access keys for repositories are generated on demand and expire after one
91-
hour.
92-
1. The generated tokens are only kept by a build agent for the duration of the
93-
step, and do not require any other persistence.
94-
1. The private key for the GitHub application is specific to the
95-
`chinmina-bridge` service. It can (and should) be rotated, an operation that
96-
is easy to perform.
97-
1. Supplied tokens are scoped to just the repositories and actions necessary for
98-
the requesting pipeline.
99-
1. Additional Buildkite configuration per repository is not required. If the
100-
application has access, the agent can request a token for it. No need to
101-
create PATs or generate keypairs, and no need to upload them in multiple
102-
places. This allows the an organization to have tighter access control on
103-
pipeline setup without creating additional support overhead.
104-
1. Tokens can enable a wider set of actions than simple Git operations (e.g. PR
105-
comments). This is not yet implemented in `chinmina-bridge`, but is a high
106-
priority for future enhancement.
107-
108-
Also, since `chinmina-bridge` uses Buildkite's OIDC tokens to authorize requests,
109-
the claims associated with the token can be used to further refine access to a token.
110-
111-
Github has some [good documentation][gh-deploy-keys] about the pros and cons of
112-
the application token approach. There are two primary downsides documented:
113-
114-
> - Additional setup is needed to create the GitHub App.
115-
> - Installation access tokens expire after 1 hour, and so need to be
116-
> re-generated, typically on-demand using code.
117-
118-
`chinmina-bridge` solves the second problem, by making token generation for a
119-
pipeline at build time trivial.
120-
121-
122-
[gh-deploy-keys]: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/managing-deploy-keys#github-app-installation-access-tokens
123-
124-
### What's right for your organization?
125-
126-
To understand what's right for your organization, consider:
127-
128-
- how many pipelines do you have? (That is, how many keys are managed?)
129-
- how easily are tokens rotated?
130-
- (related) if the secrets bucket is somehow compromised, how difficult would it be for the organization to respond?
131-
- if tokens are issued to a user, does a person leaving cause an outage in a build pipeline?
132-
- what processes/restrictions does your organization have around repository access in GitHub and pipeline creation in Buildkite?
133-
134-
## Limitations
135-
136-
- can only grant `contents:read` access
137-
- will only grant access to the repository associated with a pipeline
138-
- if the buildkite user has permissions to modify the pipeline repository, they
139-
may configure a repository that they don't have access to in GitHub (but is
140-
accessible in the app). This would allow them to potentially extract code via
141-
use of the pipeline step configuration. **BUT**:
142-
- it's OK if your organization members have read access to the same set of
143-
repositories covered by the `chinmina-bridge` GitHub application.<br>
144-
**OR**
145-
- it's OK if your organization controls the creation/configuration of
146-
pipelines: this restricts the opportunity to misconfigure a pipeline.
147-
148-
## Operations
149-
150-
See the [observability documentation](./docs/observability.md) for more details
151-
on the information provided by the system when running.
152-
153-
## Configuration
154-
155-
Requirements:
156-
157-
1. A Buildkite organization, and a user with sufficient access to create an API
158-
token that can be used to get the details of any pipeline that is expected to
159-
be built.
160-
1. A Github organization, and a user with sufficient permissions to create a
161-
Github App and install it into the organization.
162-
1. Ability to deploy a server that can be accessed by the build agents (for example, an ECS service)
163-
1. Ability to allow Buildkite agents to download and use a custom plugin _or_
164-
ability to add a plugin to the default settings of the Buildkite agents.
165-
166-
### Buildkite setup
167-
168-
Create an API key with access to the REST API **only** with access to the `read_pipelines` scope.
169-
170-
Save the key securely: it will be provided to the server in a later step. Use a
171-
"bot" user to create the token if you can.
172-
173-
### Github setup
174-
175-
1. Create an application in your Github organization
176-
- The application must request `contents:read`
177-
- Note the application ID
178-
- Create and save a private key for the application
179-
2. Install the application into the Github organization
180-
- choose the repositories the application will have access to. This is the
181-
limit of the resources that the application can vend tokens for.
182-
183-
### Configure and deploy the bridge server
184-
185-
The server is a Go application expecting to read configuration from environment
186-
variables, and can be deployed to a server or as a container.
187-
188-
#### Variables
189-
190-
**Server**
191-
192-
- `SERVER_PORT` (optional, default `8080`): the TCP port the server will listen on.
193-
- `SERVER_SHUTDOWN_TIMEOUT_SECS` (optional, default `25`): the number of seconds
194-
the server will wait when asked to terminate with `SIGINT`
195-
196-
**Authorization**
197-
198-
- `JWT_BUILDKITE_ORGANIZATION_SLUG` (**required**): the slug of your Buildkite
199-
organization. This is the identifier of your organization that appears in your
200-
Buildkite URLs.
201-
- `JWT_AUDIENCE` (optional, default=`app-token-issuer`): The expected value of the
202-
`aud` claim in the JWT. Describes the intended audience of the issued JWT
203-
token, guards against token reuse. Using a non-default value will require configuration of the credentials helper plugin.
204-
- `JWT_ISSUER_URL` (optional, default `https://agent.buildkite.com`): the
205-
expected value of the `iss` claim in the agent JWT. Also used to discover the
206-
JWKS configuration from the `.well-known` address.
207-
- `JWT_JWKS_STATIC` (optional): a local JWKS JSON file that can be used instead
208-
of Buildkite. Used to verify the JWT sent by the Buildkite agents to the
209-
server. This should only be required for server testing, as agents will only
210-
create a token using the Buildkite key.
211-
212-
**Buildkite API**
213-
214-
- `BUILDKITE_API_TOKEN` (**required**): The API token created for pipeline
215-
metadata lookups. **Store securely and provide to the container securely.**
216-
217-
**GitHub API connectivity**
218-
219-
- `GITHUB_APP_PRIVATE_KEY` (**required**): The PEM formatted private key of the
220-
created Github app. **Store securely and provide to the container securely.**
221-
This is a highly sensitive credential.
222-
- `GITHUB_APP_ID` (**required**): The application ID of the Github application
223-
created above.
224-
- `GITHUB_APP_INSTALLATION_ID` (**required**): The installation ID of the
225-
created Github application into your organization.
31+
[docs]: https://chinmina.github.io
32+
[docs-intro]: https://chinmina.github.io/introduction/
33+
[docs-started]: https://chinmina.github.io/guides/getting-started/
34+
[docs-config]: https://chinmina.github.io/reference/configuration/
22635

22736
## Contributing
22837

229-
Contributions are welcome.
230-
231-
- `direnv` is the tool for setting up the test environment
232-
- some variant of docker compose makes it easier to run locally
233-
- Run `make keygen` to create test keys
234-
- Execute `git` commands in the `.development/keys` directory. This has git
235-
configuration set up so it uses a local credential helper that will use the
236-
keys in the `.development/keys` directory.
38+
This project welcomes contributions! Take a look at the outstanding issues for
39+
something to dip your toes into, open an issue to get some input, or raise a PR
40+
if you're confident.
23741

23842
## License
23943

docs/chinmina-high-level.png

355 KB
Loading

0 commit comments

Comments
 (0)