|
1 |
| -# Chinmina Bridge: Buildkite/Github OIDC token bridge |
| 1 | +# Chinmina Bridge |
2 | 2 |
|
3 |
| -[](https://app.fossa.com/projects/git%2Bgithub.com%2Fjamestelfer%2Fchinmina-bridge?ref=badge_shield) |
| 3 | +**Connect Buildkite to GitHub with secure, short-lived tokens.** |
4 | 4 |
|
| 5 | +Chinmina Bridge allows Buildkite agents to securely generate GitHub API tokens |
| 6 | +that can be used to perform Git or other GitHub API actions. It is intended to |
| 7 | +be an alternative to the use of SSH deploy keys or long-lived Personal Access |
| 8 | +Tokens. |
5 | 9 |
|
6 |
| -Allows Buildkite agents to get valid GitHub tokens that can be used to perform |
7 |
| -Git or other GitHub API actions. It is intended to be an alternative to the use |
8 |
| -of SSH deploy keys or long-lived Personal Access Tokens. |
| 10 | + |
9 | 11 |
|
10 | 12 | The bridge itself is an HTTP endpoint that uses a [GitHub
|
11 | 13 | application][github-app] to create [ephemeral GitHub access
|
12 | 14 | tokens][github-app-tokens]. Requests are authorized with a [Buildkite
|
13 | 15 | OIDC][buildkite-oidc] token, allowing a token to be created just for the
|
14 | 16 | repository associated with an executing pipeline.
|
15 | 17 |
|
16 |
| -The token is created with `contents:read` permissions, and only has access to |
17 |
| -the repository associated with the executing pipeline. |
18 |
| - |
19 |
| -Two endpoints are exposed: `/token`, which returns a token and its expiry, and |
20 |
| -`/git-credentials`, which returns the token and repository metadata in the [Git |
21 |
| -Credentials format][git-credential-helper]. |
| 18 | +> [!NOTE] |
| 19 | +> Find out more about Chinmina Bridge is available in the [documentation][docs]. |
| 20 | +> |
| 21 | +> This has and expanded [introduction][docs-intro], a [getting |
| 22 | +> started][docs-started] guide and a detailed [configuration |
| 23 | +> reference][docs-config]. This has a more detailed description of the |
| 24 | +> implementation, and clear guidance on how to configuration and installation. |
22 | 25 |
|
23 | 26 | [github-app]: https://docs.github.com/en/apps
|
24 | 27 | [github-app-tokens]: https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app
|
25 | 28 | [buildkite-oidc]: https://buildkite.com/docs/agent/v3/cli-oidc
|
26 | 29 | [git-credential-helper]: https://git-scm.com/docs/gitcredentials#_custom_helpers
|
27 | 30 |
|
28 |
| -## Overview |
29 |
| - |
30 |
| -`chinmina-bridge` is used by jobs running on a Buildkite agent to request tokens |
31 |
| -from Github. These can be used to communicate with the GitHub API or (via Git) |
32 |
| -to enable authenticated Git actions. |
33 |
| - |
34 |
| -Git authentication is facilitated by a [Git credential |
35 |
| -helper](https://github.com/jamestelfer/github-app-auth-buildkite-plugin), which |
36 |
| -communicates with the bridge and supplies the result to Git in the appropriate |
37 |
| -format. |
38 |
| - |
39 |
| -The following sequence illustrates a Git authentication flow facilitated by |
40 |
| -`chinmina-bridge`. |
41 |
| - |
42 |
| -```mermaid |
43 |
| -sequenceDiagram |
44 |
| - box Buildkite Agent |
45 |
| - participant Buildkite Job |
46 |
| - participant Git |
47 |
| - participant Credential Helper |
48 |
| - end |
49 |
| - box Self hosted |
50 |
| - participant Chinmina Bridge |
51 |
| - end |
52 |
| - Buildkite Job->>+Git: clone |
53 |
| - Git ->>+ Credential Helper: get credentials |
54 |
| - Credential Helper->>+Buildkite API: Request Buildkite OIDC token |
55 |
| - Buildkite API->>-Credential Helper: bk-oidc |
56 |
| - Credential Helper->>+Chinmina Bridge: Request GH token (auth bk-oidc) |
57 |
| - Chinmina Bridge->>+Buildkite API: Get Pipeline Details |
58 |
| - Buildkite API-->>-Chinmina Bridge: pipeline-repository |
59 |
| - Chinmina Bridge->>+GitHub: Create Token (auth app JWT) |
60 |
| - GitHub-->>-Chinmina Bridge: app-token |
61 |
| - Chinmina Bridge->>-Credential Helper: bk-oidc |
62 |
| - Credential Helper->>-Git: "x-access-token"/app-token |
63 |
| - Git-->>-Buildkite Job: complete |
64 |
| -``` |
65 |
| - |
66 |
| -## Why? |
67 |
| - |
68 |
| -There are two options generally used to authenticate Buildkite agents to GitHub: |
69 |
| - |
70 |
| -1. Via a PAT (owned by a GitHub user) that is saved in the agent S3 secrets bucket |
71 |
| -2. Via a deploy key (registered to a single repository) that is likewise saved to |
72 |
| - S3. |
73 |
| - |
74 |
| -As the organization scales however, the overhead of managing them becomes |
75 |
| -unwieldy, and it can be quite difficult for an organisation to successfully |
76 |
| -manage a rotation scheme. |
77 |
| - |
78 |
| -Unless centralized issuance is practiced as well, both of these schemes can |
79 |
| -produce tokens that are tied to a user, leading to unexpected problems when a |
80 |
| -user leaves the organization. There is also the potential for key material to be |
81 |
| -stored or shared incorrectly, leading to increased possibility of accidental |
82 |
| -leakage. |
83 |
| - |
84 |
| -Lastly, all key material is typically stored in an S3 bucket. This is |
85 |
| -straightforward to configure and maintain, but creates a significant issue in |
86 |
| -the event of an account/bucket breach. |
87 |
| - |
88 |
| -Using a GitHub application to authenticate GitHub actions allows: |
89 |
| - |
90 |
| -1. Access keys for repositories are generated on demand and expire after one |
91 |
| - hour. |
92 |
| -1. The generated tokens are only kept by a build agent for the duration of the |
93 |
| - step, and do not require any other persistence. |
94 |
| -1. The private key for the GitHub application is specific to the |
95 |
| - `chinmina-bridge` service. It can (and should) be rotated, an operation that |
96 |
| - is easy to perform. |
97 |
| -1. Supplied tokens are scoped to just the repositories and actions necessary for |
98 |
| - the requesting pipeline. |
99 |
| -1. Additional Buildkite configuration per repository is not required. If the |
100 |
| - application has access, the agent can request a token for it. No need to |
101 |
| - create PATs or generate keypairs, and no need to upload them in multiple |
102 |
| - places. This allows the an organization to have tighter access control on |
103 |
| - pipeline setup without creating additional support overhead. |
104 |
| -1. Tokens can enable a wider set of actions than simple Git operations (e.g. PR |
105 |
| - comments). This is not yet implemented in `chinmina-bridge`, but is a high |
106 |
| - priority for future enhancement. |
107 |
| - |
108 |
| -Also, since `chinmina-bridge` uses Buildkite's OIDC tokens to authorize requests, |
109 |
| -the claims associated with the token can be used to further refine access to a token. |
110 |
| - |
111 |
| -Github has some [good documentation][gh-deploy-keys] about the pros and cons of |
112 |
| -the application token approach. There are two primary downsides documented: |
113 |
| - |
114 |
| -> - Additional setup is needed to create the GitHub App. |
115 |
| -> - Installation access tokens expire after 1 hour, and so need to be |
116 |
| -> re-generated, typically on-demand using code. |
117 |
| -
|
118 |
| -`chinmina-bridge` solves the second problem, by making token generation for a |
119 |
| -pipeline at build time trivial. |
120 |
| - |
121 |
| - |
122 |
| -[gh-deploy-keys]: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/managing-deploy-keys#github-app-installation-access-tokens |
123 |
| - |
124 |
| -### What's right for your organization? |
125 |
| - |
126 |
| -To understand what's right for your organization, consider: |
127 |
| - |
128 |
| -- how many pipelines do you have? (That is, how many keys are managed?) |
129 |
| -- how easily are tokens rotated? |
130 |
| -- (related) if the secrets bucket is somehow compromised, how difficult would it be for the organization to respond? |
131 |
| -- if tokens are issued to a user, does a person leaving cause an outage in a build pipeline? |
132 |
| -- what processes/restrictions does your organization have around repository access in GitHub and pipeline creation in Buildkite? |
133 |
| - |
134 |
| -## Limitations |
135 |
| - |
136 |
| -- can only grant `contents:read` access |
137 |
| -- will only grant access to the repository associated with a pipeline |
138 |
| -- if the buildkite user has permissions to modify the pipeline repository, they |
139 |
| - may configure a repository that they don't have access to in GitHub (but is |
140 |
| - accessible in the app). This would allow them to potentially extract code via |
141 |
| - use of the pipeline step configuration. **BUT**: |
142 |
| - - it's OK if your organization members have read access to the same set of |
143 |
| - repositories covered by the `chinmina-bridge` GitHub application.<br> |
144 |
| - **OR** |
145 |
| - - it's OK if your organization controls the creation/configuration of |
146 |
| - pipelines: this restricts the opportunity to misconfigure a pipeline. |
147 |
| - |
148 |
| -## Operations |
149 |
| - |
150 |
| -See the [observability documentation](./docs/observability.md) for more details |
151 |
| -on the information provided by the system when running. |
152 |
| - |
153 |
| -## Configuration |
154 |
| - |
155 |
| -Requirements: |
156 |
| - |
157 |
| -1. A Buildkite organization, and a user with sufficient access to create an API |
158 |
| - token that can be used to get the details of any pipeline that is expected to |
159 |
| - be built. |
160 |
| -1. A Github organization, and a user with sufficient permissions to create a |
161 |
| - Github App and install it into the organization. |
162 |
| -1. Ability to deploy a server that can be accessed by the build agents (for example, an ECS service) |
163 |
| -1. Ability to allow Buildkite agents to download and use a custom plugin _or_ |
164 |
| - ability to add a plugin to the default settings of the Buildkite agents. |
165 |
| - |
166 |
| -### Buildkite setup |
167 |
| - |
168 |
| -Create an API key with access to the REST API **only** with access to the `read_pipelines` scope. |
169 |
| - |
170 |
| -Save the key securely: it will be provided to the server in a later step. Use a |
171 |
| -"bot" user to create the token if you can. |
172 |
| - |
173 |
| -### Github setup |
174 |
| - |
175 |
| -1. Create an application in your Github organization |
176 |
| - - The application must request `contents:read` |
177 |
| - - Note the application ID |
178 |
| - - Create and save a private key for the application |
179 |
| -2. Install the application into the Github organization |
180 |
| - - choose the repositories the application will have access to. This is the |
181 |
| - limit of the resources that the application can vend tokens for. |
182 |
| - |
183 |
| -### Configure and deploy the bridge server |
184 |
| - |
185 |
| -The server is a Go application expecting to read configuration from environment |
186 |
| -variables, and can be deployed to a server or as a container. |
187 |
| - |
188 |
| -#### Variables |
189 |
| - |
190 |
| -**Server** |
191 |
| - |
192 |
| -- `SERVER_PORT` (optional, default `8080`): the TCP port the server will listen on. |
193 |
| -- `SERVER_SHUTDOWN_TIMEOUT_SECS` (optional, default `25`): the number of seconds |
194 |
| - the server will wait when asked to terminate with `SIGINT` |
195 |
| - |
196 |
| -**Authorization** |
197 |
| - |
198 |
| -- `JWT_BUILDKITE_ORGANIZATION_SLUG` (**required**): the slug of your Buildkite |
199 |
| - organization. This is the identifier of your organization that appears in your |
200 |
| - Buildkite URLs. |
201 |
| -- `JWT_AUDIENCE` (optional, default=`app-token-issuer`): The expected value of the |
202 |
| - `aud` claim in the JWT. Describes the intended audience of the issued JWT |
203 |
| - token, guards against token reuse. Using a non-default value will require configuration of the credentials helper plugin. |
204 |
| -- `JWT_ISSUER_URL` (optional, default `https://agent.buildkite.com`): the |
205 |
| - expected value of the `iss` claim in the agent JWT. Also used to discover the |
206 |
| - JWKS configuration from the `.well-known` address. |
207 |
| -- `JWT_JWKS_STATIC` (optional): a local JWKS JSON file that can be used instead |
208 |
| - of Buildkite. Used to verify the JWT sent by the Buildkite agents to the |
209 |
| - server. This should only be required for server testing, as agents will only |
210 |
| - create a token using the Buildkite key. |
211 |
| - |
212 |
| -**Buildkite API** |
213 |
| - |
214 |
| -- `BUILDKITE_API_TOKEN` (**required**): The API token created for pipeline |
215 |
| - metadata lookups. **Store securely and provide to the container securely.** |
216 |
| - |
217 |
| -**GitHub API connectivity** |
218 |
| - |
219 |
| -- `GITHUB_APP_PRIVATE_KEY` (**required**): The PEM formatted private key of the |
220 |
| - created Github app. **Store securely and provide to the container securely.** |
221 |
| - This is a highly sensitive credential. |
222 |
| -- `GITHUB_APP_ID` (**required**): The application ID of the Github application |
223 |
| - created above. |
224 |
| -- `GITHUB_APP_INSTALLATION_ID` (**required**): The installation ID of the |
225 |
| - created Github application into your organization. |
| 31 | +[docs]: https://chinmina.github.io |
| 32 | +[docs-intro]: https://chinmina.github.io/introduction/ |
| 33 | +[docs-started]: https://chinmina.github.io/guides/getting-started/ |
| 34 | +[docs-config]: https://chinmina.github.io/reference/configuration/ |
226 | 35 |
|
227 | 36 | ## Contributing
|
228 | 37 |
|
229 |
| -Contributions are welcome. |
230 |
| - |
231 |
| -- `direnv` is the tool for setting up the test environment |
232 |
| -- some variant of docker compose makes it easier to run locally |
233 |
| -- Run `make keygen` to create test keys |
234 |
| -- Execute `git` commands in the `.development/keys` directory. This has git |
235 |
| - configuration set up so it uses a local credential helper that will use the |
236 |
| - keys in the `.development/keys` directory. |
| 38 | +This project welcomes contributions! Take a look at the outstanding issues for |
| 39 | +something to dip your toes into, open an issue to get some input, or raise a PR |
| 40 | +if you're confident. |
237 | 41 |
|
238 | 42 | ## License
|
239 | 43 |
|
|
0 commit comments