forked from rust-lang/rustup
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Document the new release process for Rustup
The Rustup release process has historically been a manual process that involved copying files from S3 to the local machine and back to S3. This introduced a high risk of human error. When modifications to the existing release script became necessary, the decision was made to automate the release process (see rust-lang#3819 for details). The documentation in the dev-guide has been updated to cover the new release process, which is fully automated to produce `beta` releases using GitHub Actions and the [promote-release] tooling. [promote-release]: https://github.com/rust-lang/promote-release
- Loading branch information
Showing
1 changed file
with
207 additions
and
54 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,66 +1,219 @@ | ||
| # Making a release | ||
| # Release Process | ||
|
|
||
| Before making a release, ensure that `rustup-init.sh` is behaving correctly, | ||
| This document describes the process for making a new release of Rustup. It is | ||
| split into the two sections. The first section explains the release process, | ||
| while the second section documents for maintainers how to make a release. | ||
|
|
||
| - [About the Release Process](#about-the-release-process) | ||
| - [Historic Context](#historic-context) | ||
| - [Beta and Stable Releases](#beta-and-stable-releases) | ||
| - [Release Artifacts](#release-artifacts) | ||
| - [Automation](#automation) | ||
| - [How to Cut a Release](#how-to-cut-a-release) | ||
| - [1. Test `rustup-init.sh`](#1-test-rustup-initsh) | ||
| - [2. Update Version](#2-update-version) | ||
| - [3. Update Checksum](#3-update-checksum) | ||
| - [4. Sync `master` to `stable`](#4-sync-master-to-stable) | ||
| - [5. Test the `beta` Release](#5-test-the-beta-release) | ||
| - [6. Prepare Blog Post for `stable` Release](#6-prepare-blog-post-for-stable-release) | ||
| - [7. Request the `stable` Release](#7-request-the-stable-release) | ||
| - [8. Tag the `stable` Release](#8-tag-the-stable-release) | ||
|
|
||
| ## About the Release Process | ||
|
|
||
| This section explains the release process for Rustup. | ||
|
|
||
| ### Historic Context | ||
|
|
||
| The Rustup release process has evolved over time. In the past, the process | ||
| involved more manual steps and large parts of it were executed locally. This | ||
| required careful execution and coordination to avoid accidentally overwriting | ||
| exiting releases, since the tooling did not implement sanity checks that would | ||
| prevent destructive actions. | ||
|
|
||
| A full step-by-step guide for the old process can be found in | ||
| the [old developer guide](https://github.com/rust-lang/rustup/blob/1cf1b5a6d80c978e0dcaabbce5f10b3861612425/doc/dev-guide/src/release-process.md). | ||
| The following summarizes how release artifacts were created and copied around. | ||
|
|
||
| In the past, both beta and stable releases were started by merging a new commit | ||
| into the `stable` branch in [rust-lang/rustup]. This started a new build on | ||
| GitHub Actions that produced release | ||
| artifacts ([source](https://github.com/rust-lang/rustup/blob/1cf1b5a6d80c978e0dcaabbce5f10b3861612425/.github/workflows/ci.yaml#L144-L151)), | ||
| which were uploaded to `s3://dev-static-rust-lang-org/rustup/dist`. As new | ||
| commits were merged into `stable`, they would overwrite the artifacts from prior | ||
| builds. | ||
|
|
||
| The release artifacts were then copied to the final location by running a script | ||
| on a local machine. This script would download the artifacts from the | ||
| `dev-static` bucket on S3 and upload them to the `static` bucket. The script | ||
| also generated a new manifest with the given version, and uploaded a copy of the | ||
| files to an archive directory. | ||
|
|
||
| Given that the process was manual, involved copying files to the local machine, | ||
| and waiting between creating a `beta` and a `stable` release, there was a risk | ||
| of human error. When we had to update the script after four years without a | ||
| change, we [decided](https://github.com/rust-lang/rustup/pull/3819) to redesign | ||
| and automate the release process. | ||
|
|
||
| ### Beta and Stable Releases | ||
|
|
||
| Rustup can be released to two different environments: `beta` and `stable`. The | ||
| main difference between the two is that they use different values for the | ||
| `RUSTUP_UPDATE_ROOT` environment variable: | ||
|
|
||
| - A beta release is deployed on `https://dev-static.rust-lang.org/rustup`. | ||
| - An official release is deployed on `https://static.rust-lang.org/rustup`. | ||
|
|
||
| By switching between those two values, Rustup effectively provides two | ||
| "self update channels", making beta testing possible with `rustup self update`. | ||
|
|
||
| ### Release Artifacts | ||
|
|
||
| The release artifacts are produced by CI when a new commit is merged into the | ||
| `stable` branch, and they are uploaded to the `dev-static` bucket on S3. There, | ||
| they are put into a folder named after their commit hash, for example the | ||
| artifacts for commit `1cf1b5a` would be uploaded to | ||
| `s3://dev-static-rust-lang-org/rustup/builds/1cf1b5a`. | ||
|
|
||
| When a new `beta` release is cut, the artifacts are copied to two new locations | ||
| within the same bucket: | ||
|
|
||
| - One copy is put into an archive named after the version, e.g. | ||
| `/rustup/archive/1.0.0`. | ||
| - Another copy is put into the `/rustup/dist` folder, which is where clients | ||
| look for new versions. | ||
|
|
||
| When a new `stable` release is cut, the artifacts are copied to the `static` | ||
| bucket on S3, following the same process as the `beta` release: | ||
|
|
||
| - One copy is archived into `/rustup/archive/1.0.0`. | ||
| - Another copy is put into `/rustup/dist`. | ||
|
|
||
| This ensures backwards compatibility with the old release process, while also | ||
| ensuring that release artifacts are not overwritten by new builds. | ||
|
|
||
| ### Automation | ||
|
|
||
| The interaction with the release artifacts is fully automated. | ||
|
|
||
| First, artifacts are produced automatically by | ||
| the [`CI`](https://github.com/rust-lang/rustup/blob/master/.github/workflows/ci.yaml) | ||
| job on GitHub Actions when a new commit is merged into the `stable` branch, and | ||
| are then automatically uploaded to their respective S3 bucket by the action as | ||
| well. | ||
|
|
||
| Second, when making a release, the artifacts are copied to their final locations | ||
| by the [promote-release] tool. This reduces the risk of human error and ensures | ||
| that the release process is consistent and reliable. The tool is also run in a | ||
| secure environment on AWS CodeBuild, reducing the risk of leaking sensitive | ||
| credentials that would give write access (past) releases. | ||
|
|
||
| For a `beta` release, `promote-release` performs the following actions: | ||
|
|
||
| 1. Query GitHub's API to get the latest commit on the `stable` branch | ||
| 2. Confirm that `/rustup/builds/${commit}` exists in the `dev-static` bucket | ||
| 3. Get the new version number from the `stable` branch | ||
| 1. Download `Cargo.toml` from `stable` | ||
| 2. Parse the file and read the `version` field | ||
| 4. Confirm that `/rustup/archive/${version}` does not exist yet | ||
| 5. Copy `/rustup/builds/${commit}` to `/rustup/archive/${version}` | ||
| 6. Copy `/rustup/builds/${commit}` to `/rustup/dist` | ||
| 7. Generate a new manifest and upload it to `/rustup/dist` | ||
|
|
||
| For a new `stable` release, the process is the same. The only difference is that | ||
| the steps 4-6 copy the artifacts to the `static` bucket. | ||
|
|
||
| ## How to Cut a Release | ||
|
|
||
| This section documents the steps that a maintainer should follow to cut a new | ||
| release of Rustup. | ||
|
|
||
| ### 1. Test `rustup-init.sh` | ||
|
|
||
| Before cutting a release, ensure that `rustup-init.sh` is behaving correctly, | ||
| and that you're satisfied that nothing in the ecosystem is breaking because | ||
| of the update. A useful set of things to check includes verifying that | ||
| real-world toolchains install okay, and that `rust-analyzer` isn't broken by | ||
| the release. While it's not our responsibility if they depend on non-stable | ||
| APIs, we should behave well if we can. | ||
|
|
||
| As a maintainer, you have two options to choose from when cutting a new | ||
| release: a beta release or an official release. | ||
| The main difference between the two is that they use different values for | ||
| the `RUSTUP_UPDATE_ROOT` environment variable: | ||
| - A beta release is deployed on `https://dev-static.rust-lang.org/rustup`. | ||
| - An official release is deployed on `https://static.rust-lang.org/rustup`. | ||
| ### 2. Update Version | ||
|
|
||
| By switching between those two values, Rustup effectively provides two "self | ||
| update channels", making beta testing possible with `rustup self update`. | ||
|
|
||
| Producing the final release artifacts is a bit involved because of the way | ||
| Rustup is distributed. | ||
| Below is a list of things to be done in order to cut a new [b]eta release | ||
| or an official [r]elease: | ||
|
|
||
| 1. [b/r] In a separate PR: | ||
| 1. If the version strings in `Cargo.toml`s haven't been updated: | ||
| - Decide what the new version number `$VER_NUM` should be. | ||
| > **Note:** We always increment the *minor* number unless: | ||
| > - A major incompatibility has been introduced in this release: | ||
| > increment the *major* number instead. | ||
| > - This release is a hotfix because the last one had a defect: | ||
| > increment the *patch* number instead. | ||
| - Update `Cargo.toml` and `download/Cargo.toml` to have that same new | ||
| version number, then run `cargo build` and review `Cargo.lock` changes. | ||
| The release process gets metadata from the `Cargo.toml` file, so ensure that | ||
| the version number in `Cargo.toml` is correct. | ||
|
|
||
| 1. If the version strings in `Cargo.toml`s haven't been updated: | ||
| - Decide what the new version number `$VER_NUM` should be. | ||
| > **Note:** We always increment the *minor* number unless: | ||
| > - A major incompatibility has been introduced in this release: | ||
| > increment the *major* number instead. | ||
| > - This release is a hotfix because the last one had a defect: | ||
| > increment the *patch* number instead. | ||
| - Update `Cargo.toml` and `download/Cargo.toml` to have that same new | ||
| version number, then run `cargo build` and review `Cargo.lock` changes. | ||
| If all looks well, make a commit. | ||
| 2. Update `CHANGELOG.md` accordingly if necessary. | ||
| 2. [b/r] After merging the PR made in step 1, in a separate PR: | ||
| 1. Update the commit shasum in `rustup-init.sh` to match the latest commit | ||
| on `master`. | ||
| 3. [b/r] After merging the PR made in step 2, sync `master` to `stable` using | ||
| `--ff-only`: | ||
| - `git fetch origin master:master` | ||
| - `git checkout stable && git merge --ff-only master` | ||
| - `git push origin HEAD:stable` | ||
| 4. [b/r] While you wait for green CI on `stable`, double-check the | ||
| functionality of `rustup-init.sh` and `rustup-init` just in case. | ||
| 5. [b/r] Ensure all of CI is green on the `stable` branch. | ||
| Once it is, check through a representative proportion of the builds looking | ||
| for the reported version statements to ensure that | ||
| we definitely built something cleanly which reports as the right version | ||
| number when run `--version`. | ||
| 6. [r] Make a new PR to the [Rust Blog] adding a new release announcement post. | ||
| 7. [b/r] Ping someone in the release team to perform the actual release. | ||
| They can find instructions in `ci/sync-dist.py`. | ||
| > **Note:** Some manual testing occurs here, so hopefully they'll catch | ||
| anything egregious in which case abort the change and roll back. | ||
| 8. [b] Once the beta release has happened, post a new topic named "Seeking beta | ||
| testers for Rustup $VER_NUM" on the [Internals Forum]. | ||
| 9. [r] Once the official release has happened, prepare and push a tag on the | ||
| latest `stable` commit. | ||
| - `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG | ||
| signing the tag) | ||
| - `git push origin $VER_NUM` | ||
| 2. Update `CHANGELOG.md` accordingly if necessary. | ||
|
|
||
| Submit the changes in a PR and merge it. | ||
|
|
||
| ### 3. Update Checksum | ||
|
|
||
| After merging the PR in the previous step, update the commit SHA checksum in | ||
| `rustup-init.sh` to match the latest commit on `master`. Submit the change in a | ||
| PR and merge it. | ||
|
|
||
| ### 4. Sync `master` to `stable` | ||
|
|
||
| After merging the PR in the previous step, sync `master` to `stable` using | ||
| `--ff-only`: | ||
|
|
||
| ```shell | ||
| git fetch origin master:master | ||
| git checkout stable && git merge --ff-only master | ||
| git push origin HEAD:stable | ||
| ``` | ||
|
|
||
| This will kick off a new build on GitHub Actions, which will produce the release | ||
| artifacts, upload them to `dev-static`, and make the new `beta` release | ||
| available at `RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup`. | ||
|
|
||
| ### 5. Test the `beta` Release | ||
|
|
||
| While you wait for green CI on `stable`, double-check the functionality of | ||
| `rustup-init.sh` and `rustup-init` just in case. | ||
|
|
||
| Ensure all of CI is green on the `stable` branch. Once it is, check through a | ||
| representative proportion of the builds looking for the reported version | ||
| statements to ensure that we definitely built something cleanly which reports as | ||
| the right version number when run `--version`. | ||
|
|
||
| Once the beta release has happened, post a new topic named "Seeking beta testers | ||
| for Rustup $VER_NUM" on the [Internals Forum]. | ||
|
|
||
| ### 6. Prepare Blog Post for `stable` Release | ||
|
|
||
| Make a new PR to the [Rust Blog] adding a new release announcement post. | ||
|
|
||
| ### 7. Request the `stable` Release | ||
|
|
||
| Ping someone in the release team to perform the `stable` release. | ||
|
|
||
| They will have to start a new CodeBuild job on AWS to run the `promote-release` | ||
| tool for Rustup. | ||
|
|
||
| ### 8. Tag the `stable` Release | ||
|
|
||
| Once the official release has happened, prepare and push a tag on the latest | ||
| `stable` commit. | ||
|
|
||
| - `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG | ||
| signing the tag) | ||
| - `git push origin $VER_NUM` | ||
|
|
||
| [promote-release]: https://github.com/rust-lang/promote-release | ||
|
|
||
| [rust-lang/rustup]:https://github.com/rust-lang/rustup | ||
|
|
||
| [Rust Blog]: https://github.com/rust-lang/blog.rust-lang.org | ||
|
|
||
| [Internals Forum]: https://internals.rust-lang.org |