Skip to content
This repository was archived by the owner on Feb 9, 2021. It is now read-only.

Prebuilt Ruby ADR #49

Open
bryanmacfarlane wants to merge 5 commits into
base: adrs
Choose a base branch
from
Open

Prebuilt Ruby ADR #49

Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
eaaf2df
start prebuilt adr
bryanmacfarlane Jan 12, 2020
09a50a9
rename
bryanmacfarlane Jan 12, 2020
c04a346
update packaging options
bryanmacfarlane Jan 12, 2020
967c2ae
build and cache option
bryanmacfarlane Jan 24, 2020
45ad9d5
tweak adr
bryanmacfarlane Jan 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions adrs/0049-prebuilt-rubies-at-runtime.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# ADR-0049 Prebuilt Rubies at Runtime

***Status***: proposed

## Context
Currently, only two versions of Ruby are available at runtime as part of the hosted build VM tools cache. The setup-ruby action only supports using what's in the cache and does not support pulling other versions at runtime. It also doesn't support pulling other versions like JRuby at runtime.

This is in contrast to other setup actions like setup-node where the cache is an optimization but supports pulling any version by a specified semver at runtime.

## Goals

- Offer as many versions as possible by platforms
- Offer popular Ruby variants like JRuby
- Consistent with other setup-xxx actions patterns and Actions workflows.

## Decisions

### Prebuilt Rubies

Other tools like offer a [distribution](https://nodejs.org/dist/) which offers a [queryable endpoint](https://nodejs.org/dist/index.json) which allows the desired version [by a version spec which is semver](https://github.com/actions/setup-node#setup-node).

Other CI services like Travis support pulling a wide variety of [ruby versions offered here](http://rubies.travis-ci.org/).

Requirements:
- Offer prebuilt Ruby, JRuby, TruffleRuby versions
- Setup-Ruby exposes `ruby-version` which accepts a semver
- Rubies are discoverable and queryable by the Setup-Ruby action.
- Initially offer for all platforms offered by the hosted Actions VMs (Ubuntu 18, Mac 10.15 and Windows 2019)

NOTE: The prebuilt rubies offered will only be supported for use by GitHub Actions.

The scripts and tooling to build the Rubies will be at `actions/build-ruby` and it will be consumed by the existing action `actions/setup-ruby`. Since the capabilities are additive, there's no need to create a `v2`. There is no compat break.

A specific version of a Ruby will be offered as an individual semi-immutable build-ruby repo release similar to how the [actions runner exposes versions](https://github.com/actions/runner/releases/tag/v2.164.0) where each version offers n platforms. This is also similar to [Travis individual versions](http://rubies.travis-ci.org/ubuntu/18.04/s390x/ruby-2.6.5).

NOTE: semi-immutable means it's desirable to be immutable but it's possible to patch. Note that

Option 1: GitHub GPR Universal Package

The `actions/build-ruby` repo offers packages for [example](https://github.com/actions/setup-ruby/packages).
Copy link
Contributor

@eregon eregon Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between packages and releases?

From https://github.com/features/packages#pricing I can see 500MB (total) could quickly become too tight for hosting all Ruby versions. Releases don't have such restrictions AFAIK (just no single file >2GB).

Copy link
Member Author

@bryanmacfarlane bryanmacfarlane Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm discussing with packaging folks. But essentially, packaging has full semver support without putting that burden on the action or some other index json or api.

I think if we went the packaging route, then each distribution, per version, per arch would be an independent package (how big is a discrete ruby? 10 - 20 MB?)

Copy link
Contributor

@eregon eregon Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/eregon/ruby-install-builder/releases/tag/builds-bundler1
So it's between 10MB and 150MB. ~35MB for Ruby on Linux.

I think Rubyists are BTW more used to 2.6 rather than 2.6.x. In use-ruby-action I just use startsWith on an ordered list of versions per engine, and it's very simple in the end:
https://github.com/eregon/use-ruby-action/blob/933f49684485836830e31e48a48835cba066dc55/index.js#L77-L79


Option 2: GitHub Release

The `actions/build-ruby` repo offers releases for [example](https://github.com/actions/runner/releases/tag/v2.164.0)

NOTE: release assets are backed by a CDN

Offering each version as an individual package / release offers queryable APIs and the ability to convey whether pre-release or not by version using the packaging and release features.

This is fairly straight forward to come up with a scheme to complete automate with a workflow.
Copy link
Contributor

@eregon eregon Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not straightforward in the case building fails, as explained in #48 (comment).
How do you plan to address that? Modify the create-release and upload-release-asset to allow using them for patching an existing release?

Copy link
Member Author

@bryanmacfarlane bryanmacfarlane Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the runner runs a graph of jobs (for each arch), each job uploads an artifact and then there's a job that depends on all those which finally aggregates all and creates the release. If any job fails the job that creates the release doesn't run.

We could also easily do another approach which is to create the release in pre-release and then each job modifies the release and then either a final job node flips it or there's another canary set of workflows that validates all E2E functionality which flips from pre-release to release (runner does that).

Copy link
Contributor

@eregon eregon Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

each job uploads an artifact and then there's a job that depends on all those which finally aggregates all and creates the release.

Uploading release assets requires the release to be created already. It could be in draft mode, but there is no premade action under github/actions to turn a draft release into a public release AFAIK.
Using download-artifact/upload-artifact workarounds that at the cost of extra complexity.

I see, https://github.com/actions/runner/blob/a727194742dd7b28c265527f71e2a3669b71516a/.github/workflows/release.yml does something like you mention.
https://github.com/actions/github-script seems a way to workaround the limitations of the create-release and upload-release-asset actions, but it's more verbose and it's JavaScript in a YAML string. It also includes the list of platforms/versions twice.

Anyway, since I guess this is a part GitHub will maintain, best to leave it to you.
Readability would still be good if the community wants to help building new versions or understand how all this works.


### Setup action

Repo releases are also queryable via an [http api](https://developer.github.com/v3/repos/releases/). This allows the `actions/setup-ruby` action to query the versions. match the semver version spec against the list and resolve the latest matching the spec.
Copy link
Contributor

@eregon eregon Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting the list of releases via https://developer.github.com/v3/repos/releases/#list-releases-for-a-repository might take some time, especially if there are many releases, the JSON responses are paginated, and therefore multiple requests are issued.

Copy link
Member Author

@bryanmacfarlane bryanmacfarlane Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, another reason to use packages. However, in practice I don't believe it will be a problem. Note that the tool-cache lib queries the cache first and the VM would populate latest versions so it short circuits and never queries even if the cache can satisfy the semver. Also, exact version specs don't need to query at all.

Copy link
Member Author

@bryanmacfarlane bryanmacfarlane Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forwarded this to our teams that gen images and also our packaging folks. There's other work going on in these areas so it would be great to drive to consensus with community ❤️


The existing action offers a `ruby-version` input which accepts a [semver pattern](https://github.com/actions/setup-ruby/blob/master/action.yml#L7) via the `toolcache.find` api. The contract will remain the same so there's no breaking compat and need to version to `V2`.

The existing capability will be extended to be consistent with the setup-node action:

1. Attempt to resolve the semver pattern against the tool-cache
2. If no match is found from tool-cache, query the releases api for the `build-ruby` repo.

Setup-node queries the cache first to add reliability in the event of distruptions, self-hosted runners and eventual GHES air gap scenarios. Testing against n versions is still a scenario for self hosted and on-premises. Of course, air gapped installations would own populating the cache by versions they want to test against. The tool cache is simply an envvar pointing to a directory with tools by name, version, arch folders.

The action will enforce the supported list of platforms (ubuntu18, osx 10.15, windows 2019) regardless of hosted or self-hosted.

## More

Not done. Just starting. Add consequences, alternatives, etc.