From 66f4183dd83c77ffc22535162faf88643012571f Mon Sep 17 00:00:00 2001 From: John Boyes Date: Thu, 13 Aug 2020 14:50:49 +0700 Subject: [PATCH] Improve dependency updating process documentation (#49) --- .github/DEPENDENCIES.md | 48 +++++++++++++++++++++++++++ .github/workflows/dependabot_hack.yml | 1 + CONTRIBUTING.md | 24 +------------- README.md | 5 +++ 4 files changed, 55 insertions(+), 23 deletions(-) create mode 100644 .github/DEPENDENCIES.md diff --git a/.github/DEPENDENCIES.md b/.github/DEPENDENCIES.md new file mode 100644 index 0000000..a3eaeba --- /dev/null +++ b/.github/DEPENDENCIES.md @@ -0,0 +1,48 @@ +# Updating dependencies + +## Dependabot + +We use [GitHub Dependabot](https://docs.github.com/en/github/administering-a-repository/keeping-your-dependencies-updated-automatically) +([bought by GitHub in 2019](https://dependabot.com/blog/hello-github/) and now +[baked into GitHub](https://github.blog/2020-06-01-keep-all-your-packages-up-to-date-with-dependabot/)) +to manage our dependencies. + +Whenever possible we let Dependabot update our dependencies automatically (by +[automatically creating a PR](https://docs.github.com/en/github/administering-a-repository/managing-pull-requests-for-dependency-updates#about-github-dependabot-pull-requests) +for us to merge). + +Dependabot will +[automatically update non-Docker dependencies in our GitHub Actions](https://github.blog/2020-06-25-dependabot-now-updates-your-actions-workflows/). + +### Workaround for other dependencies + +For our other dependencies which cannot be updated automatically by Dependabot, we employ a bit of a hack. +We have a [`dependabot_hack.yml`](workflows/dependabot_hack.yml) GitHub Action which triggers a Dependabot PR when these other dependencies have a new version to update to. This GitHub Action is set to never actually run; it exists just so that Dependabot can do its thing. The `dependabot_hack.yml` documents where in our codebase that we then need to **update to the new version manually** (we then **add this manual update as another commit to the PR that Dependabot creates**). NB we are able to use this hack to **manage _any_ dependency that uses +[GitHub releases](https://docs.github.com/en/github/administering-a-repository/about-releases)** - we are not limited to just dependencies which are themselves GitHub Actions (this is because Dependabot doesn't care +whether the dependencies are valid GitHub Actions, it just parses the file and updates any versions that are +managed through GitHub releases). + +We could in theory automate this entirely (by e.g. having a GitHub Action that is triggered by Dependabot PRs, +which updates the version in the requisite files and then adds the change in a new commit to the Dependabot PR), +but that would be overkill for now. + +Eventually as Dependabot adds more features we may be able to remove this workaround. + + +## Dockerfile dependencies + +We have [pinned the linux dependencies in the devcontainer Dockerfile](https://github.com/agilepathway/hoverfly-github-action/pull/46/files), but there is no mechanism to automatically update them, currently. It looks like [it's on Dependabot's roadmap](https://github.com/dependabot/dependabot-core/issues/2129#issuecomment-511552345), so we have reminders every 6 months to +1. update the dependencies manually +2. see if Dependabot now offer this functionality + +### Updating the Dockerfile dependencies manually + +1. Temporarily unpin the versions (i.e. remove `=` from each package in the Dockerfile) +2. Execute the Dockerfile (e.g. if it's a remote container Dockerfile build the remote container) +3. Run `apt-cache policy ` for each package, to see the version installed +4. Pin all the versions, replacing any old versions with new ones + + +## Ubuntu version + +[Ubuntu releases annually in April](https://wiki.ubuntu.com/Releases). In 2020 the GitHub Actions team [supported the new version by mid June](https://github.com/actions/virtual-environments/issues/228#issuecomment-644065532), so we have an annual reminder for 1st July each year to see if we can upgrade. We can find out [here](https://docs.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#supported-runners-and-hardware-resources). When we do upgrade to the new version it involves e.g. for 2021, simply replacing every case of `ubuntu-20.04` with `ubuntu-21.04`. diff --git a/.github/workflows/dependabot_hack.yml b/.github/workflows/dependabot_hack.yml index 7e7ab6d..f230145 100644 --- a/.github/workflows/dependabot_hack.yml +++ b/.github/workflows/dependabot_hack.yml @@ -1,4 +1,5 @@ --- +# See ../DEPENDENCIES.md#workaround-for-other-dependencies name: Dependabot hack on: # yamllint disable-line rule:truthy push: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 90614a7..1275ae0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -24,30 +24,8 @@ The [tests](.github/workflows/tests.yml) are [end-to-end black box tests](http:/ There is no need for a separate language for the tests - as we are running the actual GitHub Action we are able to use the [GitHub Action workflow syntax](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions), which gives us what we need (e.g. [expressions](https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#about-contexts-and-expressions)) to write clean tests. - ## Updating dependencies -We use [GitHub Dependabot](https://docs.github.com/en/github/administering-a-repository/keeping-your-dependencies-updated-automatically) -([bought by GitHub in 2019](https://dependabot.com/blog/hello-github/) and now -[baked into GitHub](https://github.blog/2020-06-01-keep-all-your-packages-up-to-date-with-dependabot/)) -to manage our dependencies. - -Whenever possible, we let Dependabot update our dependencies automatically (by -[automatically creating a PR](https://docs.github.com/en/github/administering-a-repository/managing-pull-requests-for-dependency-updates#about-github-dependabot-pull-requests) -for us to merge). - -Dependabot will -[automatically update non-Docker dependencies in our GitHub Actions](https://github.blog/2020-06-25-dependabot-now-updates-your-actions-workflows/). - -For our other dependencies which cannot be updated automatically by Dependabot, we employ a bit of a hack. -We have a [`dependabot_hack.yml`](.github/workflows/dependabot_hack.yml) GitHub Action which triggers a Dependabot PR when these other dependencies have a new version to update to. This GitHub Action is set to never actually run; it exists just so that Dependabot can do its thing. The `dependabot_hack.yml` documents where in our codebase that we then need to **update to the new version manually** (we then **add this manual update as another commit to the PR that Dependabot creates**). NB we are able to use this hack to **manage _any_ dependency that uses -[GitHub releases](https://docs.github.com/en/github/administering-a-repository/about-releases)** - we are not limited to just dependencies which are themselves GitHub Actions (this is because Dependabot doesn't care -whether the dependencies are valid GitHub Actions, it just parses the file and updates any versions that are -managed through GitHub releases). - -We could in theory automate this entirely (by e.g. having a GitHub Action that is triggered by Dependabot PRs, -which updates the version in the requisite files and then adds the change in a new commit to the Dependabot PR), -but that would be overkill for now. +See the [DEPENDENCIES.md](.github/DEPENDENCIES.md) -Eventually as Dependabot adds more features we may be able to remove this workaround. diff --git a/README.md b/README.md index 7a7ed00..0fd3d8a 100644 --- a/README.md +++ b/README.md @@ -141,3 +141,8 @@ The project is [open source](https://opensource.guide/how-to-contribute/) and al * [How to report a bug or suggest a new feature](CONTRIBUTING.md#how-to-report-a-bug-or-suggest-a-new-feature) * [How to make a contribution](CONTRIBUTING.md#how-to-make-a-contribution) + + +## Updating dependencies + +See the [DEPENDENCIES.md](.github/DEPENDENCIES.md)