New release procedure

[diegodlh]

A series of commits have been made to support a new release procedure based on release-it which greatly simplifies the process of publishing new releases.

As I try to understand how this works, I wrote some comments and questions. @Dominic-DallOsto, I would appreciate your feedback!

commit 76a0a59

Changes the release script from zotero-plugin release to release-it

• why don't we use zotero-plugin release anymore?
• we already had release-it settings in our package.json, from when it was used by zotero-plugin-template/zotero-plugin-scaffold
  • zotero-plugin-template stopped having release-it settings in their package.json since they started using zotero-plugin-scaffold in windingwind/zotero-plugin-template@0d76086
  • as mentioned in Use zotero-plugin-scaffold? #272 (comment), zotero-plugin-scaffold "doesn't use release-it now, it uses bumpp (...) scaffold then implements the ability to create GitHub releases itself."
    • release-it had been removed from zotero-plugin-scaffold in v0.0.14.
• release-it has been configured so that build/update*.json (created by npm run build) is included in the Github release as an asset (see related discussion below).

commit 0b5d690

Adds a release job to the release workflow:

• this workflow already includes a test job. Both jobs are run on push (only that the release job is only run if the push event is for a tag starting with v). Maybe consider changing the name of the workflow to push?

commit e9dc2e1

updates README with new release instructions:

1. npm run lint to make sure the code is formatted properly

• npm run lint seems to be run automatically by the before:init hook when npm run release is run (see our release-it settings in package.json). Therefore, I think we could remove step 1 above. Also, would it make sense that we run lint before every commit too?

2. Decide a new version number vX.Y.Z using Semantic Versioning, and depending on whether it's a beta release or not.

• What about having release-it choose the version number automatically? We would just need to tell release-it whether to do a patch, minor or major version increment, and whether it's a pre-release or not (see Manage pre-releases) and let release-it take care of the rest. Right?

3. npm run release and choose this version number and the version will be automatically bumped in package.json and the release tagged. This will trigger github actions to build the code and make a new release at this version

• Run as npm run release <patch|minor|major> [preRelease=<alpha|beta|rc>]
• I think it would be useful to describe what happens when we run npm run release:
  • we are using release-it
  • lint is run automatically
  • version is bump
  • we Git commit, tag and push
  • npm run release is run again by Github actions
    • npm run build is run and update.json and/or update-beta.json are created; stable releases include both update.json and update-beta.json, both pointing to the same stable release, so beta testers also get the stable release when published (see https://github.com/windingwind/zotero-plugin-template/tree/main#about-prerelease)
    • a new release (or pre-release) is published
  • anything else?
• As far as I could understand, release-it is run both locally and on the cloud (as a workflow job; see above), with different configurations. When run locally, the version is bumped but a Github release is not created. Instead, when run on the cloud, a Github release is created (making sure that the version is not bumped again).
  • What are the benefits of having this release job running on the cloud?

4. Until integration tests have been implemented (Implement automatic tests #30), install the new version on a fresh Zotero profile and run some manual tests.

• Tests (whether manual or automatic) should be run first, before publishing a new release.

---

• As final step, we may add that a list of most important changes from the previous version should be manually added to the release that was just published automatically.
• As step 1, before anything else, do we want to remove node_modules/, run npm install and make sure any changes made to package-lock.json are committed (if appropriate)?

commit 923212c

Fixes the updateURL link:

• So this is wrong in zotero-plugin-template? Shall we report?
• But pre-releases are not latest, so their update-beta.json asset is unreachable at https://github.com/{{owner}}/{{repo}}/releases/latest/download/update-beta.json, right? Does this mean that beta-testers do not get new beta updates automatically? Contrary to what it says in https://github.com/windingwind/zotero-plugin-template/tree/main#about-prerelease

[Dominic-DallOsto]

To roughly answer:

why don't we use zotero-plugin release anymore?

• I was using release-it for another plugin and I found it easy to configure, so that's why I kept using it here. zotero-plugin release or bumpp would probably work the same.

this workflow already includes a test job. Both jobs are run on push (only that the release job is only run if the push event is for a tag starting with v). Maybe consider changing the name of the workflow to push?

• Yeah that's right. We test every push, but only release those with a version tag. Probably the workflow as a whole could have a better name than "release", for sure

npm run lint seems to be run automatically by the before:init hook when npm run release is run (see our release-it settings in package.json). Therefore, I think we could remove step 1 above. Also, would it make sense that we run lint before every commit too?

• Yeah, npm run release automatically runs npm run lint - it's just a bit annoying because it aborts the release if the linting actually changes something because then the working tree isn't clean when trying to do the release, so that's why I linted beforehand. But maybe there's a way to configure this better. Or yeah, we could also lint on every commit then we wouldn't have that problem

What about having release-it choose the version number automatically? We would just need to tell release-it whether to do a patch, minor or major version increment, and whether it's a pre-release or not (see Manage pre-releases) and let release-it take care of the rest. Right?

• Actually with deciding the version number, that's exactly what happens. release-it just asks whether this is a major, minor, or patch release, and whether or not it's a beta, then automatically chooses the version number based on this

I think it would be useful to describe what happens when we run npm run release:
(...)
anything else?

• I think you've summarised the release process well, yeah. One extra step it does it automatically reply to all issues and PRs that were solved by a particular release, to let people know there's now a release that solves their problem (eg. Updating a status name updates the status from/to dropdowns, but not the new label dropdown Dominic-DallOsto/zotero-reading-list#57)

What are the benefits of having this release job running on the cloud?

• I think the benefits of running the release in the cloud are that the build environment is repeatable - it does matter whether you or I build the plugin. And that it needs less configuration, otherwise I think we'd need to setup a token locally allowing for programmatically adding a release. I haven't checked this though. But I mainly did it like this because that's how it worked out of the box.

Tests (whether manual or automatic) should be run first, before publishing a new release.

• Yeah, now we at least have build tests, but the workflow would be: you push all the commits you want, test the release, then run npm run release

As final step, we may add that a list of most important changes from the previous version should be manually added to the release that was just published automatically.

• Yes! Adding a nice changelog would be great to do.

As step 1, before anything else, do we want to remove node_modules/, run npm install and make sure any changes made to package-lock.json are committed (if appropriate)?

• I think because we push package-lock to the repo, the same package versions we're using will be used by the build process in the cloud. Things only change if we manually decide to update our packages for some reason, which I think is OK?

So this is wrong in zotero-plugin-template? Shall we report?

• Yes actually, I didn't think to report it!

But pre-releases are not latest, so their update-beta.json asset is unreachable at https://github.com/{{owner}}/{{repo}}/releases/latest/download/update-beta.json, right? Does this mean that beta-testers do not get new beta updates automatically? Contrary to what it says in https://github.com/windingwind/zotero-plugin-template/tree/main#about-prerelease

• I think yeah, beta releases get stuck and need to be manually updated. Then once the next full release rolls out (eg. https://github.com/Dominic-DallOsto/zotero-reading-list/releases/tag/v1.5.1) it includes both update.json and update-beta.json so everyone is moved to this full release. Probably it'd be nicer to have beta releases be updated automatically too though true!

[Dominic-DallOsto]

• So this is wrong in zotero-plugin-template? Shall we report?

Actually, it could be that they have a different release setup where the latest update.json is always stored under a release called "release", and that's why the URL is different

[Dominic-DallOsto]

I just updated the release procedure so that the new release xpi is uploaded to a new release like before, but the update.json and update-beta.json are instead uploaded to a release called update.

This fixes the limitation that beta releases couldn't auto update because they don't show up under releases/latest

[diegodlh]

Or yeah, we could also lint on every commit then we wouldn't have that problem

Done in 7b23e68

[diegodlh]

As step 1, before anything else, do we want to remove node_modules/, run npm install and make sure any changes made to package-lock.json are committed (if appropriate)?

I think because we push package-lock to the repo, the same package versions we're using will be used by the build process in the cloud. Things only change if we manually decide to update our packages for some reason, which I think is OK?

I feel it has happened before that changes are made to package.json but not to package-lock.json, because npm install was not run before committing. Usually this is not such a big deal, because eventually someone will run npm install and commit changes to package-lock.json as well. But when we are publishing a new release, I guess we want to make sure our package-lock.json is up to date on the commit tagged with the new version. That's why I thought of reminding us to remove /node_modules and run npm install as a first step before running npm run release to make sure everything looks good.

I just updated the release procedure so that the new release xpi is uploaded to a new release like before, but the update.json and update-beta.json are instead uploaded to a release called update.

Good! I saw what you've done. However, I think the action you are using (meeDamian/[email protected]) is overwriting the whole "update" release each time. I mean, not only the assets. As a result, if it is a beta release, which only produces an update-beta.json manifest, any previous update.json asset would be lost, right?

I've been checking what zotero-plugin release does and I think I prefer we stick to using that instead of release-it. We are using zotero-plugin-template for most things and I think it may be simpler if we continue to use their release script too. I will submit a pull request about this.

[Dominic-DallOsto]

I feel it has happened before that changes are made to package.json but not to package-lock.json, because npm install was not run before committing. Usually this is not such a big deal, because eventually someone will run npm install and commit changes to package-lock.json as well. But when we are publishing a new release, I guess we want to make sure our package-lock.json is up to date on the commit tagged with the new version. That's why I thought of reminding us to remove /node_modules and run npm install as a first step before running npm run release to make sure everything looks good.

Hmm, could you explain an example? The only time I make changes to package.json is when I need a new package or a new version of a package, and I need to call npm install for this to actually have an effect which will then update package-lock.json. Maybe I'm missing something though?

Good! I saw what you've done. However, I think the action you are using (meeDamian/[email protected]) is overwriting the whole "update" release each time. I mean, not only the assets. As a result, if it is a beta release, which only produces an update-beta.json manifest, any previous update.json asset would be lost, right?

I don't think the release gets deleted and recreated if it already exists. Just the assets get updated - update-beta.json for a beta release and both update.json and update-beta.json for a production release. I think this is what we want right?