Does anyone have recommendations for best practices for release processes?

[dopry]

Currently, it seems like a lot of folks are doing releases manually in the python ecosystem. As a maintainer I feel like this is a lot of overhead. On most of my node.js and commercial projects I use tools like semantic release to automate versioning and changelog generation for my releases. Shifting responsibility for releases to CI/CD allows me to focus my energy on code review, testing, and community engagement. Are there any recommended tools/workflows or good example repos that do CI/CD right in the python ecosystem? Would documenting best practice for various release process (manual, tooling supported, fully automated) be something useful to the wider django commons community?

[tim-schilling]

Are there any recommended tools/workflows or good example repos that do CI/CD right in the python ecosystem?

I think "right" is subjective and depends on what facets you're trying to optimize for. There's always trade-offs. I like looking at what Adam Johnson does for CI. Regarding CD, I've been leaning towards updating the release docs and version manually, creating the git tag manually, then letting the release process handle releasing to PyPI and generating the GitHub release along with the notes.

Would documenting best practice for various release process (manual, tooling supported, fully automated) be something useful to the wider django commons community?

Yes, it would be. We've got some started here: https://github.com/django-commons/best-practices/blob/main/docs/releases.md

[browniebroke]

On most of my node.js and commercial projects I use tools like semantic release to automate versioning and changelog generation for my releases.

For my smaller OSS projects, I use Python Semantic Release which aims at replicating some of that experience for Python projects. You can tweak the level of changes to mention in your changelog and customise the template.

It works well at the level of my smaller projects, but you need to be conscious to have your commit message describe well enough the change, and sometimes it feel quite limited.

[bckohan]

Release security is not academic. I've had private stuff pilfered thanks to sloppy CD in widely trusted build tool dependencies. There are so many things you can do to tighten up CI/CD (zizmor, etc) - but I think the only hard and fast rule is to make sure a 2FA step is in front of any release process. You can get this with PyPI by enabling 2FA and using trusted releases with your GHA pipeline and requiring a review from trusted github accounts. It's also important to make sure the scope of any github CLI token is such that it can't trigger a release from start to finish. My releases are as secure as my browser, which is pretty secure, but I would prefer it if Github allowed me to configure the review approvals to require a 2FA challenge to press the button.

I've been interested in semantic release automation but I don't manage enough popular libraries to feel the need to automate to that degree yet. I also don't use build plugins that pull the package version from source - preferring to enter it thrice as a triple check that I'm issuing the version I intend. I have a just command called release that accepts a version, verifies that the pyproject.toml and source versions match and match the version passed in on the command line, then pushes a tag of that name which triggers the release workflow. The release workflow first runs all the CI and if all that passes it will enter the PyPi publish job which waits for me to login to the github UI and hit approve. I have also added a check in my release action that makes sure the git tag is cryptographically verified.

So my release process is this:

• Update pyproject.toml version
• Update source version
• Run: just release x.x.x
• Go get a coffee or do other work and wait for CI to finish
• Login to github and approve the publish to PyPi

I use this process in my two django-commons repos: django-typer and django-enum