Skip to content

When publishing multiple packages, twine should upload entire packages, not sort by file type. #1254

@pnacht

Description

@pnacht

Is there an existing issue for this?

  • I have searched the existing issues (open and closed), and could not find an existing issue

What keywords did you use to search existing issues?

sort
sorted
alphabetical

Please describe the problem you are attempting to solve with this request

When I publish multiple packages with the same call to twine, the files get sorted into two groups: first all the .whl's get published, then all the .tar.gz source balls:

$ twine upload dist/* --repository testpypi --verbose
Uploading distributions to https://test.pypi.org/legacy/
INFO     dist/bar-0.0.2-py3-none-any.whl (8.4 KB)
INFO     dist/buzz-0.0.3-py3-none-any.whl (8.4 KB)
INFO     dist/foo-0.0.1-py3-none-any.whl (8.4 KB)
INFO     dist/bar-0.0.2.tar.gz (7.7 KB)
INFO     dist/buzz-0.0.3.tar.gz (7.7 KB)
INFO     dist/foo-0.0.1.tar.gz (7.7 KB)

Assuming the release is successful, this works just fine.

But if foo-0.0.1-py3-none-any.whl fails for some reason (maybe I forgot to configure Trusted Publishing for it), then my other packages (bar and buzz) will end up in a "half-published" state, with only their .whl published, but not their source tarball.

How do you think we should solve this?

Twine should attempt to publish each package fully before moving to the next one.

This can be done by sorting the list of artifacts alphabetically, leveraging the fact that both .whls and .tar.gzs start with $pkg-$version. This way, the list of artifacts to upload becomes:

$ twine upload dist/* --repository testpypi --verbose
Uploading distributions to https://test.pypi.org/legacy/
INFO     dist/bar-0.0.2-py3-none-any.whl (8.4 KB)
INFO     dist/bar-0.0.2.tar.gz (7.7 KB)
INFO     dist/buzz-0.0.3-py3-none-any.whl (8.4 KB)
INFO     dist/buzz-0.0.3.tar.gz (7.7 KB)
INFO     dist/foo-0.0.1-py3-none-any.whl (8.4 KB)
INFO     dist/foo-0.0.1.tar.gz (7.7 KB)

This way, if foo-0.0.1-py3-none-any.whl fails, at least bar and buzz will be fully released.

Note that such a change would ever-so-slightly increases the risk of momentarily breaking users' builds. If package child depends on parent, ensuring we've published child's whl and tarball means there's a infinitesimally longer window of time where a user might pull a new version of child, only to have their build break because we haven't published the new version of parent yet.

According to #106, the "whls first" approach was taken to ensure a package's metadata arrives before its source. As it happens, sorting packages alphabetically still ensures each package's whls get published first.

Anything else you'd like to mention?

No response

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions