ci: Add GitHub artifact attestations to package distribution

[matthewfeickert]

• Add generation of GitHub artifact attestations to built sdist and wheel before upload. c.f.:
  • https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/
  • https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds
• Add verification of artifact attestation before publishing awkward to PyPI using the 'gh attestation verify' CLI API, added in v2.49.0.
  • c.f. https://github.com/cli/cli/releases/tag/v2.49.0

As an example c.f. scikit-hep/pyhf#2473.

[matthewfeickert]

Debug examples running on my fork:

• Example for deploying awkward-cpp

Attestation created for awkward-cpp-33.tar.gz@sha256:ac86c8e77e024afc75cb16d84fab16a95b343183ac47017f332ee35b64553abe
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
https://search.sigstore.dev?logIndex=95705050
Attestation uploaded to repository
https://github.com/matthewfeickert/awkward/attestations/884514

• Example for deploying awkward

Attestation created for awkward-2.6.4-py3-none-any.whl@sha256:c9dc7d953f9c48cad502543176ccc9db2634a6b52dfdcf1fd93b9a1337fbe621
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
https://search.sigstore.dev?logIndex=95694782
Attestation uploaded to repository
https://github.com/matthewfeickert/awkward/attestations/883802
Attestation created for awkward-2.6.4.tar.gz@sha256:8b95026cda9814b64bdf769c0af0c9b72a9e3d77967d45c02b78aa883a8eae51
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
https://search.sigstore.dev?logIndex=95694788
Attestation uploaded to repository
https://github.com/matthewfeickert/awkward/attestations/883803

Run gh attestation verify dist/awkward-*.tar.gz --repo matthewfeickert/awkward
matthewfeickert/awkward	https://slsa.dev/provenance/v1	.github/workflows/deploy.yml@refs/heads/debug/add-artifact-attestations
Run gh attestation verify dist/awkward-*.whl --repo matthewfeickert/awkward
matthewfeickert/awkward	https://slsa.dev/provenance/v1	.github/workflows/deploy.yml@refs/heads/debug/add-artifact-attestations

These debug examples are running on push events on a debug branch I made, but this PR implements things so that they only create attestations when the publishing workflow runs (so on workflow dispatch or release events).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.11%. Comparing base (b749e49) to head (8f01e17).
Report is 90 commits behind head on main.

Additional details and impacted files

see 85 files with indirect coverage changes

[jpivarski]

Just like scikit-hep/uproot5#1222, this looks good, too! I'll merge it now.

[agoose77]

This is fantastic! Thanks all :)

[matthewfeickert]

Nice to see for awkward-cpp v34 and awkward v2.6.5: https://github.com/scikit-hep/awkward/attestations

$ python -m pip download --no-deps awkward
Collecting awkward
  Downloading awkward-2.6.5-py3-none-any.whl.metadata (7.0 kB)
Downloading awkward-2.6.5-py3-none-any.whl (796 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 796.1/796.1 kB 5.3 MB/s eta 0:00:00
Saved ./awkward-2.6.5-py3-none-any.whl
Successfully downloaded awkward
$ gh attestation verify awkward-2.6.5-py3-none-any.whl --repo scikit-hep/awkward
Loaded digest sha256:9fdecd255b660e05369631331689ca6ffedc1047e7756aa402445b4fa96081ce for file://awkward-2.6.5-py3-none-any.whl
Loaded 1 attestation from GitHub API
✓ Verification succeeded!

sha256:9fdecd255b660e05369631331689ca6ffedc1047e7756aa402445b4fa96081ce was attested by:
REPO                PREDICATE_TYPE                  WORKFLOW                                     
scikit-hep/awkward  https://slsa.dev/provenance/v1  .github/workflows/deploy.yml@refs/tags/v2.6.5

[matthewfeickert]

Though I messed up with

awkward/.github/workflows/deploy-cpp.yml

Lines 37 to 40 in 45c708d

	- name: Generate artifact attestation for sdist and wheel
	uses: actions/attest-build-provenance@173725a1209d09b31f9d30a3890cf2757ebbff0d # v1.1.2
	with:
	subject-path: "dist/awkward-cpp-*"

as the naming scheme is awkward-cpp-*.tar.gz and awkward_cpp-*.whl so only the sdist for awkward-cpp got signed

# $ python -m pip download --no-binary :all: --no-deps awkward-cpp  # isn't working to get the sdist without build?
$ curl -sLO https://files.pythonhosted.org/packages/bd/0d/64ce9e9f6164a1bf8f0c32f86f5e705598093f16158d9b73b3e33c00ec35/awkward-cpp-34.tar.gz
$ gh attestation verify awkward-cpp-34*.tar.gz --repo scikit-hep/awkward
Loaded digest sha256:39ac814d769f38ade9a5f6c39b85b07bbc05af56342fb15742edce1237f9e049 for file://awkward-cpp-34.tar.gz
Loaded 1 attestation from GitHub API
✓ Verification succeeded!

sha256:39ac814d769f38ade9a5f6c39b85b07bbc05af56342fb15742edce1237f9e049 was attested by:
REPO                PREDICATE_TYPE                  WORKFLOW                                        
scikit-hep/awkward  https://slsa.dev/provenance/v1  .github/workflows/deploy-cpp.yml@refs/heads/main
$ python -m pip download --no-cache-dir --no-deps awkward-cpp
Collecting awkward-cpp
  Downloading awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.2 kB)
Downloading awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (694 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 694.1/694.1 kB 6.0 MB/s eta 0:00:00
Saved ./awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Successfully downloaded awkward-cpp
$ gh attestation verify awkward_cpp-34*.whl --repo scikit-hep/awkward
Loaded digest sha256:077f6ae40e2a4fdda1252665eeebb7ab52d3987924e861fe5c8f97b473b540b5 for file://awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
✗ Loading attestations from GitHub API failed

Error: failed to fetch attestations from scikit-hep/awkward: HTTP 404: Not Found (https://api.github.com/repos/scikit-hep/awkward/attestations/sha256:077f6ae40e2a4fdda1252665eeebb7ab52d3987924e861fe5c8f97b473b540b5?per_page=30)

[matthewfeickert]

A note related to cli/cli#9215, this is only working currently because there is one wheel. gh attestation verify can't operate on file globs!