Add SPDX format support for SBOM

[midnightercz]

Support for SPDX format was added to fetch-depds command and also to merge_syft_sboms.
No changes were made in particular package manager generating components which are then converted to cyclonedx format. SPDX sbom can be obtained by calling Sbom.to_spdx().
New switch sbom-type was added to merge_syfy_sboms, so user can choose which output format should be generated - default is cyclonedx. Once all tooling is ready to consume spdx sboms, cutoff changes in this repository can be started.

Maintainers will complete the following section

• Commit messages are descriptive enough
• Code coverage from testing does not decrease and new code is covered
• Docs updated (if applicable)
• Docs links in the code are still valid (if docs were updated)

Note: if the contribution is external (not from an organization member), the CI
pipeline will not run automatically. After verifying that the CI is safe to run:

• approve GitHub Actions workflows by clicking a button
• approve the Red Hat Trusted App Pipeline container build by commenting /ok-to-test
  (as is the standard for Pipelines as Code)

[brunoapimentel]

Just a quick heads-up: the utils/merge_syft_sbom.py script was removed from this repo and now lives here. So you'll need to propose the related changes there instead.

[midnightercz]

Just a quick heads-up: the utils/merge_syft_sbom.py script was removed from this repo and now lives here. So you'll need to propose the related changes there instead.

I thought I will eventually integrate the changes into new subcommand implemented here: #593 but it wasn't merged yet when I started working on this one

[MartinBasti]

typo in commit message and PR: s/SDPX/SPDX/

[midnightercz]

Thanks for all the reviews and comments. I also wanted to add spdx support into merge-sboms subcommand but I noticed that in the tests you don't really test merged output. Or am I missing something?

[a-ovchinnikov]

Mostly LGTM, a couple of minor changes is needed. Edit: you would also need to make sure UTs pass.

[a-ovchinnikov]

Thanks for all the reviews and comments. I also wanted to add spdx support into merge-sboms subcommand but I noticed that in the tests you don't really test merged output. Or am I missing something?

merge-sboms is a thin wrapper around an internal function that is tested elsewhere. It does not do much except some validation of input and then invoking a method for saving merge results (which is also tested elsewhere). There is not much value in doing this sort of test.

[a-ovchinnikov]

Is it known how a property like this will be replicated for SPDX?

@joejstuart I applied the changes I was talking about yesterday. This is how your sample component would look like in SPDX now:

            SPDXPackage(
                SPDXID="SPDXRef-Package-archive.zip-None-965cfdf16e9275d1b3b562dee596de0474cdc751ba4c30715cfc3934fab3b300",
                name="archive.zip",
                versionInfo=None,
                externalRefs=[
                    SPDXPackageExternalRefPackageManagerPURL(
                        referenceLocator="pkg:generic/archive.zip?checksum=sha256:386428a82f37345fa24b74068e0e79f4c1f2ff38d4f5c106ea14de4a2926e584&download_url=https://github.com/cachito-testing/cachi2-generic/archive/refs/tags/v2.0.0.zip",
                        referenceType="purl",
                        referenceCategory="PACKAGE-MANAGER",
                    ),
                ],
                annotations=[
                    SPDXPackageAnnotation(
                        annotator="Tool: cachi2:jsonencoded",
                        annotationDate="2021-07-01T00:00:00Z",
                        annotationType="OTHER",
                        comment='{"name": "cachi2:found_by", "value": "cachi2"}',
                    ),
                ],
                downloadLocation="NOASSERTION",
            )

[brunoapimentel]

/ok-to-test

[brunoapimentel]

nitpick: typo in commit message

s/Co-authered-by/Co-authored-by

[brunoapimentel]

This LGTM!

There are some outstanding comments about the SPDX content, but I believe this is more than enough for our first iteration.

[brunoapimentel]

Tiny nitpick that should be addressed as a follow up (if ever): if we're ok with defining these lambdas, shouldn't we just configure flake8 to ignore E731?

[a-ovchinnikov]

At some point definitely yes, currently it is only the new code in one SPDX-related commit that relies on named helper lambdas, I suggest waiting until the approach is actually adopted.

[eskultety]

as a follow up (if ever)

I vote for a follow-up :). I don't object against named lambdas in our code, but there was apparently a reason a rule was added to flake8. Okay, there's no obligation for us to be 100% conformant with flake8 rules (that's too harsh), but in that case we should disable a given rule globally in the mentioned follow up, not just in a single module as I'm not a fan of such practices, that just hinders readability and clarity.

[a-ovchinnikov]

To the best of my understanding the reason behind this rule is clarity of traceback in case something fails within a lambda vs when something fails in a named function i.e. to ensure that the ability to dynamically bind function objects to names is not abused. All I do with named lambdas is lifting them from comprehensions to appease black (which would make me go multiline with LCs otherwise which, in my opinion, is way worse).

[joejstuart]

Is it known how a property like this will be replicated for SPDX?

@joejstuart: Currently it will be transformed into a file entry which is not exactly what is needed (see my response above). I will be pushing a fix for that next.

Thank you. When you know what it will look like, can you please post? I'm working on a rule to verify the package sources. https://issues.redhat.com/browse/EC-1022

[a-ovchinnikov]

When you know what it will look like, can you please post?

@joejstuart here is how it is supposed to look like: #608 (comment)

[joejstuart]

When you know what it will look like, can you please post?

@joejstuart here is how it is supposed to look like: #608 (comment)

I completely missed that. Thanks!

[a-ovchinnikov]

/ok-to-test

[chmeliik]

Consider this a "shallow" ack - the results meet what I expect for our Konflux needs, but I didn't do a thorough review

[eskultety]

as a follow up (if ever)

I vote for a follow-up :). I don't object against named lambdas in our code, but there was apparently a reason a rule was added to flake8. Okay, there's no obligation for us to be 100% conformant with flake8 rules (that's too harsh), but in that case we should disable a given rule globally in the mentioned follow up, not just in a single module as I'm not a fan of such practices, that just hinders readability and clarity.

[joejstuart]

downloadLocation="NOASSERTION"

Here is the new policy for this. enterprise-contract/ec-policies#1270