Pick up license information from .venv when --license-full

[black-snow]

Description

I expected --license-full to walk the fs and pick up license information from .venv/lib/python3.12/site-packages, e.g., from .venv/lib/python3.12/site-packages/wsproto-1.2.0.dist-info/LICENSE.

Trivy supposedly already scans site-packages, just not in arbitrary places. So it seems to me that picking up .venv from the project should be rather easy to add. Perhaps, we could also pass in additional directory and cover most / all Python build tools.

My expectation was that with --license-full trivy would walk the whole file tree under the given path and look for package metadata it found in my poetry.lock file.

It seems like, at the moment, in order to get license information for python projects you have to build a container or to install the package locally after the build. Not impossible but unwieldy when building libs.

/edit: or if I could just point it to the generated wheel or sdist ...

/edit2: we can do this via the rootfs scanner (thanks @DmitriyLewen) - it's somewhat unexpected, though. Why should fs and rootfs differ? Why are there even two different scanning methods for the file system? Why can you give rootfs a path (shouldn't it scan from ... the root?)? We should at least make this clearer in the docs.

Target

Filesystem

Scanner

License

[DmitriyLewen]

Hello @black-snow

This is a bit unclear in the context of license scanning, but you can scan *.dist-info/METADATA files in rootfs mode (see https://aquasecurity.github.io/trivy/v0.51/docs/coverage/language/#supported-languages):

➜ trivy -d rootfs --scanners license /opt/homebrew/lib/python3.12
...
Python (license)

Total: 8 (UNKNOWN: 2, LOW: 6, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

┌──────────────┬──────────────┬────────────────┬──────────┐
│   Package    │   License    │ Classification │ Severity │
├──────────────┼──────────────┼────────────────┼──────────┤
│ Jinja2       │ BSD-3-Clause │ Notice         │ LOW      │
├──────────────┼──────────────┼────────────────┼──────────┤
│ pip          │ MIT License  │ Non Standard   │ UNKNOWN  │
├──────────────┼──────────────┼────────────────┼──────────┤
│ Werkzeug     │ BSD-3-Clause │ Notice         │ LOW      │
├──────────────┼──────────────┼────────────────┼──────────┤
│ wheel        │ MIT License  │ Non Standard   │ UNKNOWN  │
├──────────────┼──────────────┼────────────────┼──────────┤
│ Flask        │ BSD-3-Clause │ Notice         │ LOW      │
├──────────────┤              │                │          │
│ MarkupSafe   │              │                │          │
├──────────────┤              │                │          │
│ click        │              │                │          │
├──────────────┤              │                │          │
│ itsdangerous │              │                │          │
└──────────────┴──────────────┴────────────────┴──────────┘

Regards, Dmitriy

[black-snow]

Interesting, I though about trying rootfs. Will give it a shot.

[black-snow]

Thanks @DmitriyLewen this works fine. I'll reformulate the issue.

[DmitriyLewen]

Why should fs and rootfs differ? Why are there even two different scanning methods for the file system? Why can you give rootfs a path (shouldn't it scan from ... the root?)? We should at least make this clearer in the docs.

https://aquasecurity.github.io/trivy/v0.51/docs/coverage/language/#supported-languages

[black-snow]

Are you pointing towards the pre/post-build section? I don't really get it, though. At least for license scanning (or license-full), shouldn't trivy try to pick up as much information as possible? I'd expect it to look in whatever place makes sense under the given path.

[DmitriyLewen]

We use packages analyzers to detect licenses in packages(metadata files, lock files, etc). This imposes certain restrictions.
Also, enabling all analyzers can lead to undesirable consequences such as memory consumption, increased operating time, etc.

Whenever possible, we try to integrate the license detection for all analyzers (if possible) (e.g., we added license detection for pip files (fs mode) - #6782)).

But perhaps we can enable all analyzers if only license scanner is enable + --license-full

@knqyf263 wdyt?