Add ENISA EUVD provider (#915)#1156
Open
ChrisJr404 wants to merge 1 commit into
Open
Conversation
Adds an auxiliary vunnel provider that pulls from the ENISA EU Vulnerability Database public search API and normalizes records into a new EUVD vunnel schema, so downstream consumers (e.g. grype) can pivot ENISA-aligned identifiers and CVSS evaluations off NVD/GHSA matches as @westonsteimel sketched in the issue thread. Per @kzantow's note in the thread, this PR keeps the scope to the vunnel side only; how grype consumes the data and any prioritisation rules are left to a follow-up against anchore/grype#2601 once this lands and the new schema is published. What's included: * schema/vulnerability/euvd/schema-1.0.0.json - new schema, mirroring the level of detail in the upstream EUVD record but with stable ISO-8601 timestamps and proper string-array fields. The upstream payload uses locale-formatted dates (e.g. 'May 4, 2026, 1:00:23 AM') and \n-delimited string lists for aliases / references; the manager normalizes both. * src/vunnel/providers/euvd/__init__.py + manager.py - new auxiliary provider, modelled on the kev provider's shape. Tagged 'auxiliary'. Pagination uses the upstream API's page/size parameters with a conservative MAX_PAGES safety cap, and supports incremental sync via the fromUpdatedDate filter when last_updated is supplied. * src/vunnel/providers/__init__.py - register the provider in the auxiliary section alongside kev/epss/eol. * src/vunnel/schema.py - new EUVD_SCHEMA_VERSION constant + EUVDSchema factory. * 8 unit tests under tests/unit/providers/euvd/ covering happy-path normalization, missing fields, id/description validation, uneven vendor/product zip, newline-list parsing edge cases, locale-timestamp parsing edge cases, paginated download termination, and end-to-end provider-update + schema validation against a real fixture catalogue. All pre-existing 417 unit tests still pass. Closes anchore#915 Signed-off-by: ChrisJr404 <11917633+ChrisJr404@users.noreply.github.com>
Contributor
|
Hi @ChrisJr404 thanks for the PR! I am a little busy at the moment but hope to take a look at it soon, probably about 2 weeks. You also have a lot of PRs open across a few different repos. Is there a particular PR that you hope we'll review first? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #915. Chains into anchore/grype#2601.
Adds an auxiliary vunnel provider that pulls from the ENISA EU Vulnerability Database public search API (https://euvd.enisa.europa.eu/apidoc) and normalizes records into a new EUVD vunnel schema, so downstream consumers (e.g. grype) can pivot ENISA-aligned identifiers and CVSS evaluations off NVD/GHSA matches as @westonsteimel sketched in the issue thread:
Per @kzantow's note, this PR keeps the scope to the vunnel side only — how grype consumes the data and any prioritisation rules are left to a follow-up against anchore/grype#2601 once the new schema is published.
What's in here
Notes on normalization
The upstream API ships a couple of footguns the manager irons out so consumers don't have to:
Pagination
`Manager.get(last_updated=None)` walks pages of `size=100` (the upstream maximum) until either:
A `MAX_PAGES = 20000` cap protects against a misbehaving server pushing us into an infinite loop. ENISA had ~350k records when I tested, so this leaves plenty of room before the cap needs to move.
When `last_updated` is supplied, the request adds `fromUpdatedDate=YYYY-MM-DD` so subsequent runs only re-fetch deltas. This pairs naturally with vunnel's existing last-run state.
Tests
`tests/unit/providers/euvd/test_euvd.py` (8 cases):
I also re-ran the rest of the unit suite to make sure nothing else regressed, including the two version-registry asserts in `tests/unit/test_provider.py` (`test_provider_versions` + `test_provider_distribution_versions`) which I updated to register `euvd` at version 1.
Follow-ups (intentionally out of scope here)
Signed-off-by: ChrisJr404 11917633+ChrisJr404@users.noreply.github.com