Skip to content

Add ENISA EUVD provider (#915)#1156

Open
ChrisJr404 wants to merge 1 commit into
anchore:mainfrom
ChrisJr404:feat/euvd-provider-915
Open

Add ENISA EUVD provider (#915)#1156
ChrisJr404 wants to merge 1 commit into
anchore:mainfrom
ChrisJr404:feat/euvd-provider-915

Conversation

@ChrisJr404

Copy link
Copy Markdown

Summary

Closes #915. Chains into anchore/grype#2601.

Adds an auxiliary vunnel provider that pulls from the ENISA EU Vulnerability Database public search API (https://euvd.enisa.europa.eu/apidoc) and normalizes records into a new EUVD vunnel schema, so downstream consumers (e.g. grype) can pivot ENISA-aligned identifiers and CVSS evaluations off NVD/GHSA matches as @westonsteimel sketched in the issue thread:

EUVD doesn't really have any match data that grype can make use of today, only really related metadata, but I think getting that in would allow for instance an option to pivot identifiers by ENISA EUVD id where possible which would satisfy the majority of asks I suspect... [it could] just appear in `relatedVulnerabilities` section and you could pivot to raise up the ENISA CVSS evaluation over NVD's for instance.

Per @kzantow's note, this PR keeps the scope to the vunnel side only — how grype consumes the data and any prioritisation rules are left to a follow-up against anchore/grype#2601 once the new schema is published.

What's in here

File What
`schema/vulnerability/euvd/schema-1.0.0.json` new schema, mirroring upstream record detail but with stable ISO-8601 timestamps + proper string-array fields
`src/vunnel/providers/euvd/init.py` provider class, tagged `auxiliary`
`src/vunnel/providers/euvd/manager.py` pagination + per-record normalization
`src/vunnel/providers/init.py` register in the auxiliary block alongside kev/epss/eol
`src/vunnel/schema.py` `EUVD_SCHEMA_VERSION = "1.0.0"` + `EUVDSchema()` factory

Notes on normalization

The upstream API ships a couple of footguns the manager irons out so consumers don't have to:

  • `datePublished` / `dateUpdated` are locale-formatted strings (e.g. `"May 4, 2026, 1:00:23 AM"`). The manager parses them as UTC and emits ISO-8601.
  • `aliases` and `references` arrive as `\n`-delimited strings rather than JSON arrays. The manager splits them.
  • `enisaIdProduct` and `enisaIdVendor` are parallel arrays. The manager zips them by index into vendor/product/version triples and pads missing slots with `null`.
  • Records missing the mandatory `id` or `description` fields are skipped rather than failing the whole sync.

Pagination

`Manager.get(last_updated=None)` walks pages of `size=100` (the upstream maximum) until either:

  • the page is empty,
  • the running count reaches the upstream `total`, or
  • the page returns fewer items than `size` (last page).

A `MAX_PAGES = 20000` cap protects against a misbehaving server pushing us into an infinite loop. ENISA had ~350k records when I tested, so this leaves plenty of room before the cap needs to move.

When `last_updated` is supplied, the request adds `fromUpdatedDate=YYYY-MM-DD` so subsequent runs only re-fetch deltas. This pairs naturally with vunnel's existing last-run state.

Tests

`tests/unit/providers/euvd/test_euvd.py` (8 cases):

  • `test_normalize_record_full` — happy path, fixture pulled from the live API, asserts ISO-8601 conversion + array splits + product zip.
  • `test_normalize_record_handles_missing_fields` — every nullable field absent, none of the array helpers crash.
  • `test_normalize_record_rejects_records_without_id_or_description` — guards the schema's required fields.
  • `test_normalize_record_zips_uneven_product_vendor_arrays` — pads with `null`s when product/vendor counts differ.
  • `test_split_newline_list_handles_array_input` — accepts both string and array shapes.
  • `test_normalize_timestamp_handles_unparseable_input` — returns `None` rather than raising.
  • `test_manager_paginates_until_empty` — drives a fake paginated response sequence end-to-end.
  • `test_provider_schema` — runs the full provider against a 3-entry catalogue, asserts the workspace contains 3 entries and they all validate against `EUVDSchema()`.
$ pytest tests/unit/providers/euvd/ -v
8 passed in 0.21s

I also re-ran the rest of the unit suite to make sure nothing else regressed, including the two version-registry asserts in `tests/unit/test_provider.py` (`test_provider_versions` + `test_provider_distribution_versions`) which I updated to register `euvd` at version 1.

$ pytest tests/unit/ --ignore=tests/unit/providers/
417 passed in 1.22s

Follow-ups (intentionally out of scope here)

  • Wiring on the grype side per @willmurphyscode's "we're unblocked" comment in Include Support for EUVD (European Union Vulnerability Database) grype#2601 — happy to open that as a separate PR after this lands and the schema URL is live on `main`.
  • Quality-suite snapshot fixture under `tests/quality/`. The other auxiliary providers don't have one and the upstream catalogue is large enough that a snapshot would inflate the repo, so I left it out by default. Happy to add one if you'd prefer.
  • A real recorded HTTP fixture via `vcrpy` against a small fromUpdatedDate window. Same reasoning — opted for the lighter `mocker.patch` approach already used by `kev` / `eol`.

Signed-off-by: ChrisJr404 11917633+ChrisJr404@users.noreply.github.com

Adds an auxiliary vunnel provider that pulls from the ENISA EU
Vulnerability Database public search API and normalizes records into a
new EUVD vunnel schema, so downstream consumers (e.g. grype) can pivot
ENISA-aligned identifiers and CVSS evaluations off NVD/GHSA matches as
@westonsteimel sketched in the issue thread.

Per @kzantow's note in the thread, this PR keeps the scope to the
vunnel side only; how grype consumes the data and any prioritisation
rules are left to a follow-up against anchore/grype#2601 once this
lands and the new schema is published.

What's included:

* schema/vulnerability/euvd/schema-1.0.0.json - new schema, mirroring
  the level of detail in the upstream EUVD record but with stable
  ISO-8601 timestamps and proper string-array fields. The upstream
  payload uses locale-formatted dates (e.g. 'May 4, 2026, 1:00:23 AM')
  and \n-delimited string lists for aliases / references; the
  manager normalizes both.
* src/vunnel/providers/euvd/__init__.py + manager.py - new auxiliary
  provider, modelled on the kev provider's shape. Tagged 'auxiliary'.
  Pagination uses the upstream API's page/size parameters with a
  conservative MAX_PAGES safety cap, and supports incremental sync via
  the fromUpdatedDate filter when last_updated is supplied.
* src/vunnel/providers/__init__.py - register the provider in the
  auxiliary section alongside kev/epss/eol.
* src/vunnel/schema.py - new EUVD_SCHEMA_VERSION constant + EUVDSchema
  factory.
* 8 unit tests under tests/unit/providers/euvd/ covering happy-path
  normalization, missing fields, id/description validation, uneven
  vendor/product zip, newline-list parsing edge cases, locale-timestamp
  parsing edge cases, paginated download termination, and end-to-end
  provider-update + schema validation against a real fixture catalogue.
  All pre-existing 417 unit tests still pass.

Closes anchore#915

Signed-off-by: ChrisJr404 <11917633+ChrisJr404@users.noreply.github.com>
@willmurphyscode

Copy link
Copy Markdown
Contributor

Hi @ChrisJr404 thanks for the PR! I am a little busy at the moment but hope to take a look at it soon, probably about 2 weeks.

You also have a lot of PRs open across a few different repos. Is there a particular PR that you hope we'll review first?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a provider for the ENISA EUVD data

2 participants