Fingerprint observability: count missing media fingerprints, record the client, persist rejected components

[t-bhendarkar]

A media registration can end up with no perceptual fingerprint at all, silently. Nothing counts it, nothing records which client sent it, and a component the client rejected is indistinguishable from one it never sent. Every gap below was hit for real during the 2026-08-16/17 extension debugging.

What happened

Over two days, ~11 broken registrations accumulated on the local cluster , videos with 0 frames, blank frame-0 hashes, image posts with all image components dropped. None of it was visible anywhere. Every finding came from hand-joining content against phash_code in a conversation, one ctid at a time.

Two wrong conclusions were drawn along the way (an "image regression" that was actually the pre-existing bug, and a VP success misread as extension evidence) , both caused by the same root: the database does not record which client produced a fingerprint. VP vs extension could only be told apart by inferring from video frame spacing (1s vs 5s), and for images there is no signature at all.

Three additive changes

1. Count media components that arrive without a fingerprint

At ingest, when a content row has N media items but fewer than N image/video fingerprint components land, increment a metric (e.g. tip_fingerprint_media_missing_total{modality}) and log at warn with the ctid. This is the alert that would have caught all 11 on day one.

The check belongs at ingest time (content-service, after _ingestRequestFingerprints), comparing media[] length against the distinct media components in the parsed envelope.

2. Record the client on the fingerprint row

perceptual_fingerprint.pipeline already stores {package, profile}. Add the client (e.g. client: "extension" | "vp-app" | "web-sdk") , either a new envelope field the clients send, or at minimum persist the request's user-agent. Makes "how is the extension doing today" a query instead of a two-day conversation.

3. Persist rejected components instead of dropping them

tier: "reject" is a legitimate "client tried and could not fingerprint this" placeholder , but ingest.js returns null and writes no row, so "tried and failed" and "never sent" are identical in storage. Persist a row with a rejected marker (no index entries), so the two failure modes are distinguishable. This is the single fact that could not be answered during the entire investigation.

Constraints

• All three are node-local and additive: no signed bytes change (the envelope is off-DAG; only fingerprint_commit rides the tx), no _canonContent change, no state-root impact.
• Adding a client field to the envelope changes the commit input for NEW registrations only , the commit is over the exact received bytes, so old envelopes verify unchanged. If that is judged too invasive, user-agent capture at the API is the zero-contract-change fallback.
• Enforcement/rejection of incomplete fingerprints is explicitly out of scope: the envelope is optional by design and consensus cannot see it (it never rides the tx), so a hard rule at the API would be bypassable theatre. This issue is visibility only.

Related

• Add content_set_hash: order-independent lookup so content can be found from its bytes without a ctid #260 , content_set_hash (same additive node-local shape)
• The extension defects that motivated this are fixed (sampling 0.2→1 fps, decode race, component drops), verified 16-for-16 clean after the fixes , this issue is about the node never noticing.