Skip to content

proto differential: findings that combine several known schema-drift causes match no allowlist entry #60

Description

@jlucaso1

What happens

Since the bridge bump to 0.10.0 (#55), proto:round-trip reports 9 findings that no entry in KNOWN_DIVERGENCES matches, even though every individual cause in them is already documented.

npm test    # 1266/1268 — "round-trips a message through the other implementation"

Why

The bridge regenerated its proto against WhatsApp schema 2.3000.1044659339 (bridge #51/#54). baileys has not: 7.0.0-rc13 and 7.0.0-rc14 both ship the older bundle — verified, not assumed:

rc14  pollResultSnapshotMessageV3 -> field 114     (bridge: 115)
rc14  faviconMMSMetadata                            (bridge: faviconMmsMetadata)

So bumping the dev dependency does not help. The drift is small in kind and wide in reach:

  • Exactly one field is renumbered. Swept the whole surface — Message.pollResultSnapshotMessageV3, 115 here against 114 upstream, which is already tracked in KNOWN_WIRE_GAPS, in wire-fidelity.test.ts as KNOWN_DIVERGENT, and by proto-field-number-mismatch.
  • Everything else is fields one side declares and the other does not. faviconMMSMetadata/faviconMmsMetadata, statusMentionMessage, statusNotificationMessage, mediaKeyDomain and friends.

Each of those has an entry. The problem is that a generated message reaches several of them at once, and applyAllowlist picks a single entry per finding: each when asks "is my difference the whole difference", and with two causes present the answer is no for both. The finding is then reported as unknown while nothing in it is.

Measured on one round-trip finding:

only in baileyrs:  statusMentionMessage, statusNotificationMessage, faviconMmsMetadata
only in upstream:  pollResultSnapshotMessageV3, faviconMMSMetadata, mediaKeyDomain

Three documented causes, one finding, no match.

What was tried and rejected

Stripping the second cause inside the first entry's predicate (removing pollResultSnapshotMessageV3 inside the favicon entry). It made that finding pass and would have hidden any third difference in the same message. Reverted — an allowlist wider than its own argument is the defect this suite exists to avoid, and PR #55's review flagged exactly that pattern three times.

Options

1. A directional class entry for "fields only one schema declares".
Extend what sameExceptUnwrittenFields already does for DECODE_OMITTED_PATHS: compare the two decodes ignoring keys that only one side's schema has, and fail on any value that differs where both sides hold the key. The inventory can be derived rather than hand-kept — whatsapp-surface.txt on one side, the upstream namespace on the other.
For: one entry replaces a growing list; survives the next schema release without edits; keeps every real misread failing, because a differing shared key still fails.
Against: needs the two inventories available to the harness at runtime; a decoder that genuinely loses a field it does declare must not be swept in, so the direction has to be checked per key rather than per message.

2. Let the harness compose entries.
Allow several KnownDivergence entries to jointly explain one finding: subtract what each accounts for and require the remainder to be empty.
For: fixes the general problem, not this instance; every entry stays as narrow as it is today.
Against: the largest change of the three — applyAllowlist currently returns the first match, and "what an entry accounts for" is not something an entry can express yet.

3. Generate only the fields both schemas declare.
Restrict the differential's generator to the intersection.
For: removes the class at the source; nothing to allowlist.
Against: loses coverage precisely on the fields WhatsApp just added, which is where a codec bug is most likely; and the intersection shrinks with every schema release.

4. Enumerate the affected fields exactly, in the style of NOT_ENCODED_FIELDS, and require the remainder to match.
For: smallest change; explicit, and a new field fails the suite rather than joining silently.
Against: the list grows with every WhatsApp schema release, and each entry is a claim someone has to re-argue at review time.

Recommendation: 1, with 4 as the stopgap if the inventory turns out not to be reachable from the harness. 2 is the right answer if this class shows up a third time.

Notes

  • Not a regression in the bridge or in this library: the wire is identical for the renumbered field, and the rest are fields that simply did not exist when upstream last regenerated.
  • Self-resolving in part — the day upstream regenerates its proto, most of these disappear.
  • proto:mutation-agreement, proto:field-names, proto:field-numbers, proto:encode-bytes and proto:decode-parity are all green; only proto:round-trip builds messages rich enough to hit several causes at once.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions