Skip to content

fix(jsonpb): eliminate non-deterministic behaviour in unmarshalling#161

Open
aarmoa wants to merge 1 commit intocosmos:mainfrom
aarmoa:fix/jsonb-unmarshal-non-deterministic-result
Open

fix(jsonpb): eliminate non-deterministic behaviour in unmarshalling#161
aarmoa wants to merge 1 commit intocosmos:mainfrom
aarmoa:fix/jsonb-unmarshal-non-deterministic-result

Conversation

@aarmoa
Copy link

@aarmoa aarmoa commented Mar 5, 2026

Two independent sources of non-determinism existed in the JSON unmarshaller, both caused by iterating over Go maps whose key order is randomised per program execution.

Oneof conflict (critical)
sprops.OneofTypes is a map[string]*OneofProperties. When the JSON input contains multiple keys that belong to the same oneof group (e.g. {"title":"foo","salary":31000} for MsgWithOneof.union), every matched alternative overwrote the same interface field via target.Field(oop.Field).Set(nv). The winner depended on which key the runtime happened to visit last, making the decoded value non- deterministic.
The fix collects the map keys, sorts them with slices.Sort, and tracks which oneof interface fields have been set. If a second key from the same group is encountered an error is returned, consistent with the behaviour of the text-format parser (text_parser.go:589) and the proto oneof contract (at most one field may be set at a time).

Extension map iteration (low)
proto.RegisteredExtensions returns a map[int32]*ExtensionDesc. The unmarshaller iterated it without sorting. Extension names are unique, so this did not produce an incorrect result, but it was structurally inconsistent with the marshalling side (lines 401-408) which explicitly sorts extension IDs for stable output. The fix applies the same pattern: collect IDs, sort with sort.Sort(int32Slice(extIDs)), then iterate.

Tests
Following TDD, the tests were added first and confirmed to fail before the logic changes were applied:

  • Two new entries in unmarshalingShouldError cover the oneof-conflict cases (orig_name and camelCase variants). TestUnmarshalingBadInput now asserts that both inputs produce a non-nil error.
  • TestUnmarshalOneofConflictDeterminism runs the same conflicting JSON through the unmarshaller 100 times and asserts only one unique outcome is observed, directly reproducing the non-determinism that existed before the fix.

Two independent sources of non-determinism existed in the JSON
unmarshaller, both caused by iterating over Go maps whose key order is
randomised per program execution.
**Oneof conflict (critical)**
`sprops.OneofTypes` is a `map[string]*OneofProperties`. When the JSON
input contains multiple keys that belong to the same oneof group (e.g.
`{"title":"foo","salary":31000}` for `MsgWithOneof.union`), every
matched alternative overwrote the same interface field via
`target.Field(oop.Field).Set(nv)`. The winner depended on which key the
runtime happened to visit last, making the decoded value non-
deterministic.
The fix collects the map keys, sorts them with `slices.Sort`, and tracks
which oneof interface fields have been set. If a second key from the same
group is encountered an error is returned, consistent with the behaviour
of the text-format parser (`text_parser.go:589`) and the proto oneof
contract (at most one field may be set at a time).
**Extension map iteration (low)**
`proto.RegisteredExtensions` returns a `map[int32]*ExtensionDesc`. The
unmarshaller iterated it without sorting. Extension names are unique, so
this did not produce an incorrect result, but it was structurally
inconsistent with the marshalling side (lines 401-408) which explicitly
sorts extension IDs for stable output. The fix applies the same pattern:
collect IDs, sort with `sort.Sort(int32Slice(extIDs))`, then iterate.
**Tests**
Following TDD, the tests were added first and confirmed to fail before
the logic changes were applied:
- Two new entries in `unmarshalingShouldError` cover the oneof-conflict
  cases (orig_name and camelCase variants). `TestUnmarshalingBadInput`
  now asserts that both inputs produce a non-nil error.
- `TestUnmarshalOneofConflictDeterminism` runs the same conflicting JSON
  through the unmarshaller 100 times and asserts only one unique outcome
  is observed, directly reproducing the non-determinism that existed
  before the fix.
aarmoa added a commit to InjectiveLabs/gogoproto that referenced this pull request Mar 19, 2026
- Sort oneof keys and error on multiple fields in same oneof (critical).
- Sort extension IDs before iterating (consistency with marshal).
- Add unmarshalingShouldError cases and TestUnmarshalOneofConflictDeterminism.

Ref: cosmos#161
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant