fix(jsonpb): eliminate non-deterministic behaviour in unmarshalling#1
Conversation
- Sort oneof keys and error on multiple fields in same oneof (critical). - Sort extension IDs before iterating (consistency with marshal). - Add unmarshalingShouldError cases and TestUnmarshalOneofConflictDeterminism. Ref: cosmos#161
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughOneof deserialization in Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Addressing one of codex review concerns:
When a payload includes null for one member of a oneof and a real value for another, e.g. {"title":null,"salary":31000},
consumeField still returns the null token here, so setOneofFields is marked and the second member now fails with the overwrite
error. In jsonpb, null is otherwise treated as an unset field, so this change starts rejecting valid sparse JSON emitted with
explicit nulls; the conflict check should skip oneof entries whose raw value is null.
Two independent sources of non-determinism existed in the JSON unmarshaller, both caused by iterating over Go maps whose key order is randomised per program execution.
Oneof conflict (critical)
sprops.OneofTypesis amap[string]*OneofProperties. When the JSON input contains multiple keys that belong to the same oneof group (e.g.{"title":"foo","salary":31000}forMsgWithOneof.union), every matched alternative overwrote the same interface field viatarget.Field(oop.Field).Set(nv). The winner depended on which key the runtime happened to visit last, making the decoded value non- deterministic.The fix collects the map keys, sorts them with
slices.Sort, and tracks which oneof interface fields have been set. If a second key from the same group is encountered an error is returned, consistent with the behaviour of the text-format parser (text_parser.go:589) and the proto oneof contract (at most one field may be set at a time).Extension map iteration (low)
proto.RegisteredExtensionsreturns amap[int32]*ExtensionDesc. The unmarshaller iterated it without sorting. Extension names are unique, so this did not produce an incorrect result, but it was structurally inconsistent with the marshalling side (lines 401-408) which explicitly sorts extension IDs for stable output. The fix applies the same pattern: collect IDs, sort withsort.Sort(int32Slice(extIDs)), then iterate.Tests
Following TDD, the tests were added first and confirmed to fail before the logic changes were applied:
unmarshalingShouldErrorcover the oneof-conflict cases (orig_name and camelCase variants).TestUnmarshalingBadInputnow asserts that both inputs produce a non-nil error.TestUnmarshalOneofConflictDeterminismruns the same conflicting JSON through the unmarshaller 100 times and asserts only one unique outcome is observed, directly reproducing the non-determinism that existed before the fix.Ref: cosmos#161
Summary by CodeRabbit
Bug Fixes
Tests