Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opt: add decoder Option NoValidateJSON for skipping JSON faster #696

Merged
merged 22 commits into from
Sep 13, 2024

Conversation

AsterDY
Copy link
Collaborator

@AsterDY AsterDY commented Aug 26, 2024

Background

By default, sonic uses FSM algorithm to skip unnecessary JSON parsing, but it's much slower than SIMD-fast-skipping. Thus we introduce an Option NoValidateJSON to boost cases such as :

  • Mismatched-type values
  • Unknown fields
  • Omitted fields
  • Empty struct
  • Redundant array elements
  • json.Unmarshaler (json.RawMessage)

Performance

In x86, the fast-skipping is 5x times fast when handling complicated JSON value like TwitterJSON

goos: linux
goarch: amd64
pkg: github.com/bytedance/sonic/decoder
cpu: Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz
BenchmarkSkipValidate/mismatched-field/validate-32                114846             10841 ns/op
BenchmarkSkipValidate/mismatched-field/fast-32                    878814              1172 ns/op
BenchmarkSkipValidate/ommited-field/validate-32                   110401             11003 ns/op
BenchmarkSkipValidate/ommited-field/fast-32                      1000000              1073 ns/op
BenchmarkSkipValidate/redundant-array/validate-32                  52734             22754 ns/op
BenchmarkSkipValidate/redundant-array/fast-32                     111651             11742 ns/op
BenchmarkSkipValidate/unknown-field/validate-32                    97878             10872 ns/op
BenchmarkSkipValidate/unknown-field/fast-32                      1000000              1150 ns/op
BenchmarkSkipValidate/empty-struct/validate-32                    115086             11182 ns/op
BenchmarkSkipValidate/empty-struct/fast-32                       1000000              1147 ns/op
BenchmarkSkipValidate/mismatched-elem/validate-32                 104296             12210 ns/op
BenchmarkSkipValidate/mismatched-elem/fast-32                     758089              1458 ns/op

liuq19
liuq19 previously approved these changes Aug 27, 2024
@AsterDY AsterDY merged commit edc70ff into main Sep 13, 2024
49 checks passed
@AsterDY AsterDY deleted the opt/decoder_skip branch September 13, 2024 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants