Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MessagePack item serializer and validator #628

Merged
merged 35 commits into from
Jan 27, 2025
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
8395a35
Add msgpack item serializer and validator
jarmuszz Aug 11, 2024
f051c31
Merge branch 'gnieh:main' into main
jarmuszz Aug 11, 2024
52b48d7
Fix one additional argument being passed
jarmuszz Aug 13, 2024
7b4246e
Add tests to cover all fmts for msgpack serializer
jarmuszz Aug 13, 2024
9bc1452
Remove debug code in serializer test
jarmuszz Aug 13, 2024
eed74a4
Add validation test cases
jarmuszz Aug 15, 2024
54d7d55
Make msgpack item serializer omit leading zeros
jarmuszz Aug 17, 2024
1ed1f73
Make Extension tests use `ByteVector.fill`
jarmuszz Aug 18, 2024
ac14528
Refine msgpack fixpoint test
jarmuszz Aug 18, 2024
671e693
Remove redundant `padLeft`s when size is known
jarmuszz Aug 18, 2024
b732583
Reformat ValidationSpec.scala
jarmuszz Aug 18, 2024
3f30e33
Remove scaladoc from an embedded function
jarmuszz Aug 18, 2024
aa8658a
Add benchmars for msgpack item serializer
jarmuszz Aug 24, 2024
cd9782e
Merge msgpack serializers
jarmuszz Sep 5, 2024
cdc4894
Make `SerializerSpec` no longer extend `Checkers`
jarmuszz Sep 7, 2024
309569e
Make `msgpack.low` API similar to `cbor.low` API
jarmuszz Sep 7, 2024
3d717a3
Update msgpack serializer spec documentation
jarmuszz Sep 7, 2024
deede3f
Change `msgpack.low.toBinary` scaladoc
jarmuszz Sep 10, 2024
698f727
Fix msgpack doc generation
jarmuszz Sep 10, 2024
248fbc6
Add doc for `msgpack.low` public methods
jarmuszz Sep 10, 2024
4760221
Run prePR
jarmuszz Sep 10, 2024
041e135
Extract literals into constants
jarmuszz Sep 14, 2024
2be8831
Fix msgpack serialization test of negative fixint
jarmuszz Sep 14, 2024
fd845e8
Make msgpack Array and Map use Long for sizes
jarmuszz Sep 21, 2024
482bf9e
Make msgpack exceptions public
jarmuszz Sep 22, 2024
8d67768
Move Pull.pure(None) into a constant
jarmuszz Sep 22, 2024
05c4c1c
Use bit shifts instead of `Math.pow(2, n)`
jarmuszz Sep 22, 2024
fce1083
Use `.redeem` in msgpack validation spec
jarmuszz Sep 23, 2024
989ec8a
Use binary data in msgpack serializer benchmark
jarmuszz Oct 27, 2024
59d5e3f
Use binary data in msgpack parser benchmark
jarmuszz Oct 30, 2024
9e787e5
Merge branch 'gnieh:main' into main
jarmuszz Nov 15, 2024
dad4569
Make Out wrapper class an AnyVal
ybasket Jan 27, 2025
28da537
Use expect to simplify test case
ybasket Jan 27, 2025
f557332
Make overloaded test helpers DRY through delegation
ybasket Jan 27, 2025
9f5c52d
Make Out no longer an AnyVal
ybasket Jan 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use binary data in msgpack serializer benchmark
instead of relying on conversion from hex representation via scodec.

This also makes it easier to load input data into a properly chunked stream.
  • Loading branch information
jarmuszz committed Oct 27, 2024
commit 989ec8aeda017eff4393428477cad6dbcf7df1e0
Binary file not shown.
1 change: 0 additions & 1 deletion benchmarks/src/main/resources/twitter_msgpack.txt

This file was deleted.

Original file line number Diff line number Diff line change
@@ -32,38 +32,28 @@ import fs2._
@Warmup(iterations = 3, time = 2)
@Measurement(iterations = 10, time = 2)
class MsgPackItemSerializerBenchmarks {
val msgpackItems: List[fs2.data.msgpack.low.MsgpackItem] = {
val bytes =
fs2.io
.readClassLoaderResource[SyncIO]("twitter_msgpack.txt", 4096)
.through(fs2.text.utf8.decode)
.compile
.string
.map(ByteVector.fromHex(_).get)
.unsafeRunSync()

Stream
.chunk(Chunk.byteVector(bytes))
val msgpackItems: Stream[SyncIO, fs2.data.msgpack.low.MsgpackItem] =
fs2.io
.readClassLoaderResource[SyncIO]("twitter_msgpack.mp", 4096)
.through(fs2.data.msgpack.low.items[SyncIO])
.chunks
.compile
.toList
.unsafeRunSync()
}

.map(Stream.chunk)
.fold(Stream.empty)(_ ++ _)

@Benchmark
def serialize() =
Stream
.emits(msgpackItems)
msgpackItems
.through(fs2.data.msgpack.low.toNonValidatedBinary[SyncIO])
.compile
.drain
.unsafeRunSync()

@Benchmark
def withValidation() =
Stream
.emits(msgpackItems)
msgpackItems
.through(fs2.data.msgpack.low.toBinary[SyncIO])
.compile
.drain
Loading