New features:
- New filters
isNull
andisNotNull
to filter by missing values. custom
function added to FS2'sviaParquet
to use custom Parquet builders, e.g. to write a stream of Avro files to Parquet.
Improvements and bug fixes:
- Improved file rotation by
maxDuration
in FS2'sviaParquet
by giving it a higher priority than regular writes. Thanks to that, rotation by time happens immediately instead of waiting until the pending backlog of data is written first. - Improved custom readers in Akka/Pekko and FS2s to better accommodate issues with reading Java Protobuf from Parquet.
Updates:
- Scala:
- 2.12 to 2.12.20
- 2.13 to 2.13.15
- 3.3 to 3.3.4
- Parquet to 1.14.3
- Cats Effect to 3.5.5
- FS2 to 3.11.0
- Pekko to 1.1.2