Skip to content

Add support for LZ4 decompression in the IPC reader #727

@paleolimbot

Description

@paleolimbot

As reported in duckdb-nanoarrow: paleolimbot/duckdb-nanoarrow#13

ZSTD is implemented here: https://github.com/apache/arrow-nanoarrow/blob/bdb7d0cad6676bb93762f5e1ae6180303503457b/src/nanoarrow/ipc/codecs.c

...and the PR implementing ZSTD was #693

I think this should be fairly straightforward but the default ArrowSerialIpcDecompressor() might have an inefficient implementation for LZ4 (because it doesn't share decompression state across decompression requests).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions