Skip to content

Conversation

osztenkurden
Copy link

@osztenkurden osztenkurden commented Aug 22, 2025

Reference: #279
Use case for user-provider Uint8Array is mostly when decompressing a lot of packages one after the other. I also a bit over-used napi-rs type overwrite, to create better return type.

Benchmark table:

┌─────────┬────────────────────────────────┬───────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ (index) │ Task name                      │ Latency avg (ns)  │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├─────────┼────────────────────────────────┼───────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0       │ 'snappy-uncompress'            │ '97095 ± 1.71%'   │ '81500 ± 10200'  │ '11592 ± 0.43%'        │ '12270 ± 1696'         │ 10300   │
│ 1       │ 'snappy-alloc-uncompress'      │ '90846 ± 1.12%'   │ '67400 ± 11900'  │ '13294 ± 0.65%'        │ '14837 ± 3020'         │ 11009   │
│ 2       │ 'snappy-sync-uncompress'       │ '96091 ± 1.17%'   │ '98100 ± 12100'  │ '11681 ± 0.56%'        │ '10194 ± 1160'         │ 10407   │
│ 3       │ 'snappy-sync-alloc-uncompress' │ '44030 ± 1.25%'   │ '39600 ± 1500.1' │ '24331 ± 0.13%'        │ '25252 ± 994'          │ 22712   │
│ 4       │ 'snappy-v6-uncompress'         │ '290066 ± 23.38%' │ '248150 ± 37450' │ '4567 ± 1.21%'         │ '4030 ± 541'           │ 3448    │
│ 5       │ 'gzip-uncompress'              │ '510540 ± 1.25%'  │ '491400 ± 61700' │ '2036 ± 0.72%'         │ '2035 ± 283'           │ 1959    │
│ 6       │ 'deflate-uncompress'           │ '459887 ± 1.06%'  │ '413900 ± 14700' │ '2241 ± 0.59%'         │ '2416 ± 89'            │ 2175    │
│ 7       │ 'brotli-uncompress'            │ '562070 ± 0.86%'  │ '518250 ± 19850' │ '1818 ± 0.59%'         │ '1930 ± 76'            │ 1780    │
└─────────┴────────────────────────────────┴───────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘

The biggest improvement we see from uncompress-sync to sync-alloc-uncompress

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for user-provided output buffers in decompression functions to improve performance when processing multiple packages sequentially. It also enhances TypeScript type definitions with detailed overloads for better type safety.

  • Added output field to DecOptions for user-provided Uint8Array buffers
  • Updated return types to support returning decompressed length when using provided buffers
  • Enhanced TypeScript type definitions with comprehensive function overloads

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/lib.rs Core implementation of user-provided buffer support and improved type definitions
benchmark/bench.ts Added benchmarks to test performance improvements with user-provided buffers

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

if let Some(ref mut opts) = self.options {
if let Some(ref mut output_buffer) = opts.output {
let decompressed_len = self
.inner
Copy link
Preview

Copilot AI Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of unsafe to get a mutable reference to the output buffer is potentially dangerous. Consider using safe alternatives or adding proper safety documentation explaining why this unsafe block is necessary and what invariants must be maintained.

Copilot uses AI. Check for mistakes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Brooooooklyn a question from me - I see no other way than using the unsafe in this small part. Would adding:

          // SAFETY: We know the buffer is valid for the lifetime of this function
          // and we're not extending beyond its bounds

comment be sufficient enough? I might be missing something, as Im pretty new to Rust.

Repository owner deleted a comment from cursor bot Aug 29, 2025
Repository owner deleted a comment from cursor bot Aug 29, 2025
Repository owner deleted a comment from cursor bot Sep 11, 2025
@osztenkurden
Copy link
Author

@Brooooooklyn would you mind chiming on the unsafe usage here? I'm afraid there is no way around it to actually achieve the goal of the #279, but I don't want to resolve the cursor comment without direct approval 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants