Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streams memory leak #12198

Open
oguimbal opened this issue Jun 27, 2024 · 0 comments
Open

Streams memory leak #12198

oguimbal opened this issue Jun 27, 2024 · 0 comments
Labels
bug Something isn't working needs triage

Comments

@oguimbal
Copy link
Contributor

oguimbal commented Jun 27, 2024

What version of Bun is running?

1.1.17+bb66bba1b

What platform is your computer?

Darwin 23.1.0 arm64 arm

What steps can reproduce the bug?

Execute the below code, and watch memory usage:

setInterval(() => {
    Bun.gc(true);
}, 100);

const p = Bun.spawn(['cat'], {
    stdin: 'pipe',
    stdout: 'pipe',
})


const r = p.stdout.getReader()
while (true) {
    const buf = new Uint8Array(1_000_000);
    await p.stdin.write(buf);

    let i = 0;
    while (true) {
        const { value } = await r.read();
        i += value?.length ?? 0;
        if (i >= buf.length) {
            break;
        }
    }
}

What is the expected behavior?

Each iteration just wait until the buffer is streamed through cat, so I'd expect memory not to increase.

What do you see instead?

CPU 100%, memory increases rapidly.

Interestingly enough, when every byte has been streamed, if I add another .read() (which rightfuly returns no data), the memory leak disapears:

setInterval(() => {
    Bun.gc(true);
}, 100);

const p = Bun.spawn(['cat'], {
    stdin: 'pipe',
    stdout: 'pipe',
})


const r = p.stdout.getReader()
while (true) {
    const buf = new Uint8Array(1_000_000);
    await p.stdin.write(buf);

    let i = 0;
    while (true) {
        const { value } = await r.read();
        i += value?.length ?? 0;
        if (i >= buf.length) {
           // 👉  this fixes the leak ! (and never throws)
            const { value: value2 } = await r.read();
            if (value2?.length) {
                throw new Error('Expected EOF');
            }
            break;
        }
    }
}

Additional information

This issue is kind of linked to #12194 (which I discovered investigating this leak)

@oguimbal oguimbal added bug Something isn't working needs triage labels Jun 27, 2024
@oguimbal oguimbal changed the title FileSink => memory leak Streams memory leak Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage
Projects
None yet
Development

No branches or pull requests

1 participant