Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible memory issue #100

Open
r-teplov opened this issue Jan 14, 2020 · 2 comments
Open

Possible memory issue #100

r-teplov opened this issue Jan 14, 2020 · 2 comments

Comments

@r-teplov
Copy link

Hello.

There is possible issue while saving data with write function that could lead to high memory usage by Node.js process. uploadProgress function at https://github.com/sffc/socketio-file-upload/blob/master/server.js#L382 doesn't properly handle return value of FileStream write function.
According to Node documentaion here - https://nodejs.org/api/stream.html#stream_writable_write_chunk_encoding_callback
When write function returns false, futher attemts to write data should stop until drain event is emitted. Though data still written, Node buffers all written chunks that could cause issues uploading large files.

@sffc
Copy link
Owner

sffc commented Feb 10, 2020

Thanks for the report. Ultimately what's happening is that we are attempting to push data into the write stream as it gets sent to us from the socket.io connection. Typically network connections are much slower than file I/O, so my expectation is that the write buffer doesn't usually hit its high water mark. However, it would still be beneficial to properly handle this case.

What made you find this issue? Were you having performance problems and found this via debugging, or are you reporting it as a more theoretical problem?

@r-teplov
Copy link
Author

Hi.

I had some non relevant issues while saving files on my local machine. So i checked out source code and stumbled upon code mentioned above. I played with emitChunkFailed prop and it does really emitted error event which meant stream highWaterMark was hit. Same result for staging server.

I have not done any stress test for now so i cant confidently report this to be an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants