Skip to content

test: build large uploads in one piece - #362

Merged
dimdenGD merged 2 commits into
dimdenGD:mainfrom
nigrosimone:fix-multer-test-slow-file
Jul 31, 2026
Merged

test: build large uploads in one piece#362
dimdenGD merged 2 commits into
dimdenGD:mainfrom
nigrosimone:fix-multer-test-slow-file

Conversation

@nigrosimone

@nigrosimone nigrosimone commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Two tests build their upload out of one blob part per byte, which is slow enough to trip a uWS limit and makes them fail depending on how fast the machine is.

multer

const bigFile = new File(new Array(Math.floor(1024 * 1024 * 0.2)).fill(0), "big.txt");

new Array(209715).fill(0) is an array of 209715 zeros, and File treats each one as its own blob part. So this builds a file out of 209715 parts rather than 209715 bytes, and undici needs about 19 seconds to encode that as a multipart body on my machine.

express-fileupload with temp file

const arr = new Uint8Array(1024 * 128);
const file = new File(arr, "test.txt");

fileBits is an iterable of blob parts, so handing it a Uint8Array directly iterates the buffer and stringifies every byte into its own part. The intended 128kb of binary became roughly 337kb of decimal digits across 131072 parts.

Why they fail

uWS wants a request body to keep arriving at 16KB/s or better: HttpContext.h only resets the 10 second idle timeout after another HTTP_RECEIVE_THROUGHPUT_BYTES * HTTP_IDLE_TIMEOUT_S bytes, which is 160KB. Neither upload is anywhere near that rate, so the socket is reset while the client is still sending and the test fails with ECONNRESET or UND_ERR_SOCKET.

Whether they pass comes down to how fast the machine can serialise all those parts, which is why they are green on CI and red here: multer failed 6 times out of 6 locally, and the run that did pass took 51 seconds.

Reduced to just the timing, with no upload machinery involved:

result
ultimate-express, body stalled 5s 200
ultimate-express, body stalled 12s fails after 11.8s
Express 4, body stalled 12s 200 after 12.0s

Fix

Pass a single part in both cases. Same bytes on the wire, assertions untouched, and both tests now upload what they claim to: multer goes from ~51s to under a second, express-fileupload from ~51s to ~0.5s, and their output matches Express 4 and Express 5 exactly.

The 16KB/s floor itself is not a bug, it is uWS refusing to hold sockets open for slow clients. #363 documents it.

new Array(209715).fill(0) makes a File out of 209715 separate blob parts
rather than 209715 bytes, and encoding that as multipart takes about 19
seconds here. uWS wants request bodies to arrive at 16kb/s or better and
resets the connection after 10 idle seconds, so the upload died with
ECONNRESET before the client had finished sending it.

Whether the test passes therefore depends on how fast the machine can
serialise those parts, which is why it goes green on CI and red locally.

A single Uint8Array uploads the same 209715 bytes, keeps the assertion
unchanged, and takes the test from ~51s to under a second.
new File(arr, ...) takes fileBits as an iterable of blob parts, so
passing a Uint8Array directly iterates it and turns every byte into its
own part, each stringified. The 128kb buffer became about 337kb of
decimal digits spread over 131072 parts, which is slow enough to trip
the same 16kb/s limit as the multer test.

Wrapping it uploads the intended 128kb of binary, and the test goes from
51 seconds to under one.
@nigrosimone nigrosimone changed the title test: build the large multer upload in one piece test: build large uploads in one piece Jul 31, 2026
@dimdenGD
dimdenGD merged commit 77ca8a2 into dimdenGD:main Jul 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants