Vertx v2.1.0 running out of memory for 50mb file concurrent file uploads.

[iamakashk]

Hi

I have use case where I upload 50 mb of zip file with lots of images. While testing this functionality when 3 or more users uploads 50mb size zip file for the further operation gateway goes out of memory and stops. Is there any way we can solve this issue ?

Current vertx gateway memory configuration is -Xms1024m -Xmx1024m . Here is the grafana dashboard for memory consumption of the docker container.

[iamakashk]

Hi @msavy Could you please provide any suggestions here ?

[msavy]

I would suggest you run a java flight recorder session and see what's actually happening in terms of heap usage when you try to proxy those requests.

• What's in the policy chain for the request(s)?
• What is the Transfer-Encoding you are using?

I took a week-long break as I've been working non-stop for a very extended period :-).

[msavy]

Made any progress on this?

[iamakashk]

Hi @msavy

I could not spend much time on this. I have planned this in coming days. Will try out with java flight recorder as you suggested and get back to you.

[msavy]

Cool, good luck. Interested to see what you discover.

It may be that the consumer (backend API) is slower than the gateway so a lot of data ends up being buffered in memory, but the Vert.x Gateway uses Vert'x backpressure mechanism which usually ensures that you can't flood/bloat the gateway when the consumer is slow.

It could also be an interaction with one of the policies that you're using in combination with a slow consumer.

[iamakashk]

Made any progress on this?

Hi @msavy ,
Long time no see 😅😅! I hope you are doing well.
I managed to spend some time on this issue and figured out vertx was not getting OOM while upload process but it was getting OOM due to download process which was taking huge amount of time than expected. This process is now streamlined and download ( process ) time is within limit and I am no longer getting this issue now.