Reaching max memory when running as lambda layer

[changma2]

Hi there,

We have mediainfo binary running in a lambda, and the file we are processing is around 7GB with extension .mov, the file is accessed via presigned URL, our lambda is having 8GB memory configured, but the handler is throwing runtime error saying memory used up. We tested locally and monitoring the memory used for this specific file and it is using around 12 GB to extract tech metadata out of this file. Questions we want to ask are:

1. For most of the files we processed with mediainfo, it just fetch the header info, which is relatively small to the file, but for this one, seems like it fetch the whole file content, so we are wondering In which cases, mediainfo will need to fetch the complete file content in order to get the needed tech metadata?
2. Since lambda can only config memory up to 10GB, is there a way to control the file size that mediainfo used to get tech metadata?
3. What could be the extra memory used here, since the file is just 7GB, it's understandable if memory is a little higher, but 12GB seems too much?

Looking forward to hearing from you.

Bests,
Sean

[JeromeMartinez]

so we are wondering In which cases, mediainfo will need to fetch the complete file content in order to get the needed tech metadata?

In theory, never.
Please provide a URL (or the file) so we could test on our side and find the bug.
drop us an email at [email protected] if it can not be public.

[changma2]

Hi Jerome, thanks for the reply, I will discuss with the team on how can we securely share the file with you to do further investigation, will keep you posted!