GenerateContentAsync - OutOfMemoryException

[ermak-smartcat]

Bug: OutOfMemoryException in GenerateContentResponse fromVertex aI API when response contains large inline image data

Environment

Detail	Value
SDK	Google.GenAI (NuGet)
SDK Version	1.2.0 (also likely reproducible on 1.3.0)
Runtime	.NET 8.0 (Linux container)
API	Vertex AI (vertexAI: true)
Model	gemini-3.1-flash-image-preview
Response modality	Image (inline data)

Container
replicas: 1
resources:
requests:
cpu: 1000m
memory: 896Mi
limits:
cpu: 1200m
memory: 1024Mi

Summary

GenerateContentAsync throws System.OutOfMemoryException when the Vertex AI response contains a large image (PNG) as base64 inline data. The exception originates inside the SDK's response deserialization code (GenerateContentResponseFromVertex), which internally calls JsonNode.ToJsonString() on the entire response payload, triggering a massive contiguous memory allocation for the JSON-escaped base64 string.

This is not an issue with the caller's code or the request payload — the OOM occurs after the HTTP response has been received, during the SDK's internal JSON transformation of the response.

This issue can be reproduced under the load with a few images ~4-6 with resolution about 2k webp. For only one image it works fine.

Steps to Reproduce

1. Create a Vertex AI client:

var client = new Client(
    project: "my-project",
    location: "global",
    vertexAI: true,
    httpOptions: new HttpOptions { Timeout = 120000, BaseUrl = "https://..." },
    credential: GoogleCredential.FromAccessToken("...")
);

2. Call GenerateContentAsync requesting an image response with a high resolution:

var response = await client.Models.GenerateContentAsync(
    model: "gemini-3.1-flash-image-preview",
    contents:
    [
        new Content
        {
            Role = "user",
            Parts =
            [
                new Part
                {
                    FileData = new FileData
                    {
                        FileUri = "https://signed-s3-url-to-source-image.webp",
                        MimeType = "image/webp"
                    }
                }
            ]
        }
    ],
    config: new GenerateContentConfig
    {
        SystemInstruction = new Content
        {
            Parts = [new Part { Text = "Replace text in this image with another text..." }]
        },
        ResponseModalities = [Modality.Image.ToString()],
        ImageConfig = new ImageConfig
        {
            ImageSize = "4K",       // or "2K" — larger sizes increase OOM probability
            AspectRatio = "16:9",
        },
        Temperature = 0.0,
        Seed = 42,
    }
);

3. The call throws OutOfMemoryException during response processing.

Key factor: The larger the requested ImageSize, the larger the base64-encoded PNG in the response, and the more likely the OOM. With "4K" the response image can be 10–20 MB raw PNG, ~25–30 MB base64.

Full Stack Trace

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.GC.AllocateNewArray(IntPtr typeHandle, Int32 length, GC_ALLOC_FLAGS flags)
   at System.Buffers.SharedArrayPool`1.Rent(Int32 minimumLength)
   at System.Text.Json.Utf8JsonWriter.WriteStringEscapeValue(ReadOnlySpan`1 utf8Value, Int32 firstEscapeIndexVal)
   at System.Text.Json.JsonDocument.WriteString(DbRow& row, Utf8JsonWriter writer)
   at System.Text.Json.JsonDocument.WriteComplexElement(Int32 index, Utf8JsonWriter writer)
   at System.Text.Json.JsonDocument.WriteElementTo(Int32 index, Utf8JsonWriter writer)
   at System.Text.Json.Nodes.JsonArray.WriteTo(Utf8JsonWriter writer, JsonSerializerOptions options)
   at System.Text.Json.Nodes.JsonNode.ToJsonString(JsonSerializerOptions options)
   at Google.GenAI.Common.ToJsonNode(Object value)
   at Google.GenAI.Common.SetValueByPath(JsonObject jsonObject, String[] path, Object value)
   at Google.GenAI.Models.GenerateContentResponseFromVertex(JsonNode fromObject, JsonObject parentObject, JsonNode rootObject)
   at Google.GenAI.Models.PrivateGenerateContentAsync(String model, List`1 contents, GenerateContentConfig config, CancellationToken cancellationToken)
   at Google.GenAI.Models.GenerateContentAsync(String model, List`1 contents, GenerateContentConfig config, CancellationToken cancellationToken)