Excessive memory usage in Iceberg connector since version 468

[vburenin]

I just recently did an upgrade from Trino 466 to 470 and operation like 'show columns from' or 'show create table' for Iceberg table started to fail miserably.
The table itself has just several top level fields and one of the fields is a deeply nested data structure with around 7k projected fields.

After digging deeper, I discovered an excessive memory usage in Iceberg module:

It appears there is an array with 7k record where each record is 2.5MB on average.

I can't share a data structure details due to the sensitivity of the content, however, I hope there is something obvious.
This is a critical bug for us as it makes Trino cluster fully unusable.

[ebyhr]

@vburenin Can you confirm the behavior on 467, 468 and 469? We want to know the exact version we introduced the issue.

[vburenin]

Trino 468 has this issue.

[vburenin]

467 does not. So, Trino 468 it is where the issue starts to happen. And it seems like it is iteratively got worse at 470.

[vburenin]

The notable change in Trino 468 is Iceberg Upgrade to 1.7.1 from 1.7.0

[vburenin]

Downgraded Iceberg to 1.7.0 in Trino 470. The problem didn't go away.

[vburenin]

I have set iceberg.bucket-execution to false and the problem seems to be gone.

[ebyhr]

cc: @dain @raunaqmorarka