Iceberg parquet data file 'created_by' field is empty

[PingLiuPing]

Describe the problem you faced

• A clear and concise description of the problem.
• Was this working before or is this a first try?
• If this worked before, then what has changed that recently?
• Provide table DDLs and EXPLAIN ANALYZE for your Presto queries.

When using Presto iceberg connector and create a iceberg table with parquet format. After insert some data I checked the parquet data file metadata by parquet-tools inspect and found that the created_by field is empty.

Is this works as design?
This does not affect anything though, just other engines do write this field.
Such as Trino and Spark.

Environment Description

• Presto version used:
• Storage (HDFS/S3/GCS..):
• Data source and connectors or catalogs used:
• Deployment (Cloud or On-prem):
• Pastebin link to the complete debug logs:

Steps To Reproduce

Steps to reproduce the behavior:
1.
2.
3.
4.

Expected behavior

Additional context

Stacktrace

[PingLiuPing]

Trino:
created_by = parquet-mr-trino version testversion (build n/a)

[PingLiuPing]

Spark:
created_by = parquet-mr version 1.13.1 (build db4183109d5b734ec5930d870cdae161e408ddba)

[imjalpreet]

I have made this change as part of my ongoing Hive 4.x Upgrade PR. After the upgrade, this change became necessary in certain cases, which is why I introduced it. You can find it added here:

https://github.com/imjalpreet/presto/blob/c0b40b52c90c6617ae16ee9d85b2463d205b527e/presto-parquet/src/main/java/com/facebook/presto/parquet/writer/ParquetWriter.java#L281-L286

[PingLiuPing]

@imjalpreet , thanks.
Do you know the history of why this field is empty at the begining?
And I also noticed that presto parquet writer doesn't write column_orders. Is this because of parquet version in Presto?

[imjalpreet]

Do you know the history of why this field is empty at the begining?

Most likely because it wasn’t a required field in the Parquet spec, so it wasn’t included in the initial implementation.

And I also noticed that presto parquet writer doesn't write column_orders. Is this because of parquet version in Presto?

It could be related to the Parquet version used when the native writer was first introduced, or simply because column_orders is optional and Presto’s writer never implemented it. That said, Presto has been on a Parquet version supporting column_orders for at least 4–5 years.

[PingLiuPing]

@imjalpreet Thanks.

Possibly Presto java might need to write column_orders too in future.

[PingLiuPing]

Will be resolved by https://github.com/prestodb/presto/pull/24571/files#diff-76147f4c95410c485cbd8a0bc4f4a7aaf6458bf17d822950ad06a9e12aa9ff62R285.