feat(iceberg): bump iceberg-rust to v0.8.0 and add write support #19200
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Upgrade iceberg-rust from v0.4.0 to v0.8.0 with breaking API changes - Add IcebergFileIO wrapper to adapt new FileIO API to OperatorRegistry trait - Update catalog builders to use new CatalogBuilder trait with load() method - Re-enable standalone_iceberg_tpch CI tests that were disabled for arm64 - Upgrade hive_metastore from v0.1.0 to v0.2.0 - Simplify generate_catalog_meta to generate_default_catalog_meta
github-actions
bot
added
the
pr-chore
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 827a035a51
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- Handle file:// and memory:// URIs that don't have a host/bucket - Fix gcs.project-id mapping to project_id (was incorrectly default_storage_class) - Use Error::other() to fix clippy io_other_error lint
- Basic insert with multiple rows - Multiple insert statements - Various data types (int, bigint, float, double, string, date, boolean) - NULL value handling - Partitioned table writes - INSERT SELECT from another iceberg table - Aggregation queries on inserted data
- Map iceberg 'table not found' errors to ErrorCode::UnknownTable - This allows DROP TABLE IF EXISTS to work correctly for iceberg tables - Fix column type annotation in base.test (ITI -> TAT) - Remove write tests since iceberg tables don't support INSERT yet
- Test CREATE TABLE with various types (int, bigint, double, string, date, boolean) - Test CREATE TABLE with partition by clause - Test INSERT statements (expected to fail with error 1002 since writes not yet supported) - Test DROP TABLE cleanup
- Implement IcebergDataFileWriter for writing data blocks to parquet files - Add IcebergCommitSink for committing data via Transaction API - Support both partitioned and non-partitioned table writes - Handle multi-field partitioning with FanoutWriter - Add type conversion from Databend scalars to Iceberg literals - Include cache invalidation after successful commits - Update tests to verify write functionality works correctly
sundy-li
changed the title
github-actions
bot
added
the
pr-feature
- Suppress deprecated as_slice/from_slice warnings in hash.rs and jwk.rs - Add missing rand::Rng import in sized_spsc.rs tests - Add allow for diverging_sub_expression in raft_state_machine_impl.rs
The alpine variant doesn't support ARM64 architecture.
The polygon ring starting point changed after dependency upgrade. The polygon is geometrically equivalent - same shape, different vertex order.
- Add astral-sh/setup-uv@v7 to iceberg tpch test action - Update float scientific notation format (e308 -> e+308) in tests
KKould
reviewed
Jan 9, 2026
KKould
reviewed
Jan 9, 2026
KKould
reviewed
Jan 9, 2026
tests/sqllogictests/suites/query/functions/02_0060_function_geometry.test
Show resolved
Hide resolved
b41sh
reviewed
Jan 9, 2026
youngsofun
reviewed
Jan 9, 2026
youngsofun
approved these changes
Jan 9, 2026
b41sh
approved these changes
Jan 9, 2026
Member
|
@codex review |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
Upgrade iceberg-rust from v0.4.0 to v0.8.0 and implement write support for Iceberg tables.
Dependency Upgrades
iceberg-rustfrom v0.4.0 to v0.8.0 with breaking API changeshive_metastorefrom v0.1.0 to v0.2.0IcebergFileIOwrapper to adapt new FileIO API toOperatorRegistrytrait (iceberg-rust removed built-in support)CatalogBuildertrait withload()methodNew Feature: Iceberg Table Write Support
IcebergDataFileWriterfor writing data blocks to Parquet filesIcebergCommitSinkfor committing data via Transaction APIFanoutWriterOther Changes
standalone_iceberg_tpchCI tests that were temporarily disabled for arm64 compatibilitygenerate_catalog_metatogenerate_default_catalog_metaTests
Added comprehensive SQL logic tests for:
INSERT INTO)Type of change
This change is