Skip to content

XTable performs incorrect schema sync during Delta to Iceberg conversion when a column is renamed in the Delta table. #711

@brishi19791

Description

@brishi19791

Search before asking

  • I had searched in the issues and found no similar issues.

Please describe the bug 🐞

I'm performing a Delta to Iceberg conversion using XTable. I renamed a column in the Delta table from "name" to "FullName". In the generated Iceberg schema, XTable removed the original "name" field (field ID: 2) and added a new field "FullName" with a new field ID: 5. Below is the Iceberg schema XTable generated:
"schemas" : [ {
"type" : "struct",
"schema-id" : 0,
"fields" : [ {
"id" : 1,
"name" : "id",
"required" : false,
"type" : "int"
}, {
"id" : 2,
"name" : "name",
"required" : false,
"type" : "string"
}, {
"id" : 3,
"name" : "role",
"required" : false,
"type" : "string"
}, {
"id" : 4,
"name" : "address",
"required" : false,
"type" : "string"
} ]
}, {
"type" : "struct",
"schema-id" : 1,
"fields" : [ {
"id" : 1,
"name" : "id",
"required" : false,
"type" : "int"
}, {
"id" : 3,
"name" : "role",
"required" : false,
"type" : "string"
}, {
"id" : 4,
"name" : "address",
"required" : false,
"type" : "string"
}, {
"id" : 5,
"name" : "FullName",
"required" : false,
"type" : "string"
} ]
} ]

XTable should have retained the original field ID (2) and simply updated the column name to "FullName" to reflect a rename, not a new field addition.

Are you willing to submit PR?

  • I am willing to submit a PR!
  • I am willing to submit a PR but need help getting started!

Code of Conduct

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions