apache / apache/incubator-xtable
XTable performs incorrect schema sync during Delta to Iceberg conversion when a column is renamed in the Delta table.
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 212
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 16
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-xtable/issues?q=is%3Aissue) and found no similar issues.
### Please describe the bug 🐞
I'm performing a Delta to Iceberg conversion using XTable. I renamed a column in the Delta table from "name" to "FullName". In the generated Iceberg schema, XTable removed the original "name" field (field ID: 2) and added a new field "FullName" with a new field ID: 5. Below is the Iceberg schema XTable generated:
"schemas" : [ {
"type" : "struct",
"schema-id" : 0,
"fields" : [ {
"id" : 1,
"name" : "id",
"required" : false,
"type" : "int"
}, {
"id" : 2,
"name" : "name",
"required" : false,
"type" : "string"
}, {
"id" : 3,
"name" : "role",
"required" : false,
"type" : "string"
}, {
"id" : 4,
"name" : "address",
"required" : false,
"type" : "string"
} ]
}, {
"type" : "struct",
"schema-id" : 1,
"fields" : [ {
"id" : 1,
"name" : "id",
"required" : false,
"type" : "int"
}, {
"id" : 3,
"name" : "role",
"required" : false,
"type" : "string"
}, {
"id" : 4,
"name" : "address",
"required" : false,
"type" : "string"
}, {
"id" : 5,
"name" : "**FullName**",
"required" : false,
"type" : "string"
} ]
} ]
XTable should have retained the original field ID (2) and simply updated the column name to "FullName" to reflect a rename, not a new field addition.
### Are you willing to submit PR?
- [x] I am willing to submit a PR!
- [ ] I am willing to submit a PR but need help getting started!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.