REST Catalog does not validate "to" identifier on rename table
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Query engine
Spark
### Question
# Background
Spark will pass `catalog` name to `renameTable` operations as part of its `to` identifier, and if that `catalog` name is not handled (i.e. stripped), then it will be treated as part of the `namespace` name. This will cause spark to rename the table into undesirable namespaces (or even just encounter namespace not exist issue and fail).
Traditional catalog like HiveCatalog has special pre-validation in renameTable operation for this purpose (ref: https://github.com/apache/iceberg/pull/1156), while RESTCatalog and JdbcCatalog (ref REST server implementation is adapter on JdbcCatalog) does not have it. This is causing spark command like `ALTER TABLE ${tbl} RENAME TO ${tbl_rename}` to fail on RESTCatalog/JdbcCatalog such as in this integration test: https://github.com/apache/iceberg/blob/316f0a1/spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestAlterTable.java#L296
Part of https://github.com/apache/iceberg/issues/11079
# Questions
1. Should RESTCatalog (client) validate `to` identifier in `renameTable` operation just like what HiveCatalog is doing?
2. If RESTCatalog, like hive, always strip the first level of namespace for `to` identifier, then what if the intention is to rename to a legitimate multi-level namespace?
Contributor guide
Research direction
Start with the linked TestAlterTable.java integration case and compare renameTable handling in HiveCatalog, RESTCatalog, and JdbcCatalog. Determine the intended treatment of the catalog component in the to identifier, including legitimate multi-level namespaces; done means the chosen behavior is implemented and covered by the relevant integration test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100