[Spark Integration Tests] TestCreateTable::testCreateTableCommitProperties won't work on RESTCatalog
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
None
### Query engine
None
### Please describe the bug 🐞
Part of: https://github.com/apache/iceberg/issues/11079
## Intro
The test `TestCreateTable::testCreateTableCommitProperties` will only work on Hadoop/Hive Catalog but not on REST Catalog. Code link: https://github.com/apache/iceberg/blame/0a705b0637db484730eb4eece69ae6c4d52fd9da/spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestCreateTable.java#L353
## Error message
```
TestCreateTable > testCreateTableCommitProperties() > catalogName = testrest, implementation = org.apache.iceberg.spark.SparkCatalog, config = {type=rest, cache-enabled=false, uri=http://localhost:34885/} FAILED
java.lang.AssertionError:
Expecting actual throwable to be an instance of:
org.apache.iceberg.exceptions.ValidationException
but was:
org.apache.iceberg.exceptions.BadRequestException: Malformed request: Table property commit.retry.num-retries must have integer value
at org.apache.iceberg.rest.ErrorHandlers$DefaultErrorHandler.accept(ErrorHandlers.java:208)
at org.apache.iceberg.rest.ErrorHandlers$TableErrorHandler.accept(ErrorHandlers.java:118)
at org.apache.iceberg.rest.ErrorHandlers$TableErrorHandler.accept(ErrorHandlers.java:102)
...(164 remaining lines not displayed - this can be changed with Assertions.setMaxStackTraceElementsDisplayed)
at org.apache.iceberg.spark.sql.TestCreateTable.testCreateTableCommitProperties(TestCreateTable.java:366)
```
## Root cause
REST client will throw `BadRequestException` (`HTTP 400`) where a `ValidationException` will be thrown by Hive/Hadoop Catalog in similar situation. REST client will simply categorize `ValidationException` as `HTTP 400` and throw `BadRequestException` instead. This is currently determined by REST's `ErrorHandlers`: https://github.com/apache/iceberg/blob/0a705b0637db484730eb4eece69ae6c4d52fd9da/core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java#L202
## Further consideration
Two options going forward:
1. Unify Hive/Hadoop/REST interface so all of them throw the same type of exception in the cases that this test is covering for
2. Accept that Hive/Hadoop/REST could throw different type of exception in these cases and modify the test itself
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
Contributor guide
Assessment
This issue has not been assessed yet.