apache / apache/kyuubi

[Bug] Iceberg ranger test case compatible error

Open
#7,070 15 comments 1 reaction 0 assignees View on GitHub
kind:bug priority:major
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.

### Describe the bug

Currently iceberg ranger test use hadoop type catalog to test the permission, but the hadoop type has some compatible questions with alter table rename command and create view command
test case
```
test("RENAME TABLE for Iceberg") {
val table = s"$catalogV2.$namespace1.partitioned_table"
val newTable = s"$catalogV2.$namespace1.renamed_table"
withCleanTmpResources(Seq((table, "table"), (newTable, "table"))) {
doAs(
admin,
sql(
s"CREATE TABLE $table (id int NOT NULL, name string, city string) USING iceberg"))
val renameTableSql = s"ALTER TABLE $table RENAME TO $newTable"
interceptEndsWith[AccessControlException] {
doAs(someone, sql(renameTableSql))
}(s"does not have [alter] privilege on [$namespace1/partitioned_table]")
doAs(admin, sql(renameTableSql))
}
}

test("CREATE VIEW for Iceberg") {
val table = s"$catalogV2.$namespace1.partitioned_table"
val view = s"$catalogV2.$namespace1.test_view"
withCleanTmpResources(Seq((table, "table"), (view, "view"))) {
doAs(
admin,
sql(
s"CREATE TABLE $table (id int NOT NULL, name string, city string) USING iceberg"))
doAs(admin, sql(s"INSERT INTO $table VALUES (1, 'test', 'city')"))
val createViewSql = s"CREATE VIEW $view AS SELECT * FROM $table"
interceptEndsWith[AccessControlException] {
doAs(someone, sql(createViewSql))
}(s"does not have [create] privilege on [$namespace1]")
doAs(admin, sql(createViewSql))
}
}
```

### Affects Version(s)

master

### Kyuubi Server Log Output

```logtalk
# Hadoop Type Catalog
Cannot rename Hadoop tables
java.lang.UnsupportedOperationException: Cannot rename Hadoop tables
---------------------------
Creating a view is not supported by catalog: local
java.lang.UnsupportedOperationException: Creating a view is not supported by catalog: local

# cancel Hadoop Type Catalog, use hive, code could see additional context
org.apache.iceberg.hive.RuntimeMetaException: Failed to connect to Hive Metastore
....
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
....
Caused by: MetaException(message:Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=/var/folders/dz/wcf2hy112t1_zw142j3lqr6c0000gn/T/hms-f1c6c83d-7435-4f0d-9641-7dbb392eee87;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
java.sql.SQLException: Failed to start database '/var/folders/dz/wcf2hy112t1_zw142j3lqr6c0000gn/T/hms-f1c6c83d-7435-4f0d-9641-7dbb392eee87' with class loader sun.misc.Launcher$AppClassLoader@18b4aac2, see the next exception for details.
....
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /private/var/folders/dz/wcf2hy112t1_zw142j3lqr6c0000gn/T/hms-f1c6c83d-7435-4f0d-9641-7dbb392eee87.
```

### Kyuubi Engine Log Output

```logtalk

```

### Kyuubi Server Configurations

```yaml

```

### Kyuubi Engine Configurations

```yaml

```

### Additional context

remove the hadoop
```
override def beforeAll(): Unit = {
spark.conf.set(
s"spark.sql.catalog.$catalogV2",
"org.apache.iceberg.spark.SparkCatalog")
spark.conf.set(
s"spark.sql.catalog.$catalogV2.warehouse",
Utils.createTempDir("iceberg-hadoop").toString)
```

### Are you willing to submit PR?

- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [x] No. I cannot submit a PR at this time.

Contributor guide

Open the contributing guide

Research direction

Start with the Iceberg Ranger tests named “RENAME TABLE for Iceberg” and “CREATE VIEW for Iceberg,” then inspect their beforeAll setup and the spark.sql.catalog.$catalogV2 configuration. Run these tests and compare the Hadoop, Hive, and local catalog behavior. Done means the permission assertions pass without the reported rename and view compatibility failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.