apache / apache/iceberg-rust

Support hierarchical namespaces in Datafusion integration

Open
#1,976 1 comment 0 reactions 1 assignee Claimed by @eickler View on GitHub
enhancement
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Is your feature request related to a problem or challenge?

Currently, while Iceberg and iceberg-rust support hierarchical namespaces, the Datafusion integration in iceberg-rust [only makes tables in top-level namespaces](https://github.com/apache/iceberg-rust/blob/99ca196a7a81343ac35d993d9db2ec59c626f389/crates/integrations/datafusion/src/catalog.rs#L54) available. My use case includes tenancy and fairly complex schemas with thousands of tables, so our lake is quite much nested.

### Describe the solution you'd like

A fairly simple addition to the IcebergCatalogProvider can make deeply nested tables using a dot notation (i.e., the namespace is presented as `parent.child.…` and is available for querying using `select … from catalog."parent.child".table`). This is [apparently also how the same use case is solved in Datafusion itself](https://github.com/apache/datafusion/issues/13822) currently.

### Willingness to contribute

I can contribute to this feature independently

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.