Support hierarchical namespaces in Datafusion integration
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Is your feature request related to a problem or challenge?
Currently, while Iceberg and iceberg-rust support hierarchical namespaces, the Datafusion integration in iceberg-rust [only makes tables in top-level namespaces](https://github.com/apache/iceberg-rust/blob/99ca196a7a81343ac35d993d9db2ec59c626f389/crates/integrations/datafusion/src/catalog.rs#L54) available. My use case includes tenancy and fairly complex schemas with thousands of tables, so our lake is quite much nested.
### Describe the solution you'd like
A fairly simple addition to the IcebergCatalogProvider can make deeply nested tables using a dot notation (i.e., the namespace is presented as `parent.child.…` and is available for querying using `select … from catalog."parent.child".table`). This is [apparently also how the same use case is solved in Datafusion itself](https://github.com/apache/datafusion/issues/13822) currently.
### Willingness to contribute
I can contribute to this feature independently
Contributor guide
Assessment
This issue has not been assessed yet.