Iceberg View Support
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
> We haven't talked about [Iceberg Views](https://iceberg.apache.org/view-spec/) yet and I'm not sure if you would like to support them in the future. I'm actually really interested in using views.
>
> I don't want to include view support in this PR but I would like to future proof the catalog design to easily support views in the future.
>
> The REST catalog is in the [process](https://github.com/apache/iceberg/pull/7913) of adding view support. However, I have the feeling that the initial REST catalog API wasn't designed with views in mind. There is one issue that comes up with the current design that we might be able to avoid for our catalog design. I will try to explain the issue.
>
> Imagine a query engine wants to perform a query like the following:
>
> ```sql
> SELECT first_name, last_name, age FROM users WHERE age > 18;
> ```
>
> Without the catalog information the query engine doesn't know whether `users` refers to a view or a table. So if the catalog only exposes a `load_table` and `load_view` operation, the query engine doesn't know which one to call. I would prefer a `load_tablelike` (the naming is not important) operation that returns an
>
> ```rust
> enum TableLike {
> Table(Table),
> View(View)
> }
> ```
>
> All iceberg catalogs except for the REST catalog return a "table metadata location" and it would be easy to distinguish between Tables and Views based on the metadata.
>
> Let me know what you think.
_Originally posted by @JanKaul in https://github.com/apache/iceberg-rust/issues/54#issuecomment-1702390543_
Contributor guide
Research direction
Start by reading the Iceberg View Spec linked in the issue and the referenced REST catalog view-support pull request. Then inspect the catalog API design and the discussion to determine whether a future-proof table-or-view operation is wanted; there is currently no named file, test, or agreed implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100