apache / apache/polaris

Federation of Catalog-level entities to remote catalogs

Open
#540 1 comment 0 reactions 1 assignee Assigned to @dennishuo View on GitHub
proposal
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 17h
Merged PRs (30d)
137

Description

### Is your feature request related to a problem? Please describe.

As Iceberg users sometimes find different Iceberg-catalog sources of truth being created organically over time, it can be difficult to consolidate or centralize workloads onto a new target architecture or onboard new catalog implementations like Polaris to unlock its RBAC features without disrupting ongoing workloads.

Being able to have one Catalog instance serve as a combined "pane of glass" serving the contents of a legacy Catalog alongside new datasets enables an incremental migration or even simply enabling different workloads to leverage different catalog features without a stop-the-world migration.

### Describe the solution you'd like

Read-through federation of Catalog entities (e.g. Namespaces, Tables, Views) would allow a Polaris instance to link in an existing remote Catalog as a source-of-truth while exposing a Polaris entry-point along with Polaris RBAC features without performing a disruptive migration of Catalog contents.

Ideally, this federation would require minimal setup of the initial catalog connection while still benefiting from all the fine-grained access control features expected from normal Polaris catalogs; it should intelligently pass through API operations but also allow defining privilege grants on entities without requiring prior setup/syncing of those entities into the Polaris federation layer.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.