Federate to an Iceberg JDBC catalog (new JDBC connection type)
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 140
Description
### Is your feature request related to a problem? Please describe.
**Requirement**
Polaris supports federating to Hadoop, Hive, and BigQuery metastore catalogs, but not to an
Iceberg JDBC catalog. Many organizations keep Iceberg metadata in a relational database (commonly
PostgreSQL) via Iceberg's `JdbcCatalog`. We would like Polaris to federate to such a catalog so the
database remains the source of truth for table metadata while Polaris brokers access, policies, and
multi-engine connectivity — consistent with the existing federation types.
**Why**
- JDBC/relational Iceberg catalogs are a common deployment; there is currently no way to bring them
under Polaris governance without migration.
- It rounds out the federation extension family (hadoop/hive/bigquery → +jdbc).
**Out of scope (separate proposal)**
- Sub-catalog (namespace/table-level) RBAC over federated JDBC catalogs.
**Environment:** Apache Polaris `main`; JDK 21.
### Describe the solution you'd like
**Proposed behavior (for discussion)**
- A new `JDBC` connection type and an optional `polaris-extensions-federation-jdbc` extension
(enabled with `-PNonRESTCatalogs=JDBC`), mirroring the existing federation extensions.
- A `PASSWORD` connection authentication type (username + password), with the password offloaded to
the Polaris user secrets manager (never stored inline), plus support for `IMPLICIT` auth.
- Driver-agnostic: no bundled JDBC driver; drivers come from the deployment classpath.
### Describe alternatives you've considered
* Migrate JDBC-catalog metadata into Polaris-managed Iceberg catalogs — rejected: requires a one-time migration and gives up the existing DB as source of truth, which defeats the point of federation.
* Front the JDBC catalog with an Iceberg REST catalog and use existing REST federation — viable for some, but adds an extra service to deploy/operate and doesn't help orgs that only run JdbcCatalog.
* Bundle common JDBC drivers in the extension — rejected in favor of sourcing drivers from the deployment classpath, to avoid license/version-pinning issues.
### Additional context
I have a working implementation of this along the lines described and am happy to open a PR / discuss the design if there's interest. Related existing federation connection types: Hadoop, Hive, BigQuery.
Contributor guide
Research direction
Start by reviewing the existing Hadoop, Hive, and BigQuery federation connection types and the proposed polaris-extensions-federation-jdbc extension. Examine how connection authentication and user secrets are handled, then define the JDBC connection and PASSWORD/IMPLICIT behavior without bundling a driver. Done means JDBC federation is supported consistently with the existing federation extensions and the out-of-scope RBAC limitation remains clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100