apache / apache/polaris

[Proposal] Multi-DataSource Support via DataSourceResolver

Open
#4,032 3 comments 0 reactions 0 assignees View on GitHub
enhancement stale
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 22h
Merged PRs (30d)
137

Description

### Is your feature request related to a problem? Please describe.

Currently, Polaris JDBC uses a single global DataSource for all operations. This creates two issues: * **Internal Interference:** Heavy metrics/events traffic can saturate the connection pool and slow down core metadata operations. * **Deployment Rigidity:** There is no programmatic way to route specific realms/tenants to dedicated physical databases without separate JVM clusters.

### Describe the solution you'd like

Introduce a DataSourceResolverSPI that resolutions the correct DataSource based onRealmContextandStoreType (METASTORE, METRICS, EVENTS).
**Objectives:** * **Workload Isolation:** Enable dedicated connection pools for isolated telemetry and metadata workloads. * **Stateless Scaling:** Allow operators to dynamically shard or migrate tenants at runtime via a custom resolver.

### Describe alternatives you've considered

Separate JVM clusters (inefficient for multi-tenancy) or static configs (too rigid for dynamic sharding maps).

### Additional context

Foundation for PR #3960. It specifically resolves transactional consistency concerns by partitioning isolated workloads (Metrics/Events) that do not require cross-database joins.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing single-DataSource flow and PR #3960, then trace how RealmContext and the METASTORE, METRICS, and EVENTS store types are represented. Define the resolver contract and its integration points before implementing it. Done means callers can select isolated DataSources programmatically while preserving the stated transactional boundaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.