apache / apache/polaris

Loading catalog config via IRC API results using multiple JDBC connections

Open
#5,087 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
2d 16m
Merged PRs (30d)
139

Description

A simple IRC operation like loading the catalog config apparently uses as many as 7 JDBC connections.

One (representative) stack trace:

```
java.lang.Throwable: sample
at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
at com.intellij.rt.debugger.MethodInvoker.invokeInternal(MethodInvoker.java:223)
at com.intellij.rt.debugger.MethodInvoker.invoke1(MethodInvoker.java:35)
at org.apache.polaris.persistence.relational.jdbc.DatasourceOperations.borrowConnection(DatasourceOperations.java:465)
at org.apache.polaris.persistence.relational.jdbc.DatasourceOperations.lambda$executeSelectOverStream$2(DatasourceOperations.java:176)
at org.apache.polaris.persistence.relational.jdbc.DatasourceOperations.withRetries(DatasourceOperations.java:389)
at org.apache.polaris.persistence.relational.jdbc.DatasourceOperations.executeSelectOverStream(DatasourceOperations.java:174)
at org.apache.polaris.persistence.relational.jdbc.DatasourceOperations.executeSelect(DatasourceOperations.java:155)
at org.apache.polaris.persistence.relational.jdbc.JdbcMetaStoreManagerFactory_ProducerMethod_produceDatasourceOperations__yn4Mb9y-33mIeOz3zQKnFmQpKU_ClientProxy.executeSelect(Unknown Source)
at org.apache.polaris.persistence.relational.jdbc.JdbcBasePersistenceImpl.lookupEntities(JdbcBasePersistenceImpl.java:484)
at org.apache.polaris.core.persistence.AtomicOperationMetaStoreManager.loadGrantsToGrantee(AtomicOperationMetaStoreManager.java:1416)
at org.apache.polaris.core.persistence.AtomicOperationMetaStoreManager.loadGrantsToGrantee(AtomicOperationMetaStoreManager.java:1384)
at org.apache.polaris.core.persistence.ServiceProducers_ProducerMethod_polarisMetaStoreManager_2QgveElh41S2GrgpagbE0iTcmng_ClientProxy.loadGrantsToGrantee(Unknown Source)
at org.apache.polaris.service.auth.DefaultAuthenticator.loadPrincipalGrants(DefaultAuthenticator.java:237)
at org.apache.polaris.service.auth.DefaultAuthenticator.resolvePrincipalRoles(DefaultAuthenticator.java:169)
at org.apache.polaris.service.auth.DefaultAuthenticator.authenticate(DefaultAuthenticator.java:101)
at org.apache.polaris.service.auth.DefaultAuthenticator_ClientProxy.authenticate(Unknown Source)
at org.apache.polaris.service.auth.ServiceProducers_ProducerMethod_authenticator_oPZhnnXSCvvtb9FlHY14q-AvJLY_ClientProxy.authenticate(Unknown Source)
at org.apache.polaris.service.auth.AuthenticatingAugmentor.authenticatePolarisPrincipal(AuthenticatingAugmentor.java:75)
at org.apache.polaris.service.auth.AuthenticatingAugmentor.lambda$augment$0(AuthenticatingAugmentor.java:61)
at io.quarkus.vertx.core.runtime.security.VertxBlockingSecurityExecutor$1$1.call(VertxBlockingSecurityExecutor.java:43)
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$4(ContextImpl.java:192)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:270)
at io.vertx.core.impl.ContextImpl$1.execute(ContextImpl.java:221)
at io.vertx.core.impl.WorkerTask.run(WorkerTask.java:56)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$15.runWith(VertxCoreRecorder.java:695)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2651)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2630)
at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1622)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1589)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
```

Ideally each REST API request should use one JDBC connection for the sake of consistency and efficiency.

Contributor guide

Open the contributing guide

Research direction

Start by tracing connection borrowing in DatasourceOperations.java and the persistence calls from JdbcBasePersistenceImpl.java through authentication in DefaultAuthenticator. Run the relevant REST or IRC API tests and inspect connection counts; done means a request uses one JDBC connection while preserving consistent behavior and passing regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.