Source Redshift - error trying to get schema from the source (Since version 0.3.12)
- Langage dominant
- Python
- Étoiles
- 22.1k
- Forks
- 5.3k
- Métriques de merge des PR
- Métriques de PR en attente
Description
## Environment
- **Airbyte version**: 0.39.14-alpha
- **OS Version / Instance**: AWS EKS
- **Deployment**: Kubernetes (On Plural)
- **Source Connector and version**: Redshift 0.3.12
- **Step where error happened**: Sync Job
## Current Behavior
Currently, trying to create a new connection from Redshift to Postgres and the creation fails in the sync schema step.
## Expected Behavior
Retrieve all tables from the schema to create the connection.
## Logs
> m (plural-airbyte-3), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,230 INFO: no action. I am (plural-airbyte-2), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,238 INFO: no action. I am (plural-airbyte-4), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,246 INFO: no action. I am (plural-airbyte-5), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,260 INFO: no action. I am (plural-airbyte-7), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,267 INFO: no action. I am (plural-airbyte-1), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,280 INFO: no action. I am (plural-airbyte-0), the leader with the lock
2022-10-14 07:09:24,274 INFO: no action. I am (plural-airbyte-6), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:27.964 38 LOG {ticks: 0, maint: 0, retry: 0}
2022-10-14 07:07:07 [32mINFO[m i.a.w.p.KubePodProcess(close):714 - (pod: airbyte / redshift-discover-ecffb8a4-66ef-47a2-9562-f3d8cd7f40fa-0-swrgr) - Closed all resources for pod
Log4j2Appender says: (pod: airbyte / redshift-discover-ecffb8a4-66ef-47a2-9562-f3d8cd7f40fa-0-swrgr) - Closed all resources for pod
2022-10-14 07:07:07 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):161 - Completing future exceptionally...
io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:76) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:26) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Log4j2Appender says: Completing future exceptionally...
2022-10-14 07:07:07 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...
Log4j2Appender says: Stopping cancellation check scheduling...
2022-10-14 07:07:07 [33mWARN[m i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=81f39ae7-6663-39d3-a3bc-e85f5f61d972, activityType=Run, attempt=1
java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.discover.catalog.DiscoverCatalogActivityImpl.run(DiscoverCatalogActivityImpl.java:84) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor831.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:76) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:26) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
... 1 more
Log4j2Appender says: Activity failure. ActivityId=81f39ae7-6663-39d3-a3bc-e85f5f61d972, activityType=Run, attempt=1
2022-10-14 07:07:07 [1;31mERROR[m i.t.i.s.WorkflowExecuteRunnable(logWorkflowExecutionException):125 - Workflow execution failure WorkflowId=9f629641-fc9d-435f-9232-0f5ee43aefc7, RunId=93ac5c88-deee-4d36-a159-ad942a5a653b, WorkflowType=DiscoverCatalogWorkflow
io.temporal.failure.ActivityFailure: scheduledEventId=5, startedEventId=6, activityType='Run', activityId='81f39ae7-6663-39d3-a3bc-e85f5f61d972', identity='1@airbyte-worker-65ffd94695-mm6gz', retryState=RETRY_STATE_MAXIMUM_ATTEMPTS_REACHED
at java.lang.Thread.getStackTrace(Thread.java:1610) ~[?:?]
at io.temporal.internal.sync.ActivityStubBase.execute(ActivityStubBase.java:48) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.ActivityInvocationHandler.lambda$getActivityFunc$0(ActivityInvocationHandler.java:77) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.ActivityInvocationHandlerBase.invoke(ActivityInvocationHandlerBase.java:70) ~[temporal-sdk-1.8.1.jar:?]
at jdk.proxy2.$Proxy79.run(Unknown Source) ~[?:?]
at io.airbyte.workers.temporal.discover.catalog.DiscoverCatalogWorkflowImpl.run(DiscoverCatalogWorkflowImpl.java:28) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor829.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation$RootWorkflowInboundCallsInterceptor.execute(POJOWorkflowImplementationFactory.java:317) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java:292) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.WorkflowExecuteRunnable.run(WorkflowExecuteRunnable.java:72) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java:137) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:101) [temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java:111) [temporal-sdk-1.8.1.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.temporal.failure.ApplicationFailure: message='io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1', type='java.util.concurrent.ExecutionException', nonRetryable=false
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.discover.catalog.DiscoverCatalogActivityImpl.run(DiscoverCatalogActivityImpl.java:84) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor831.invoke(Unknown Source:0) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.temporal.failure.ApplicationFailure: message='Discover job subprocess finished with exit code 1', type='io.airbyte.workers.exception.WorkerException', nonRetryable=false
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:76) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:26) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]
Terminated
## Steps to Reproduce
1. Set 0.3.12 version in Redhisft Source
2. Create a new connection
3. Wait for retrieve the schema from the source
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.