airbytehq / airbytehq/airbyte

Source Redshift - error trying to get schema from the source (Since version 0.3.12)

Ouverte
#17,988 5 commentaires 4 réactions 0 personnes assignées Voir sur GitHub
area/connectors community connectors/source/redshift frozen team/db-dw-sources type/bug
Langage dominant
Python
Étoiles
22.1k
Forks
5.3k
Métriques de merge des PR
Métriques de PR en attente

Description

## Environment
- **Airbyte version**: 0.39.14-alpha
- **OS Version / Instance**: AWS EKS
- **Deployment**: Kubernetes (On Plural)
- **Source Connector and version**: Redshift 0.3.12
- **Step where error happened**: Sync Job

## Current Behavior
Currently, trying to create a new connection from Redshift to Postgres and the creation fails in the sync schema step.

## Expected Behavior
Retrieve all tables from the schema to create the connection.

## Logs

> m (plural-airbyte-3), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,230 INFO: no action. I am (plural-airbyte-2), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,238 INFO: no action. I am (plural-airbyte-4), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,246 INFO: no action. I am (plural-airbyte-5), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,260 INFO: no action. I am (plural-airbyte-7), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,267 INFO: no action. I am (plural-airbyte-1), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:24,280 INFO: no action. I am (plural-airbyte-0), the leader with the lock
2022-10-14 07:09:24,274 INFO: no action. I am (plural-airbyte-6), a secondary, and following a leader (plural-airbyte-0)
2022-10-14 07:09:27.964 38 LOG {ticks: 0, maint: 0, retry: 0}
2022-10-14 07:07:07 INFO i.a.w.p.KubePodProcess(close):714 - (pod: airbyte / redshift-discover-ecffb8a4-66ef-47a2-9562-f3d8cd7f40fa-0-swrgr) - Closed all resources for pod
Log4j2Appender says: (pod: airbyte / redshift-discover-ecffb8a4-66ef-47a2-9562-f3d8cd7f40fa-0-swrgr) - Closed all resources for pod
2022-10-14 07:07:07 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):161 - Completing future exceptionally...
io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:76) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:26) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Log4j2Appender says: Completing future exceptionally...
2022-10-14 07:07:07 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...
Log4j2Appender says: Stopping cancellation check scheduling...
2022-10-14 07:07:07 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=81f39ae7-6663-39d3-a3bc-e85f5f61d972, activityType=Run, attempt=1
java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.discover.catalog.DiscoverCatalogActivityImpl.run(DiscoverCatalogActivityImpl.java:84) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor831.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:76) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:26) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
... 1 more
Log4j2Appender says: Activity failure. ActivityId=81f39ae7-6663-39d3-a3bc-e85f5f61d972, activityType=Run, attempt=1
2022-10-14 07:07:07 ERROR i.t.i.s.WorkflowExecuteRunnable(logWorkflowExecutionException):125 - Workflow execution failure WorkflowId=9f629641-fc9d-435f-9232-0f5ee43aefc7, RunId=93ac5c88-deee-4d36-a159-ad942a5a653b, WorkflowType=DiscoverCatalogWorkflow
io.temporal.failure.ActivityFailure: scheduledEventId=5, startedEventId=6, activityType='Run', activityId='81f39ae7-6663-39d3-a3bc-e85f5f61d972', identity='1@airbyte-worker-65ffd94695-mm6gz', retryState=RETRY_STATE_MAXIMUM_ATTEMPTS_REACHED
at java.lang.Thread.getStackTrace(Thread.java:1610) ~[?:?]
at io.temporal.internal.sync.ActivityStubBase.execute(ActivityStubBase.java:48) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.ActivityInvocationHandler.lambda$getActivityFunc$0(ActivityInvocationHandler.java:77) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.ActivityInvocationHandlerBase.invoke(ActivityInvocationHandlerBase.java:70) ~[temporal-sdk-1.8.1.jar:?]
at jdk.proxy2.$Proxy79.run(Unknown Source) ~[?:?]
at io.airbyte.workers.temporal.discover.catalog.DiscoverCatalogWorkflowImpl.run(DiscoverCatalogWorkflowImpl.java:28) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor829.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation$RootWorkflowInboundCallsInterceptor.execute(POJOWorkflowImplementationFactory.java:317) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java:292) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.WorkflowExecuteRunnable.run(WorkflowExecuteRunnable.java:72) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java:137) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:101) [temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java:111) [temporal-sdk-1.8.1.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.temporal.failure.ApplicationFailure: message='io.airbyte.workers.exception.WorkerException: Discover job subprocess finished with exit code 1', type='java.util.concurrent.ExecutionException', nonRetryable=false
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.discover.catalog.DiscoverCatalogActivityImpl.run(DiscoverCatalogActivityImpl.java:84) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor831.invoke(Unknown Source:0) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.temporal.failure.ApplicationFailure: message='Discover job subprocess finished with exit code 1', type='io.airbyte.workers.exception.WorkerException', nonRetryable=false
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:76) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.general.DefaultDiscoverCatalogWorker.run(DefaultDiscoverCatalogWorker.java:26) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.14-alpha.jar:?]
at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]
Terminated

## Steps to Reproduce
1. Set 0.3.12 version in Redhisft Source
2. Create a new connection
3. Wait for retrieve the schema from the source

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.