airbytehq / airbytehq/airbyte

Source Snowflake - JDBC timeout

Open
#21,768 3 comments 0 reactions 0 assignees View on GitHub
area/connectors community connectors/source/snowflake frozen lang/java team/db-dw-sources type/bug
Dominant language
Python
Stars
22.1k
Forks
5.3k
PR merge metrics
PR metrics pending

Description

## Environment
- **Airbyte version**: 0.40.23 / Airbyte Cloud
- **OS Version / Instance**: AWS EC2 / Airbyte Cloud
- **Deployment**: Docker / Airbyte Cloud
- **Source Connector and version**: Snowflake 0.1.29
- **Destination Connector and version**: S3 0.3.19
- **Step where error happened**: Sync job

## Current Behavior
The initial sync does not complete. The sync runs for ~7h30m and then decides to throw an timeout error.

```
at io.airbyte.integrations.source.snowflake.SnowflakeSource.main(SnowflakeSource.java:47) Caused by: net.snowflake.client.jdbc.SnowflakeSQLLoggedException: JDBC driver internal error: Timeout waiting for the download of #chunk164(Total chunks: 1147) retry=1.
```

This appears to be a problem because the initial sync volume is really large.

To make sure it was not related to the local EC2 instance it was running, the same thing was tried on Airbyte Cloud. The same issue occurred which appears to confirm it's a issue with the Snowflake source.

## Expected Behavior
The initial sync should complete.

## Logs
The full log file was already shared on the Airbyte slack channel `airbyte-help`. It's 102MB so do not want to share here due to the size of the log.

## Steps to Reproduce
1. Set up a snowflake source.
2. Set up a S3 destination.
3. Set up an sync and ensure the table size is really large.
4. The initial sync should fail.

## Are you willing to submit a PR?
Sure! If I can be pointed in the right direction!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.