airbytehq / airbytehq/airbyte

[source-mysql] Sync Become Stalled

Ouverte
#35,804 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
area/connectors community connectors/source/mysql frozen team/db-dw-sources type/bug
Langage dominant
Python
Étoiles
22.1k
Forks
5.3k
Métriques de merge des PR
Métriques de PR en attente

Description

### Connector Name

source-mysql

### Connector Version

3.3.8

### What step the error happened?

During the sync

### Relevant information

Hi Airbyte Team,

Hope you are well.

I have established a data pipeline for loading Change Data Capture (CDC) from MySQL 5 into Snowflake.
Typically, this pipeline operates smoothly, completing the load within 2-3 minutes. However, intermittently, the loading process becomes stalled, resulting in a duration exceeding 1 hour. Currently, the pipeline is set to trigger the connection every hour via API. Before triggering the connection, it checks the connection state, in the event that the connection is in a 'running' state, it triggers an error, which is the current situation.
I'm using Airbyte Version 0.50.50 deployed in a EC2 Instance.

Attached is a screenshot of the job history showing the average loading time.
I'm also sharing the logs of the latest two jobs that took over 1 hour to complete.

Thanks!

![Screenshot 2024-02-28 090718](https://github.com/airbytehq/airbyte/assets/130519892/d9c1780c-b1d4-4b5f-a4b7-863aa9d5d295)
[production_job_48141_txt.txt](https://github.com/airbytehq/airbyte/files/14487960/production_job_48141_txt.txt)
[production_logs_48138_txt.txt](https://github.com/airbytehq/airbyte/files/14487961/production_logs_48138_txt.txt)

### Relevant log output

_No response_

### Contribute

- [ ] Yes, I want to contribute

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.