Gorgias source - incremental sync re-syncing duplicates, runs for hours
- 主要語言
- Python
- 星號
- 22.1k
- 分支
- 5.4k
- 平均合併
- 5 小時
- 30 天內合併 PR
- 671
描述
### Connector Name
Gorgias
### Connector Version
0.1.42
### What step the error happened?
During the sync
### Relevant information
We're experiencing an issue with our Gorgias source connector where incremental sync is not behaving incrementally. Each sync re-pulls records that were already synced in prior runs, producing large volumes of duplicate data in the destination and causing syncs to run for several hours instead of completing quickly.
Connection details:
- Source: Gorgias
- Destination: BigQuery
- Affected stream(s): [e.g. messages, tickets]
- Sync mode: [Incremental | Append OR Incremental | Append + Deduped]
- Cursor field: [ created_datetime]
- Environment: [Airbyte Cloud]
- Gorgias source connector version: 0.1.42
What we expect:
After an initial full sync, subsequent incremental runs should only fetch records created or updated since the last cursor value, and complete in minutes.
What's actually happening:
- Every sync re-fetches a large overlapping set of records that were already present from previous runs (duplicates).
- Sync durations have grown to several hours.
- The cursor / state does not appear to be advancing or filtering correctly between runs.
We're currently deduplicating downstream in BigQuery as a workaround, but we'd like the incremental sync itself to behave correctly to reduce sync time and cost.
### Relevant log output
```shell
```
### Contribute
- [ ] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12944
貢獻指南
研究方向
The report identifies the Gorgias source connector version 0.1.42, incremental sync state and cursor handling, and affected streams such as messages and tickets, but provides no logs or file paths. Start by reproducing an incremental run after a full sync and inspect whether the created_datetime cursor advances and filters records. Done means later runs fetch only records created or updated after the saved cursor without duplicates.
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- data-engineering
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 冷清
- 描述清晰度
- 基本清楚
- 新手友好度
- 38/100