Gorgias source - incremental sync re-syncing duplicates, runs for hours
- Ngôn ngữ chính
- Python
- Star
- 22.1k
- Fork
- 5.3k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
### Connector Name
Gorgias
### Connector Version
0.1.42
### What step the error happened?
During the sync
### Relevant information
We're experiencing an issue with our Gorgias source connector where incremental sync is not behaving incrementally. Each sync re-pulls records that were already synced in prior runs, producing large volumes of duplicate data in the destination and causing syncs to run for several hours instead of completing quickly.
Connection details:
- Source: Gorgias
- Destination: BigQuery
- Affected stream(s): [e.g. messages, tickets]
- Sync mode: [Incremental | Append OR Incremental | Append + Deduped]
- Cursor field: [ created_datetime]
- Environment: [Airbyte Cloud]
- Gorgias source connector version: 0.1.42
What we expect:
After an initial full sync, subsequent incremental runs should only fetch records created or updated since the last cursor value, and complete in minutes.
What's actually happening:
- Every sync re-fetches a large overlapping set of records that were already present from previous runs (duplicates).
- Sync durations have grown to several hours.
- The cursor / state does not appear to be advancing or filtering correctly between runs.
We're currently deduplicating downstream in BigQuery as a workaround, but we'd like the incremental sync itself to behave correctly to reduce sync time and cost.
### Relevant log output
```shell
```
### Contribute
- [ ] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12944
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.