airbytehq / airbytehq/airbyte

Gorgias source - incremental sync re-syncing duplicates, runs for hours

Đang mở
#80,326 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
area/connectors autoteam community needs-triage team/extensibility team/use type/bug
Ngôn ngữ chính
Python
Star
22.1k
Fork
5.3k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

### Connector Name

Gorgias

### Connector Version

0.1.42

### What step the error happened?

During the sync

### Relevant information

We're experiencing an issue with our Gorgias source connector where incremental sync is not behaving incrementally. Each sync re-pulls records that were already synced in prior runs, producing large volumes of duplicate data in the destination and causing syncs to run for several hours instead of completing quickly.

Connection details:
- Source: Gorgias
- Destination: BigQuery
- Affected stream(s): [e.g. messages, tickets]
- Sync mode: [Incremental | Append OR Incremental | Append + Deduped]
- Cursor field: [ created_datetime]
- Environment: [Airbyte Cloud]
- Gorgias source connector version: 0.1.42

What we expect:
After an initial full sync, subsequent incremental runs should only fetch records created or updated since the last cursor value, and complete in minutes.

What's actually happening:
- Every sync re-fetches a large overlapping set of records that were already present from previous runs (duplicates).
- Sync durations have grown to several hours.
- The cursor / state does not appear to be advancing or filtering correctly between runs.

We're currently deduplicating downstream in BigQuery as a workaround, but we'd like the incremental sync itself to behave correctly to reduce sync time and cost.

### Relevant log output

```shell

```

### Contribute

- [ ] Yes, I want to contribute

---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12944

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.