[source-postgres] Incremental sync skips records when cursor field timestamp is extremely close to airbyte run start time
- Linguagem predominante
- Python
- Estrelas
- 22.1k
- Forks
- 5.3k
- Métricas de merge de PRs
- Métricas de PR pendentes
Descrição
### Connector Name
source-postgres
### Connector Version
3.7.0
### What step the error happened?
During the sync
### Relevant information
**Summary**
During an incremental sync from Postgres, the connector missed two records that were inserted into the source database at almost the exact time the Airbyte sync started. The sync completed successfully with no errors, but those two records did not appear in the destination until we ran a full refresh.
**Environment**
- Airbyte deployment: OSS
- Source: Postgres
- Connector: source-postgres v3.7.0
- Destination: S3 Data Lake v0.3.44
**Sync configuration**
- Sync mode: Incremental
- Destination sync mode: Incremental - Append + Deduped
- Replication method:
- Standard (cursor based)
- Cursor field: `created_at`
**Whats happening + details**
Two records were created in the source table at the following cursor field (created_at) identical timestamps:
1. 2026-06-12 16:31:32.869
2. 2026-06-12 16:31:32.869
Here are the cursor values from the airbyte logs:
cursor value: 2026-06-12T16:01:35.607000 --------- 2026-06-12 12:31:33 run
cursor value: 2026-06-12T16:31:33.680000 --------- 2026-06-12 13:01:37 run
**Expected behavior**
All rows that exist in the source at or before the end of the incremental sync window should be replicated to the destination. Rows inserted around the sync time should not be permanently missed; they should either appear in that sync or in a subsequent incremental sync, without requiring a manual full refresh.
**Actual behavior**
- The incremental sync completed successfully with no errors.
- Two rows inserted at (or very close to) the sync start time were not replicated.
- Subsequent incremental syncs did not pick them up.
- Only a manual full refresh caused those rows to appear in the destination.
**Evidence / verification**
- Verified in Postgres that the two rows existed before and after the incremental sync.
- Verified in the destination that:
- They were missing after the incremental sync.
- They appeared only after the full refresh.
- No obvious errors or warnings in the Airbyte logs for the incremental sync.
**Logs**
[clinicaldataservice_logs_64692_txt.txt](https://github.com/user-attachments/files/29022914/clinicaldataservice_logs_64692_txt.txt)
### Relevant log output
```shell
```
### Contribute
- [ ] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12889
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.