[source-snowflake] Incremental syncs silently skip rows on TIMESTAMP_NTZ cursors when the Snowflake session timezone is not UTC
- 主要言語
- Python
- スター
- 22.1k
- フォーク
- 5.3k
- PR マージ指標
- PR 指標を取得中
説明
### Connector Name
source-snowflake
### Connector Version
1.1.0 (latest). Also reproduced on 1.0.5, 1.0.8, and 1.0.9.
### What step the error happened?
During the sync
### Relevant information
**Context:** `source-snowflake` supports cursor-based incremental sync. On each sync, the connector reads the maximum cursor value present in the table, uses it as the upper bound for the rows it selects in that sync, and persists it as the new sync state so the next sync only selects rows newer than that checkpoint. This report covers what happens when the cursor field is a `TIMESTAMP_NTZ` column.
**Root cause:** the connector computes `cursorUpperBound = MAX(cursor)` at the start of each incremental read, then queries rows in the range `(previous_checkpoint, cursorUpperBound)`. The upper bound is bound into the query via `PreparedStatement.setTimestamp()` with no explicit UTC `Calendar` (in `SnowflakeFieldTypes.kt`'s `SnowflakeLocalDateTimeAccessor`, or the equivalent generic CDK `TimestampAccessor` in earlier versions). Because no explicit timezone is passed, the effective value depends on the Snowflake session, user, or account `TIMEZONE` parameter. When that parameter isn't UTC, the bound upper-bound value used in the SQL comparison is shifted backward, relative to the true max, by the session's UTC offset.
By contrast, the sibling `SnowflakeOffsetDateTimeFieldType` (used for `TIMESTAMP_TZ`/`TIMESTAMP_LTZ`) anchors both read and write through `Instant`/UTC and does not exhibit this problem, which is why it's specific to `TIMESTAMP_NTZ` cursors.
**Impact:** any record whose cursor value falls within a trailing window, sized to the session's UTC offset, below the batch's true maximum is excluded from the sync that should have captured it. Under steady, continuous incoming data, this manifests as a systematic delay: the record is captured once enough newer data pushes the max far enough ahead, typically arriving roughly one full UTC-offset's worth of hours later than it should have. If the cursor value stops advancing entirely (no further inserts or updates to the stream), the trailing window of records at the time inflow stops remains permanently unsynced. No error, warning, or failed-sync log entry is produced in either case.
**Workaround:** forcing the Snowflake session into UTC (for example via a `TIMEZONE=UTC` session, user, or account parameter, or the equivalent JDBC connection parameter) removes the asymmetry since the read and write then agree.
**Reproduction:**
*Part 1, isolate the mechanism (single batch):*
1. Set the Snowflake session, user, or account `TIMEZONE` to a non-UTC zone with a known offset (e.g. `America/Los_Angeles`, UTC-8 in winter).
2. Seed a table with a `TIMESTAMP_NTZ` cursor column and run an initial sync to establish a baseline checkpoint.
3. Insert or update a batch of rows with cursor values spread across a window at least as wide as the timezone's offset, ending at a new maximum (e.g. one row every few minutes across a 10-hour span, ending at the new max).
4. Run an incremental sync.
5. Compare source rows to destination rows for that batch. Expected: every row present. Actual: rows within the trailing window (sized to the offset) below the new max are missing, and the boundary is sharp.
6. Run a second, no-op sync (no data change) and confirm the missing rows stay missing rather than catching up.
7. Push the cursor far enough ahead in a later update (well beyond one offset-window) and observe the previously missing rows finally appear, while whichever row now holds the new max becomes the next one silently excluded.
*Part 2, production-representative pattern (steady trickle):*
1. Same non-UTC session setup as Part 1.
2. Continuously insert or update rows over many sync cycles (e.g. a steady drip of new/changed rows every cycle, rather than one large batch).
3. Compare, per cycle, each record's actual update time against the time it appears in the destination.
4. Expected: each record appears in the sync immediately following its update. Actual: each record consistently arrives roughly one full session-UTC-offset later than expected, a steady rolling delay rather than a one-time loss.
### Relevant log output
None. This is a silent data-loss/delay bug with no error, warning, or failed-sync log entry. Affected rows are simply missing or arrive late in the destination.
### Contribute
- [x] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/13255
---
**Edit (2026-08-13): the root cause described above is wrong. The bug is real, the mechanism is not what I wrote.**
My original verification was invalid. It ran against a source that still had `jdbc_url_params=TIMEZONE=UTC` set from an unrelated experiment, which suppresses this bug regardless of connector code. With that removed I re-tested properly, and the read/write `Calendar` explanation does not survive.
What is actually happening. The column is `TIMESTAMP_NTZ`, a naive wall clock with no timezone. Suppose the true max cursor value is `2026-02-01 20:00:00` and the session is `America/Los_Angeles` (UTC-8):
1. `SELECT MAX(MODIFICATION_DATE)` returns `2026-02-01 20:00:00`, which is correct.
2. The connector binds that value back as the upper bound of the next query, `WHERE MODIFICATION_DATE <= ?`.
3. The driver sends every `java.sql.Timestamp` parameter according to `CLIENT_TIMESTAMP_TYPE_MAPPING`, which defaults to `TIMESTAMP_LTZ`. The parameter therefore leaves as an absolute instant, `2026-02-01T20:00:00Z`, chosen without reference to the column it will be compared against.
4. To compare an LTZ instant against an NTZ column, Snowflake renders the instant into a naive wall clock using the session timezone. `20:00Z` in Los Angeles is `12:00`.
5. The filter effectively becomes `<= 2026-02-01 12:00:00` instead of `<= 20:00:00`, so everything in that 8 hour window is skipped.
So there is a genuine asymmetry, but it is between a JVM-anchored read and a session-anchored write coercion, not between the two `Calendar` arguments.
I confirmed this from the other direction: setting `CLIENT_TIMESTAMP_TYPE_MAPPING=TIMESTAMP_NTZ` and nothing else, leaving the session at `America/Los_Angeles`, eliminates the loss completely. The bind type is the lever, not the timezone.
One clarification on impact: the connector never emits an incorrect timestamp. The affected rows are simply not returned by the query, so the shift exists only in the `WHERE` bound and never in the data itself.
The original report is preserved above, unedited. Its reproduction steps and workaround still hold; only the root cause paragraph is wrong.
コントリビューションガイド
評価
この issue はまだ評価されていません。