airbytehq / airbytehq/airbyte

[source-netsuite] bug- incremental sync silently drops records modified between sync time and account-local midnight

オープン
#79,653 コメント 4 件 リアクション 0 件 担当者 0 名 GitHub で見る
area/connectors autoteam community connectors/source/netsuite hyd-fix hyd-review needs-triage team/use type/bug
主要言語
Python
スター
22.1k
フォーク
5.4k
平均マージ
5時間
マージ済み PR(30日)
671

説明

### Connector Name

source-netsuite

### Connector Version

0.1.27

### What step the error happened?

During the sync

### Relevant information

## Summary

`source-netsuite` incremental streams permanently lose records whose
`lastModifiedDate` falls between the sync run time and midnight in the
NetSuite account's configured timezone.

## Steps to reproduce

1. Configure a NetSuite connection with an incremental stream (e.g. `journalentry`)
2. Schedule daily syncs that run before midnight UTC (e.g. a sync running at
03:00 UTC on a PDT account, where local midnight = 07:00 UTC)
3. A record is created or modified after the sync runs but before local midnight
(e.g. at 04:15 UTC)
4. The next day's sync does not return the record

## Expected behavior

All records modified since the previous sync cursor are returned on the next sync.

## Actual behavior

Records modified in the window `(sync_run_time, account_local_midnight)` UTC are
never returned. They are permanently absent from the destination.

## Root cause

`stream_slices` calls `.date()` on the cursor, stripping the time component, and
formats slice bounds using `NETSUITE_INPUT_DATE_FORMATS` (date-only strings such as
`"03/15/2026"`). NetSuite interprets bare date strings in the account's configured
timezone, so `AFTER "03/15/2026"` on a PDT account resolves to
`2026-03-15T07:00:00Z` — not UTC midnight.

A record at `04:15Z` therefore falls in a dead zone:
- The previous sync ran at `03:00Z` and could not see it (it didn't exist yet)
- The cursor advances to `03:00Z`, which `.date()` truncates to March 15
- The next sync starts its slice from `"03/15/2026"` = `07:00Z`
- The record at `04:15Z` is before `07:00Z` and is never queried

The dead zone is `[sync_run_time, account_local_midnight)` UTC — up to 12 hours
wide depending on the account's timezone offset.

## Proposed fix

Replace `.date()` + locale format with `.replace(hour=0,...)` +
`NETSUITE_OUTPUT_DATETIME_FORMAT`, producing explicit UTC datetime bounds
(`"2026-03-15T00:00:00Z"`), unambiguous regardless of account timezone.

## Environment

- Connector: `source-netsuite:0.1.27`
- Affected streams: all `IncrementalNetsuiteStream` subclasses
- NetSuite account timezone: reproduced on `America/Los_Angeles`; affects any
account whose UTC offset means local midnight > sync run time

### Relevant log output

```shell

```

### Contribute

- [x] Yes, I want to contribute

---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12846

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。