[source-netsuite] bug- incremental sync silently drops records modified between sync time and account-local midnight
- Lingua principale
- Python
- Stelle
- 22.1k
- Fork
- 5.4k
- Merge medio
- 5h
- PR unite (30g)
- 671
Descrizione
### Connector Name
source-netsuite
### Connector Version
0.1.27
### What step the error happened?
During the sync
### Relevant information
## Summary
`source-netsuite` incremental streams permanently lose records whose
`lastModifiedDate` falls between the sync run time and midnight in the
NetSuite account's configured timezone.
## Steps to reproduce
1. Configure a NetSuite connection with an incremental stream (e.g. `journalentry`)
2. Schedule daily syncs that run before midnight UTC (e.g. a sync running at
03:00 UTC on a PDT account, where local midnight = 07:00 UTC)
3. A record is created or modified after the sync runs but before local midnight
(e.g. at 04:15 UTC)
4. The next day's sync does not return the record
## Expected behavior
All records modified since the previous sync cursor are returned on the next sync.
## Actual behavior
Records modified in the window `(sync_run_time, account_local_midnight)` UTC are
never returned. They are permanently absent from the destination.
## Root cause
`stream_slices` calls `.date()` on the cursor, stripping the time component, and
formats slice bounds using `NETSUITE_INPUT_DATE_FORMATS` (date-only strings such as
`"03/15/2026"`). NetSuite interprets bare date strings in the account's configured
timezone, so `AFTER "03/15/2026"` on a PDT account resolves to
`2026-03-15T07:00:00Z` — not UTC midnight.
A record at `04:15Z` therefore falls in a dead zone:
- The previous sync ran at `03:00Z` and could not see it (it didn't exist yet)
- The cursor advances to `03:00Z`, which `.date()` truncates to March 15
- The next sync starts its slice from `"03/15/2026"` = `07:00Z`
- The record at `04:15Z` is before `07:00Z` and is never queried
The dead zone is `[sync_run_time, account_local_midnight)` UTC — up to 12 hours
wide depending on the account's timezone offset.
## Proposed fix
Replace `.date()` + locale format with `.replace(hour=0,...)` +
`NETSUITE_OUTPUT_DATETIME_FORMAT`, producing explicit UTC datetime bounds
(`"2026-03-15T00:00:00Z"`), unambiguous regardless of account timezone.
## Environment
- Connector: `source-netsuite:0.1.27`
- Affected streams: all `IncrementalNetsuiteStream` subclasses
- NetSuite account timezone: reproduced on `America/Los_Angeles`; affects any
account whose UTC offset means local midnight > sync run time
### Relevant log output
```shell
```
### Contribute
- [x] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12846
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.