BigQuery: `last_modified` underreports freshness for streamed sources
Open
- Dominant language
- Python
- Stars
- 8
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
Just an observation from our implementation
**Issue summary:**
On BigQuery, the adapter derives freshness from `Table.modified` in `clients/dbt_state/src/dbt_state/adapters/bigquery.py`. For tables fed by streaming ingestion (`tabledata.insertAll` or the Storage Write API), `Table.modified` doesn't advance per streamed row — it moves on buffer flush, load jobs, or DML.
This means that the state adapter may reuse models when new rows are available in a source's streaming buffer (in our case this occurs, it may be circumventable on the DE side by flushing more often).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.