[adapters] Delta connector: decouple lateness from transaction size.
Open
@ryzhyk is already working on this.
Since Dec 27, 2025.
connectors
high priority
user-reported
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 154
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 133
Description
Currently, when transaction_mode is set to snapshot or always, the delta connector initiates one transaction per lateness-size batch of inputs when reading the table snapshot. The user may prefer coarser transactions for performance reasons. To allow this, we show allow the user to configure transaction size independently of lateness. The transaction size can be expressed in terms of the number of records per transaction or event time range for tables with timestamps.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.