Hash and range flushes resend pending tuples
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Feature Summary
Python hash and range partitioners emit pending batches during `flush` and `flush_state` but do not clear those batches afterward. Later control markers can resend the same tuples, unlike the round-robin, broadcast, and one-to-one partitioners.
Expected behavior:
A batch emitted across a control or state boundary is cleared exactly once.
Reproduction evidence:
Add one tuple below the batch-size threshold, flush its receiver twice, and inspect both outputs.
Before: first flush sends the tuple and marker, second flush sends the same tuple and marker again
After: first flush sends the tuple and marker, second flush sends only the marker
The same stale-batch behavior occurs after `flush_state` in both partitioners.
Version and commit evidence:
Current `main` at 70c21145887920528d7d5540e3fb790b43e8b759.
**Commit Hash (Optional)**
`70c21145887920528d7d5540e3fb790b43e8b759`
### Proposed Solution or Design
Expected behavior:
### Affected Area
Workflow Engine (Amber)
Contributor guide
Assessment
This issue has not been assessed yet.