Python DirectRunner does not emit data at GC time
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
The following should succeed but does not:
```
test_options = PipelineOptions(flags=['--allow_unsafe_triggers'])
with TestPipeline(options=test_options)
as pipeline:
pcoll = (
pipeline
| beam.Create([(1, 1), (1, 2), (1, 3), (1, 4)])
|
WindowInto(
window.GlobalWindows(),
trigger=trigger.AfterCount(5),
accumulation_mode=trigger.AccumulationMode.ACCUMULATING)
| beam.GroupByKey())
assert_that(pcoll, equal_to([(1, [1, 2, 3, 4])]))
```
However, it currently fails, because pcoll will be empty. It appears that the Direct Runner drops data if the trigger never fired.
Imported from Jira [BEAM-13078](https://issues.apache.org/jira/browse/BEAM-13078). Original Jira may contain additional context.
Reported by: zhoufek.
Contributor guide
Assessment
This issue has not been assessed yet.