Triggers not working for bounded data
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
pCollection
.apply(Window.into(FixedWindows.of(Duration.millis(100)))
.triggering(Repeatedly.forever(AfterPane.elementCountAtLeast(1000)))
.discardingFiredPanes().withAllowedLateness(Duration.ZERO))
.apply(TextIO.write().withWindowedWrites().withNumShards(1).to('gs-path'));
Here pCollection is a *bounded* PCollection. I'm trying to break it into files of 1000 roughly, but all I get is 2 files one having 1000 other having the rest of the data.
If instead I do:-
pCollection
.apply(new GlobalWindow())
.triggering(Repeatedly.forever(AfterPane.elementCountAtLeast(1000)))
.discardingFiredPanes().withAllowedLateness(Duration.ZERO))
.apply(TextIO.write().withWindowedWrites().withNumShards(1).to('gs-path'));
I get just one file.
Both of the above cases should have conceptually divided the records into chucks of 1000 to be written in a file.
Imported from Jira [BEAM-6743](https://issues.apache.org/jira/browse/BEAM-6743). Original Jira may contain additional context.
Reported by: aditya.guru149.
Contributor guide
Assessment
This issue has not been assessed yet.