Create another version of groupWithin() which won't create a singleton chunk when elements are available in bulk after being idle for a long time
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 2.5k
- Forks
- 636
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 7
Description
Current groupWithin() has a feature which doesn't fit our requirements.
An example scenario:
- The stream is started, but no element is available for a long time, longer than the timeout
- Then a batch of elements are available
- Since the stream is in "timed out" state, it will emit a chunk right away. It will pick the first element that arrives, and create a singleton chunk.
We need another version of this method . Same scenario:
- The stream is started, but no element is available for a long time, longer than the timeout
- Then a batch of elements are available. Let's say the available time of the first element is T0
- The stream will not emit right away, until the number of elements available reaches
chunkSize, or when the time is atT0 + timeout
This is very desirable if the chunk processing logic prefers a big batch, instead of just a few elements.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the existing groupWithin() implementation and its tests, then read how timeout and chunk-size behavior are specified. Add coverage for a stream idle past the timeout followed by a batch of elements, and define the new method as done when it waits for chunkSize elements or until timeout from the first element instead of emitting a singleton immediately.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100