typelevel / typelevel/fs2

Create another version of groupWithin() which won't create a singleton chunk when elements are available in bulk after being idle for a long time

Open
#3,608 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
2.5k
Forks
636
Avg merge
2d 4h
Merged PRs (30d)
7

Description

Current groupWithin() has a feature which doesn't fit our requirements.

An example scenario:

  • The stream is started, but no element is available for a long time, longer than the timeout
  • Then a batch of elements are available
  • Since the stream is in "timed out" state, it will emit a chunk right away. It will pick the first element that arrives, and create a singleton chunk.

We need another version of this method . Same scenario:

  • The stream is started, but no element is available for a long time, longer than the timeout
  • Then a batch of elements are available. Let's say the available time of the first element is T0
  • The stream will not emit right away, until the number of elements available reaches chunkSize, or when the time is at T0 + timeout

This is very desirable if the chunk processing logic prefers a big batch, instead of just a few elements.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the existing groupWithin() implementation and its tests, then read how timeout and chunk-size behavior are specified. Add coverage for a stream idle past the timeout followed by a batch of elements, and define the new method as done when it waits for chunkSize elements or until timeout from the first element instead of emitting a singleton immediately.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.