Add Flow#onBackpressureDrop() operator
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 211
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 89
Description
Motivation:
I want to do some rate limiting and just fail the single task, eg when combined with `mapAsync`.
Currently seems I can't observe the backpressure.
`onBackpressure` is only been called once downstream backpressure.
```scala
queue = Source.queue(10240)
.groupedWeightedWithin(
dynamicDictionaryConfigs.getMaxBatchTokens(), //最大的字符数,1000 个以内
dynamicDictionaryConfigs.getMaxBatchSize(), //最大的批量大小,50 个以内
costFn, //字符数计算
Duration.ofMillis(dynamicDictionaryConfigs.getMaxBatchIntervalInMillis())) //最大的聚合时间,比如 3ms
.onBackpressure(task -> task.fail(...)) //
.buffer(dynamicDictionaryConfigs.getOvsQpsLimit() * 2, OverflowStrategy.backpressure())
.toMat(Sink.foreach(this::batchTranslate), Keep.left())
....
.run(actorSystem);
```
What do you think?
In reactor-core, there is:
- `reactor.core.publisher.Flux#onBackpressureDrop(java.util.function.Consumer)`
This is what I would like to make use.
Contributor guide
Research direction
Start by reading the existing Flow#onBackpressure operator and comparing its behavior with reactor-core's Flux#onBackpressureDrop(java.util.function.Consumer). Trace how groupedWeightedWithin, mapAsync, and buffer propagate backpressure, then define the dropped-element callback and verify the intended behavior with focused stream tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100