Replace bufferedSubStreamOutput in Http2Demux with direct access to stream state machine
- Dominant language
- Scala
- Stars
- 196
- Forks
- 55
- Avg merge
- 4d 52m
- Merged PRs (30d)
- 74
Description
Right now, we dispatch substreams as soon as they are incoming / we detect them into `bufferedSubStreamOutput`. Instead, we can let them be pulled and query the state machine whether a stream is ready to be dispatched. In particular, this will automatically avoid allocating infrastructure for streams that might never materialize (e.g. because they are cancelled before they will be pulled).
One way to do it would be to introduce a kind of "holding position" state. In that state, we could also already start collecting incoming data (which we have to buffer anyway up to the default window size). In the best case, we have already seen `endStream = true` for the request side, so that we can avoid setting up the expensive streaming infrastructure (this will mean that in overload cases we will "automatically" reduce work because we collect buffers early on when there is backpressure from the stream handling flow).
Contributor guide
Research direction
Start by tracing Http2Demux and its bufferedSubStreamOutput path, then read the stream state machine and how substreams are pulled and dispatched. Define the holding-position behavior, including early incoming-data buffering and endStream handling. Done means dispatch is driven by direct state-machine readiness and avoids unnecessary streaming infrastructure for cancelled or fully received streams.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend, networking
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100