python-hyper / python-hyper/h2

[RFC]: Lazy state changes.

Open
#228 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Question
Dominant language
Python
Stars
1k
Forks
187
PR merge metrics
No merged PRs in 30d

Description

Inspired by @njsmith in njsmith/h11#4.

Right now there is a wart in the h2 API when working on a busy connection. The problem exists when working with receive_data. For example, it is possible that you might encounter the following set of events returned from receive_data: [DataReceived(stream_id=1), WindowUpdated(stream_id=1), StreamReset(stream_id=1)].

In response to that WindowUpdated you may want to send another data frame on stream 1, but if you do so without processing the entire event list h2 will throw a ProtocolError at you because the stream is now in the CLOSED state. However, the user doesn't know that: they're processing the events serially, and haven't gotten to the event that actually closed the stream.

The outcome of this is not too bad: while the ProtocolError does get thrown, the connection state is still fine and no damage has been done to your ability to continue to use the connection. However, the exception may cause an incautious user to run into trouble: if they're not expecting and handling this exception it'll quite probably cause them to lose a lot of application state.

In njsmith/h11#4, @njsmith has proposed a split API for receiving data: a single receive_data call returns no events, but instead you need to call next_event to advance the state machine. This proposal is essentially a "split" API much like we have for sending.

Making that change is a pretty substantial API change and I don't know how I feel about it. However, it turns out that hyper-h2 is capable of emulating that change with a very simple change to the logic of receive_data.

Right now, receive_data returns a list of events. However, it would be a trivial change for receive_data to return a generator instead. That generator would then iterate across events, one event at a time. The effect of doing that would be to avoid updating the state machines until the user has an event in hand, and would essentially immediately change hyper-h2 to an API that lazily updates stream state. This would ensure that a user cannot see a ProtocolError for sending data on a closed stream unless they have iterated past an event that closed the stream (or they closed it themselves).

I'm interested in what @python-hyper/contributors think, as well as our wider user-base. I think making such a change would be extremely valuable, but it represents a breaking API change (receive_data is documented as returning a list).

Do people think this change is sensible?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the documented H2Connection.receive_data API and the issue's event sequence; compare its current list-returning contract with the proposed lazy generator behavior. Done means reaching a project decision on whether to accept the breaking API change and recording the resulting direction; no implementation file or test is named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.