python-hyper / python-hyper/h2

Add support for sendfile() based DATA frames.

Open
#236 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement Performance Question
Dominant language
Python
Stars
1k
Forks
187
PR merge metrics
No merged PRs in 30d

Description

Right now there is no support for HTTP/2 using sendfile to send DATA frames. This is potentially inefficient for implementations that are able to send really sizeable DATA frames. Given that sendfile allows users to send fixed length data, it would be nice to provide some way to say "send these bytes, then do a sendfile with this fobj and this length".

@njsmith's h11 library has this capacity, but this relies on the fact that h11 has no internal buffer: each "event" call returns the bytes required for that event directly without writing into a buffer. This allows for the subtle changing of return value in comparison to send, which is not so naturally achievable with h2.

Coming up with a good design here is a bit tricky. It may be that this should be an optional switch on the H2Connection class that affects the return value of data_to_send(), changing it to be an iterable of bytes and sentinel objects where each sentinel object is . Alternatively, we could go further and say that not only does the optional switch need to be enabled but there is also a separate "get the data I need" function that conforms to this new API.

Another possible option that leaps out to me is to have a subclass (eww) or some other type that implements this support as a wrapper around the base H2Connection object.

The final possible option is a dramatic API change. That changes the signature of receive_data to return two values: the events iterable and any bytes that may need to be sent. If we do that, we can then remove hyper-h2's internal buffer and then delegate entirely to the calling code by having all do_event() type functions simply return the data they generate rather than storing it internally. That's a very large API break, but it allows supporting this use-case more cleanly by simply emulating what h11 does.

I'd like opinions from @python-hyper/contributors. Any thoughts on API choices? Is this worth supporting at all?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining H2Connection, data_to_send(), and receive_data(), which the issue identifies as the API points affected. Compare the proposed buffering, iterable-sentinel, wrapper, and API-break approaches; the work is done when contributors agree on a design that supports sendfile-based DATA frames.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.