CONNECT, Upgrade, and HTTP/2 bi-directional streaming.

Originally opened by @tomchristie on 2020-08-08 11:29:35 in encode/httpx

At some point we ought to consider a low-level Transport API for exposing the functionality offered by the HTTP CONNECT method, the HTTP Upgrade header, and HTTP/2's bi-directional streaming. (Eg. as used by gRPC)

All there of these essentially offer the same thing - a means of obtaining a raw connection over HTTP.
The connection itself might either be the actual TCP connection (HTTP/1.1) or just a single stream (HTTP/2), but this should be ~transparent to the user, except for the fact that we might want to expose some functionality such as "now start TLS on this connection" that is only appropriate in the HTTP/1.1 case.

One thing that's potentially a bit fiddly about the API, is that the connect request can either be accepted or rejected, and we've got different kinds of stream interfaces we want to expose in each case. (Ie if it's rejected, then we want to return a regular byte stream representing the HTTP response, otherwise we want to return some kind of Connection interface.)

This issue is related to WebSockets support (which uses the HTTP Upgrade mechanism), but I think we want to provide a dedicated API for that use case, rather than building our websockets support on top of a low level connect mechanism.

Would be useful to hear from any folks with use cases in this area.

The issue proposes a low-level Transport API covering CONNECT, HTTP Upgrade, and HTTP/2 bi-directional streaming, but names no files, tests, or entry points. Start by reviewing the existing transport API and the related WebSockets issue (#304), then identify use cases and resolve how accepted and rejected CONNECT responses should expose different stream interfaces. Done means a settled API design with an implementation and tests for the described protocols.