NVIDIA / NVIDIA/nvcf

worker: allow the stateful tunnel to run over TCP instead of QUIC

Open
#1,219 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 12h
Merged PRs (30d)
427

Description

Problem

The worker's stateful tunnel to the proxy runs over QUIC. Edge proxy CPU is proportional to bytes moved, and on high-throughput workloads every byte is currently encrypted and decrypted twice: once on the worker to edge proxy leg, and again on the edge proxy to grpc-proxy leg.

On a measured high-volume workload the edge proxy achieved roughly 33 MB/s per core, reproduced across three runs at two different scales. At sustained multi-GB/s throughput that becomes the dominant cost, and horizontal scaling does not relieve it because QUIC tunnels do not migrate between proxy pods once established.

Proposal

Allow the tunnel to be carried over TCP, selected per pod by an environment variable so it can be enabled for a single function and reverted by removing the variable.

The endpoint is derived from the HTTP/3 connection config the proxy already sends, so no control plane change is needed. The connection is made in two nested steps: an authority-form CONNECT to the regional TCP entry point, which the edge proxy turns into an opaque byte pipe to the target pod, and then the ordinary POST inside that pipe. Because the edge proxy does not parse what flows inside the pipe, HTTP/1.1's rule that a response may not begin before the request body completes does not apply, which is what makes a long-lived bidirectional tunnel workable over TCP.

grpc-proxy needs no change: it receives a plain HTTP/1.1 request on a real TCP connection, so the existing hijack path works unmodified.

Failure falls through to the existing QUIC path, so enabling the variable cannot take a worker offline.

Status

Experimental and off by default. Unit tests cover the endpoint derivation. The network path has not yet been exercised end to end; that requires a matching edge proxy route and will be measured against the existing QUIC baseline before any wider use.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the worker's stateful tunnel and the HTTP/3 connection configuration used to derive its endpoint. Verify the environment-variable selection, TCP CONNECT path, QUIC fallback, and existing endpoint-derivation unit tests; completion also requires exercising the network path with a matching edge proxy route and comparing it with the QUIC baseline.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.