imroc / imroc/req

Upstream Sync Report 2026-09-10: HTTP/2 Framer idle buffer pooling (memory optimization)

Open
#536 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

http2 performance priority:low
Dominant language
Go
Stars
4.9k
Forks
409
Avg merge
6h 47m
Merged PRs (30d)
7

Description

Upstream Changes

Go stdlib net/http landed a series of memory optimizations (tracking golang/go#80735) that reduce memory pinned by idle connections. Three of them live in src/net/http/internal/http2/frame.go and apply to req's inlined copy (internal/http2/frame.go, which tracks golang.org/x/net/http2):

Commit Subject Assessment
94819a5 internal/http2: turn Framer.getReadBuf func field into a method Pure refactor, no behavior change; preparation for the two pooling changes below
3127f7b internal/http2: don't pin frame-sized read buffers on idle conns Applies to req — read buffer released to a sync.Pool while blocked waiting for a frame header
64c0b14 internal/http2: don't pin frame-sized write buffers on idle conns Applies to req — frame-sized write buffer released to a sync.Pool in endWrite

Two more commits landed on the server side only and do not affect req (req does not inline net/http server code, and internal/http2/ has no server.go):

  • a14b543 net/http: populate Request.TLS from HTTP/2 connection state (only internal/http2/server.go)
  • e2a74e8 net/http: don't pin bufio buffers on idle HTTP/1 keep-alive connections (only net/http/server.go)

As of today golang.org/x/net/http2 has not picked these up yet (latest commit still 55577aa, 2026-08-28), so the stdlib commits are the reference to port from — same situation as the internal-http2 commits tracked in #527.

Impact on req

req is a client library; each ClientConn holds one Framer, and the readLoop calls cc.fr.ReadFrame() (internal/http2/transport.go:2457):

  • Read side: after receiving a large frame, Framer.readBuf stays at frame size (16 kB with the default max frame size) for the whole connection lifetime, including while the readLoop is blocked waiting for the next 9-byte frame header — potentially minutes or hours on pooled idle connections. Upstream measured 32715 B → 16363 B heap per idle connection with the fix.
  • Write side: Framer.wbuf grows to the largest frame written (typically a full DATA frame) and is never released. Upstream measured an additional ~4.9 kB saved per connection.

For workloads holding many idle HTTP/2 connections (the common scraping / API-client scenario req targets), this is a meaningful per-connection memory reduction. Buffers up to 1 kB stay attached to the Framer, so connections exchanging only small control frames pay nothing for the pooling.

Suggested sync

Port the three frame.go commits as a series into internal/http2/frame.go (merge upstream http2: <date>(<hash>) style). Port notes:

  • req's copy has a single ReadFrame (no ReadFrameForHeader split), so releaseReadBuf() goes at the top of ReadFrame(), before readFrameHeader blocks.
  • The req-specific customizations in ReadFrame (currentRequest, dump integration) and the cc *ClientConn field are orthogonal to buffer pooling and must be preserved as-is.
  • Adds bufio/sync imports, readBufP/wbufP fields, readBufPool/writeBufPool, and the 1 kB maxIdleReadBufCap/maxIdleWriteBufCap constants.
  • No public API change; no behavior change beyond buffer lifetime.

Not security-related, no workaround needed — pure memory optimization, low priority.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with internal/http2/frame.go and compare it with the three referenced Go stdlib commits, preserving the req-specific ReadFrame customizations and cc field. Inspect the readLoop call at internal/http2/transport.go:2457. Done means the read and write buffers use the described pooling behavior without a public API change or altered request behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking, performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.