port-forward uses a SPDY-only dialer and fails behind proxies that do not support SPDY
Nobody has claimed this yet.
- Dominant language
- Makefile
- Stars
- 597
- Forks
- 227
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 16
Description
Bug report
General Information
- Cilium CLI version: v0.16.24 (the code below is unchanged on
mainand in v0.19.4) - Kubernetes: kubectl v1.31+, any apiserver version
- Platform: environment-independent — reproduces whenever an HTTP proxy / gateway /
load balancer that does not support SPDY sits in front of the kube-apiserver
Problem
cilium port-forward (e.g. cilium hubble port-forward) fails with
error upgrading connection: ... Upgrade request required when an HTTP proxy that
does not support SPDY is in front of the kube-apiserver.
The dialer in pkg/k8s/portforward/portforward.go (vendored from cilium/cilium)
is SPDY-only, with no WebSocket attempt and no fallback:
roundTripper, upgrader, err := spdy.RoundTripperFor(pf.config)
dialer := spdy.NewDialer(upgrader, &http.Client{Transport: roundTripper}, http.MethodPost, req.URL())
SPDY has been deprecated since 2015 (KEP-4006) and is no longer supported by many
proxies/gateways/load balancers. kubectl made WebSocket the default for port-forward
in v1.31, with SPDY as a fallback.
Why exec is unaffected
The exec path already uses the apimachinery WebSocket/SPDY fallback dialer
(createDialer, added in #37538). port-forward was never migrated.
#38988 later set exec to SPDY-primary with WebSocket fallback, due to hardcoded
idle-timeout issues in the upstream remotecommand WebSocket executor. That is
specific to remotecommand and does not apply to port-forward, which uses a separate
path (portforward.NewSPDYOverWebsocketDialer / the tunneling dialer).
Fix
Use the apimachinery fallback dialer in portforward.go with WebSocket as primary
and SPDY as fallback, matching kubectl's port-forward behaviour since v1.31:
spdyRoundTripper, upgrader, err := spdy.RoundTripperFor(pf.config)
if err != nil {
return nil, err
}
spdyDialer := spdy.NewDialer(upgrader, &http.Client{Transport: spdyRoundTripper}, http.MethodPost, req.URL())
tunnelingDialer, err := portforward.NewSPDYOverWebsocketDialer(req.URL(), pf.config)
if err != nil {
return nil, err
}
dialer := portforward.NewFallbackDialer(tunnelingDialer, spdyDialer, func(err error) bool {
return httpstream.IsUpgradeFailure(err) || httpstream.IsHTTPSProxyError(err)
})
SPDY remains an automatic fallback for older API servers, so the change is
backward-compatible.
How to reproduce
- Place an HTTP proxy that does not support SPDY in front of the kube-apiserver.
- Run
cilium hubble port-forward(or anyciliumcommand that port-forwards). - The SPDY upgrade is rejected and the command fails.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pkg/k8s/portforward/portforward.go, where the SPDY-only dialer is created, and compare the exec path's createDialer with Kubernetes port-forward behavior. Verify the WebSocket-primary, SPDY-fallback behavior through an HTTP proxy that rejects SPDY, while preserving compatibility with older API servers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- cli, networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100