envoyproxy / envoyproxy/envoy

health check: use the ALPN-negotiated protocol for HTTP health checks

Open
#46,848 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.9k
Forks
5.6k
Avg merge
1d 20h
Merged PRs (30d)
437

Description

### Summary

When an HTTP health check connection negotiates a protocol via ALPN, Envoy discards the result and speaks whatever `codec_client_type` says. This proposes using the negotiated protocol to select the health check codec, with `codec_client_type` retained as the value used when nothing is negotiated.

### Current behavior

`codec_client_type` fixes the health check codec at config load. It applies to every host in the cluster and every check attempt, and there is no way to vary it per host.

ALPN negotiation does happen on health check connections whenever TLS is in use — Envoy offers a protocol list (from `tls_options.alpn_protocols`, or the cluster's TLS context when that is unset) and the server selects one. Envoy then ignores the selection and uses the configured codec regardless.

Data-plane connections behave differently: a cluster configured with `AutoHttpConfig` advertises multiple protocols and uses whichever one ALPN settles on. Health checking is the one place where the negotiated protocol is thrown away.

### Motivation

**1. `auto_config` clusters cannot be health checked representatively.** When the data plane negotiates h2-or-http/1.1 per connection, no single `codec_client_type` matches what it will actually use. The health check exercises one protocol while real traffic may use the other.

**2. Heterogeneous and mid-migration upstreams break outright.** If some hosts in a cluster serve h2 and others only http/1.1 — a mixed fleet, or an upstream partway through an HTTP/1.1 → h2 rollout — a pinned codec marks every host of the other protocol unhealthy. Since the codec cannot vary per host, the options are splitting the cluster or flipping health check config in lockstep with the rollout, with a window where checks fail either way. Using the negotiated protocol makes both cases work with no config change.

**3. Contradictory config resolves itself instead of failing at runtime.** Today this loads cleanly and then fails forever:

```yaml
http_health_check:
path: /healthz
codec_client_type: HTTP2
tls_options:
alpn_protocols: ["http/1.1"] # can never negotiate h2
```

The handshake settles on `http/1.1`, Envoy sends HTTP/2, and every check fails — no config rejection, and nothing in the logs pointing at ALPN as the cause. Honoring the negotiated protocol makes the two fields consistent by construction.

### Proposed behavior

- When the health check connection negotiates an ALPN protocol, use it to select the codec: `h2` → HTTP/2, `http/1.1` → HTTP/1.1.
- When nothing is negotiated — plaintext health checks, or a peer that does not do ALPN — fall back to `codec_client_type`. The field keeps a well-defined meaning rather than becoming dead config.
- When `tls_options` is unset, default the offered ALPN list from `codec_client_type` (`http/1.1` or `h2`), so the handshake advertises something consistent with the configured intent instead of inheriting unrelated cluster TLS settings. Operators who want real negotiation set `alpn_protocols: ["h2", "http/1.1"]` explicitly.
- Applies to HTTP/1.1 and HTTP/2. `codec_client_type: HTTP3` keeps today's behavior, and gRPC health checks are unaffected since gRPC requires HTTP/2.

*Relevant Links*:

- `codec_client_type`: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/health_check.proto#envoy-v3-api-field-config-core-v3-healthcheck-httphealthcheck-codec-client-type
- `tls_options`: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/health_check.proto#envoy-v3-api-msg-config-core-v3-healthcheck-tlsoptions
- Data-plane equivalent (`AutoHttpConfig`): https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/upstreams/http/v3/http_protocol_options.proto#extensions-upstreams-http-v3-httpprotocoloptions-autohttpconfig

Contributor guide

Open the contributing guide

Research direction

Start by tracing HTTP health-check codec selection from codec_client_type and the TLS health-check handling for tls_options and ALPN. Compare that flow with the data-plane AutoHttpConfig behavior, then verify the proposed h2, http/1.1, plaintext, HTTP3, and gRPC cases; done means negotiated protocols select the matching codec while the documented fallback behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.