gRPC response body and trailers stripped through tunnel even with TLS+ALPN+h2 origin (cloudflared 2026.3.0)

# gRPC response body and trailers stripped through Cloudflare Tunnel even with TLS+ALPN+h2 origin (cloudflared 2026.3.0)

## Summary

Through a fully-documented gRPC-tunnel configuration (origin uses TLS, advertises `h2` via ALPN, `originRequest.http2Origin: true`, `noTLSVerify: true`), responses lose their body and trailers between the origin and the external client. The same origin returns full responses when reached directly inside the cluster — only the path through the tunnel is affected.

## Environment

- cloudflared: `2026.3.0` (also reproduced on `2025.8.1`)
- Tunnel type: locally-managed (config.yaml, not Dashboard)
- Origin: nginx 1.27 with `grpc_pass` to a Tonic (Rust) gRPC server, listening on TLS:50443 with self-signed cert and `h2` ALPN
- Client: `grpcurl 1.9.1` and `grpc-java-netty 1.80.0`
- Tunnel transport: QUIC (default), 4 connections to fra03/fra16/fra17/fra18 edge

## cloudflared config

```yaml
tunnel:
credentials-file: /etc/cloudflared/creds/credentials.json
metrics: 0.0.0.0:2000
no-autoupdate: true
ingress:
- hostname: grpc.example.com
service: https://grpc-proxy.namespace.svc.cluster.local:50443
originRequest:
http2Origin: true
noTLSVerify: true
- service: http_status:404
```

The origin nginx config:

```nginx
events { worker_connections 1024; }
http {
client_max_body_size 64m;
grpc_read_timeout 600s;
grpc_send_timeout 600s;
server {
listen 50443 ssl;
http2 on;
ssl_certificate /etc/nginx/tls/tls.crt;
ssl_certificate_key /etc/nginx/tls/tls.key;
location / {
grpc_pass grpc://tonic-backend.namespace.svc.cluster.local:50051;
grpc_set_header authorization $http_authorization;
}
}
}
```

## Symptom

External client through Cloudflare:

```
$ curl -s -o /dev/null -w "status=%{http_code} ct=%{content_type} size=%{size_download}\n" \
--http2 -X POST 'https://grpc.example.com/some.Service/Method' \
-H 'content-type: application/grpc' -H 'te: trailers'
status=200 ct=application/grpc size=0
```

```
$ grpcurl -import-path /p -proto x.proto -H "authorization: Bearer …" \
-d '{"lat":48.8566,"lng":2.3522}' grpc.example.com:443 some.Service/Method
ERROR:
Code: Internal
Message: server closed the stream without sending trailers
```

The HTTP/2 response status is 200 with the correct `content-type: application/grpc`, but **the body is empty (0 bytes) and no `grpc-status` / `grpc-message` trailers are sent**. The gRPC client treats this as `Internal: server closed the stream without sending trailers` (or `Received unexpected EOS on empty DATA frame from server` in grpc-java).

## Same origin works directly

`kubectl port-forward` to the in-cluster proxy Service, then identical grpcurl call locally:

```
$ grpcurl -insecure -import-path /p -proto x.proto -H "authorization: Bearer …" \
-d '{"lat":48.8566,"lng":2.3522}' localhost:50443 some.Service/Method
{
"continentCode": "EU",
"countryCode": "FR"
}
```

Full response body, proper `grpc-status: 0` trailer.

In-cluster pod (different from the cloudflared pod) calling the same Service DNS name with TLS:

```
$ kubectl run … --image fullstorydev/grpcurl:v1.9.1 \
-- -insecure -import-path /proto -proto geocoding.proto … \
grpc-proxy.namespace.svc.cluster.local:50443 some.Service/Method
{ "continentCode": "EU", "countryCode": "FR" }
```

So the proxy is correctly emitting HTTP/2 responses with body + trailers. Confirmed in nginx access log:

```
… "POST /some.Service/Method HTTP/2.0" status=200 sent=71 upstream=…
```

## What we ruled out

- **Origin not speaking HTTP/2**: nginx access log shows it received the request as `HTTP/2.0` from cloudflared and emitted a 71-byte response. The in-cluster grpcurl test confirms the proxy returns full bodies + trailers.
- **`http2Origin: true` not taking effect**: with this config (HTTPS origin + the flag), nginx access log explicitly shows HTTP/2.0. (For comparison: with `service: http://…` + same flag, nginx access log shows `HTTP/1.1` from cloudflared, matching the documented "HTTP/2 origin requires SSL" behavior — that's a separate finding, not the one in this report.)
- **Origin failing mid-stream**: backend returns proper responses to in-cluster clients; nginx is healthy, no errors.
- **gRPC zone setting**: API token doesn't have permission to read it, but the response status is `200` (not `403 Forbidden` as the docs say a disabled zone returns).
- **cloudflared version**: same behavior on `2025.8.1` and `2026.3.0`.

## Hypothesis

cloudflared (or the Cloudflare edge over the tunnel transport) is dropping HTTP/2 DATA frames and/or HEADERS-with-trailers when proxying gRPC responses back to the client. The headers (`:status`, `content-type: application/grpc`) are preserved but DATA / trailers are not.

## Repro steps

1. Deploy a Tonic gRPC server (or any HTTP/2-prior-knowledge server) on `:50051` cleartext.
2. Deploy nginx with the config above (`grpc_pass` to the Tonic backend, listen TLS+h2 on `:50443`).
3. Configure cloudflared to point at `https://nginx:50443` with `originRequest.http2Origin: true` and `noTLSVerify: true`.
4. From outside the cluster, run grpcurl against the public hostname.
5. Observe: status 200, content-type `application/grpc`, but empty body and no trailers.
6. From inside the cluster (port-forward or in-cluster pod), run the same grpcurl against the same nginx Service.
7. Observe: full response body + `grpc-status: 0` trailer.

## What we'd like

Either confirmation that this is a known bug with a fix in flight, or guidance on what we should configure differently. If the answer is "this can't currently work for HTTP/2-prior-knowledge origins," that's worth documenting on the gRPC docs page (which currently implies any HTTP/2 origin should work).