cloudflare / cloudflare/pingora

Incorrectly parsing gRPC-Web requests to native gRPC

Open
#513 2 comments 0 reactions 1 assignee Claimed by @drcaramelsyrup View on GitHub
bug enhancement
Dominant language
Rust
Stars
27.4k
Forks
1.7k
Avg merge
6h 22m
Merged PRs (30d)
3

Description

## Describe the bug

The GrpcWeb module does not properly convert from `application/grpc-web-text` to `application/grpc`, which results in the error: `protocol error: received message with invalid compression flag: 65 (valid flags are 0 and 1), while sending request`.

After attempting to set up the `GrpcWebModule` following the [reference example](https://github.com/cloudflare/pingora/blob/main/pingora-proxy/examples/grpc_web_module.rs), I was unable to forward requests from gRPC-Web to native gRPC. After some investigation, I discovered that the module doesn't upgrade the connection from HTTP/1 to HTTP/2 properly. Specifically, it doesn't change the content type as expected: `application/grpc-web-text` was converted to `application/grpc-text`. The issue arises from [this line](https://github.com/cloudflare/pingora/blob/main/pingora-core/src/protocols/http/bridge/grpc_web.rs#L113).

I assume that the error with the compression flag is related to the content-type parsing, as mentioned in this [comment](https://github.com/hyperium/tonic/issues/100#issuecomment-547176921).

## Pingora info

**Pingora version**: `0.4.0`
**Rust version**: `cargo 1.83.0 (5ffbef321 2024-10-29)`
**Operating system version**: macOS Sonoma 14.7.1

## Steps to reproduce

I created a [repository](https://github.com/davihsg/pingora-grpc-web) with all the artifacts necessary to reproduce the error.

First, I have a simple [Hello gRPC server](https://github.com/davihsg/pingora-grpc-web/blob/main/server/src/main.rs) written in Rust with `tokio 0.10.2`. It runs on port 50051.

Then, I have the Pingora proxy. It also runs a simple [CORS service](https://github.com/davihsg/pingora-grpc-web/blob/main/proxy/src/config/grpc_web_preflight.rs) to handle preflight requests:

```rust
impl ServeHttp for GrpcWebPreflightHttpApp {
async fn response(&self, _http_session: &mut ServerSession) -> Response> {
let buffer = "Grpc Web OK".as_bytes().to_vec();

Response::builder()
.status(200)
.header(
http::header::ACCESS_CONTROL_ALLOW_HEADERS,
"content-type,x-grpc-web,x-user-agent",
)
.header(http::header::ACCESS_CONTROL_ALLOW_METHODS, "POST")
.header(http::header::CONTENT_LENGTH, buffer.len())
.header(http::header::ACCESS_CONTROL_ALLOW_ORIGIN, "null")
.header(http::header::ACCESS_CONTROL_EXPOSE_HEADERS, "*")
.header(http::header::ACCESS_CONTROL_MAX_AGE, 1728000)
.body(buffer)
.unwrap()
}
}
```

The proxy configuration is as follows. The GrpcWeb module is added in `init_downstream_modules` and initialized for every request in `early_request_filter`, as shown in the reference example. For the [upstream_peer](https://github.com/davihsg/pingora-grpc-web/blob/main/proxy/src/main.rs#L51), it returns the GrpcWebPreflight peer whenever the method is `OPTIONS`, or the Hello server peer otherwise. The ALPN version is set to H2 for the Hello server peer.

```rust
async fn upstream_peer(
&self,
session: &mut Session,
_ctx: &mut Self::CTX,
) -> Result, Box> {
let method = &session.req_header().method;

if method == "OPTIONS" {
return Ok(create_grpc_web_preflight_peer());
}

// gRPC server
let mut peer = Box::new(HttpPeer::new(
String::from("0.0.0.0:50051"),
false,
String::from(""),
));

peer.options.alpn = ALPN::H2;

Ok(peer)
}
```

The proxy runs at `0.0.0.0:6193`.

gRPC requests can be sent with [grpcurl](https://github.com/davihsg/pingora-grpc-web/blob/main/client/grpc/request.sh):

```bash
ADDRESS="localhost:6193"

grpcurl \
-plaintext \
-d '{}' \
-import-path ../../protos \
-proto ../../protos/helloworld.proto \
-vv \
$ADDRESS \
helloworld.Greeter/SayHello
```

For gRPC-Web, there is a simple [JavaScript client](https://github.com/davihsg/pingora-grpc-web/tree/main/client/grpc-web) that sends a Hello request. You can open [index.html](https://github.com/davihsg/pingora-grpc-web/blob/main/client/grpc-web/dist/index.html) in the browser and provide the proxy URL.

![Image](https://github.com/user-attachments/assets/a2461114-5111-4647-a50d-d795044cc1fd)

## Expected results

Native gRPC requests should be ignored by the module and forwarded normally.

gRPC-Web requests with content type `application/grpc-web-text` should be parsed as `application/grpc` before being sent to the upstream peer, and the response should be parsed back to `application/grpc-web+{proto | json | ... }`.

## Observed results

Native gRPC requests are being ignored by the module and forwarded normally, as expected.

gRPC-Web requests with content type `application/grpc-web-text` are being incorrectly parsed as `application/grpc-text`, which results in the error `protocol error: received message with invalid compression flag: 65 (valid flags are 0 and 1), while sending request`.

## Additional context

`grpc-web` should be treated differently from `grpc-web-text`. As outlined in the [gRPC documentation](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md#protocol-differences-vs-grpc-over-http2), `grpc-web-text` is text-encoded and should be parsed before being converted to native gRPC.

This is similar to how Envoy handles it [here](https://github.com/envoyproxy/envoy/blob/9a04f4c96372b222cd8366aca9f7ebf6122c653a/source/extensions/filters/http/grpc_web/grpc_web_filter.cc#L169) with its gRPC-Web filter.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.