apache / apache/arrow-rs

Add gRPC reflection support to arrow flight

Open
#1,706 0 comments 0 reactions 0 assignees View on GitHub
arrow-flight enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

We like to use the gRPC service reflection, via `grpcurl --plaintext localhost:8082 list`, to list gRPC services in IOx

We have a flight based service that does not appear in the service list (see https://github.com/influxdata/influxdb_iox/issues/4543)

I think this is because the arrow-flight library doesn't create the appropriate gRPC descriptors.

**Describe the solution you'd like**
Whatever is needed so that https://crates.io/crates/tonic-reflection can be used to provide reflection services to arrow-flight

@tustvold suggests:

I think we would likely need to make the generated descriptor set available as a constant within the produced binary, so that it can be passed to the reflection https://docs.rs/tonic-reflection/0.4.0/tonic_reflection/server/struct.Builder.html.

Effectively all gRPC reflection is, is a service that returns the compiled descriptor sets, which in turn are just a parsed version of the proto files, represented as protobuf - https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/descriptor.proto

**Additional context**
Here is an attempt (that didn't work): https://github.com/apache/arrow-rs/pull/1686

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the failed attempt in apache/arrow-rs#1686 and the Arrow Flight generated descriptor path. Check how the generated descriptor set could be made available to tonic-reflection, then verify the result with grpcurl --plaintext localhost:8082 list. Done means the Arrow Flight services appear in the reflection service list.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, rust
Domain
api, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.