apache / apache/arrow

[FlightRPC][C++] Support implementing simple endpoints with async API

Open
#34,221 5 comments 1 reaction 0 assignees View on GitHub
Component: C++ Component: FlightRPC Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the enhancement requested

For high-QPS (queries per second) servers, gRPC recommends implementing services using the Async APIs over the Sync API ([docs](https://grpc.io/docs/guides/performance/#c)). In some Flight designs, particularly ones where there is a single coordinator node that all clients go to before talking to the data nodes, the `ListFlights` and `GetFlightInfo` may see many queries per second. We should allow defining most endpoints using the callback gRPC API (completion queue is another option, but probably overly complex).

We can probably keep methods that exchange Arrow data out-of-scope, since they are meant to be optimized for high-throughput rather than high QPS.

I wonder if the signature of the methods could be something like:

```cpp
Future> GetFlightInfo(const ServerCallContext& context,
const FlightDescriptor& request);
```

Though not sure if there is a straight-forward way to hook our `Future`s into the gRPC callback API.

### Component(s)

C++, FlightRPC

Contributor guide

Open the contributing guide

Research direction

Start with the FlightRPC C++ implementations of ListFlights and GetFlightInfo, then review the gRPC callback API and how Arrow Futures could connect to it. Keep data-exchanging methods out of scope unless the design requires otherwise. Done means simple endpoints can be implemented asynchronously with a documented, workable API.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.