apache / apache/pinot

Support multiple response formats in Broker SQL API

Open
#8,437 6 comments 5 reactions 1 assignee Claimed by @KKcorps View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

This issue extends upon https://github.com/apache/pinot/issues/6939.

Currently, for a query, broker can only return JSON response. However, for better performance on client side, it might be desirable to get response in a compact, fast, binary format such as Avro, ProtoBuf, FlatBuffers etc.

There are two changes that can be done to support this request -

* Create a gRPC client for existing `GrpcBrokerRequestHandler` and provide it to users.

* Allow a `format` parameter to be passed in `/query` API and return response in the specified format.

```
curl --location --request POST 'http://localhost:8000/query/sql' \
--header 'Content-Type: application/json' \
--data-raw '{"sql" : "SELECT * FROM baseballStats LIMIT 10",
"format" : "avro"
}'
```

This will reduce the deserialization cost on the client end.

What is not desirable is simply converting json response to any other format as that just transfers the cost for serialization/deserialization to Pinot.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.