apache / apache/pinot

Support query response dump in different formats and location

Open
#6,939 3 comments 2 reactions 0 assignees View on GitHub
feature good first issue
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

From the query endpoint, Pinot currently only supports customized JSON format.

It would be great to support:
1. user can specify the response format, e.g. CSV
2. user can specify a file to dump the query response to, e.g. local or remote(s3) file or directory.
3. user can specify the compression algorithm for output file.

This can be implemented in broker query API or we can do all the reformat and response dump using org.apache.pinot.tools.admin.command.PostQueryCommand

Contributor guide

Open the contributing guide

Research direction

Start by comparing the broker query API with org.apache.pinot.tools.admin.command.PostQueryCommand, the two implementation options named in the issue. Define how response format, local or remote output location, and compression are exposed, then verify that query responses can be dumped in each requested format and destination.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.