Support query response dump in different formats and location
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
From the query endpoint, Pinot currently only supports customized JSON format.
It would be great to support:
1. user can specify the response format, e.g. CSV
2. user can specify a file to dump the query response to, e.g. local or remote(s3) file or directory.
3. user can specify the compression algorithm for output file.
This can be implemented in broker query API or we can do all the reformat and response dump using org.apache.pinot.tools.admin.command.PostQueryCommand
Contributor guide
Research direction
Start by comparing the broker query API with org.apache.pinot.tools.admin.command.PostQueryCommand, the two implementation options named in the issue. Define how response format, local or remote output location, and compression are exposed, then verify that query responses can be dumped in each requested format and destination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100