apache / apache/pinot

Improve GcsPinotFs to Read Credentials Directly from User Configurations

Open
#8,517 1 comment 0 reactions 1 assignee Claimed by @zhtaoxiang View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Now, the [GcsPinotFs](https://github.com/apache/pinot/blob/master/pinot-plugins/pinot-file-system/pinot-gcs/src/main/java/org/apache/pinot/plugin/filesystem/GcsPinotFS.java) can only [read credentials from a file] (https://github.com/apache/pinot/blob/master/pinot-plugins/pinot-file-system/pinot-gcs/src/main/java/org/apache/pinot/plugin/filesystem/GcsPinotFS.java#L78) specified in the user configuration.

This needs to be improved for the following reasons:
1. Users need to create a file first, then provide this file name in the configuration, which is not so convenient.
2. If users don't have access to the container running pinot, then users cannot provide credentials.

Ideally, GcsPinotFs should be similar to [S3PinotFS](https://github.com/apache/pinot/blob/master/pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java), allowing users to provide credential directly in configuration.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.