apache / apache/arrow

[C++] Make S3Filesystem consume key-value options

Open
#50,126 0 comments 0 reactions 1 assignee Claimed by @raulcd View on GitHub
Component: C++ Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

This is a follow up issue from:
- https://github.com/apache/arrow/issues/46369

Which added the key-value options to `FileSystemFromUri`. This issue makes `S3FileSystem` actually consume the options.

This is required in order to split S3Filesystem out of libarrow and into its own dedicated module (see: https://github.com/apache/arrow/issues/49399) because once s3 is on its own module users can no longer create their own S3Options and must use `FileSystemFromUri`.

We should expose, through the options channel, the `S3Options` settings that are not already expressible in the URI. The exception is `access_key`/`secret_key`, which can already be set in the URI (`s3://access_key:secret_key@bucket`); exposing them as options as well provides a path to set credentials without putting them in the URI (which is a security smell).

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.