[FEA] Propagate relevant `storage_options` to KvikIO when reading from s3 with cuDF and kvikio s3 is enabled
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
The current approach to using KviKIO enabled s3 reads is to export all AWS specific credentials/auth options via env variables for kvikio and the `storage_options` provided aren't used/mapped to the read.
**Describe the solution you'd like**
Ideally cuDF could map `storage_options` into kvikio compliant options and propagate them down to the kvikio layer if kvikio s3 reads are enabled.
**Describe alternatives you've considered**
Current solution requires setting the options in a couple of places (storage_options+env vars) to fully leverage kvikio s3 + cudf reads.
**Additional context**
Related to #19757 (and possibly the solution is similar) but this request specific to the cuDF api.
Contributor guide
Assessment
This issue has not been assessed yet.