apache / apache/arrow-rs-object-store
Support Bearer Authorisation for `http` based object stores
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
It would be great to support Bearer Authorization for e.g. PMTiles URLs.
[Databricks Files API](https://docs.databricks.com/api/workspace/files/download) requires setting Bearer headers.
Adding the ability to specify the header would allow to load a PMTiles file from [Unity Catalog Volumes](https://docs.databricks.com/aws/en/volumes/volume-files).
**Describe the solution you'd like**
I think the simplest solution would be to allow configuring the following for http stores:
https://github.com/apache/arrow-rs-object-store/blob/95b0a94150acaa40ef2265adaba5052cafc594d6/src/client/mod.rs#L486
The difficult part here is mainly the "how do I configure this in the config file" part.
I am not sure if the ideas I have there are good and would like to hear others oppinons first ^^
**Describe alternatives you've considered**
Alternatively, this functionality could be made public api:
https://github.com/apache/arrow-rs-object-store/blob/95b0a94150acaa40ef2265adaba5052cafc594d6/src/client/builder.rs#L154-L155
Not sure what is the best approach.
**Additional context**
This issue was originally raised by @danielsparing in https://github.com/maplibre/martin/issues/2353
Contributor guide
Research direction
Start by reading src/client/mod.rs around the referenced configuration and src/client/builder.rs around the referenced public API. Determine how Bearer headers should be configured for HTTP object stores, including the config-file approach discussed in the issue; done means PMTiles and other HTTP store URLs can use the required Bearer Authorization header.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100