apache / apache/arrow-rs-object-store

Support Bearer Authorisation for `http` based object stores

Open
#539 2 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

It would be great to support Bearer Authorization for e.g. PMTiles URLs.

[Databricks Files API](https://docs.databricks.com/api/workspace/files/download) requires setting Bearer headers.
Adding the ability to specify the header would allow to load a PMTiles file from [Unity Catalog Volumes](https://docs.databricks.com/aws/en/volumes/volume-files).

**Describe the solution you'd like**

I think the simplest solution would be to allow configuring the following for http stores:

https://github.com/apache/arrow-rs-object-store/blob/95b0a94150acaa40ef2265adaba5052cafc594d6/src/client/mod.rs#L486

The difficult part here is mainly the "how do I configure this in the config file" part.
I am not sure if the ideas I have there are good and would like to hear others oppinons first ^^

**Describe alternatives you've considered**

Alternatively, this functionality could be made public api:

https://github.com/apache/arrow-rs-object-store/blob/95b0a94150acaa40ef2265adaba5052cafc594d6/src/client/builder.rs#L154-L155

Not sure what is the best approach.

**Additional context**

This issue was originally raised by @danielsparing in https://github.com/maplibre/martin/issues/2353

Contributor guide

Open the contributing guide

Research direction

Start by reading src/client/mod.rs around the referenced configuration and src/client/builder.rs around the referenced public API. Determine how Bearer headers should be configured for HTTP object stores, including the config-file approach discussed in the issue; done means PMTiles and other HTTP store URLs can use the required Bearer Authorization header.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.