apache / apache/arrow-rs-object-store

Impossible to access `default_headers` from `ClientOptions` for custom `HttpConnector`

Open
#559 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

object_store: Impossible to access `default_headers` from ClientOptions; thus impossible to pass on to a custom HTTPService. We should make sure that all values from ClientOptions are accessible for custom http implementations.

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

When trying to implement a custom [`HttpConnector`](https://docs.rs/object_store/latest/object_store/client/trait.HttpConnector.html), the `connect` method takes in a `&ClientOptions` to be used for creating an `HttpClient`.

This implies that you should be able access all properties of `ClientOptions` in order to define the creation of your custom `HttpClient`.

However the only public API to access data from a `ClientOptions` is [`get_config_value`](https://docs.rs/object_store/latest/object_store/client/struct.ClientOptions.html#method.get_config_value), which is only defined for values of [`ClientConfigKey`](https://docs.rs/object_store/latest/object_store/client/enum.ClientConfigKey.html). This doesn't include (at least) [`default_headers`](https://docs.rs/object_store/latest/object_store/client/struct.ClientOptions.html#method.with_default_headers). There may be other properties not included.

**Describe the solution you'd like**

Perhaps we should create accessors for all struct fields inside `ClientOptions`?

**Describe alternatives you've considered**

Not forward on all configuration options to user-defined HTTP connectors.

**Additional context**

I'm [prototyping](https://github.com/developmentseed/obstore/pull/596) integrating the [`HttpConnector`](https://docs.rs/object_store/latest/object_store/client/trait.HttpConnector.html) into [`obstore`](https://github.com/developmentseed/obstore) — Python bindings to `object_store` — so that someone can define an arbitrary Python-based HTTP client to make the requests initiated by `object_store`.

Contributor guide

Open the contributing guide

Research direction

Start with ClientOptions::get_config_value, ClientOptions::with_default_headers, and HttpConnector::connect; compare the fields exposed through ClientConfigKey with those needed by custom HTTP implementations. Decide the public access strategy, then verify that a custom connector can receive default_headers and the relevant ClientOptions values.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.