apache / apache/arrow-rs-object-store
Refactoring the configuration system
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Each per-store config key in object_store is currently duplicated across ~8 hand-maintained locations:
- the XxxConfigKey variant,
- its doc + "Supported keys:" list,
- the builder field,
- the AsRef arm,
- the FromStr arm (with all aliases),
- the with_config write arm,
- the get_config_value read arm,
- and the with_ convenience setter.
That is a lot of manual sync points which drift independently.
Keeping it in sync by review alone is not realistic if we are honest.
**Describe the solution you'd like**
A small internal proc-macro derive that makes the struct field the single source of truth:
```rust
#[derive(ObjectStoreConfig)]
#[object_store(config_key = ClientConfigKey, error_path = crate::Error::UnknownConfigurationKey, error_store =
"HTTP")]
pub struct ClientOptions {
/// Set timeout for the overall request.
timeout: Option>,
#[config(setter = skip)] // hand-written setter clears http2_only too
http1_only: ConfigValue,
#[config(skip)]
secrete_internal_type: (),
// ... etc
}
```
The code noted above is auto-generated, so the user does not need to deal with this.
**Describe alternatives you've considered**
I considered
- a `macro_rules!` proc-macro, but i could not quite make the API make the same "level of sense" as the derive-macro one.
- DataFusion's `config_namespace!` approach which seems overkill. It pulls in a ConfigField trait + visitor infrastructure for the nested-namespace case we don't have...
This comes with the downside of needing another crate, not sure if this is a deal breaker.
**Additional context**
I think a good path for such a thing is to do it in an iterative way, first starting with `ClientOptions` and many tests to ensure that everything works as intendet.
Contributor guide
Research direction
Start by locating ClientOptions and the existing per-store configuration key handling. Review the current generated-equivalent paths and design the derive macro incrementally for ClientOptions, including the documented skip and setter behavior. Add many tests to confirm the generated configuration behavior stays synchronized and works as intended.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100