chore: Explore combining `SessionConfig` and `RuntimeConfig` into same framework
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
A followup on #23316
Can we please explore if runtime config can be part of `SessionConfig`? In `SessionConfig` there is already a framework that enforces type checking.
Runtime configs looks looks finite according to `set_runtime_variable`, so should fit into main configs rather than having separate config set
One difference was that the `RuntimeConfig` is potentially shared across multiple `SessionConfigs` (and maybe can also be adjusted during plan execution)
Trying to consolidate `RuntimeConfig` and `SessionConfig` should be considered as an independent project
_Originally posted by @alamb in https://github.com/apache/datafusion/issues/23316#issuecomment-4881517563_
Contributor guide
Research direction
Start by tracing SessionConfig, RuntimeConfig, and set_runtime_variable in the DataFusion source. Compare the existing type-checking framework with runtime configuration, including sharing across multiple SessionConfigs and adjustment during plan execution. Done means documenting whether consolidation is viable and resolving those lifecycle differences in a concrete design.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100