Make `arrow-csv` and `arrow-json` optional for applicable crates
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Currently importing crates like `datafusion-expr` or `datafusion-common` pulls in `arrow` with its default features, which includes the rather heavy arrow parts `arrow-csv` and `arrow-json`. Since these parts are really only required during physical execution of the respective operators & data sources, that seems rather unnessary.
### Describe the solution you'd like
I see two options:
- set `default-features = false` for the workspace-level `arrow` dependency and then request the right features in the `Cargo.toml` files of the respective datafusion crates
- **better:** do NOT depend on the `arrow` umbrella/meta crate at all but directly on the subcrates that we need
### Describe alternatives you've considered
Keeping the dependencies simple but instead suffer from longer compile times and larger build artifacts.
### Additional context
_No response_
Contributor guide
Research direction
Start by reviewing the workspace-level Arrow dependency and the Cargo.toml files for datafusion-expr, datafusion-common, and the applicable DataFusion crates. Determine which crates require arrow-csv or arrow-json, then verify that unaffected crates no longer pull them in while the relevant builds and tests continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100