Remove `object_store` dependency - Extract object store related code
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 168
Description
We treat `object_store` as a public API surface of parquet and avro (even if the traits used didn't change eg. from 0.13 to 0.14, presumably because things like how to contruct them has changed).
We should extract all code that uses `object_store` from the crates in this repository since it prevents us from upgrading `object_store`, especially when none of those pieces of functionality are being used.
- Related to https://github.com/apache/arrow-rs/issues/9878#issuecomment-4792008921
See also
- https://github.com/apache/arrow-rs/issues/10501
@alamb
Contributor guide
Research direction
Start by locating all crates and code in this repository that use object_store, with particular attention to parquet and avro. Read the related issues #9878 and #10501 for the intended boundaries and compatibility concerns. Done means object_store-dependent code is extracted from those crates so the dependency can be upgraded independently when unused functionality is not needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100