apache / apache/arrow-rs-object-store
Slow local file reads when using GetResult.into_stream
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Describe the bug**
See https://github.com/apache/datafusion/issues/21450
Root cause: there's a `spawn_blocking` call per each 8KiBs read from the file, adding significant context switch overhead
**To Reproduce**
See https://github.com/apache/datafusion/issues/21446
For the tests I've used a c7a.16xlarge ec2 instance, with a trimmed down version of hits.json to 51G (original has 217 GiB), with a warm cache (by running cat hits_50.json > /dev/null)
**Expected behavior**
A more efficient implementation (e.g. tokio uses a buffer size of 2MiBs when reading files)
**Additional context**
https://github.com/apache/datafusion/pull/21478#discussion_r3078305447
Contributor guide
Research direction
Start at GetResult.into_stream and trace the per-8KiB spawn_blocking path described in the report. Compare its read behavior with the 2MiB buffering noted for tokio, then use the linked DataFusion reproductions (issues 21446 and 21450) with the warm-cache 51G file to verify that reads no longer incur excessive context-switch overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100