Add an API to provide the reason / what is being requested in IO
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
So we basically have a 2nd file parser here 🤔 I'm wondering if instead of creating yet-another-parser -- even though it's partial -- we could ask the decoder to provide us a "reason" or a "trace" on the individual read requests. For example, we could extend `AsyncFileReader` with an implemented-by-default method:
```rust
trait AsyncFileReader {
// all the current methods stay!
fn get_bytes_with_trace(&mut self, range: Range, trace: Trace) -> BoxFuture<'_, parquet::errors::Result> {
// ignore trace by default
self.get_bytes(range)
}
// same for the other two methods...
}
// bikeshed whatever `Trace` is, maybe use :http::Extensions?
```
_Originally posted by @crepererum in https://github.com/apache/arrow-rs/pull/7971#discussion_r2262364717_
Contributor guide
Research direction
Start with the AsyncFileReader trait and the discussion in PR #7971 referenced by the issue. Determine the agreed shape for a trace or reason and which read methods should accept it; done means the API design is resolved and the requested IO context can reach individual reads without another parser.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100