apache / apache/arrow-rs-object-store
Add ObjectStore::list_opts
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
We should add an ObjectStore::list_opts to allow
* Specifying both a start offset and delimiter apache/arrow-rs-object-store#291
* Providing `Extensions` apache/arrow-rs#7213
**Describe the solution you'd like**
I _think_ the following API should work and eventually allow providing default implementations of `list` and `list_with_delimiter` that call through to `list_opts`
```
struct ListOptions {
pub offset: Option,
pub delimiter: bool,
pub extensions: Extensions
}
fn list_opts(&self, opts: ListOptions) -> BoxStream<'static, Result>
```
**Describe alternatives you've considered**
We could expose an API that more explicitly exposes the paginated nature of the underlying APIs, however, such APIs are quite cumbersome to use, and wouldn't obviously translate to LocalFileSystem.
**Additional context**
Contributor guide
Research direction
Start by locating the ObjectStore trait and its existing list and list_with_delimiter APIs, then review the context in apache/arrow-rs-object-store#291 and apache/arrow-rs#7213. Done means the proposed ListOptions and list_opts API are integrated with the requested offset, delimiter, and Extensions inputs, with the existing listing behavior accounted for.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100