apache / apache/arrow-rs-object-store
List requests do not map HTTP status codes to error variants, so 403 and 404 both become `Error::Generic`
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
### What happens
A `list` that fails with 403 or 404 comes back as `Error::Generic`. The same failure on `get` or `head` comes back as `PermissionDenied` or `NotFound`.
This means you cannot tell "I am not allowed to read this bucket" apart from "this bucket does not exist" apart from "the network is down", unless you parse the error string.
### Why
In `src/aws/client.rs`, only two variants are passed to the status-code mapper:
```rust
match err {
Error::CompleteMultipartRequest { source, path } => source.error(STORE, path),
Error::DeleteObjectsRequest { source, paths } => source.error(STORE, paths.join(",")),
_ => Self::Generic { store: STORE, source: Box::new(err) },
}
```
`Error::ListRequest` hits the `_` arm, so it never reaches `RetryError::error()`, which is the function that turns 404 into `NotFound`, 403 into `PermissionDenied` and 401 into `Unauthenticated`.
`src/gcp/client.rs` has the same problem. There, `GetRequest` and `Request` are mapped and `ListRequest` falls through.
### How to reproduce
Reproduced with obstore 0.9.2, against real S3:
```python
import asyncio, obstore
from obstore.store import S3Store
async def main():
# a private bucket -> S3 returns 403
s = S3Store(bucket="", region="us-east-1", skip_signature=True)
try:
await obstore.list_with_delimiter_async(s)
except Exception as e:
print(type(e).__name__) # GenericError
try:
await obstore.head_async(s, "probe")
except Exception as e:
print(type(e).__name__) # PermissionDeniedError
asyncio.run(main())
```
Same bucket, same 403, two different error variants.
A bucket that does not exist returns 404, and `list` reports that as generic too.
### Suggested fix
Add a `ListRequest` arm that goes through `source.error(...)`, the same way #365 / #366 did for the HTTP backend.
I am happy to open a PR if this looks right.
Contributor guide
Research direction
Start in src/aws/client.rs and src/gcp/client.rs, comparing the existing CompleteMultipartRequest, DeleteObjectsRequest, GetRequest, and Request mappings with the ListRequest fall-through. Trace RetryError::error() and use the provided Python reproduction against S3 to verify that list failures now report the same 403, 404, and 401 variants as other requests instead of GenericError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, gcp, rust
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- Half a day
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100