apache / apache/arrow-rs-object-store

List requests do not map HTTP status codes to error variants, so 403 and 404 both become `Error::Generic`

Open Beginner friendly
#851 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

### What happens

A `list` that fails with 403 or 404 comes back as `Error::Generic`. The same failure on `get` or `head` comes back as `PermissionDenied` or `NotFound`.

This means you cannot tell "I am not allowed to read this bucket" apart from "this bucket does not exist" apart from "the network is down", unless you parse the error string.

### Why

In `src/aws/client.rs`, only two variants are passed to the status-code mapper:

```rust
match err {
Error::CompleteMultipartRequest { source, path } => source.error(STORE, path),
Error::DeleteObjectsRequest { source, paths } => source.error(STORE, paths.join(",")),
_ => Self::Generic { store: STORE, source: Box::new(err) },
}
```

`Error::ListRequest` hits the `_` arm, so it never reaches `RetryError::error()`, which is the function that turns 404 into `NotFound`, 403 into `PermissionDenied` and 401 into `Unauthenticated`.

`src/gcp/client.rs` has the same problem. There, `GetRequest` and `Request` are mapped and `ListRequest` falls through.

### How to reproduce

Reproduced with obstore 0.9.2, against real S3:

```python
import asyncio, obstore
from obstore.store import S3Store

async def main():
# a private bucket -> S3 returns 403
s = S3Store(bucket="", region="us-east-1", skip_signature=True)

try:
await obstore.list_with_delimiter_async(s)
except Exception as e:
print(type(e).__name__) # GenericError

try:
await obstore.head_async(s, "probe")
except Exception as e:
print(type(e).__name__) # PermissionDeniedError

asyncio.run(main())
```

Same bucket, same 403, two different error variants.

A bucket that does not exist returns 404, and `list` reports that as generic too.

### Suggested fix

Add a `ListRequest` arm that goes through `source.error(...)`, the same way #365 / #366 did for the HTTP backend.

I am happy to open a PR if this looks right.

Contributor guide

Open the contributing guide

Research direction

Start in src/aws/client.rs and src/gcp/client.rs, comparing the existing CompleteMultipartRequest, DeleteObjectsRequest, GetRequest, and Request mappings with the ListRequest fall-through. Trace RetryError::error() and use the provided Python reproduction against S3 to verify that list failures now report the same 403, 404, and 401 variants as other requests instead of GenericError.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, gcp, rust
Domain
cloud
Issue type
Bug
Difficulty
2/5
Estimated time
Half a day
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.