developmentseed / developmentseed/multistore
List of a prefix 503s when a single object key contains an empty path segment (//)
- Dominant language
- Rust
- Stars
- 19
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
### Summary
When a ListObjectsV2 page returned by the backend contains a single object whose key has an **empty path segment** (a `//`), `multistore` fails the **entire** list request with `503 ServiceUnavailable` instead of listing the other (valid) keys. One malformed object key takes out the whole prefix.
`object_store`'s strict `Path` parser rejects empty segments, and multistore propagates that as a fatal backend error for the page rather than skipping the offending key.
### Observed error
Proxy log:
```
[WARN] multistore::proxy: request failed {
error=backend error: Encountered object with invalid path:
Path "raw//" contained empty path segment,
status=503, s3_code=ServiceUnavailable
}
```
Client just sees the generic:
```
An error occurred (ServiceUnavailable) when calling the ListObjectsV2 operation
(reached max retries: 2): Service unavailable
```
### Reproduction
An object genuinely exists in S3 with a `//` in its key (double slash after `raw/`):
```
s3://.../raw//b.e21.BW.f09_g17.SSP245-TSMLT-GAUSS-DELAYED-2045.001/atm/proc/tseries/day_1/....nc
```
`aws s3 ls --no-sign-request` against the raw bucket lists it fine (S3 keys are opaque byte strings and `//` is legal). Listing the same prefix through multistore 503s.
Only prefixes that *contain* such a key fail; sibling prefixes list normally.
### Impact
A single stray upload with a double slash (or a zero-byte folder-marker object ending in `//`) makes an entire prefix unlistable through the gateway, with an error that gives the operator no hint about which object or why. S3 itself accepts and serves these keys, so the gateway is stricter than the backend it fronts.
### Suggested fix
On list, don't let one unparseable key fail the page. Options:
- **Skip** keys that fail strict `Path::parse`, ideally logging the raw key at `warn` so operators can find and fix it; or
- Parse list keys leniently (`Path::from` / `from_url_path`, which tolerate empty segments) instead of strict `parse`.
At minimum, surface the offending raw key in the error so it's diagnosable without correlating backend logs.
### Environment
- multistore 0.6.3 (features: `azure`), via `multistore-cf-workers` on Cloudflare Workers
- Backend: S3
Contributor guide
Research direction
Start by tracing ListObjectsV2 page handling to the strict Path::parse call described in the issue, then inspect existing list-related tests. Reproduce with a key containing // and verify that one invalid key no longer causes the whole prefix listing to return 503, while the offending key is either skipped with a warning or handled leniently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100