apache / apache/iceberg

AWS, REST: Remote signer client's cache causes 403 errors

Open
#15,166 5 comments 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.10.1 (latest release)

### Query engine

None

### Please describe the bug 🐞

The `S3V4RestSignerClient` caches signed responses based on a key that only includes method, region, and uri, but not request headers.

When a cached signed response is reused for a _different_ request with the same method/region/uri but different headers, the `reconstructHeaders` method overwrites the signed headers with the original request headers, invalidating the signature.

## Example Scenario

### Request A (PUT object with content hash `abc123`):

- Method: PUT
- URI: `s3://bucket/path/to/file.avro`
- Headers: `x-amz-content-sha256: abc123`

Server signs this request and returns signed headers.
Client caches the response with key `{PUT, us-east-1, s3://bucket/path/to/file.avro}`.

### Request B (PUT different content to the same URI with content hash `def456`):

- Method: PUT
- URI: `s3://bucket/path/to/file.avro`
- Headers: `x-amz-content-sha256: def456`

Cache hit, same key `{PUT, us-east-1, s3://bucket/path/to/file.avro}`.
`reconstructHeaders` takes the cached signed headers from Request A.
The signature was computed for `x-amz-content-sha256: abc123`, but the request now has `x-amz-content-sha256: def456`.
S3 rejects the request with 403 Forbidden because the signature doesn't match the request

## Impact

This bug causes intermittent 403 Forbidden errors when:

* Multiple requests target the same URI with different content
* The requests occur within the 30-second cache TTL
* The server returns `Cache-Control: private` (enabling caching)

## Suggested Fix

The cache key should include headers that are part of the signature calculation, particularly `x-amz-content-sha256`.

Alternatively, the caching logic should verify that the cached signed headers are compatible with the current request headers before reusing them.

### Willingness to contribute

- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.