AWS, REST: Remote signer client's cache causes 403 errors
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
1.10.1 (latest release)
### Query engine
None
### Please describe the bug 🐞
The `S3V4RestSignerClient` caches signed responses based on a key that only includes method, region, and uri, but not request headers.
When a cached signed response is reused for a _different_ request with the same method/region/uri but different headers, the `reconstructHeaders` method overwrites the signed headers with the original request headers, invalidating the signature.
## Example Scenario
### Request A (PUT object with content hash `abc123`):
- Method: PUT
- URI: `s3://bucket/path/to/file.avro`
- Headers: `x-amz-content-sha256: abc123`
Server signs this request and returns signed headers.
Client caches the response with key `{PUT, us-east-1, s3://bucket/path/to/file.avro}`.
### Request B (PUT different content to the same URI with content hash `def456`):
- Method: PUT
- URI: `s3://bucket/path/to/file.avro`
- Headers: `x-amz-content-sha256: def456`
Cache hit, same key `{PUT, us-east-1, s3://bucket/path/to/file.avro}`.
`reconstructHeaders` takes the cached signed headers from Request A.
The signature was computed for `x-amz-content-sha256: abc123`, but the request now has `x-amz-content-sha256: def456`.
S3 rejects the request with 403 Forbidden because the signature doesn't match the request
## Impact
This bug causes intermittent 403 Forbidden errors when:
* Multiple requests target the same URI with different content
* The requests occur within the 30-second cache TTL
* The server returns `Cache-Control: private` (enabling caching)
## Suggested Fix
The cache key should include headers that are part of the signature calculation, particularly `x-amz-content-sha256`.
Alternatively, the caching logic should verify that the cached signed headers are compatible with the current request headers before reusing them.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
Contributor guide
Assessment
This issue has not been assessed yet.