HTTPFS curl client normalizes leading double slash in S3 path, causing AWS SigV4 SignatureDoesNotMatch
- Dominant language
- C++
- Stars
- 60
- Forks
- 100
- Avg merge
- 1h 50m
- Merged PRs (30d)
- 25
Description
## Cause
The behavior appears to come from `HTTPFSCurlClient::NormalizePathToBeAdded`, which strips leading `//` from paths before passing them to curl:
https://github.com/duckdb/duckdb-httpfs/blob/52afb4204a3238d6ee132e83340f8d68c40ee91c/src/httpfs_curl_client.cpp#L117-L127
That normalized path is then used when constructing the curl URL for GET requests:
https://github.com/duckdb/duckdb-httpfs/blob/52afb4204a3238d6ee132e83340f8d68c40ee91c/src/httpfs_curl_client.cpp#L241-L245
For S3, the leading slash is part of the object key. So these two paths are not equivalent:
```text
//prefix/file.csv.gz ->
/prefix/file.csv.gz
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with src/httpfs_curl_client.cpp, especially NormalizePathToBeAdded and the GET URL construction around the cited lines. Trace how an S3 path with a leading double slash reaches curl, then verify that the resulting request preserves the object key and no longer produces an AWS SigV4 SignatureDoesNotMatch error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100