duckdb / duckdb/duckdb-httpfs

HTTPFS curl client normalizes leading double slash in S3 path, causing AWS SigV4 SignatureDoesNotMatch

Open Beginner friendly
#333 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
60
Forks
100
Avg merge
1h 50m
Merged PRs (30d)
25

Description

## Cause

The behavior appears to come from `HTTPFSCurlClient::NormalizePathToBeAdded`, which strips leading `//` from paths before passing them to curl:

https://github.com/duckdb/duckdb-httpfs/blob/52afb4204a3238d6ee132e83340f8d68c40ee91c/src/httpfs_curl_client.cpp#L117-L127

That normalized path is then used when constructing the curl URL for GET requests:

https://github.com/duckdb/duckdb-httpfs/blob/52afb4204a3238d6ee132e83340f8d68c40ee91c/src/httpfs_curl_client.cpp#L241-L245

For S3, the leading slash is part of the object key. So these two paths are not equivalent:

```text
//prefix/file.csv.gz ->
/prefix/file.csv.gz

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with src/httpfs_curl_client.cpp, especially NormalizePathToBeAdded and the GET URL construction around the cited lines. Trace how an S3 path with a leading double slash reaches curl, then verify that the resulting request preserves the object key and no longer produces an AWS SigV4 SignatureDoesNotMatch error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cloud
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.