awslabs / awslabs/aws-sdk-rust
Endpoint URL with path component causes incorrect request URLs for S3-compatible services
- Dominant language
- Rust
- Stars
- 3.3k
- Forks
- 290
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 3
Description
### Describe the bug
When using S3-compatible services that expose their S3 API behind a path prefix (e.g., `https://project.supabase.co/storage/v1/s3`), the AWS SDK for Rust does not work, making it incompatible with these services. Using libraries like boto3 works.
This is similar to issues reported in other AWS SDK implementations:
- AWS SDK PHP: [Issue #3037](https://github.com/aws/aws-sdk-php/issues/3037) and [Issue #3099](https://github.com/aws/aws-sdk-php/issues/3099)
- AWS SDK JavaScript: [Issue #4061](https://github.com/aws/aws-sdk-js-v3/issues/4061
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
When configuring an endpoint URL with a path component:
```rust
let config = aws_sdk_s3::Config::builder()
.endpoint_url("https://project.supabase.co/storage/v1/s3")
.region(Region::new("us-east-1"))
.credentials_provider(credentials)
.build();
let client = aws_sdk_s3::Client::from_conf(config);
```
Accessing bucket `my-bucket` and key `file.parquet` should construct the URL:
```
https://project.supabase.co/storage/v1/s3/my-bucket/file.parquet
```
### Current Behavior
Currently this works with `https://project.supabase.co/storage/v1/s3/` but not `https://project.supabase.co/storage/v1/s3`
### Reproduction Steps
I scrubbed out my credentials, but running this fails
```rust
let credentials = Credentials::new(
access_key_id,
secret_access_key,
None,
None,
"static",
);
let config = Config::builder()
.endpoint_url(endpoint)
.force_path_style(true)
.region(Region::new("us-east-1"))
.credentials_provider(credentials.clone())
.build();
let client = Client::from_conf(config);
match client
.head_object()
.bucket(bucket)
.key(key)
.send()
.await
{
Ok(output) => {
println!("SUCCESS!");
}
Err(e) => {
println!("FAILED!");
}
}
```
and rerunning the same with `let endpoint = format!("{}/", endpoint);` succeeds.
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### Version
```text
; cargo tree | grep aws-
│ │ ├── aws-credential-types v1.2.10
│ │ │ ├── aws-smithy-async v1.2.6
│ │ │ ├── aws-smithy-runtime-api v1.9.2
│ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ ├── aws-smithy-types v1.3.4
│ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-lc-rs v1.14.1
│ │ │ │ │ │ ├── aws-lc-sys v0.32.0
│ │ │ │ │ │ ├── aws-lc-rs v1.14.1 (*)
│ │ │ ├── aws-config v1.8.1
│ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ ├── aws-runtime v1.5.16
│ │ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ │ ├── aws-sigv4 v1.3.6
│ │ │ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ │ │ ├── aws-smithy-eventstream v0.60.13
│ │ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ │ ├── aws-smithy-http v0.62.5
│ │ │ │ │ │ │ ├── aws-smithy-eventstream v0.60.13 (*)
│ │ │ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ ├── aws-smithy-eventstream v0.60.13 (*)
│ │ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ │ ├── aws-smithy-runtime v1.9.4
│ │ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ │ │ ├── aws-smithy-http-client v1.1.4
│ │ │ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ │ ├── aws-smithy-observability v0.1.4
│ │ │ │ │ │ │ └── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-types v1.3.10
│ │ │ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ ├── aws-sdk-sso v1.74.0
│ │ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ │ ├── aws-runtime v1.5.16 (*)
│ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ │ ├── aws-smithy-json v0.61.7
│ │ │ │ │ │ └── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-smithy-runtime v1.9.4 (*)
│ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-types v1.3.10 (*)
│ │ │ │ ├── aws-sdk-ssooidc v1.75.0
│ │ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ │ ├── aws-runtime v1.5.16 (*)
│ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ │ ├── aws-smithy-json v0.61.7 (*)
│ │ │ │ │ ├── aws-smithy-runtime v1.9.4 (*)
│ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-types v1.3.10 (*)
│ │ │ │ ├── aws-sdk-sts v1.76.0
│ │ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ │ ├── aws-runtime v1.5.16 (*)
│ │ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ │ ├── aws-smithy-json v0.61.7 (*)
│ │ │ │ │ ├── aws-smithy-query v0.60.7
│ │ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-smithy-runtime v1.9.4 (*)
│ │ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ │ ├── aws-smithy-xml v0.60.12
│ │ │ │ │ ├── aws-types v1.3.10 (*)
│ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ ├── aws-smithy-json v0.61.7 (*)
│ │ │ │ ├── aws-smithy-runtime v1.9.4 (*)
│ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ ├── aws-types v1.3.10 (*)
│ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ ├── aws-sdk-s3 v1.115.0
│ │ │ │ ├── aws-credential-types v1.2.10 (*)
│ │ │ │ ├── aws-runtime v1.5.16 (*)
│ │ │ │ ├── aws-sigv4 v1.3.6 (*)
│ │ │ │ ├── aws-smithy-async v1.2.6 (*)
│ │ │ │ ├── aws-smithy-checksums v0.63.11
│ │ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ ├── aws-smithy-eventstream v0.60.13 (*)
│ │ │ │ ├── aws-smithy-http v0.62.5 (*)
│ │ │ │ ├── aws-smithy-json v0.61.7 (*)
│ │ │ │ ├── aws-smithy-runtime v1.9.4 (*)
│ │ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
│ │ │ │ ├── aws-smithy-types v1.3.4 (*)
│ │ │ │ ├── aws-smithy-xml v0.60.12 (*)
│ │ │ │ ├── aws-types v1.3.10 (*)
│ │ │ ├── aws-smithy-http-client v1.1.4 (*)
│ │ │ ├── aws-smithy-runtime-api v1.9.2 (*)
```
### Environment details (OS name and version, etc.)
MacOS version 15.6
### Logs
_No response_
Contributor guide
Research direction
Start at Config::builder endpoint_url handling and trace how Client::from_conf builds the head_object request when force_path_style is enabled. Add regression coverage for an endpoint with and without a trailing slash, and verify that the bucket and key are appended to the expected path without changing existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- api, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100