S3 Object Integrity Algorithms and Presigned URLs
- Dominant language
- Ruby
- Stars
- 3.7k
- Forks
- 1.2k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 5
Description
### Describe the bug
Hey there - thanks for making a great library!
I'm trying to use the SDK to generate a presigned S3 PUT URL that enforces object integrity as [documented](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html) using the `SHA256` checksum algorithm.
After looking through the source, it appears that the [Aws::S3::Presigner](https://github.com/aws/aws-sdk-ruby/blob/46114c4e00174a05727eb3f1635954ccbc98305b/gems/aws-sdk-s3/lib/aws-sdk-s3/presigner.rb) does not support generating presigned URLs that specify the `x-amz-checksum-{algorithm}` header as a signed header value.
At first glance this _feels_ like a defect, therefore submitting as a bug. Happy to treat this as a feature request though - thanks!
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
Calling `Aws::S3::Presigner#presigned_url` with a `checksum_algorithm` value generates a URL that specifies the necessary checksum headers as _signed_ headers.
### Current Behavior
Calling `Aws::S3::Presigner#presigned_url` with a `checksum_algorithm` value generates a URL with the checksum headers as _unsigned_ headers - meaning that the recipient of the URL can't utilize object integrity checks.
### Reproduction Steps
Here's a minimal script that demonstrates the issue:
```ruby
require 'bundler/inline'
require 'net/http'
gemfile do
source 'https://rubygems.org/'
gem 'aws-sdk-s3', '~> 1.182.0'
gem 'marcel', '~> 1.0.4'
gem 'nokogiri', '~> 1.18.5'
gem 'rest-client', '~> 2.1.0'
end
class FileAttributes
attr_reader :bytesize, :checksum, :mime_type
def initialize(path)
@bytesize = File.read(path).bytesize
@checksum = digest(path)
@mime_type = Marcel::MimeType.for(Pathname.new(path))
end
private
def digest(path)
Digest::SHA2.new.tap do |sha|
File.open(path) do |f|
while chunk = f.read(256)
sha << chunk
end
end
end.base64digest
end
end
def upload_file(path, attributes, url, headers: {})
puts "==> Uploading #{path}..."
puts "Request: PUT #{url}"
response = RestClient::Request.execute(
url: url,
method: :put,
headers: { 'Content-Type' => attributes.mime_type }.merge(headers),
payload: File.new(path)
)
puts "Response: #{response.code}"
rescue RestClient::ExceptionWithResponse => e
puts "Error: #{e.response.body}"
end
attributes = FileAttributes.new('file_one.png')
presigner = Aws::S3::Presigner.new
presigned_url = presigner.presigned_url(
:put_object,
bucket: ENV.fetch('BUCKET_NAME'),
key: 'test',
checksum_algorithm: 'SHA256',
checksum_sha256: attributes.checksum,
expires_in: 3600,
content_type: attributes.mime_type,
content_length: attributes.bytesize
)
# Request succeeds as expected
upload_file('file_one.png', attributes, presigned_url)
# Uploading with a _different_ checksum using the same URL succeeds, but SHOULD fail
upload_file('file_two.jpeg', attributes, presigned_url)
# Specifying x-amz-checksum-sha256 fails with 403 (HeadersNotSigned)
upload_file(
'file_one.png',
attributes,
presigned_url,
headers: { 'x-amz-checksum-sha256' => attributes.checksum }
)
```
Save the script as `script.rb` and provide two different test files in the same directory (`file_one.png` and `file_two.jpeg`).
Execute the script as follows:
```bash
BUCKET_NAME=bucket ruby script.rb
```
### Possible Solution
Avoid [removing the checksum handler](https://github.com/aws/aws-sdk-ruby/blob/46114c4e00174a05727eb3f1635954ccbc98305b/gems/aws-sdk-s3/lib/aws-sdk-s3/presigner.rb#L200) and don't hoist the checksum headers when the presigned URL is being generated.
### Additional Information/Context
It's worthwhile to note that the `Content-MD5` header is treated as a signed header for presigned URLs. This is different than the behaviour when a different algorithm is specified.
### Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version
aws-sdk-s3, 1.182.0
### Environment details (Version of Ruby, OS environment)
Ruby 3.3.6, MacOS 15.3.2
Contributor guide
Assessment
This issue has not been assessed yet.