aws / aws/aws-sdk-ruby

S3 Object Integrity Algorithms and Presigned URLs

Open
#3,217 4 comments 0 reactions 0 assignees View on GitHub
investigating service-api
Dominant language
Ruby
Stars
3.7k
Forks
1.2k
Avg merge
1d 4h
Merged PRs (30d)
5

Description

### Describe the bug

Hey there - thanks for making a great library!

I'm trying to use the SDK to generate a presigned S3 PUT URL that enforces object integrity as [documented](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html) using the `SHA256` checksum algorithm.

After looking through the source, it appears that the [Aws::S3::Presigner](https://github.com/aws/aws-sdk-ruby/blob/46114c4e00174a05727eb3f1635954ccbc98305b/gems/aws-sdk-s3/lib/aws-sdk-s3/presigner.rb) does not support generating presigned URLs that specify the `x-amz-checksum-{algorithm}` header as a signed header value.

At first glance this _feels_ like a defect, therefore submitting as a bug. Happy to treat this as a feature request though - thanks!

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

Calling `Aws::S3::Presigner#presigned_url` with a `checksum_algorithm` value generates a URL that specifies the necessary checksum headers as _signed_ headers.

### Current Behavior

Calling `Aws::S3::Presigner#presigned_url` with a `checksum_algorithm` value generates a URL with the checksum headers as _unsigned_ headers - meaning that the recipient of the URL can't utilize object integrity checks.

### Reproduction Steps

Here's a minimal script that demonstrates the issue:

```ruby
require 'bundler/inline'
require 'net/http'

gemfile do
source 'https://rubygems.org/'
gem 'aws-sdk-s3', '~> 1.182.0'
gem 'marcel', '~> 1.0.4'
gem 'nokogiri', '~> 1.18.5'
gem 'rest-client', '~> 2.1.0'
end

class FileAttributes
attr_reader :bytesize, :checksum, :mime_type

def initialize(path)
@bytesize = File.read(path).bytesize
@checksum = digest(path)
@mime_type = Marcel::MimeType.for(Pathname.new(path))
end

private

def digest(path)
Digest::SHA2.new.tap do |sha|
File.open(path) do |f|
while chunk = f.read(256)
sha << chunk
end
end
end.base64digest
end
end

def upload_file(path, attributes, url, headers: {})
puts "==> Uploading #{path}..."
puts "Request: PUT #{url}"

response = RestClient::Request.execute(
url: url,
method: :put,
headers: { 'Content-Type' => attributes.mime_type }.merge(headers),
payload: File.new(path)
)

puts "Response: #{response.code}"
rescue RestClient::ExceptionWithResponse => e
puts "Error: #{e.response.body}"
end

attributes = FileAttributes.new('file_one.png')
presigner = Aws::S3::Presigner.new
presigned_url = presigner.presigned_url(
:put_object,
bucket: ENV.fetch('BUCKET_NAME'),
key: 'test',
checksum_algorithm: 'SHA256',
checksum_sha256: attributes.checksum,
expires_in: 3600,
content_type: attributes.mime_type,
content_length: attributes.bytesize
)

# Request succeeds as expected
upload_file('file_one.png', attributes, presigned_url)

# Uploading with a _different_ checksum using the same URL succeeds, but SHOULD fail
upload_file('file_two.jpeg', attributes, presigned_url)

# Specifying x-amz-checksum-sha256 fails with 403 (HeadersNotSigned)
upload_file(
'file_one.png',
attributes,
presigned_url,
headers: { 'x-amz-checksum-sha256' => attributes.checksum }
)
```

Save the script as `script.rb` and provide two different test files in the same directory (`file_one.png` and `file_two.jpeg`).

Execute the script as follows:

```bash
BUCKET_NAME=bucket ruby script.rb
```

### Possible Solution

Avoid [removing the checksum handler](https://github.com/aws/aws-sdk-ruby/blob/46114c4e00174a05727eb3f1635954ccbc98305b/gems/aws-sdk-s3/lib/aws-sdk-s3/presigner.rb#L200) and don't hoist the checksum headers when the presigned URL is being generated.

### Additional Information/Context

It's worthwhile to note that the `Content-MD5` header is treated as a signed header for presigned URLs. This is different than the behaviour when a different algorithm is specified.

### Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3, 1.182.0

### Environment details (Version of Ruby, OS environment)

Ruby 3.3.6, MacOS 15.3.2

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.