aws / aws/aws-sdk-ruby

Thread leak on error in `Aws::S3::TransferManager`

Open
#3,419 0 comments 0 reactions 0 assignees View on GitHub
bug needs-triage
Dominant language
Ruby
Stars
3.7k
Forks
1.2k
Avg merge
1d 4h
Merged PRs (30d)
5

Description

### Describe the bug

Similarly to #3408, after investigating an issue reported on our project (https://github.com/mastodon/mastodon/issues/40407), I came to the conclusion that missing error handling causes `aws-sdk-s3` to leak threads when an error is encountered in certain methods.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

`Aws::S3::TransferManager` always closes threads it opens.

### Current Behavior

`Aws::S3::TransferManager` seems to leak threads when an error occurs.

This is because, when no `executor` is provided as an option, it creates its own, spawning multiple threads, and tell them to close once processing is finished. However, that last `executor.shutdown` is not called if an error occurs during processing.

`Aws::S3::TransferManager#upload_file` is the most obvious offender, but the same problematic pattern is used in several methods.

### Reproduction Steps

Upload a file using Multipart File Upload with very low timeouts so that uploading fails and leaks threads.

### Possible Solution

`executor.close` should probably be called in an `ensure` block (see monkey-patch that seems to solve the issue: https://github.com/mastodon/mastodon/issues/40407#issuecomment-5615399056).

### Additional Information/Context

https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L163-L166
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L248-L251
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L363-L366
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L447-L456
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L514-L522

### Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3 1.229.0

### Environment details (Version of Ruby, OS environment)

ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-linux]

Contributor guide

Open the contributing guide

Research direction

Start in gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb, especially the referenced ranges around upload_file and the other methods that create executors. Reproduce the failure with Multipart File Upload and very low timeouts, then verify that every affected path closes its internally created executor when processing raises.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, ruby
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.