Thread leak on error in `Aws::S3::TransferManager`
- Dominant language
- Ruby
- Stars
- 3.7k
- Forks
- 1.2k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 5
Description
### Describe the bug
Similarly to #3408, after investigating an issue reported on our project (https://github.com/mastodon/mastodon/issues/40407), I came to the conclusion that missing error handling causes `aws-sdk-s3` to leak threads when an error is encountered in certain methods.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
`Aws::S3::TransferManager` always closes threads it opens.
### Current Behavior
`Aws::S3::TransferManager` seems to leak threads when an error occurs.
This is because, when no `executor` is provided as an option, it creates its own, spawning multiple threads, and tell them to close once processing is finished. However, that last `executor.shutdown` is not called if an error occurs during processing.
`Aws::S3::TransferManager#upload_file` is the most obvious offender, but the same problematic pattern is used in several methods.
### Reproduction Steps
Upload a file using Multipart File Upload with very low timeouts so that uploading fails and leaks threads.
### Possible Solution
`executor.close` should probably be called in an `ensure` block (see monkey-patch that seems to solve the issue: https://github.com/mastodon/mastodon/issues/40407#issuecomment-5615399056).
### Additional Information/Context
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L163-L166
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L248-L251
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L363-L366
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L447-L456
https://github.com/aws/aws-sdk-ruby/blob/50686d67937307b0637d0d24d6c9c552c523f6f3/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb#L514-L522
### Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version
aws-sdk-s3 1.229.0
### Environment details (Version of Ruby, OS environment)
ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-linux]
Contributor guide
Research direction
Start in gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb, especially the referenced ranges around upload_file and the other methods that create executors. Reproduce the failure with Multipart File Upload and very low timeouts, then verify that every affected path closes its internally created executor when processing raises.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, ruby
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100