Thread leak on error in `Aws::S3::TransferManager`

[ClearlyClaire]

Describe the bug

Similarly to #3408, after investigating an issue reported on our project (mastodon/mastodon#40407), I came to the conclusion that missing error handling causes aws-sdk-s3 to leak threads when an error is encountered in certain methods.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Aws::S3::TransferManager always closes threads it opens.

Current Behavior

Aws::S3::TransferManager seems to leak threads when an error occurs.

This is because, when no executor is provided as an option, it creates its own, spawning multiple threads, and tell them to close once processing is finished. However, that last executor.shutdown is not called if an error occurs during processing.

Aws::S3::TransferManager#upload_file is the most obvious offender, but the same problematic pattern is used in several methods.

Reproduction Steps

Upload a file using Multipart File Upload with very low timeouts so that uploading fails and leaks threads.

Possible Solution

executor.close should probably be called in an ensure block (see monkey-patch that seems to solve the issue: mastodon/mastodon#40407 (comment)).

Additional Information/Context

aws-sdk-ruby/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb

Lines 163 to 166 in 50686d6

	executor = @executor || DefaultExecutor.new(max_threads: options.delete(:thread_count))
	downloader = DirectoryDownloader.new(client: @client, executor: executor, logger: @logger)
	result = downloader.download(destination, bucket: bucket, **options)
	executor.shutdown unless @executor

aws-sdk-ruby/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb

Lines 248 to 251 in 50686d6

	executor = @executor || DefaultExecutor.new(max_threads: download_opts.delete(:thread_count))
	downloader = FileDownloader.new(client: @client, executor: executor)
	downloader.download(destination, download_opts)
	executor.shutdown unless @executor

aws-sdk-ruby/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb

Lines 363 to 366 in 50686d6

	executor = @executor || DefaultExecutor.new(max_threads: options.delete(:thread_count))
	uploader = DirectoryUploader.new(client: @client, executor: executor, logger: @logger)
	result = uploader.upload(source, bucket, **options.merge(http_chunk_size: resolve_http_chunk_size(options)))
	executor.shutdown unless @executor

aws-sdk-ruby/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb

Lines 447 to 456 in 50686d6

	executor = @executor || DefaultExecutor.new(max_threads: upload_opts.delete(:thread_count))
	uploader = FileUploader.new(
	multipart_threshold: upload_opts.delete(:multipart_threshold),
	http_chunk_size: http_chunk_size,
	client: @client,
	executor: executor
	)
	response = uploader.upload(source, upload_opts)
	yield response if block_given?
	executor.shutdown unless @executor

aws-sdk-ruby/gems/aws-sdk-s3/lib/aws-sdk-s3/transfer_manager.rb

Lines 514 to 522 in 50686d6

	executor = @executor || DefaultExecutor.new(max_threads: upload_opts.delete(:thread_count))
	uploader = MultipartStreamUploader.new(
	client: @client,
	executor: executor,
	tempfile: upload_opts.delete(:tempfile),
	part_size: upload_opts.delete(:part_size)
	)
	uploader.upload(upload_opts, &block)
	executor.shutdown unless @executor

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3 1.229.0

Environment details (Version of Ruby, OS environment)

ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-linux]

[bhoradc]

Hi @ClearlyClaire,

Thanks for the detailed report. I was able to reproduce the thread leak on the failure path in TransferManager#upload_file.

The behavior appears to have started with aws-sdk-s3 1.200.0, when the executor handling was introduced. We'll investigate further and follow up here with next steps.

Regards,
Chaitanya

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[bhoradc]

Hi @ClearlyClaire - This is fixed via #3427 (merged), will ship in the upcoming release. Please let us know if you hit any issues with the fixed version. Thanks again for reporting the issue.