Skip to content

Fix :thread_count handling in TransferManager and Object transfer methods - #3428

Open
jterapin wants to merge 2 commits into
version-3from
thread-count-executor-fixes
Open

jterapin wants to merge 2 commits into
version-3from
thread-count-executor-fixes

Conversation

@jterapin

@jterapin jterapin commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Note

Targeted at #3427 branch, so the diff here shows only this change.
Will retarget to version-3 once #3427 merges.

Description

Every TransferManager documentation states that :thread_count is "Only used when no custom executor when provided" but the current implementation does not honor that:

  • upload_file raises ArgumentError when :thread_count and a custom executor: are combined, because || short-circuits past the delete. The other four ignore the option.
  • Object#download_file has never honored :thread_count; it deletes an Array key that never matches.

Fix

Hoist the delete above the || at all five TransferManager sites, and delete the Symbol rather than the Array in customizations/object.rb:

thread_count = upload_opts.delete(:thread_count)
executor = @executor || DefaultExecutor.new(max_threads: thread_count)

Testing

Full gems/aws-sdk-s3 suite: 1678 examples, 0 failures, 3 pending (1665 before).

🤖 AI-assisted and manually reviewed by @jterapin ✨

bhoradc and others added 2 commits September 23, 2026 16:59
…hods

The five TransferManager transfer methods build their executor as
`@executor || DefaultExecutor.new(max_threads: opts.delete(:thread_count))`.
Ruby's `||` short-circuits, so when a custom executor was given to the
constructor the `delete` never runs and `:thread_count` stays in the options
hash. On `upload_file` it reaches the unfiltered `put_object` call on the
single-part path and raises `ArgumentError: unexpected value at
params[:thread_count]`, while the multipart path filters the key out, so the
same call fails or succeeds depending on file size.

Hoist the `delete` above the `||` so the key is always removed. Behavior is
now uniform: `:thread_count` is honored when no custom executor was provided
and ignored when one was, matching the existing docstrings.

Separately, `Object#download_file` deleted `[:thread_count]`, an Array that
never matches the Symbol, so it always used the default thread count.
@jterapin
jterapin marked this pull request as ready for review September 24, 2026 17:31
@jterapin
jterapin requested a review from a team as a code owner September 24, 2026 17:31
Base automatically changed from fix/gh-3419-transfer-manager-executor-shutdown to version-3 September 24, 2026 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants