Repository navigation
CUDA backend: delete each output handle once when a program returns one tensor twice - #24442
Closed
pytorchbot wants to merge 1 commit into
Closed
pytorchbot wants to merge 1 commit into
pytorchbot wants to merge 1 commit into
Conversation
…ne tensor twice (#24420) ### What is wrong today When a CUDA program returns the same tensor more than once, for example `y.detach()` twice, AOTI (AOTInductor, PyTorch's ahead-of-time compiler that generates the CUDA code) puts one output handle in each of those output slots: ``` output_handles[0] = buf0.release(); output_handles[1] = output_handles[0]; ``` The CUDA backend copied each output into its ExecuTorch tensor and then deleted that slot's handle. So it deleted the shared handle twice, and the process crashed, for example with: ``` SlimTensor storage_offset must not be negative, got -9007650542493611503 Failed to D2D copy GPU output 1 into ETensor free(): double free detected in tcache 2 ``` ### What this change does A handle is now deleted only at the last output slot that holds it, as PyTorch's own AOTI runner does. The cleanup on error paths follows the same rule. With CUDA graphs on, a successful capture keeps every output handle for the graph and deletes none of them, so nothing changes there. If ending the capture or instantiating the graph fails, the cleanup deletes the handles by the same rule. A comment in `ReplaceViewCopyWithViewPass` said that a view would make two outputs share one handle. That was misleading, because two `alias_copy` outputs of one tensor share one handle too, so the comment no longer says it. ### What was tested `test_same_tensor_returned_twice` returns one tensor in two output slots that are not next to each other, with CUDA graphs on: three eager calls, a graph capture and a replay, each compared with eager. With torch 2.14 on an A100 it passes, and it crashes without this change. The other `backends/cuda` tests that CI runs in the same job still pass. (cherry picked from commit 5c05ad4)
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24442
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 204 PendingAs of commit acd31c5 with merge base 4db778e ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
Contributor
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is wrong today
When a CUDA program returns the same tensor more than once, for example
y.detach()twice, AOTI (AOTInductor, PyTorch's ahead-of-time compiler that generates the CUDA code) puts one output handle in each of those output slots:The CUDA backend copied each output into its ExecuTorch tensor and then deleted that slot's handle. So it deleted the shared handle twice, and the process crashed, for example with:
What this change does
A handle is now deleted only at the last output slot that holds it, as PyTorch's own AOTI runner does. The cleanup on error paths follows the same rule. With CUDA graphs on, a successful capture keeps every output handle for the graph and deletes none of them, so nothing changes there. If ending the capture or instantiating the graph fails, the cleanup deletes the handles by the same rule.
A comment in
ReplaceViewCopyWithViewPasssaid that a view would make two outputs share one handle. That was misleading, because twoalias_copyoutputs of one tensor share one handle too, so the comment no longer says it.What was tested
test_same_tensor_returned_twicereturns one tensor in two output slots that are not next to each other, with CUDA graphs on: three eager calls, a graph capture and a replay, each compared with eager. With torch 2.14 on an A100 it passes, and it crashes without this change. The otherbackends/cudatests that CI runs in the same job still pass.