Fix compile leaking multi-output arrays - #4183
Draft
AKnassa wants to merge 1 commit into
Draft
Conversation
compile_dfs deep copies the traced tape, so the originals of any multi-output node are meant to die once the tape and the parents map have been rewired. They did not: the outputs of a multi-output primitive hold each other through siblings, and array::~array() only breaks that cycle when it is the last external reference. Moving the tape still left the parents map holding them, and the parents map then released them through operator=, which has no such cycle breaking. The group was left orphaned, holding its inputs, which pinned any array the traced function had captured. Keep one member of each replaced group alive until both have been rewired, then drop it, so the destructor runs when its reference count test is actually valid. A group the caller still holds stays above the threshold and is left untouched. Compiling a function with an mx.split over 8 captured 256x256 weights retained 2 MB per call before this, and nothing after. Fixes ml-explore#3932
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Fixes #3932.
A compiled function containing a multi-output primitive leaks every array the
traced function captured, once per call:
Replacing the
mx.splitwith plain slicing (a single-output op) leaks nothing,which is what pointed at the multi-output path.
Cause
compile_dfsdeep copies the traced tape, so the originals of any multi-outputnode should die once the new tape and the parents map are in place. They cannot:
siblings(
array::make_arrays), which is a strong reference cycle by construction.array::~array()is the only thing that breaks that cycle, and only when thedestructing array is the last external holder (
use_count() == n + 1).tape = std::move(new_tape)the old arrays are destroyed, but the parentsmap still holds copies of them, so the count is too high and nothing is
detached.
array::operator=, whichreseats
array_desc_and has no cycle-breaking counterpart.The group ends up orphaned at
use_count() == 1each, referenced only by eachother, still holding
ArrayDesc::inputs— which pins the captured weights.That also explains why all three ingredients are needed to see it: single-output
originals have no cycle,
compile_dfsis the only place doing this deep copyplus
operator=rewiring, and an already-evaluated capture is shared into thenew tape rather than copied, so the orphan holds a real buffer.
Fix
Rather than adding a new liveness rule, keep one member of each replaced group
alive until both the tape and the parents map have been rewired, then drop it,
so the existing destructor test runs at a point where its arithmetic is valid.
This is deliberately conservative. If the caller still holds the group — for
example a lazy
mx.splitcreated before tracing and captured by the closure —the reference count stays above the threshold and nothing is mutated at all. I
checked that case explicitly against
mainand against this branch, and bothbehave identically. Nothing is force-detached, so the compile cache's record of
each node's primitive is untouched.
Also worth a line:
compile_simplify'smerge_onerewires parents through thesame
array::operator=andsplit_oneclones nodes, so a similar shape mayexist there. I have not investigated it and am not claiming simplify is clean.
Testing
New
test_multi_output_leakinpython/tests/test_compile.py, which loops so asingle-cycle fluke cannot pass it. It fails before this change and passes after.
The existing
test_leaksgates onmx.metal.is_available(), so it is a no-opon a CPU-only build; this one does not.
Built CPU-only (
MLX_BUILD_METAL=OFF); the GPU suite was not run on my machine.Checklist
pre-commit run --all-filesto format my code / installed pre-commit prior to committing changes