Skip to content

Apply the allocator gc-gate fix to SD21/SDXL#25

Closed
TroyHernandez wants to merge 2 commits into
feature/zimagefrom
feature/sd-gc-gates
Closed

Apply the allocator gc-gate fix to SD21/SDXL#25
TroyHernandez wants to merge 2 commits into
feature/zimagefrom
feature/sd-gc-gates

Conversation

@TroyHernandez

Copy link
Copy Markdown
Contributor

Stacked on #24 (needs .flux_gc_gates() from that branch; will retarget
to main when #24 merges).

load_pipeline() now routes through the same allocator gc-gate fix as
the FLUX/Z-Image pipelines whenever any component sits on CUDA. The
legacy SD path had the identical pathology: the torch allocated/reserved
ratio gate is chronically exceeded under backend:native, so R gc fired
on nearly every CUDA allocation.

Measured at 50 steps, warm generation (native modules, decoder on CPU,
RTX 5060 Ti 16 GB):

Model Before After R gc share
SDXL 1024x1024 234 s 100 s 209 s -> 75 s
SD 2.1 768x768 101 s 26 s 88 s -> 15 s

Notes:

  • The reduced gc cadence lets the garbage sawtooth ride higher between
    collections (SD21 peak 5.4 -> 13.3 GB alloc on the 16 GB card). The
    gate constants are tuned for 16 GB; smaller cards may need different
    sizing — measurement tracked separately.
  • The all-GPU SDXL layout at 1024px peaks ~14 GB with native modules
    and can OOM on 16 GB regardless of gates (decoder was on CPU for
    these measurements). Pre-existing, not changed here.
  • README's SDXL/FLUX.2 comparison table numbers were measured under a
    different (undocumented) component layout and are not restated here.

Full suite 692/692.

🤖 Generated with Claude Code

load_pipeline() now routes through .flux_gc_gates() when any component
is on CUDA. Measured at 50 steps (native modules, decoder on CPU,
RTX 5060 Ti 16 GB), warm generation:

  SDXL 1024x1024: 234 -> 100 s (R gc 209 -> 75 s)
  SD 2.1 768x768: 101 -> 26 s  (R gc 88 -> 15 s)

The reduced gc cadence lets the garbage sawtooth ride higher (SD21
peak 5.4 -> 13.3 GB alloc), so smaller cards may need different gate
sizing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant