Skip to content

[AUTOMATED] [PROPOSAL] Recover runtime stack allocations across control flow - #1235

Draft
thotypous wants to merge 15 commits into
Noelo-Lab:mainfrom
thotypous:fix/dynamic-stack-allocation
Draft

thotypous wants to merge 15 commits into
Noelo-Lab:mainfrom
thotypous:fix/dynamic-stack-allocation

Conversation

@thotypous

@thotypous thotypous commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Runtime-sized stack buffers can become addresses below a fixed C local, so recompiling the output corrupts memory across calls. This proposal adds an explicit stackalloc on option to preserve function-lifetime allocations across control flow.

cat > allocate.c <<'C'
__attribute__((noinline)) int observe(char *p, int n) {
    p[n-1] = 9; return (unsigned char)p[0] + (unsigned char)p[n-1];
}
int allocate(int n) {
    char *p = __builtin_alloca((unsigned)n);
    p[0] = 7; return observe(p, n);
}
C
clang -O2 -shared -fPIC -nostdlib -Wl,-Bsymbolic allocate.c -o allocate.so
kuna decompile allocate.so allocate --option stackalloc on \
  --assert 'prototype allocate int allocate(int n)' \
  --assert 'prototype observe int observe(char *p,int n)'

Without recovery, the buffer addresses a fixed local. With it, generated C uses __builtin_alloca and returns 16 for n > 1, matching the native function.

  • Follow the selected ABI's SP SSA across branches, loops, alignment, paired pointer/counter page probes, consecutive allocations and restored frame bases. Reserve outgoing arguments and SPARC V9 register-window bias; preserve dominance when unifying equivalent addresses.
  • Default off; decline unknown effects, unbounded SP growth, stepped stack-clash probes, and loops restoring an earlier SP. Every enclosing loop latch must carry the allocation through its SP PHI; graph/SSA proof budgets bound analysis work. Scoped VLAs remain unsupported because alloca cannot reproduce their lifetime.
  • Lower after active call-argument recovery: doing it earlier erased SP relationships and invented caller/callee parameters. An unasserted-prototype regression checks both arities. Preserve the maintainer's bounded analysis and displaced-address fixes.

Validation: 24 focused Clang allocation tests; eight targets (x86/AMD64, ARM/AArch64, RISC-V32/64, SPARC32/64), 98,496 native/recompiled comparisons at O0/O2, checking values, helper calls and input digests. Scoped VLAs and unbounded SP loops must decline. CI installs Clang, LLD, QEMU and the SPARC linker, and reruns these tests with missing tools treated as failures. SPARC uses noreturn_disc off to isolate existing no-return inference. Independent IR regressions cover zero and unrelated alignment masks. Catalog checks pass; full suite left to CI. Design, ABI references and timing limits: notes/dynamic-stack-allocation.md.

thotypous and others added 9 commits October 21, 2026 00:40
…decline stepped probes

With stackalloc on:
- Bounds propagation ran up to ops.len() sweeps before widening, so a
  nonconvergent pointer bound cost O(n^2): dash readtoken1 (O0) went from
  2.5 s to over 400 s. Widen after 64 sweeps, at most eight times.
- An address built as base + 8 - n beside SP = base - n (clang -O2's
  `neg %rax; mov %edx,8(%rcx,%rax,1)`, and x86 SIB `disp + base + index`
  placing the constant first) kept addressing the old frame, emitting
  `__builtin_dwarf_cfa()[-n] = 7`. Treat a constant-first INT_ADD as a
  constant step and rewrite dominated displaced copies from the buffer.
- GCC -fstack-clash-protection lowers SP a page per iteration and then
  subtracts the residual; recovering that residual as the allocation
  emitted __builtin_alloca(size & 4095) and smashed the stack at
  size >= 4096. Decline a base carried around a loop by a nonzero step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BEr111CDLJsQMzY3YSyWXD
@mahaloz mahaloz closed this Oct 22, 2026
@mahaloz mahaloz reopened this Oct 22, 2026
@mahaloz

mahaloz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

[AUTOMATED] Pushed 02ccabb (plus a merge of main to clear an integration.rs conflict). It fixes three problems that only show up with stackalloc on:

  • Bounds propagation could take up to ops.len() sweeps before widening. On dash (O0) readtoken1 that meant 2.5 s became more than 400 s, ending on the per-function budget error. It now widens after 64 sweeps, at most eight times.
  • clang -O2 writes a field as neg %rax; mov %edx,8(%rcx,%rax,1). x86 SIB lifts this as 8 + base + index, with the constant first, so the address was never matched to the allocation and the output wrote to __builtin_dwarf_cfa()[-n]. A constant-first add now counts as a constant step, and dominated copies at a constant displacement are rewritten from the buffer.
  • GCC's -fstack-clash-protection moves SP down one page per iteration and then subtracts size & 4095. That residual was treated as the whole allocation, so the output emitted __builtin_alloca(size & 4095) and smashed the stack once size reached 4096. These allocations are now declined. Every allocation the option recovered across a 48-binary decbench/fixture sample was this shape.

New cases: elf64_displaced_allocation_address_roundtrip and elf64_stepped_stack_probe_is_declined. Both fail before the fix and pass after it.

@mahaloz

mahaloz commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

[AUTOMATED] The default-off path is byte-identical and the option's counts and docs check out. Fixes pushed in 02ccabb: bounded the quadratic bounds propagation, handled the constant-first SIB address, and declined gcc's stepped stack-clash probes. Two option-on outputs are still wrong, so this stays open:

  1. A scoped VLA in a loop becomes __builtin_alloca inside the loop with no restore. Native code restores SP every iteration; the recompiled function grows the stack by about 80 MB and segfaults:
printf '__attribute__((noinline)) int obs(char*p,int n){p[n-1]^=(char)n;return (unsigned char)p[0]+(unsigned char)p[n-1];}\nint f(int n){int t=0;for(int i=0;i<20000;i++){char d[n+4000];d[0]=(char)i;t+=obs(d,n+4000);}return t;}\n' > s.c
clang -O0 -shared -fPIC -nostdlib -Wl,-Bsymbolic s.c -o s.so
kuna decompile s.so f --option stackalloc on

Either decline when the loop back edge restores SP instead of deriving it from the allocation, or emit a block-scoped VLA.
2. Without --assert prototype, the body's own allocate.so repro prints int allocate(unsigned int a0,unsigned long a1,unsigned long a2) and observe(v1,a0,a2,&Stackfffffffffffffff8); with the option off it prints allocate(a0).

Also, all 8 stack_allocations_isa tests skip in CI (missing ld.lld), so the cross-ISA native/recompiled comparisons never run there.

@thotypous

thotypous commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

[AUTOMATED] Addressed in 8177465 (with current main merged in 67f1c0a):

  • Scoped VLA lifetime: recovery now requires every enclosing natural-loop latch to carry the allocation through its SP PHI. Restored earlier SPs, missing proofs and irreducible loops decline recovery. Bounded graph/SSA walks prevent unbounded analysis. The 20,000-iteration scoped VLA is a negative regression rather than an alloca round trip.
  • Unasserted call arity: moved lowering after ActionActiveParam. Lowering earlier erased the SP relationships needed by argument recovery; the new unasserted-prototype regression checks that the caller retains one argument and the observer two.
  • Cross-ISA CI coverage: install Clang, LLD, QEMU and the SPARC GNU linker. A dedicated run sets KUNA_REQUIRE_STACKALLOC_ISA=1, so unavailable tools fail instead of skipping.

Preserved your bounded propagation, displaced-address and stepped stack-clash-probe fixes. Local focused validation passes: 24 allocation tests, eight ISA tests with 98,496 native/recompiled comparisons, and five catalog checks. No full local suite was run; CI is pending for the updated merge.

CI follow-up in 1cac0c4: refresh the listing snapshot for the intentional stackalloc move, and page-align the existing MIPS replay fixture’s RW data after its RX harness. Installing QEMU exposed overlapping segment-page permissions: the native reference faulted at its entry point before execution. All 10 focused schedule checks and seven call-wide tests now pass locally, including native/recompiled MIPS replay; separate native O0/O2 runs pass with declared and DWARF contracts. The earlier catalog baseline-label mismatch was corrected in 7e5f3f7. Full suites remain with CI.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants