diff --git a/.github/workflows/gc-native-roots.yml b/.github/workflows/gc-native-roots.yml index e806f353f5..89203832f1 100644 --- a/.github/workflows/gc-native-roots.yml +++ b/.github/workflows/gc-native-roots.yml @@ -52,9 +52,9 @@ # # x86-64 was refused outright until #7349 taught the runtime to derive an # SP-relative base from the CFA; it is a first-class arm of the matrix now. The -# old note said (#7324) that a run there would only -# binary that crashes under collection, so an x86-64 run of this matrix would -# test nothing but the refusal — no longer true, and that job is gone. +# note that used to sit here — that an x86-64 run could only ever produce a +# binary crashing under collection, so such a run would test nothing but the +# refusal — described the world before #7349 and is no longer true. # # The same walk is unsound on aarch64 **Linux** too, where it is merely the # non-default path: #7333. @@ -80,10 +80,18 @@ # PERRY_GC_SAFEPOINT_ONLY -> native-roots-rs4gc, "safepoint-only" steps # PERRY_STACKMAP_WALKER -> native-roots-rs4gc, "both non-default walkers" # PERRY_RS4GC -> native-roots-rs4gc -# PERRY_STATEPOINT_REPORT -> deleted. It was a second spelling of -# `--statepoint-report`; the flag is now the only -# entry point and the "fails closed" step is its -# arm. +# PERRY_STATEPOINT_REPORT -> not a knob. It survives as the driver's +# internal handoff to the rayon module workers, +# and `run_pipeline.rs` `remove_var`s it when the +# flag is absent so a value inherited from the +# user's environment cannot switch reporting on. +# The env *spelling* was deleted (#7314); the +# `--statepoint-report` flag is the only entry +# point, and the "fails closed" step is its arm. +# Said precisely because this block is a ledger: +# an entry reading "deleted" for a name still +# greppable in the tree makes the whole list +# look stale. name: gc-native-roots on: # Must run where it can actually gate something. Branch-scoped triggers were @@ -103,9 +111,27 @@ jobs: # Mach-O underscore convention in eh_walker). Windows covers x86-64 + PE/COFF # with the RtlVirtualUnwind walker (#7354) — the one walker with no Itanium # unwinder under it, which is why its arm alone carries the - # `--require-locations` telemetry gate below. ARM64 Linux would cover a - # fifth corner, but those runners queue for hours here, and its two - # components are each covered above. + # `--require-locations` telemetry gate below. + # + # ARM64 Linux is the fourth arm, and the note that used to sit here — that + # its "two components are each covered above" — was the exact compositional + # fallacy `word_width_for` in `gc_map.rs` exists to warn about. `.word` is + # not a fixed size: GNU `as` defines it as the target's natural machine word, + # so LLVM's AArch64 **ELF** backend spells every 32-bit stack-map field + # `.word`, while both previously-covered arms spell it `.long` (Mach-O uses + # `.long` on aarch64; on x86 `.word` means *two* bytes, so LLVM will not use + # it for a 32-bit field). The directive width is a property of the + # intersection, not of either component. + # + # To be precise about what this arm adds, because overclaiming here is how + # #7321's wrong explanation survived into an issue and a job name: the + # `.word` spelling IS unit-tested, by `aarch64_elf_word_directives_decode_to_ + # the_right_root` and `word_width_is_load_bearing_not_cosmetic`, and those + # run on every arm. What they use is a hand-written sample. What no arm has + # ever exercised is the end-to-end chain on this target — real LLVM asm + # output, real ELF linking, real runtime walking — where the failure mode is + # not a parse error but a wrong answer: two bytes of drift per field + # silently relocates every root that follows, and every arm stays green. native-roots-rs4gc: strategy: fail-fast: false @@ -120,6 +146,9 @@ jobs: - os: windows-latest arch: x86-64 format: PE + - os: ubuntu-24.04-arm + arch: aarch64 + format: ELF runs-on: ${{ matrix.os }} # The ubuntu/macos steps were written for bash and windows-latest defaults # to pwsh; one explicit default keeps a single script dialect per step. @@ -273,19 +302,26 @@ jobs: "$py" scripts/statepoint_report_assert.py /tmp/rs4gc-report.json \ --only-backend rs4gc - # #7354: the Windows walker liveness gate. It is the one walker with - # no Itanium unwinder under it and no verify-mode cross-check, and a - # walker that visits zero frames still lets most probes print the - # right answer (other root sources cover them). Non-zero - # frames/records/locations telemetry is the only proof it ran. - if [ "$RUNNER_OS" = "Windows" ]; then - PERRY_GC_TRACE=1 PERRY_RS4GC=1 \ - PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1 \ - PERRY_GC_HEAP_LIMIT=8 PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off \ - "/tmp/rs4gc-04_dead_after_deep_stack$exe" > /dev/null 2> /tmp/rs4gc-trace.err - "$py" scripts/gc_walker_trace_assert.py /tmp/rs4gc-trace.err \ - --require-locations - fi + # Walker liveness, on EVERY arm. A walker that visits zero frames + # still lets most probes print the right answer, because other root + # sources cover them — so a green matrix is not evidence the walker + # ran. Only non-zero frames/records/locations telemetry is. + # + # This used to be Windows-only (#7354) for a good reason: it was the + # only arm that could pass it. Measured on `04_dead_after_deep_stack`, + # macOS and Linux reported 7 frames and ZERO locations, because every + # probe in the suite collected from a shallow stack at exit. Windows + # only walked deep by accident of heap sizing. + # + # `11_collect_at_depth` collects at maximum recursion depth with a + # live root in every frame, so all three arms now walk a real stack — + # 228 frames and 221 locations on macOS, where the old best was 0. + PERRY_GC_TRACE=1 PERRY_RS4GC=1 \ + PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1 \ + PERRY_GC_HEAP_LIMIT=8 PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off \ + "/tmp/rs4gc-11_collect_at_depth$exe" > /dev/null 2> /tmp/rs4gc-trace.err + "$py" scripts/gc_walker_trace_assert.py /tmp/rs4gc-trace.err \ + --require-locations # #7327. Everything above pins PERRY_LLVM_OPT + PERRY_LLVM_CLANG to one # brew install, because RS4GC piped IR through an external `opt` and a diff --git a/benchmarks/gc_ratchet/probes/11_collect_at_depth.ts b/benchmarks/gc_ratchet/probes/11_collect_at_depth.ts new file mode 100644 index 0000000000..bee2056a4f --- /dev/null +++ b/benchmarks/gc_ratchet/probes/11_collect_at_depth.ts @@ -0,0 +1,70 @@ +// GC ratchet probe: collect WHILE the stack is deep, with a live root in +// every frame. +// +// Every other probe in this suite calls `gc()` at the end, from a shallow +// stack, so the native-root walker has almost nothing to walk. Measured on the +// same suite: `04_dead_after_deep_stack` reports 7 frames visited and **zero** +// root locations on macOS and Linux — those arms would pass unchanged with a +// walker that visited nothing at all. The Windows arm only exercises a deep +// walk (5,626 frames, 5,449 records) by accident of heap sizing, not by design. +// +// This probe makes that coverage deliberate and portable. `descend` holds a +// heap value live ACROSS its recursive call, and the collection happens at the +// deepest point — so at collection time there is one live root per frame, all +// of them mid-frame rather than in the leaf. A walker that stops early, or a +// map whose bases are wrong, loses roots that are still read afterwards, and +// the checksum diverges from the oracle instead of the run merely being slower. +// +// Under `PERRY_GC_FORCE_EVACUATE=1` every survivor also MOVES, so a stale +// pointer is a wrong value rather than a lucky one. + +declare function gc(): void; + +const DEPTH = 220; +const ROUNDS = 3; + +class Payload { + tag: number; + body: string; + constructor(tag: number) { + this.tag = tag; + this.body = "d" + tag; + } + value(): number { + return (this.tag + this.body.length) | 0; + } +} + +let escape: Payload | null = null; + +function descend(depth: number): number { + // Live across the recursive call below, which is where the collection + // happens. Its contents are read after that call returns, so the collector + // must have found and relocated this slot. + const mine = new Payload(depth); + + if (depth === 0) { + // Deepest frame: collect with ~DEPTH live roots resident on the stack. + escape = mine; + gc(); + escape = null; + return mine.value(); + } + + const deeper = descend(depth - 1); + return (mine.value() + deeper) | 0; +} + +let checksum = 0; +for (let round = 0; round < ROUNDS; round++) { + checksum = (checksum + descend(DEPTH)) | 0; +} + +gc(); +const mu = process.memoryUsage(); + +console.log("probe:11_collect_at_depth"); +console.log("checksum:" + checksum); +console.error("#gcmetric heap_used_bytes=" + mu.heapUsed); +console.error("#gcmetric heap_total_bytes=" + mu.heapTotal); +console.error("#gcmetric rss_bytes=" + mu.rss); diff --git a/changelog.d/7359-deep-stack-collect-probe.md b/changelog.d/7359-deep-stack-collect-probe.md new file mode 100644 index 0000000000..3923cbd363 --- /dev/null +++ b/changelog.d/7359-deep-stack-collect-probe.md @@ -0,0 +1,35 @@ +### Tests + +**GC ratchet: a probe that collects at stack depth, and walker-liveness gating on every platform.** + +The native-root stack walker had no probe that exercised it. Every probe in +`benchmarks/gc_ratchet/probes/` calls `gc()` at the end, from a shallow stack, +so on macOS and Linux even `04_dead_after_deep_stack` reported +`frames_visited: 7, locations_visited: 0` — **zero** root locations. Both arms +would have passed the whole suite unchanged with a walker that visited nothing, +because other root sources covered those probes. Windows walked a deep stack +(5,626 frames) only by accident of heap sizing, and that accident is the sole +reason the `--require-locations` gate added in #7354 could be applied there and +nowhere else. This is the fourth failure mode in CLAUDE.md's list: the gate ran, +but its subject never did. + +`11_collect_at_depth` makes the coverage deliberate. `descend` holds a heap +value live *across* its recursive call and collects at the deepest point, so at +collection time there is one live root per frame, all mid-frame rather than in +the leaf. Each slot is read after the collection returns, so a walker that stops +early — or a map with a wrong base register — yields a wrong checksum rather +than a merely slower run; under `PERRY_GC_FORCE_EVACUATE=1` every survivor +moves, so a stale pointer cannot be accidentally correct. + +Measured against the pinned Node oracle, byte-identical on both: + +| arm | frames | locations | before | +|---|---|---|---| +| macOS aarch64 | 228 | 221 | 7 / 0 | +| x86-64 Linux | 231 | 221 | 7 / 0 | + +With both Unix arms now walking a real stack, `gc_walker_trace_assert.py +--require-locations` moves off the Windows-only branch in +`.github/workflows/gc-native-roots.yml` and gates all three arms. Full ratchet +suite: 11/11 byte-identical under `PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1 +PERRY_GC_VERIFY_EVACUATION=1`.