gc: Track A2's premise is wrong — six force_full_scan() sites, four automatic, and PERRY_CONSERVATIVE_STACK_SCAN=full fails 134 runtime tests

[proggeramlug]

Track A2 of the repsel/GC plan calls for deleting the conservative native-stack
scanner, on the premise that it is "not merely unused in production" — that
production resolves Auto -> SkipDisabled and only two force_full_scan()
sites remain, both on the explicit gc() path.

That premise does not hold. The test-build half is real and is fixed in
#7147. The production half is a different change than the plan describes, and
this issue records the inventory and the soundness argument so the decision is
made on the facts.

There are six force_full_scan() sites, not two

#	site	path	automatic?
1	gc/policy.rs:1300	OldReclaim direct full mark-sweep at an allocation point	yes
2	gc/policy.rs:1404	nursery-churn direct minor at an allocation point	yes (fallback, see below)
3	gc/mod.rs:336	gc_try_emergency_reclaim()	yes (OOM path)
4	gc/pressure.rs:54	perry/gc pressure API (host callback)	host-driven
5	gc/policy.rs:2164	manual_gc_collect_now() — explicit gc()	explicit
6	gc/policy.rs:2199	js_gc_module_minor() — perry/gc minor()	explicit

Only 5 and 6 are the gc() paths the plan names. Four of the six are reached
without any user calling gc().

They are not test scaffolding — the code says why

Each site documents a non-safepoint rationale:

• policy.rs:1404 — "at an arbitrary alloc point a value mid-construction may
  live only in registers, which the conservative scan retains."
• policy.rs:1300 — "anything still referenced from the stack/registers at
  this allocation point (e.g. the temporary currently being built) is
  retained."
• mod.rs:336 — "this runs at an arbitrary allocation site where locals of the
  current call chain may not be spilled to shadow slots."
• pressure.rs:54 — "a host may deliver the callback with unspilled locals on
  the native frames above us."

And policy.rs draws the distinction explicitly, in its own words:

Unlike gc_scavenge_enabled() (which skips the conservative scan HERE —
sound only if the alloc point is precise), the loop-polls path never
reaches the skip: it always defers to a real safepoint, so it is sound by
construction.

So skipping the scan at these points is already recognised in-tree as
conditional on a precision property the alloc point does not have in general.
PERRY_GC_SCAVENGE is the knob that does exactly this at site 2, and it is
labelled "Phase-1 de-risking, OFF by default" — the project has deliberately
not flipped it. Removing site 2's force_full_scan() unconditionally is that
flip, for every path at once, including three that have no safepoint
alternative.

What does soften the risk

gc_moving_loop_polls_enabled() is default ON
(policy.rs:369 — unset resolves true), so the shipped default defers the
alloc-point collection to js_gc_loop_safepoint -> gc_safepoint_moving_minor,
a precise-root safepoint. Site 2's force_full_scan() is therefore the
safety-valve fallback taken when the deferral slack is exhausted — a
mega-expression that reached no poll — not the common path. That is a narrower
exposure than it first appears, but it is also the sharpest case: the fallback
fires precisely when no precise safepoint was reachable.

(Note: fable-audit-perry-gc.md lists PERRY_GC_MOVING_LOOP_POLLS as "default
off". The code says otherwise. Stale doc, worth correcting separately.)

Evidence that already exists

The matrix's cons_scan_off / cons_scan_off_force arms set
PERRY_CONSERVATIVE_STACK_SCAN=off, and the env value beats the per-thread
override — so those arms already disable all six sites and are green. Any
re-run of the matrix against a code-removal branch reproduces that result by
construction; it is not independent confirmation. The real question is whether
a 22-program corpus is sufficient basis to make it the default for everyone,
against six documented soundness comments.

A separate, unambiguous kill-policy violation

PERRY_CONSERVATIVE_STACK_SCAN=full fails 134 of 1574 perry-runtime tests on
main (verified against pristine main roots.rs at a3b31c0d8; measured on
darwin-arm64, release, --test-threads=1 — 1440 pass / 134 fail). The env value
beats the per-thread override, so the isolation guards cannot opt back out and
tests that require collection of native-stack-held objects break.

This is a shipped escape hatch in a state nobody has verified. Worth resolving
independently of the Track A2 decision.

(Correction to an earlier revision of this issue: the gc_ratchet baseline is
not measured under =full. benchmarks/gc_ratchet/README.md uses =full as
the deliberate sensitivity arm that proves the gate can fail — the control
arm reproduced the baseline exactly and the =full arm produced 60 regression
rows.)

The scan is also expensive — this is the RSS argument, quantified

That same validation table is the strongest quantitative case for removing the
scan, and it is already in-tree
(benchmarks/gc_ratchet/README.md). Turning the conservative scan on costs,
per probe, heap_used_bytes +364% to +5371%, and takes minor_cycles to
zero on all eight probes — it does not merely slow the copying minor down,
it stops it running at all:

Probe	baseline	with conservative scan	Δ	minor_cycles
01_nursery_churn	807,000	3,742,800	+364%	14 → 0
02_survivor_promotion	5,022,864	25,242,760	+403%	10 → 0
03_cross_gen_writes	705,320	6,494,536	+821%	22 → 0
04_dead_after_deep_stack	1,000,728	9,187,088	+818%	104 → 0
05_closure_capture	1,040,208	7,336,208	+605%	26 → 0
06_string_retention	746,056	4,746,240	+536%	64 → 0
07_array_grow_evacuate	1,816,232	99,362,360	+5371%	80 → 0
08_map_set_sidetables	457,872	6,754,560	+1375%	84 → 0

So on the four automatic sites, every collection that takes the
force_full_scan() path is a collection that runs no copying minor. That
raises the stakes on both sides: the upside of removal is large and measured,
and it is also why the fallback must not simply be made imprecise — the right
end state is probably that the fallback stops being reachable (precise-safepoint
deferral always available) rather than that it collects without the scan.

Recommendation

Split the decision:

1. Retire the full value of PERRY_CONSERVATIVE_STACK_SCAN (134 failing
   tests, no CI arm — a clean kill-policy case), and re-check what the ratchet
   baseline is actually measured under.
2. Treat "remove the six force_full_scan() sites" as its own change, with
   the evidence it deserves: either land the precise-safepoint deferral as the
   only path (so the fallback stops existing rather than becoming imprecise), or
   flip PERRY_GC_SCAVENGE on and soak it first. It is not a mode-deletion
   cleanup, and CLAUDE.md's kill-policy does not by itself settle it — the
   losing mode here is load-bearing at four automatic sites.

Filed from the Track A2 investigation; #7147 delivers the part that is
unambiguously correct (the test build now runs production's scan mode).

[proggeramlug]

PR #7166 takes this on. Per-site disposition, and the cross-link this issue's
end state depends on.

The enforcement mechanism already exists — on exp/stackmap-viability

This issue asks for the fallback to become unreachable rather than
imprecise. "Unreachable" is only a real property if something checks it, and
the statepoint experiment already built that check. From
docs/statepoint-gc-experiment.md (branch exp/stackmap-viability, section
"Follow-up: the explicit-safepoint collection contract"):

A collection that skips the conservative stack scan consumes only precise
roots; with native stack maps active, precise frame roots exist only at
mapped PCs. Therefore such a collection may only begin at a declared
safepoint (a loop back-edge poll or the outermost microtask-pump boundary) —
anywhere else it must scan conservatively. The runtime already routes moving
minors to those safepoints (the #7024 deferral machinery), so today the
property is emergent: it holds because every possibly-collecting call
happens to be mapped. The contract makes it enforced — a thread-local
declared-safepoint flag plus a check at the root-scan subphase — and
enforcement is what makes it sound to stop mapping call sites.

Two enforcement levels there: PERRY_GC_SAFEPOINT_ONLY=1 (heal — an undeclared
precise-root cycle gets the conservative scan forced for that cycle) and
=strict (panic — the gate mode that proves the enforcement is live, per the
four-ways-a-gate-cannot-fail rule). Getting that contract working found three
implementation bugs, each caught by a gate, and the binding constraint it
identified is re-entry, not allocation: on batch.ts, of 217 statepoints
~85 are property-access diamonds where a getter can re-enter, ~40 more are
coercion/setter/throw paths, and only ~25-30 are pure-allocation sites the
contract can unmap.

That branch's "Work deliberately left gated" section names this issue's work
as the next experiment after the #7114 prerequisite (which has landed):

Once that prerequisite lands, the next experiment is to assert that moving
collections occur only at declared safepoints and then measure whether the
conservative scanner can be deleted.

PR #7166 is the reachability census for that contract, on main. It counts
the sites where the contract's "anywhere else" arm is taken, and counts the
precise-safepoint drains that replace them. Census first, enforcement second:
the counts say whether the arm is ever taken; the contract's check is what makes
zero provable rather than observed. Adopting the enforcement flag itself
belongs with the statepoint branch's landing, not ahead of it — on main there
are no stack maps, so the flag's strict mode would be policing a property that
the shadow stack already provides by other means, and the honest first step is
to find out whether the arm is reached at all.

Per-site disposition in #7166

#	site	disposition	pressure-spike answer
1	OldReclaim policy.rs	defer — gc_safepoint_moving_minor now drains it as the same full mark-sweep with precise roots	bounded by a slack in the trigger's own unit (old-gen in-use + external side bytes, one 32 MB growth quantum), then the old conservative path, counted
2	nursery-churn policy.rs	keep as the bounded valve, counted	it is the answer — reaching it means no safepoint was reachable
3	emergency reclaim mod.rs	keep, justified, observable	vacuous: allocation has already failed and the caller's next act is to panic, so there is no next safepoint
4	host pressure pressure.rs	defer when a generated frame is live; collect precisely, no scan when the shadow stack is empty	the lowered arena trigger is untouched, so the alloc-point arm still collects at the next check — identical to the pre-existing blocked path
5	gc()	keep, observable	n/a — user request, synchronous semantics
6	perry/gc minor()	keep, observable	n/a — returns the freed byte count

★ On site 1's unit choice, because this issue is downstream of the #7024
lesson: measuring the old-gen slack in arena bytes would be that bug inverted.
#5476's workload is >16 KB temporaries born straight into the old arena, which
barely move arena_total_bytes(); an arena-unit slack would never expire and
the valve would never fire.

★ On site 4, deferring unconditionally would have been wrong in the direction
that matters. OS memory pressure arrives when a process is idle, and an idle
process reaches no loop back-edge and pumps no microtasks — a blanket deferral
would shed nothing at all before the jetsam. Hence the split on
shadow_stack_has_active_frame().

PERRY_CONSERVATIVE_STACK_SCAN=full

Fixed, not deleted, and the rationale is deliberate: =full is gc-ratchet's
validated sensitivity arm — the run that produced the 60 regression rows proving
that gate can actually fail. Deleting the mode would delete a gate's only
end-to-end proof, which is the failure the kill-policy exists to prevent, one
level up. The 134 failures were never soundness failures: a collector test's
central assertion is "this object should have been collected", and an ambient
conservative scan retains whatever the native stack looks like a pointer to.
Since #7147 the test build has one declared scan mode and the isolation guards
are its authority, so in the test build only a pinned override now beats an
env request for Full. =0 still beats a pinned Full, so the bisection hatch
is intact.

Not done here

ConservativeStackScanMode is not deleted — this issue's own circularity
argument stands, and sites 3/5/6 still need it. Removing those requires precise
roots at an arbitrary PC, which is the statepoint contract's job.

Adjacent and not touched: #7164 (FFI scan_slot SIGSEGV). Not reproduced by
this work.

[proggeramlug]

Correction to my disposition table above, and the measurements

Site 1's disposition changed from "defer" to "keep + add a competing precise
path", and the full release suite is what changed it. Recording that here
because the reasoning generalises to the other sites.

The first implementation deferred the allocation-point old-gen reclaim exactly
like the nursery arm. The release suite failed
check_trigger_drives_old_reclaim_to_completion_without_host_stepping — #5476's
regression test, which asserts that a single gc_check_trigger call (what
every allocation does) drives the reclaim to completion, because the workload it
was filed for is a compute-only loop that reaches no host step. Two reasons that
verdict is right rather than an awkward test:

1. This issue's headline RSS number does not apply at that site. The
   +364%..+5371% figure is the cost of the copying minor becoming ineligible —
   minor_cycles → 0. The old-gen arm runs a full mark-sweep, non-moving
   with or without the scan. What the scan costs there is one cycle of
   conservative retention, which is much smaller and which nobody has
   measured.
2. It trades a measured RSS guarantee for an unmeasured one. Memory not free in benchmark #5476's title
   is RSS climbs unbounded.

So the arm is unchanged and counted, and gc_safepoint_moving_minor instead
gained the same full mark-sweep with precise roots. Programs that reach a
safepoint get their old-gen reclaim precisely and no later than before. The
fallback is attacked by adding a competing earlier precise path, not by
postponing the collection — which I think is a better reading of "unreachable,
not imprecise" than the deferral was.

★ The census answers this issue's central question, and the answer is not the dramatic one

Every gc_ratchet probe, branch build, PERRY_GC_DIAG=1:

probe	old_reclaim_alloc_point	nursery_churn_valve	emergency	host_pressure	manual gc()	drain: nursery_minor
01_nursery_churn	0	0	0	0	1	14
02_survivor_promotion	0	0	0	0	1	10
03_cross_gen_writes	0	0	0	0	3	22
04_dead_after_deep_stack	0	0	0	0	1	104
05_closure_capture	0	0	0	0	1	26
06_string_retention	0	0	0	0	1	64
07_array_grow_evacuate	0	0	0	0	1	80
08_map_set_sidetables	0	0	0	0	1	84

The drain column is identical to the pinned baseline's minor_cycles
(14/10/22/104/26/64/80/84), so the zeros are not "the program did nothing" —
every collection went down the precise path.

The four automatic sites were already unreachable on this corpus before the
change. The only non-zero site is the explicit gc() each probe ends with —
which is why the census separates automatic from explicit; an undifferentiated
count would have shown 8 non-zero rows and read as alarming.

That reframes the issue's economics. The +364%..+5371% is what the fallback
costs when taken, and on the ratchet corpus it is never taken, so RSS is
flat (−0.92%..+1.46%, gated-and-ok on pinned_host; heap_used_bytes
bit-identical on all eight). No RSS win was available here, and reporting one
would require a workload that actually reaches an automatic site. Building that
workload is the obvious follow-up and I have not done it.

Gate results

• cargo test --release -p perry-runtime --lib -- --test-threads=1: 1598 pass
  / 0 fail / 3 ignored.
• Same under PERRY_CONSERVATIVE_STACK_SCAN=full: 1598 / 0 / 3. The
  134-failure item in this issue is 134 → 0.
• gc-ratchet 7 repeats: shared_ci PASS (exit 0), 95 ok rows;
  pinned_host fails on wall_ms alone, on a host at load 40–88 — every
  retention, GC-accounting and RSS row ok. Quiet-host wall clock is
  unmeasured.

PERRY_GC_MOVING_LOOP_POLLS and #7161

#7161 proposes flipping the polls default OFF. This work is written so the flip
cannot break it and every claim is tested under both defaults. Site 1 and the
host-pressure precise collection never read that gate; the host-pressure
deferral keeps two gate-independent drains. Site 2's dependence is pre-existing:
with polls off, gc_check_trigger skips the deferral and every nursery
collection runs behind a forced scan — which is the minor_cycles → 0 regime
this issue's own table quantifies. The census now makes that measurable, which
may be useful to whoever decides #7161.

Adjacent

#7164 (FFI scan_slot SIGSEGV) was not reproduced by any of this work. No
TypeError: value is not a function shape (#7154) appeared in the census,
ratchet or suite runs either.