fix(gc): a budgeted incremental cycle that nothing drives now completes (#6978)

changelog.d/7015-gc-safepoint-cycle-completion.md

@@ -0,0 +1,56 @@
+### Fixed
+
+- **GC: a budgeted incremental cycle that nothing drives now completes (#6978).**
+  The budgeted stepper had a budget but no completion guarantee. It is driven
+  by exactly two things — allocation-point mutator assists (`gc_check_trigger`)
+  and host safepoints (`gc_runtime_safepoint`, from the microtask checkpoint
+  and the stdlib pump) — so a program that stops allocating stopped driving it
+  and the cycle parked for the life of the process.
+
+  Measured on `test_gap_repsel_canonical_i32` under `PERRY_GC_HEAP_LIMIT=8`
+  (release, macOS arm64): `gc_check_trigger` runs **twice in the whole
+  process**. The trigger is `ArenaBytes` — this program never calls
+  `gc_malloc` at all (`malloc_count=0` against a 100 000 trigger), so the
+  malloc-trigger mechanism #6978 hypothesised is not involved. The second call
+  arms the cycle and pays one 256-unit assist, and no allocation-point
+  opportunity ever comes again; the host safepoints that follow each pay the
+  fixed 2 048-unit slice and get the cycle through `BuildValidPointerSet` and
+  one step into `RootScan` before the process exits. The assist *budget* was
+  never the binding constraint — the number of opportunities was, and a cycle
+  has seven resumable phases with one `step()` advancing at most one of them,
+  so completion needs a bounded number of calls rather than more units.
+
+  A parked cycle is not inert: nothing is reclaimed, the arming trigger is
+  never re-baselined, every subsequent allocation is born **black** so the
+  parked cycle can never collect it, the incremental mark barrier stays armed
+  on every store, and `gc_safepoint_moving_minor` (the precise-root copying
+  minor at the outermost microtask boundary) early-returns on
+  `gc_budgeted_cycle_active()` — so the parked cycle also disabled the
+  collector's own moving path. Net effect on a compiled program in the shipped
+  configuration: zero completed collections.
+
+  The host safepoint now carries the guarantee — the one point in the process
+  where the mutator has yielded and the JS stack has unwound. A cycle may span
+  `GC_CYCLE_HOST_SAFEPOINT_LIMIT` (2) safepoints on the ordinary bounded
+  budget; at the next one the safepoint drives it to completion, in a loop
+  bounded exactly like `gc_drain_active_budgeted_cycle`. Kill switch
+  `PERRY_GC_SAFEPOINT_FINISH=0`.
+
+  The limit does not bind on healthy workloads, measured: on
+  `test_gap_repsel_gc_stress` the budgeted cycles complete on mutator assists
+  alone (20 / 25 / 35 assist steps, **zero** host safepoints), and an async
+  probe that yields every 256 iterations while allocating reports
+  `safepoint_steps=0` in every configuration.
+
+  `PERRY_GC_TRACE=1` cycle counts under `PERRY_GC_HEAP_LIMIT=8` with no other
+  GC env var: `test_gap_repsel_canonical_i32` 0 → 1,
+  `test_gap_repsel_ptr_shape_locals` 0 → 1, `test_gap_repsel_gc_stress`
+  21 → 22, all still byte-exact against node 26.5.0.
+  `scripts/gc_repsel_matrix.sh --arms all --pressure 8` (440 cells, same
+  binaries A/B'd through the kill switch): `PASS=229 UNVER=190 XFAIL=1
+  FAIL=20` → `PASS=400 UNVER=19 XFAIL=1 FAIL=20`, the only cell transition
+  being `UNVER → PASS` × 171, with an identical FAIL set (#6981) and no
+  `PASS → non-PASS`. Costs one collection's worth of work that previously
+  never ran (+1.2 – 2.7 % wall clock where a heap budget makes a trigger due,
+  0 % without one) and does **not** regress max pause: the added cycle's
+  largest step is 16.0 ms against the run's pre-existing 54.6 ms maximum.

crates/perry-runtime/src/gc/policy.rs

@@ -1757,6 +1757,12 @@ fn gc_finish_budgeted_cycle(mut cycle: BudgetedGcCycle) -> JsGcStepResult {
         }
     }
     GC_BUDGETED_CYCLE_ACTIVE.with(|active| active.set(false));
+    // #6978: the host-safepoint allowance is charged PER CYCLE. Reset it on
+    // every completion, whatever completed the cycle — a mutator assist can
+    // finish one cycle and arm the next entirely between two safepoints, and
+    // the next cycle must not inherit its predecessor's spent allowance and be
+    // finished on its very first safepoint.
+    gc_reset_host_safepoint_starvation();
     gc_step_result(
         JS_GC_STEP_STATUS_COMPLETED,
         GcCyclePhase::Complete.ffi_code(),
@@ -1884,12 +1890,54 @@ fn gc_mutator_assist_step_work_units_inner_with_progress(
     gc_budgeted_step_work_units_inner_with_progress(work_units, start_progress_kind)
 }
 
+/// Host safepoint: the mutator has yielded (microtask checkpoint / stdlib
+/// pump) and the JS stack has unwound.
+///
+/// Ordinarily this pays one `NormalIncremental`-budgeted slice. #6978: once a
+/// cycle has been offered `GC_CYCLE_HOST_SAFEPOINT_LIMIT` such slices without
+/// completing, nothing else in the process is obliged to finish it — the
+/// allocation-point assists that would have are only emitted by allocation
+/// the program may never do again — so finish it here. One `step()` advances
+/// at most one phase even with an unbounded budget, so completion is a loop,
+/// bounded exactly like `gc_drain_active_budgeted_cycle`: seven phases plus
+/// slack, and a blocked stepper (suppression / unsafe zone / root lock)
+/// reports SKIPPED without progress, so bail then rather than spin.
 pub(crate) fn gc_runtime_safepoint() -> JsGcStepResult {
     let budget = gc_progress_contract().budget_for(GcProgressKind::NormalIncremental);
     let Some(work_units) = budget.work_units else {
         return gc_budgeted_status_result();
     };
-    gc_budgeted_step_work_units_inner_with_progress(work_units, GcProgressKind::NormalIncremental)
+    if !gc_host_safepoint_starvation_due() {
+        return gc_budgeted_step_work_units_inner_with_progress(
+            work_units,
+            GcProgressKind::NormalIncremental,
+        );
+    }
+    if std::env::var_os("PERRY_GC_DIAG").is_some() {
+        eprintln!(
+            "[gc-safepoint] finishing a cycle parked across {GC_CYCLE_HOST_SAFEPOINT_LIMIT} host safepoints"
+        );
+    }
+    let mut result = gc_budgeted_step_work_units_inner_with_progress(
+        usize::MAX,
+        GcProgressKind::NormalIncremental,
+    );
+    for _ in 0..64 {
+        if result.status != JS_GC_STEP_STATUS_ACTIVE {
+            break;
+        }
+        result = gc_budgeted_step_work_units_inner_with_progress(
+            usize::MAX,
+            GcProgressKind::NormalIncremental,
+        );
+    }
+    // A completion resets the allowance in `gc_finish_budgeted_cycle`. Do NOT
+    // reset it here on any other exit: if the stepper was blocked (suppression
+    // / unsafe zone / root lock) or the phase loop ran out, the cycle is still
+    // parked, and clearing the count would send it back through another
+    // `GC_CYCLE_HOST_SAFEPOINT_LIMIT` bounded slices before retrying — on a
+    // program with few safepoints, possibly never.
+    result
 }
 
 fn write_gc_step_result(out: *mut JsGcStepResult, result: JsGcStepResult) -> u32 {

crates/perry-runtime/src/gc/progress.rs

@@ -26,6 +26,95 @@ pub const GC_MUTATOR_ASSIST_SOFT_PAUSE_US: u64 = 500;
 /// workloads never see the scaled assists.
 pub const GC_ASSIST_DEBT_BYTES_PER_WORK_UNIT: u64 = 32;
 
+/// COMPLETION GUARANTEE (#6978): the most host safepoints one budgeted cycle
+/// may span on the ordinary bounded budget before the safepoint finishes it
+/// outright.
+///
+/// The budgeted stepper has a *budget* but no *completion guarantee*, and it
+/// is driven by exactly two things: allocation-point mutator assists
+/// (`gc_check_trigger`) and host safepoints (`gc_runtime_safepoint`, called
+/// from the microtask checkpoint and the stdlib pump). A program that stops
+/// allocating stops driving it. Measured on `test_gap_repsel_canonical_i32`
+/// under `PERRY_GC_HEAP_LIMIT=8` (release, macOS arm64): `gc_check_trigger`
+/// runs **twice in the whole process** — the second call arms an `ArenaBytes`
+/// cycle and pays one 256-unit assist, and no allocation-point opportunity
+/// ever comes again. The host safepoints that follow each paid the fixed
+/// `GC_NORMAL_INCREMENTAL_WORK_UNITS` slice, which got the cycle through
+/// `BuildValidPointerSet` and one step into `RootScan` before the process
+/// exited. Note a fatter per-step budget cannot rescue this on its own: a
+/// cycle has seven resumable phases and one `step()` advances at most one of
+/// them, so completion needs a bounded *number of calls*, not more units.
+///
+/// A PARKED CYCLE IS NOT INERT. Until it completes:
+///   * nothing is reclaimed and the arming trigger is never re-baselined;
+///   * every subsequent allocation is born BLACK (`gc_birth_extra_flags`), so
+///     the parked cycle can never collect it;
+///   * the incremental mark barrier stays armed on every store; and
+///   * `gc_safepoint_moving_minor` — the precise-root copying minor at the
+///     outermost microtask boundary — returns early on
+///     `gc_budgeted_cycle_active()`, so the parked cycle also disables the
+///     collector's own moving path for the rest of the process.
+/// Net effect on a compiled program in the shipped configuration: ZERO
+/// completed collections.
+///
+/// A host safepoint is where the mutator has yielded and the JS stack has
+/// unwound — the cheapest and safest point in the process to finish a
+/// collection. So: a cycle may span this many host safepoints on the ordinary
+/// bounded budget; at the next one the safepoint drives it to completion.
+/// Measured cost on the corpus member that actually collects
+/// (`test_gap_repsel_gc_stress`, `--pressure 8`): its cycles complete on
+/// mutator assists alone (20 / 25 / 35 assist steps, **zero** host
+/// safepoints), so a healthy allocating workload never reaches this limit and
+/// pays nothing. The bound binds only when the mutator has stopped driving
+/// the collector — precisely when parking forever is the alternative.
+///
+/// Kill switch: `PERRY_GC_SAFEPOINT_FINISH=0` (also `off` / `false`).
+pub const GC_CYCLE_HOST_SAFEPOINT_LIMIT: u32 = 2;
+
+std::thread_local! {
+    /// How many host safepoints the currently-active budgeted cycle has
+    /// already been offered. Reset whenever a safepoint finds no active cycle
+    /// — which covers completion through every path (mutator assist, host
+    /// safepoint, the drain before a synchronous collection).
+    static GC_CYCLE_HOST_SAFEPOINTS: std::cell::Cell<u32> =
+        const { std::cell::Cell::new(0) };
+}
+
+/// Kill switch for the #6978 completion guarantee. Default ON; `0` / `off` /
+/// `false` restores the pre-fix behaviour, where every host safepoint takes
+/// one bounded step and a cycle nobody drives parks for the life of the
+/// process.
+fn gc_safepoint_finish_enabled() -> bool {
+    static CACHED: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
+    *CACHED.get_or_init(|| {
+        !matches!(
+            std::env::var("PERRY_GC_SAFEPOINT_FINISH").as_deref(),
+            Ok("0") | Ok("off") | Ok("false")
+        )
+    })
+}
+
+/// Count this host safepoint against the active budgeted cycle and report
+/// whether the cycle has now outlived `GC_CYCLE_HOST_SAFEPOINT_LIMIT` of them
+/// — i.e. whether this safepoint must finish it instead of taking another
+/// bounded slice.
+pub(super) fn gc_host_safepoint_starvation_due() -> bool {
+    if !super::gc_budgeted_cycle_active() || !gc_safepoint_finish_enabled() {
+        gc_reset_host_safepoint_starvation();
+        return false;
+    }
+    let seen = GC_CYCLE_HOST_SAFEPOINTS.with(|seen| {
+        let next = seen.get().saturating_add(1);
+        seen.set(next);
+        next
+    });
+    seen > GC_CYCLE_HOST_SAFEPOINT_LIMIT
+}
+
+pub(super) fn gc_reset_host_safepoint_starvation() {
+    GC_CYCLE_HOST_SAFEPOINTS.with(|seen| seen.set(0));
+}
+
 /// Runtime-visible classification for GC progress.
 ///
 /// Only `NormalIncremental` and `MutatorAssist` satisfy the low-pause

crates/perry-runtime/src/gc/tests/host_safepoints.rs

@@ -141,6 +141,135 @@ fn repeated_runtime_safepoints_complete_cycle_rebaseline_debt_and_preserve_roots
     }
 }
 
+/// #6978: a budgeted cycle that nothing drives must still complete.
+///
+/// The stepper has a budget but no completion guarantee. Its only two drivers
+/// are allocation-point mutator assists and host safepoints, so a program
+/// that stops allocating parks its cycle — measured on a compiled program,
+/// `gc_check_trigger` ran twice in the whole process, and the seven host
+/// safepoints that followed each paid one fixed `NormalIncremental` slice and
+/// left the cycle in `RootScan`. Parked is not inert: nothing is reclaimed,
+/// the arming trigger is never re-baselined, every later allocation is born
+/// black, and `gc_safepoint_moving_minor` early-returns on
+/// `gc_budgeted_cycle_active()` for the rest of the process.
+///
+/// So the host safepoint carries the guarantee: a cycle may span
+/// `GC_CYCLE_HOST_SAFEPOINT_LIMIT` safepoints on the ordinary bounded budget,
+/// and the next one finishes it. Both halves are asserted — the early
+/// safepoints must NOT finish it (that would make incremental mode
+/// synchronous), and the one past the limit must, WITHOUT any further
+/// allocation to assist it.
+#[test]
+fn a_cycle_that_nothing_drives_is_finished_within_a_bounded_number_of_host_safepoints() {
+    let _guard = CopyingNurseryTestGuard::new(1);
+    let trigger_guard = GcTriggerThresholdTestGuard::suppress_automatic_triggers();
+    reset_old_reclaim_pressure();
+    make_arena_pressure(&trigger_guard, b"host_safepoint_starved_live");
+
+    let before = gc_collection_count();
+    let started = gc_runtime_safepoint();
+    assert_eq!(started.status, JS_GC_STEP_STATUS_ACTIVE);
+    for offered in 1..=GC_CYCLE_HOST_SAFEPOINT_LIMIT {
+        let result = gc_runtime_safepoint();
+        assert_eq!(
+            result.status, JS_GC_STEP_STATUS_ACTIVE,
+            "host safepoint {offered} of {GC_CYCLE_HOST_SAFEPOINT_LIMIT} must stay an \
+             ordinary bounded step -- finishing early would make incremental mode synchronous"
+        );
+        assert_eq!(gc_collection_count(), before);
+    }
+
+    // No allocation has happened since the cycle armed, so no mutator assist
+    // can ever come. This safepoint is the completion guarantee.
+    let completed = gc_runtime_safepoint();
+    assert_eq!(
+        completed.status, JS_GC_STEP_STATUS_COMPLETED,
+        "a cycle offered more than {GC_CYCLE_HOST_SAFEPOINT_LIMIT} host safepoints without \
+         completing must be finished by the safepoint, not parked"
+    );
+    assert_eq!(completed.completed, 1);
+    assert_eq!(completed.active, 0);
+    assert!(gc_collection_count() > before);
+    assert_eq!(
+        js_gc_step_status(std::ptr::null_mut()),
+        JS_GC_STEP_STATUS_IDLE,
+        "no budgeted cycle may remain parked after the completion guarantee fires"
+    );
+    assert!(
+        GC_NEXT_TRIGGER_BYTES.with(|trigger| trigger.get()) > crate::arena::arena_total_bytes(),
+        "the finished cycle must re-baseline the arming trigger like any other completion"
+    );
+
+    let live_after = (js_shadow_slot_get(0) & POINTER_MASK) as *const crate::StringHeader;
+    unsafe {
+        assert_string_bytes(live_after, b"host_safepoint_starved_live");
+    }
+}
+
+/// #6978 follow-up: the host-safepoint allowance is charged PER CYCLE.
+///
+/// A mutator assist can finish one cycle and arm the next entirely between two
+/// host safepoints. If the counter were only cleared when a safepoint happens
+/// to observe an idle collector, the second cycle would inherit its
+/// predecessor's spent allowance and be driven to completion on its very first
+/// safepoint — silently synchronous. The FFI stepper stands in here for the
+/// mutator assist: it drives cycles without going through
+/// `gc_runtime_safepoint`, which is exactly the off-safepoint path.
+#[test]
+fn the_host_safepoint_allowance_is_charged_per_cycle_not_carried_across_cycles() {
+    let _guard = CopyingNurseryTestGuard::new(1);
+    let trigger_guard = GcTriggerThresholdTestGuard::suppress_automatic_triggers();
+    reset_old_reclaim_pressure();
+
+    // Cycle A: spend the whole safepoint allowance without completing it.
+    make_arena_pressure(&trigger_guard, b"host_safepoint_cycle_a_live");
+    assert_eq!(gc_runtime_safepoint().status, JS_GC_STEP_STATUS_ACTIVE);
+    for _ in 0..GC_CYCLE_HOST_SAFEPOINT_LIMIT {
+        assert_eq!(gc_runtime_safepoint().status, JS_GC_STEP_STATUS_ACTIVE);
+    }
+
+    // ...then finish A OFF-safepoint, and arm + start B off-safepoint too, so
+    // no safepoint ever observes the collector idle in between.
+    let mut status = JsGcStepResult::default();
+    let mut finished_off_safepoint = false;
+    for _ in 0..64 {
+        if js_gc_step_work_units(u64::MAX, &mut status) != JS_GC_STEP_STATUS_ACTIVE {
+            finished_off_safepoint = true;
+            break;
+        }
+    }
+    assert!(
+        finished_off_safepoint,
+        "cycle A should complete through the host-driven stepper"
+    );
+    make_arena_pressure(&trigger_guard, b"host_safepoint_cycle_b_live");
+    assert_eq!(
+        js_gc_step_work_units(1, &mut status),
+        JS_GC_STEP_STATUS_ACTIVE,
+        "cycle B should start off-safepoint, the way a mutator assist starts one"
+    );
+
+    // Cycle B must get its OWN allowance, not A's leftovers.
+    for offered in 1..=GC_CYCLE_HOST_SAFEPOINT_LIMIT {
+        assert_eq!(
+            gc_runtime_safepoint().status,
+            JS_GC_STEP_STATUS_ACTIVE,
+            "safepoint {offered} of the SECOND cycle must still be an ordinary bounded \
+             step -- the allowance is per cycle, not per process"
+        );
+    }
+    assert_eq!(
+        gc_runtime_safepoint().status,
+        JS_GC_STEP_STATUS_COMPLETED,
+        "and the guarantee must still fire for the second cycle"
+    );
+
+    let live_after = (js_shadow_slot_get(0) & POINTER_MASK) as *const crate::StringHeader;
+    unsafe {
+        assert_string_bytes(live_after, b"host_safepoint_cycle_b_live");
+    }
+}
+
 #[test]
 fn microtask_runner_tail_pays_bounded_safepoint_under_pressure() {
     let _guard = CopyingNurseryTestGuard::new(1);

docs/src/internals/memory-model.md

@@ -117,7 +117,8 @@ Idle nursery blocks observed empty for 2 GC cycles are `dealloc`'d back to the O
 | `PERRY_GC_FORCE_EVACUATE=1` | With generated write barriers active and policy evacuation allowed, stress-copy every marked non-pinned nursery object instead of only tenured survivors. |
 | `PERRY_GC_VERIFY_EVACUATION=1` | After an evacuation that actually forwards objects, panic if any mutable live slot still points at a forwarded nursery object after rewrite. |
 | `PERRY_WRITE_BARRIERS=0` / `off` / `false` | Disable codegen-emitted write barriers at compile time and runtime exact helper barriers at runtime for benchmark/debug bisection. Unset, `=1`, `=on`, and `=true` keep barriers enabled. |
-| `PERRY_GC_DIAG=1` | Print per-cycle diagnostics, including one evacuation-policy line for cycles where evacuation was considered and for `barriers_inactive` skips. |
+| `PERRY_GC_SAFEPOINT_FINISH=0` / `off` / `false` | Disable the host-safepoint completion guarantee (#6978). A budgeted incremental cycle is driven only by allocation-point mutator assists and host safepoints; a program that stops allocating stops driving it, and a parked cycle reclaims nothing, keeps the mark barrier armed, makes every later allocation born-black, and blocks the moving safepoint minor for the rest of the process. By default a cycle may span `GC_CYCLE_HOST_SAFEPOINT_LIMIT` host safepoints on the ordinary bounded budget and the next one finishes it. Bisection only. |
+| `PERRY_GC_DIAG=1` | Print per-cycle diagnostics, including one evacuation-policy line for cycles where evacuation was considered and for `barriers_inactive` skips, and a `[gc-safepoint] finishing a cycle parked across N host safepoints` line whenever the completion guarantee above fires. |
 
 ## Why this design