From 28131a3dcf807379f112a8dc279a7537b9b59d31 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Sun, 4 Oct 2026 11:40:05 -0700 Subject: [PATCH] fix(compaction): keep growth latch after under-threshold folds A fold that only lands just under 60% used to clear the latch, so the next 60% crossing compacted again. The 20% resume gap already exists for still-over folds; it now applies after every compact. --- docs/ARCHITECTURE.md | 4 +-- src/agent/compaction.test.ts | 66 +++++++++++++++++++++++++++++++--- src/agent/compaction.ts | 22 ++++++------ src/provider/context-window.ts | 39 ++++++++++---------- 4 files changed, 97 insertions(+), 34 deletions(-) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 5d02d8112..dfa101708 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -203,12 +203,12 @@ The agent maintains an optional **`manage_tasks`** list (create/update via the h When a cycle's input tokens cross a threshold, the director compacts the inference-facing history (the full run is always retained in the context store). The threshold is **model-aware** — roughly 60% of the active model's real context window — so small-window models compact early enough to avoid provider context-overflow while large-window models do not compact prematurely. The compacted prefix is **append-only across passes**: the existing compacted user turn stays byte-identical; new folds become later summary turns with a harness-inserted assistant spacer (identified by reserved `model: "harness"`, plus a visible sentinel; persisted `[compaction]` tokens without a producer still freeze) between them so the prompt head can remain in the provider KV cache. Model-emitted copies of the spacer are not frozen. Spacer-only model replies are incomplete: ChatDirector nudges, then falls through loop-protection, workflow-idle, and open-task rails rather than empty-settling with work still open. The governor covers five cases: -- **Threshold at a tool pause** — Once over threshold, the follow-up `infer` after a tool batch is swapped for a `compact` cycle, and inference resumes via a host continuation message. After a compact that remains over the high watermark, the governor latches: it does not re-arm on growth alone until usage climbs a wide resume gap (~20% of the window, a full warning band) past the post-compact measurement, or until a summarizing fold gets usage back at or under the threshold. A consecutive-compact cap holds across tool-call occupancy as a second rail. +- **Threshold at a tool pause** — Once over threshold, the follow-up `infer` after a tool batch is swapped for a `compact` cycle, and inference resumes via a host continuation message. After every compact (threshold, operator, overflow), the governor snapshots post-compact usage (`tokensAtLastCompact`) and keeps that growth latch. It does not re-arm the proactive path on growth alone; re-arm requires a wide resume gap (~20% of the window, a full warning band) past that snapshot. Usage back at or under the 60% threshold restores the consecutive-compact cap and the overflow-recovery budget; it does not drop the growth latch. A fold that landed well under (snapshot + 20% already past the 60% trigger) still re-arms at the next 60% crossing because the gap predicate is already satisfied. A fold that only just got under (typical live ~45% after a thin-tail fold) does not sawtooth on the next 60% crossing; it waits until post-compact + 20%. A consecutive-compact cap holds across tool-call occupancy as a second rail. - **Idle (end-of-turn)** — An interactive turn can end with a reply and then sit idle with no tool batch to intercept; the governor requests a continuation at that pause and compacts when it arrives. An operator message that races the continuation still compacts first, then re-enters inference to answer it. - **Idle recompress past the provider TTL** — Cache expiry does not compact. Stored turns stay. When settings set `anthropicCachePrompt`, and the saved Anthropic-protocol cache write is at least 5 minutes old, the pre-inference context transform stubs old tool-result bodies on the outgoing prompt only. `turns.jsonl` is not rewritten. The setting defaults off. - **Operator `/compact`** — A first-class slash command folds now without waiting for the occupancy governor. Optional trailing instructions go to the summarizer (empty uses the default structured fold) and are stored on the compact record so later auto-folds still see them. Director rebuild (`/model`, interrupt, resume) restores those instructions from the latest compact record's parameters. An in-flight turn uses the same pair-safe compact-then-continue hop as threshold compact; an idle compact shows the fold and does not start a new turn. -- **Overflow recovery** — A `context_overflow` inference error would otherwise become a terminal error reply; the governor compacts and retries instead, bounded so a history the compactor cannot shrink does not loop forever. The overflow compact itself always fires, but the recovery counts against the same latch: only an under-threshold fold restores the budget, and the proactive path stays subject to the latched re-arm rule. +- **Overflow recovery** — A `context_overflow` inference error would otherwise become a terminal error reply; the governor compacts and retries instead, bounded so a history the compactor cannot shrink does not loop forever. The overflow compact itself always fires, then snapshots like any other compact. Only an under-threshold fold restores the overflow-recovery budget; that restore does not drop the growth latch. The proactive path re-arms only when usage has grown the resume gap past the post-compact snapshot. The compaction control flow is shaped by a reactor invariant: a `compact` action runs in its own cycle (it cannot be paired with `infer`), and **the reactor delivers no event after a compact cycle**. A director that simply emitted `compact` in place of the follow-up `infer` would leave the loop idle forever — the cause of an earlier stall. Instead the governor pairs `compact` with an emit action (`custom.compaction.continue`), and the host answers that emission by delivering a content-less inbound message (`buildCompactionContinuationMessage()`) through the serial operation queue, generation-guarded like every other deliver. A hop superseded because interrupt already bumped generation and enqueued a rebuild is re-queued onto the replacement agent rather than consumed on the outgoing liveAgent. That message adds no turn (`createInboundTurn` returns `null` for empty content) but re-enters the loop, where the director issues the follow-up `infer` against the freshly truncated history. Each emission is answered at most once (a replayed duplicate of an already-answered emission is ignored), and an unsolicited continuation with neither resume flag set answers `wait` rather than burning a billable inference. The legacy `requestContinuation` closure survives only on the sub-agent path, which delivers directly with no host emit hop; idle arming keeps the two channels exclusive (closure fires and the arming flag stays false, or no closure and the flag reports the arming) so a caller honoring both cannot double-deliver. diff --git a/src/agent/compaction.test.ts b/src/agent/compaction.test.ts index 4e1ae0944..bb050a0c3 100644 --- a/src/agent/compaction.test.ts +++ b/src/agent/compaction.test.ts @@ -593,25 +593,58 @@ describe("compaction governor", () => { expect(governor.usingEstimate).toBe(false); }); - test("clears the latch once usage drops under the high watermark", () => { + test("a deep post-compact fold still re-arms at the next 60% crossing", () => { const governor = createCompactionGovernor(() => undefined); governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); expect( governor.interceptActions(toolDone(), inferAction, capabilities), ).not.toBeNull(); - governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); + // Post-compact measurement is a deep fold, far under the watermark. + // hasWideResumeGap is already satisfied at the next 60% crossing + // (1000 + 0.2*window << threshold), so that crossing still arms. + governor.noteInferenceDone(inferenceDone(1000), tenTurns); expect( governor.interceptActions(toolDone(), inferAction, capabilities), ).toBeNull(); - governor.noteInferenceDone(inferenceDone(1000), tenTurns); + governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); + const actions = governor.interceptActions( + toolDone(), + inferAction, + capabilities, + ); + expect(actions).not.toBeNull(); + expect(actions?.some((a) => a.type === "compact")).toBe(true); + }); + + test("a marginal post-compact fold does not re-arm at the next 60% crossing", () => { + const governor = createCompactionGovernor(() => undefined); + // ~45% of a 128k window; under the 60% high watermark, but close enough + // that overThreshold - postCompact < wideDelta. + const marginalInput = Math.floor(compactionThresholdFor("m") * 0.75); + governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); + expect( + governor.interceptActions(toolDone(), inferAction, capabilities), + ).not.toBeNull(); + + governor.noteInferenceDone(inferenceDone(marginalInput), tenTurns); expect( governor.interceptActions(toolDone(), inferAction, capabilities), ).toBeNull(); - // Next crossing of high arms immediately — no growth delta required. + // Next 60% crossing must not re-arm — the growth latch survived the + // under-threshold fold. governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); + expect( + governor.interceptActions(toolDone(), inferAction, capabilities), + ).toBeNull(); + + // Re-arm only once usage has climbed a wide resume gap past the snapshot. + governor.noteInferenceDone( + inferenceDone(marginalInput + wideDelta), + tenTurns, + ); const actions = governor.interceptActions( toolDone(), inferAction, @@ -638,6 +671,31 @@ describe("compaction governor", () => { expect(actions?.some((a) => a.type === "compact")).toBe(true); }); + test("overflow still compacts while an under-threshold latch blocks the proactive path", () => { + const governor = createCompactionGovernor(() => undefined); + const marginalInput = Math.floor(compactionThresholdFor("m") * 0.75); + governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); + expect( + governor.interceptActions(toolDone(), inferAction, capabilities), + ).not.toBeNull(); + + // Post-compact measurement is a marginal under-threshold fold. The + // growth latch must survive it so the next 60% crossing stays inert. + governor.noteInferenceDone(inferenceDone(marginalInput), tenTurns); + expect( + governor.interceptActions(toolDone(), inferAction, capabilities), + ).toBeNull(); + + governor.noteInferenceDone(inferenceDone(overThreshold), tenTurns); + expect( + governor.interceptActions(toolDone(), inferAction, capabilities), + ).toBeNull(); + + const actions = governor.interceptOverflow(overflowError(), capabilities); + expect(actions).not.toBeNull(); + expect(actions?.some((a) => a.type === "compact")).toBe(true); + }); + test("spacer-echo terminal does not arm idle compact", () => { let continuations = 0; const governor = createCompactionGovernor(() => continuations++); diff --git a/src/agent/compaction.ts b/src/agent/compaction.ts index 9d5785bf4..892daac91 100644 --- a/src/agent/compaction.ts +++ b/src/agent/compaction.ts @@ -169,11 +169,12 @@ export function createCompactionGovernor( let usingEstimate = false; let lastModel: string | undefined; let turnCount = 0; - // Latch after a compact that remained over the high watermark: snapshot the - // post-compact infer's usage, then do not re-arm on growth alone — only a - // wide resume gap past the snapshot (hasWideResumeGap) or usage back at or - // under the threshold clears it. Every compact sets this latch (threshold, - // operator, and overflow alike); only fold evidence moves past it. + // Growth latch: snapshot the first inference.done after a compact, then do + // not re-arm on growth alone — only a wide resume gap past the snapshot + // (hasWideResumeGap) re-arms the proactive path. Every compact sets this + // latch (threshold, operator, and overflow alike). Under-threshold folds + // keep the snapshot; they restore consecutive/overflow/non-converged rails + // but do not drop the latch. let tokensAtLastCompact: number | undefined; let awaitingPostCompactMeasurement = false; // Set when a fold's post-compact measurement is still at or above the @@ -279,12 +280,13 @@ export function createCompactionGovernor( consecutiveThresholdCompacts = MAX_CONSECUTIVE_THRESHOLD_COMPACTS; } } - // Fold evidence: usage back at or under the threshold clears the latch - // and restores both rails (consecutive threshold compacts, overflow - // recoveries). Nothing else resets them — neither tool-call occupancy nor - // a still-over measurement — or compact→infer→compact would loop forever. + // Fold evidence: usage back at or under the threshold restores both rails + // (consecutive threshold compacts, overflow recoveries) and clears + // foldNonConverged. The growth latch stays — only a wide resume gap + // re-arms the proactive path. Nothing else resets the rails — neither + // tool-call occupancy nor a still-over measurement — or + // compact→infer→compact would loop forever. if (isAtOrUnderCompactThreshold(contextTokens, lastModel)) { - tokensAtLastCompact = undefined; consecutiveThresholdCompacts = 0; overflowRecoveries = 0; foldNonConverged = false; diff --git a/src/provider/context-window.ts b/src/provider/context-window.ts index 6a0580a89..ab74b626f 100644 --- a/src/provider/context-window.ts +++ b/src/provider/context-window.ts @@ -161,13 +161,14 @@ export const COMPACTION_WINDOW_FRACTION = 0.6; // keep 80 in warning and start danger at 81. export const CONTEXT_METER_DANGER_FRACTION = 0.8; -// Wide resume gap after a compact that stayed over the high watermark: the -// governor does not re-arm on growth alone until usage climbs this fraction -// of the window past the post-compact measurement. Anchored to the meter -// bands — danger (0.8) minus threshold (0.6) — so a still-over session must -// climb a full warning band before another fold instead of looping a compact -// on every small growth step. Re-arming sooner requires fold evidence: usage -// back at or under the threshold (isAtOrUnderCompactThreshold). +// Wide resume gap after a compact: the governor does not re-arm on growth +// alone until usage climbs this fraction of the window past the post-compact +// measurement. Anchored to the meter bands — danger (0.8) minus threshold +// (0.6) — so a session that folded only marginally under the high watermark +// (or stayed over it) must climb a full warning band before another fold +// instead of looping a compact on every small growth step. Under-threshold +// usage restores consecutive/overflow/non-converged rails; it does not skip +// this gap. export const COMPACTION_WIDE_RESUME_FRACTION = 0.2; export type ContextMeterBand = "quiet" | "warning" | "danger"; @@ -191,8 +192,8 @@ export function compactionThresholdFor(model: string | undefined): number { return Math.floor(window * COMPACTION_WINDOW_FRACTION); } -/** Tokens of growth past the last post-compact measurement before a - * still-over session may re-arm. */ +/** Tokens of growth past the last post-compact measurement before the + * latched proactive path may re-arm. */ export function compactionWideResumeDeltaFor( model: string | undefined, ): number { @@ -201,9 +202,10 @@ export function compactionWideResumeDeltaFor( return Math.floor(window * COMPACTION_WIDE_RESUME_FRACTION); } -/** Low-watermark clear for the post-compact latch: usage back at or under the - * compaction threshold is fold evidence — the summarizing fold got under — so - * the governor re-arms on the next crossing with no gap required. */ +/** Fold evidence for restoring consecutive/overflow/non-converged rails: + * usage back at or under the compaction threshold means the summarizing fold + * got under. It does not drop the growth latch — re-arming still requires + * hasWideResumeGap past the post-compact snapshot. */ export function isAtOrUnderCompactThreshold( contextTokens: number, model: string | undefined, @@ -211,12 +213,13 @@ export function isAtOrUnderCompactThreshold( return contextTokens <= compactionThresholdFor(model); } -/** Shared re-arm rule for a session latched above the threshold after a - * compact (automatic, operator, or overflow recovery): growth alone never - * re-arms — only a wide resume gap past the post-compact measurement does. - * The proactive threshold path routes through this single predicate; the - * overflow path shares the reset rule (under-threshold folds restore the - * recovery budget) but fires on overflow errors regardless of this latch. */ +/** Shared re-arm rule for a session latched after a compact (automatic, + * operator, or overflow recovery), whether the fold landed under or over + * the threshold: growth alone never re-arms — only a wide resume gap past + * the post-compact measurement does. The proactive threshold path routes + * through this single predicate; the overflow path shares the reset rule + * (under-threshold folds restore the recovery budget) but fires on overflow + * errors regardless of this latch. */ export function hasWideResumeGap( postCompactTokens: number, contextTokens: number,