Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ The agent maintains an optional **`manage_tasks`** list (create/update via the h

#### Context compaction (the compaction governor)

When a cycle's input tokens cross a threshold, the director compacts the inference-facing history (the full run is always retained in the context store). The threshold is **model-aware** — roughly 60% of the active model's real context window — so small-window models compact early enough to avoid provider context-overflow while large-window models do not compact prematurely. The compacted prefix is **append-only across passes**: the existing compacted user turn stays byte-identical; new folds become later summary turns with a harness-inserted assistant spacer (identified by reserved `model: "harness"`, plus a visible sentinel; persisted `[compaction]` tokens without a producer still freeze) between them so the prompt head can remain in the provider KV cache. Model-emitted copies of the spacer are not frozen. Spacer-only model replies are incomplete: ChatDirector nudges, then falls through loop-protection, workflow-idle, and open-task rails rather than empty-settling with work still open. The live tail drops `thinking` and `redacted_thinking` blocks so they are not billed against occupancy; full thinking stays in the archived transcript. The governor covers five cases:
When a cycle's input tokens cross a threshold, the director compacts the inference-facing history (the full run is always retained in the context store). The threshold is **model-aware** — roughly 60% of the active model's real context window — so small-window models compact early enough to avoid provider context-overflow while large-window models do not compact prematurely. The compacted prefix is **append-only across passes**: the existing compacted user turn stays byte-identical; new folds become later summary turns with a harness-inserted assistant spacer (identified by reserved `model: "harness"`, plus a visible sentinel; persisted `[compaction]` tokens without a producer still freeze) between them so the prompt head can remain in the provider KV cache. Model-emitted copies of the spacer are not frozen. Spacer-only model replies are incomplete: ChatDirector nudges, then falls through loop-protection, workflow-idle, and open-task rails rather than empty-settling with work still open. The live tail drops `thinking` and `redacted_thinking` blocks except on the last live assistant that still has a `tool_call`; full thinking stays in the archived transcript. The governor covers five cases:

- **Threshold at a tool pause** — Once over threshold, the follow-up `infer` after a tool batch is swapped for a `compact` cycle, and inference resumes via a host continuation message. After every compact (threshold, operator, overflow), the governor snapshots post-compact usage (`tokensAtLastCompact`) and keeps that growth latch. It does not re-arm the proactive path on growth alone; re-arm requires a wide resume gap (~20% of the window, a full warning band) past that snapshot. Usage back at or under the 60% threshold restores the consecutive-compact cap and the overflow-recovery budget; it does not drop the growth latch. A fold that landed well under (snapshot + 20% already past the 60% trigger) still re-arms at the next 60% crossing because the gap predicate is already satisfied. A fold that only just got under (a marginal fold, not a deep one) does not sawtooth on the next 60% crossing; it waits until post-compact + 20%. A consecutive-compact cap holds across tool-call occupancy as a second rail.
- **Idle (end-of-turn)** — An interactive turn can end with a reply and then sit idle with no tool batch to intercept; the governor requests a continuation at that pause and compacts when it arrives. An operator message that races the continuation still compacts first, then re-enters inference to answer it.
Expand Down
295 changes: 284 additions & 11 deletions src/context-compactor.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1187,7 +1187,7 @@ describe("CL-9489 zero-verbatim fold", () => {
);
});

test("a newest-pair thinking block is dropped from the live tail", async () => {
test("the last live tool-use assistant keeps thinking unmodified", async () => {
const thinking = "t".repeat(160_000);
const filler = "x".repeat(1_500);
const turns: ConversationTurn[] = [];
Expand Down Expand Up @@ -1241,17 +1241,25 @@ describe("CL-9489 zero-verbatim fold", () => {
"Folded dummy pairs. Next: resume the newest tool.",
}).apply(turns, mockStrategyCtx);

const liveTypes = result.output.flatMap((t) =>
t.content.map((b) => b.type),
);
expect(liveTypes).not.toContain("thinking");
expect(liveTypes).not.toContain("redacted_thinking");
expect(estimateContextTokens(result.output)).toBeLessThan(12_800);
expect(
result.output.some((t) =>
const lastToolUse = result.output.find(
(t) =>
t.role === "assistant" &&
t.content.some((b) => b.type === "tool_call" && b.id === "c-newest"),
),
).toBe(true);
);
expect(lastToolUse).toBeDefined();
expect(lastToolUse?.content).toEqual([
{
type: "thinking",
thinking,
signature: "sig-thinking-1",
},
{
type: "tool_call",
id: "c-newest",
name: "read_file",
arguments: { path: "src/app.ts" },
},
]);
expect(
result.output.some((t) =>
t.content.some(
Expand All @@ -1264,6 +1272,271 @@ describe("CL-9489 zero-verbatim fold", () => {
);
});

test("older live-tail tool-use thinking is dropped when a newer last assistant has a tool call", async () => {
const olderThinking = "older-thinking-block";
const newerThinking = "newer-thinking-block";
const filler = "x".repeat(1_500);
const turns: ConversationTurn[] = [];
for (let i = 0; i < 6; i++) {
turns.push(
makeTurn({
role: "user",
content: [{ type: "text", text: `dummy ask ${i} ${filler}` }],
}),
);
turns.push(
makeTurn({
role: "assistant",
content: [{ type: "text", text: `dummy reply ${i} ${filler}` }],
}),
);
}
turns.push(
makeTurn({
role: "assistant",
content: [
{
type: "thinking",
thinking: olderThinking,
signature: "sig-older",
},
{
type: "tool_call",
id: "c-older",
name: "read_file",
arguments: { path: "src/old.ts" },
},
],
}),
);
turns.push(
makeTurn({
role: "user",
content: [
{
type: "tool_result",
callId: "c-older",
content: [{ type: "text", text: "export const old = 1;\n" }],
},
],
}),
);
turns.push(
makeTurn({
role: "assistant",
content: [
{
type: "thinking",
thinking: newerThinking,
signature: "sig-newer",
},
{
type: "tool_call",
id: "c-newer",
name: "read_file",
arguments: { path: "src/new.ts" },
},
],
}),
);
turns.push(
makeTurn({
role: "user",
content: [
{
type: "tool_result",
callId: "c-newer",
content: [{ type: "text", text: "export const neu = 1;\n" }],
},
],
}),
);

const result = await createPruningCompactor({
summarize: async () =>
"Folded dummy pairs. Next: resume the newest tool.",
}).apply(turns, mockStrategyCtx);

const older = result.output.find(
(t) =>
t.role === "assistant" &&
t.content.some((b) => b.type === "tool_call" && b.id === "c-older"),
);
const newer = result.output.find(
(t) =>
t.role === "assistant" &&
t.content.some((b) => b.type === "tool_call" && b.id === "c-newer"),
);
expect(older).toBeDefined();
expect(newer).toBeDefined();
expect(older?.content.some((b) => b.type === "thinking")).toBe(false);
expect(older?.content).toEqual([
{
type: "tool_call",
id: "c-older",
name: "read_file",
arguments: { path: "src/old.ts" },
},
]);
expect(newer?.content).toEqual([
{
type: "thinking",
thinking: newerThinking,
signature: "sig-newer",
},
{
type: "tool_call",
id: "c-newer",
name: "read_file",
arguments: { path: "src/new.ts" },
},
]);
});

test("a text-only last assistant drops all live-tail thinking", async () => {
const thinking = "t".repeat(20_000);
const filler = "x".repeat(1_500);
const turns: ConversationTurn[] = [];
for (let i = 0; i < 6; i++) {
turns.push(
makeTurn({
role: "user",
content: [{ type: "text", text: `dummy ask ${i} ${filler}` }],
}),
);
turns.push(
makeTurn({
role: "assistant",
content: [{ type: "text", text: `dummy reply ${i} ${filler}` }],
}),
);
}
turns.push(
makeTurn({
role: "assistant",
content: [
{
type: "thinking",
thinking,
signature: "sig-text-only",
},
{ type: "text", text: "here is the answer" },
],
}),
);

const result = await createPruningCompactor({
summarize: async () =>
"Folded dummy pairs. Next: resume after a text reply.",
}).apply(turns, mockStrategyCtx);

const liveTypes = result.output.flatMap((t) =>
t.content.map((b) => b.type),
);
expect(liveTypes).not.toContain("thinking");
expect(liveTypes).not.toContain("redacted_thinking");
expect(
result.output.some(
(t) =>
t.role === "assistant" &&
t.content.some(
(b) => b.type === "text" && b.text === "here is the answer",
),
),
).toBe(true);
});

test("earlier live tool-use thinking is dropped when the last assistant is text-only", async () => {
const toolThinking = "tool-use-thinking-block";
const lastThinking = "text-only-thinking-block";
const filler = "x".repeat(1_500);
const turns: ConversationTurn[] = [];
for (let i = 0; i < 6; i++) {
turns.push(
makeTurn({
role: "user",
content: [{ type: "text", text: `dummy ask ${i} ${filler}` }],
}),
);
turns.push(
makeTurn({
role: "assistant",
content: [{ type: "text", text: `dummy reply ${i} ${filler}` }],
}),
);
}
turns.push(
makeTurn({
role: "assistant",
content: [
{
type: "thinking",
thinking: toolThinking,
signature: "sig-tool-use",
},
{
type: "tool_call",
id: "c-earlier",
name: "read_file",
arguments: { path: "src/app.ts" },
},
],
}),
);
turns.push(
makeTurn({
role: "user",
content: [
{
type: "tool_result",
callId: "c-earlier",
content: [{ type: "text", text: "export const app = 1;\n" }],
},
],
}),
);
turns.push(
makeTurn({
role: "assistant",
content: [
{
type: "thinking",
thinking: lastThinking,
signature: "sig-text-only",
},
{ type: "text", text: "here is the answer" },
],
}),
);

const result = await createPruningCompactor({
summarize: async () =>
"Folded dummy pairs. Next: resume after a text reply.",
}).apply(turns, mockStrategyCtx);

const liveTypes = result.output.flatMap((t) =>
t.content.map((b) => b.type),
);
expect(liveTypes).not.toContain("thinking");
expect(liveTypes).not.toContain("redacted_thinking");
expect(
result.output.some(
(t) =>
t.role === "assistant" &&
t.content.some((b) => b.type === "tool_call" && b.id === "c-earlier"),
),
).toBe(true);
expect(
result.output.some(
(t) =>
t.role === "assistant" &&
t.content.some(
(b) => b.type === "text" && b.text === "here is the answer",
),
),
).toBe(true);
});

test("a thinking-only live turn is replaced with THINKING_ONLY_OMITTED", async () => {
const thinking = "t".repeat(20_000);
const filler = "x".repeat(1_500);
Expand Down
Loading
Loading