Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions src/claude/inbound.ts
Original file line number Diff line number Diff line change
Expand Up @@ -150,10 +150,17 @@ function blockedSkillCallIds(messages: readonly unknown[], blocked: readonly str

/**
* Claude Code (observed 2026-07-11, real CLI smoke) sends `role:"system"` entries in
* `messages` despite the published API having no system role. Map them to Responses
* instructions text: the native ChatGPT backend rejects system message items in
* `input` ("System messages are not allowed", verified live), so folding into
* `instructions` is the only shape that works on every route.
* `messages` despite the published API having no system role. They are emitted as
* chronological `role:"developer"` input items, which keeps the timeline intact and
* leaves `instructions` owned solely by the top-level Anthropic `system` field.
*
* The original mapping folded them into `instructions` because the native ChatGPT
* backend rejects `role:"system"` items in `input` ("System messages are not allowed",
* verified live). That constraint is real and still respected — but it only rules out
* `system`, not `developer`, which every Responses route accepts. Folding meant each
* mid-conversation reminder mutated the prompt head, invalidating the upstream KV
* prefix and rotating the Desktop `prompt_cache_key` fallback below on every turn
* (#4148).
*/
function systemMessageText(content: unknown): string {
if (typeof content === "string") return content;
Expand Down Expand Up @@ -333,7 +340,12 @@ function translateAnthropicRequest(raw: unknown, cc: OcxClaudeCodeConfig | undef
else if (msg.role === "assistant") assistantMessageToItems(msg.content, input, budget);
else if (msg.role === "system") {
const text = systemMessageText(msg.content);
if (text.length > 0) systemParts.push(text);
// Keep it where the client put it. `developer` is first-class in the Responses
// schema and survives parseRequest as a chronological message, where `system`
// would be re-hoisted back onto the system prompt and defeat the point.
if (text.length > 0) {
input.push({ type: "message", role: "developer", content: [{ type: "input_text", text }] });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid splitting tool calls from their results

When a role:"system" entry occurs between an assistant tool_use and its matching user tool_result, inserting this developer item breaks their adjacency. On the Ollama-native route, buildNativeMessages treats the developer item as a hard boundary and calls flushPending(), which throws because the following result has not been processed yet; the Anthropic and Google adapters similarly synthesize a missing result and later downgrade the real result to an orphan. This request shape worked before because the system entry was folded out of the message timeline, so defer such reminders until after the result batch or make each affected adapter preserve the pending pair across this barrier.

AGENTS.md reference: src/AGENTS.md:L19-L19

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retain cache affinity without a top-level system field

For an accepted request that has in-message system entries but neither top-level system nor metadata.user_id, moving the text here leaves systemParts empty, so the fallback at the later else if (systemParts.length > 0) no longer emits any prompt_cache_key. The same request previously received a stable system-derived key, meaning clients that express all system context through messages now lose cache routing entirely. Track that this request contained system reminders and derive a stable model/tool cohort key that excludes their growing timeline text.

Useful? React with 👍 / 👎.

}
}
else throw new AnthropicRequestError(`unsupported message role: ${String(msg.role)}`);
}
Expand Down
85 changes: 77 additions & 8 deletions tests/claude-integration/claude-inbound.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,23 @@ import { createResponsesPassthroughAdapter } from "../../src/adapters/openai-res
import { withTestTranslatorBudget } from "../helpers/translator-budget";
import type { OcxProviderConfig } from "../../src/types";

// The translator returns an untyped wire body. These aliases name just the fields the
// system-message cases assert on, so the assertions read as a contract instead of a cast.
type TranslatedInputItem = {
type?: string;
role?: string;
content?: Array<{ type?: string; text?: string }>;
};
type TranslatedBody = {
instructions?: string;
prompt_cache_key?: string;
input: TranslatedInputItem[];
};

function translatedBody(raw: Record<string, unknown>): TranslatedBody {
return anthropicToResponsesBody(raw) as TranslatedBody;
}

// Full Claude Code-shaped request: system array, tool cycle, image, thinking, options.
function claudeCodeRequest(): Record<string, unknown> {
return {
Expand Down Expand Up @@ -310,25 +327,77 @@ describe("claude inbound translation", () => {
expect(() => parseRequest(body)).not.toThrow();
});

test("system role messages fold into instructions (real Claude Code sends them; native backend rejects system items)", () => {
const body = anthropicToResponsesBody({
// Claude Code sends role:"system" entries in `messages`. They used to be folded into
// `instructions` alongside the top-level system field; now each one becomes a
// chronological role:"developer" input item, so `instructions` belongs to the
// top-level Anthropic `system` field alone and the prompt head stops moving
// mid-conversation.
test("in-messages system role becomes a chronological developer item, never a system item", () => {
const body = translatedBody({
model: "m", max_tokens: 10,
system: "top-level",
messages: [
{ role: "system", content: "be terse" },
{ role: "system", content: [{ type: "text", text: "block form" }] },
{ role: "user", content: "hi" },
],
}) as any;
expect(body.instructions).toBe("top-level\n\nbe terse\n\nblock form");
// No system message items in input — native ChatGPT backend 400s on them.
expect((body.input as any[]).every(item => item.role !== "system")).toBe(true);
expect(body.input).toHaveLength(1);
expect(body.input[0].role).toBe("user");
});
// Only the top-level system reaches instructions now.
expect(body.instructions).toBe("top-level");
// Still no system message items in input — the native ChatGPT backend 400s on them.
expect(body.input.every(item => item.role !== "system")).toBe(true);
expect(body.input.map(item => item.role)).toEqual(["developer", "developer", "user"]);
expect(body.input[0]?.content).toEqual([{ type: "input_text", text: "be terse" }]);
expect(body.input[1]?.content).toEqual([{ type: "input_text", text: "block form" }]);
expect(() => responsesRequestSchema.parse(body)).not.toThrow();
expect(() => parseRequest(body)).not.toThrow();
});

// #4148: a client that injects a fresh reminder each turn used to rewrite the prompt
// head every time, which invalidates the upstream KV prefix and — with no
// metadata.user_id — rotated the Desktop prompt_cache_key fallback along with it.
test("a mid-conversation system message leaves the cache prefix and cache key alone", () => {
const turn = (messages: unknown[]) =>
translatedBody({ model: "m", max_tokens: 10, system: "S", messages });

const turn1 = turn([
{ role: "user", content: "u1" },
{ role: "system", content: "r1" },
]);
const turn2 = turn([
{ role: "user", content: "u1" },
{ role: "system", content: "r1" },
{ role: "assistant", content: "a1" },
{ role: "user", content: "u2" },
{ role: "system", content: "r2" },
]);

// The prompt head is the whole point: identical across turns, and equal to the
// top-level system field on its own.
expect(turn1.instructions).toBe("S");
expect(turn2.instructions).toBe("S");

expect(turn1.input.map(item => item.role)).toEqual(["user", "developer"]);
expect(turn2.input.map(item => item.role))
.toEqual(["user", "developer", "assistant", "user", "developer"]);
expect(turn2.input.map(item => item.content?.[0]?.text))
.toEqual(["u1", "r1", "a1", "u2", "r2"]);
expect(turn2.input.every(item => item.role !== "system")).toBe(true);

// Turn 1's items are still a prefix of turn 2's, which is what the KV cache matches on.
expect(turn2.input.slice(0, 2)).toEqual(turn1.input);

// No metadata.user_id, so the Desktop cohort fallback applies. It hashes the
// post-translation system text, which no longer absorbs the injected reminders.
expect(turn1.prompt_cache_key).toBeDefined();
expect(turn2.prompt_cache_key).toBe(turn1.prompt_cache_key);

for (const body of [turn1, turn2]) {
expect(() => responsesRequestSchema.parse(body)).not.toThrow();
expect(() => parseRequest(body)).not.toThrow();
}
});

test("tool_result is_error and string content", () => {
const body = anthropicToResponsesBody({
model: "m", max_tokens: 10,
Expand Down
Loading