Skip to content

Name a vendor's error as the vendor's to a Bot running its own loop, as the in-process door does - #569

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:name-vendor-error-on-callback
Sep 15, 2026
Merged

davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:name-vendor-error-on-callback

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

What this changes

A vendor can say no in two ways. It can throw, or it can answer { isError: true } with a sentence, which is how an MCP server refuses a call. The second answer reaches a Bot's model through one of two doors to the same pluginStore.callTool:

  • In process (grantedTools in server/src/plugins/tools.ts), for a Bot running here. Since Answer from Google Drive, as the person asking #97 it hands the model The vendor reported an error: <sentence>. The comment there says why: handing the sentence over as content "cost a diagnosis". Google's "The caller does not have permission" read as a result, and the model told the person it had no access to their Drive.
  • Callback (/api/agent-tools/call in server/src/app.ts), for a Bot running its own loop. It answered with the bare text.

Neither framework Bot adds the wording itself:

  • agent-langgraph passes an isError answer on untouched. tool-answer.test.ts pins that: "a refusal the store made is passed on untouched".
  • agent-langgraph-agui returns body["text"] and nothing else.

So on those Bots, a vendor's refusal still reached the model as an ordinary result. That is the failure the in-process comment describes, through the other door.

The wording moves out of grantedTools into an exported vendorAnswer beside REFUSAL_MARKER, with its comment, and both doors call it. What stays the same:

  • A result that is not an error passes through as written.
  • This deployment's own refusals keep REFUSAL_MARKER. They are thrown as PluginRefusedError and answered in the route's catch, so they never reach this helper.
  • The host-tool branch (deploymentToolCaller) is not touched.

Where it runs

  • New state that outlives a request? None. A pure function on the answer.
  • What happens on the second replica? The same sentence on every replica. The two doors already answered differently on one machine.
  • Anything serialised? No.
  • Anything fanned out to a browser? No.
  • New listener, port, or schedule? No.

Boundary and audit

  • Every acting call still goes through the gateway: resolve, decide, audit, then act. callTool and its audit rows are unchanged; only the text of an answer already on its way to the model changes.
  • New refusals and new failures each write a row. None are added. A vendor's isError answer is recorded by callTool exactly as before.
  • Nothing new is trusted from the client that the server can resolve itself.

Changelog

  • A line in CHANGELOG.md under Unreleased.

This entry sits at the top of Unreleased, which is also where #568 inserts. If one lands first, I'm happy to rebase the other.

Proof

Two tests added to server/tests/agent-callback-token.test.ts, in the route's existing describe. Each builds one fake store and asks both doors about the same call: createApp(...).request("/api/agent-tools/call") with a signed run, and grantedTools(...)[0].execute({}).

  • "a vendor's error answer is named as the vendor's, the way the in-process door names it": the store resolves { text: "The caller does not have permission.", isError: true }.
  • "a vendor's result that is not an error reaches the model as the vendor wrote it": a guard that the helper does not word a success.

On main, the first fails and the guard passes:

error: expect(received).toBe(expected)

Expected: "The vendor reported an error: The caller does not have permission."
Received: "The caller does not have permission."

      at <anonymous> (server/tests/agent-callback-token.test.ts:585:27)
(fail) the tool-call route a callback token guards > a vendor's error answer is named as the vendor's, the way the in-process door names it
 2 pass
 1 fail

With the fix:

  • bun test tests/agent-callback-token.test.ts tests/host-access-callback-route.test.ts: 33 pass, 0 fail. This includes the route's existing tests that a deployment fault is not relayed, and that a refusal and a thrown vendor sentence still are.
  • bun run typecheck (app, server, worker): exit 0.
  • bunx biome check on the three changed files: clean.

Both branches of vendorAnswer, the route's answer line and grantedTools' execute all run in these tests. grantedTools' other tests are Postgres integration tests, which I could not run locally. No test anywhere asserted the bare callback text or the inline prefix.

🤖 Generated with Claude Code

…as the in-process door does

A vendor that says no by answering `{ isError: true }` with a sentence,
the way an MCP server refuses, reaches a Bot's model through one of two
doors to the same `callTool`. The in-process door (`grantedTools`) has
named that sentence as the vendor's since CopilotKit#97, because handing it over
as content cost a diagnosis: Google's "The caller does not have
permission" read as a result, and the model told the person it had no
access to their Drive.

`/api/agent-tools/call`, the door a framework Bot calls back through,
answered with the bare text. Neither framework Bot words it on the way
through: the LangGraph Bot passes an `isError` answer on untouched, and
the Python LangGraph Bot reads only `text`. So on those Bots the model
was still handed the vendor's refusal as an ordinary result.

The wording now lives in one helper, `vendorAnswer`, and both doors use
it. A result that is not an error is unchanged, and so are this
deployment's own refusals, which keep `REFUSAL_MARKER` and never reach
this helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-verified clean; CI green on this sha.

@davidmckayv
davidmckayv merged commit 0ad74c2 into CopilotKit:main Sep 15, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants