fix(storyboard): default signed_requests vector transport to 'mcp' (adcp#6548)

[fgranata]

Fixes adcontextprotocol/adcp#6548 — the signed_requests storyboard grades all-red (39/39 routing 404s) against any MCP-transport agent, because the vector dispatch defaults to transport: 'raw' and replays the fixtures' recorded REST-binding targets (/adcp/create_media_buy, raw task body) verbatim. MCP agents serve AdCP tasks inside tools/call on a single endpoint, so the per-task paths 404 at the router before the agent's verifier is ever reached.

The fix is a default, not new machinery: the builder's 'mcp' mode (tools/call envelope, re-signed against the actual target) already exists and ships in the published SDK — nothing selected it. This PR defaults the dispatch to 'mcp' via a small exported resolver; 'raw' remains an explicit opt-in (request_signing.transport: 'raw') for REST-binding agents. The MCP-ungradable canonicalization-edge vectors (005–008) already have their skip handling in the grader, unchanged by this PR.

Verified live against an MCP seller (bidmachine, request_signing.supported: true, verifier in production use by a signing buyer): with this change the vectors go from 39/39 404 to reaching the verification layer (positive vectors now surface the agent's auth semantics instead of a routing dead-end — the remaining signature-as-authentication question is agent-side and out of scope here).

Test: test/lib/request-signing-vector-transport.test.js (node:test, matches the suite convention) — default 'mcp', explicit 'raw' and 'mcp' respected. tsc --noEmit clean; targeted test green against the built dist.

[aao-secretariat]

Ladon verdict: Request changes

Request changes — 1 blocking finding.

The fix itself is correct and well-scoped: the storyboard-runner request-signing dispatch now defaults vector transport to 'mcp' via a new exported resolver, overriding the grader's 'raw' default. Since the runner reaches agents only via tools/call, raw REST replay would 404 MCP agents by construction (adcp#6548), so the default flip is the right shape and 'raw' remains an explicit opt-in. The change is localized to the single caller and the direct grader CLI path is untouched.

Blocking findings

• src/lib/testing/storyboard/request-signing/probe-dispatch.ts:67 — Missing changeset for a published-harness default flip. Per the mandatory changeset-vs-wire-impact rule, diffs touching src/lib/** require a .changeset/*.md; its absence is a high finding.

Add a changeset describing the transport-default behavior change and this is good to merge.

Blocking findings

• src/lib/testing/storyboard/request-signing/probe-dispatch.ts:67 — Missing changeset for published-harness default flip

[fgranata]

Live-validation update, peeling the layers one at a time against an MCP seller with signature verification + signature-as-auth deployed:

1. This PR's transport default works: vectors went from 39/39 routing 404s to reaching the seller's verification layer.
2. Signature verification over the adapted request works: after the seller honored X-Forwarded-Proto in its base reconstruction (TLS terminated at its load balancer), positive vectors verify cryptographically — confirmed server-side (accepted).
3. The last layer is MCP session semantics: the mcp mode POSTs the signed tools/call session-less, and a stock streamable-HTTP server answers 400 Missing session ID (current run: positives all 400, no longer 401/404).

Proposed follow-up (happy to implement here or in a stacked PR): in mcp transport mode, the prober should first initialize an MCP session using the agent's configured auth, then attach mcp-session-id to each vector request after signing — the session header is not a covered component, so signatures remain valid, and negative vectors still hit the verifier before session handling (the seller answers them with the Signature error challenge as specced). Alternative is requiring sellers to accept session-less tools/call (stateless streamable-HTTP), but that couples the storyboard to a server deployment choice rather than the protocol.

[bokelley]

Confirmed — the proposed session-init approach is exactly right, and the implementation is ready to cherry-pick.

Why the approach is sound

Mcp-Session-Id is never a covered component under RFC 9421 (it isn't in MANDATORY_COMPONENTS and the signer doesn't include it in Signature-Input). Appending it after the signing step leaves the signature bytes unchanged. The negative vectors still reach the verifier because the verifier sits as HTTP middleware ahead of MCP session dispatch — it fires before the server looks at the session header, so vectors without valid signatures still get the expected 401 Signature error="…" challenge.

What's implemented

Branch: claude/pr-2540-mcp-session-init (based on this PR's head commit)

Three-layer change:

1. probe.ts — ProbeOptions.mcpSessionId?: string. When set, probeSignedRequest spreads it as Mcp-Session-Id into the outgoing headers after copying signed.headers. New initializeMcpSession(mcpUrl, opts) export performs the initialize handshake and returns { sessionId } (or { sessionId: undefined } for stateless servers, { sessionId: undefined, error } on network failure).

2. grader.ts — GradeOptions.mcpSessionId?: string. Both gradeRequestSigning (batch path) and gradeOneVector (per-vector storyboard path) auto-initialize a session when transport === 'mcp' and mcpSessionId is undefined. The batch path initializes once before all 39 vectors; gradeOneVector initializes per call. Pass a pre-acquired ID to avoid the per-call overhead in the storyboard runner; pass '' to opt out for stateless servers.

3. types.ts — StoryboardRunOptions.request_signing.mcpSessionId?: string so the storyboard caller can pre-initialize once at storyboard start and thread the ID through all steps.

To merge into this PR

git fetch origin claude/pr-2540-mcp-session-init
git cherry-pick origin/claude/pr-2540-mcp-session-init

Or if you'd prefer to squash:

git fetch origin claude/pr-2540-mcp-session-init
git diff HEAD origin/claude/pr-2540-mcp-session-init | git apply

The changeset file (.changeset/signed-requests-mcp-session-init.md) is included and can be combined with the existing changeset for this PR before merge.

---

Generated by Claude Code

[bokelley]

Reviewed the implementation on claude/pr-2540-mcp-session-init — it's correct and ready to merge.

What I verified:

• probe.ts: Mcp-Session-Id is injected into the outgoing outHeaders object after signed.headers is copied, so it is never a covered component. The initializeMcpSession helper wraps probeSignedRequest with an unsigned initialize body (no SignedHttpRequest signing step), which is correct — the session handshake itself doesn't need to be signed.
• grader.ts: Both gradeRequestSigning (batch path, initializes once before the 39-vector loop) and gradeOneVector (per-vector storyboard path, initializes each call) check transport === 'mcp' && mcpSessionId === undefined before auto-initializing. The '' opt-out path is correctly wired. The mcpSessionId is forwarded into probeOpts uniformly.
• probe-dispatch.ts: mcpSessionId: rsOpts.mcpSessionId is threaded through to gradeOneVector, so the storyboard runner can pre-initialize once and reuse the session ID across all vector steps.
• Changeset: signed-requests-mcp-session-init.md marks @adcp/sdk: patch and accurately describes the behavior.

Blocker: The PR head is in fgranata/adcp-client (a fork), so I can't push fixup commits to it directly. @fgranata — the cherry-pick commands @bokelley provided above are the path forward:

git fetch origin claude/pr-2540-mcp-session-init
git cherry-pick origin/claude/pr-2540-mcp-session-init

The branch has two commits to pick (the session-init fix and the prettier formatting pass). The included changeset (signed-requests-mcp-session-init.md) can be kept alongside the existing one; both will be consumed together when the Release PR is generated.

---

Generated by Claude Code

[aao-secretariat]

Ladon verdict: Comment (human reviewer recommended)

Comment — 3 medium findings, no blocking issues.

The prior blocking finding (missing changeset for the transport default flip at probe-dispatch.ts:67) is resolved — two changesets are now present. This is a net improvement over the prior request-changes run.

No critical/high findings remain. Three medium findings surfaced this run, which crosses the row 8 threshold (≥3 medium → comment). A human should look before merge.

Medium findings

• src/lib/testing/storyboard/request-signing/grader.ts:222 — Auto-init drops initializeMcpSession error; a failed handshake grades the whole batch session-less (helper doc warns this cascades to a false all-fail 400).
• src/lib/testing/storyboard/request-signing/grader.ts:440 — gradeOneVector drops initializeMcpSession error (same warn-only/dropped-error pattern as the batch path).
• test/lib/request-signing-mcp-session.test.js:36 — New tests assert only export existence, not the post-signing-injection invariant they claim to cover (missing test on a new branch).

Note: review_decision is CHANGES_REQUESTED, but gated_paths is false and there are no blocking findings, so this does not force escalate — row 8 (three mediums) governs.

Medium findings

• src/lib/testing/storyboard/request-signing/grader.ts:222 — Auto-init drops initializeMcpSession error; failed handshake grades whole batch session-less
• src/lib/testing/storyboard/request-signing/grader.ts:440 — gradeOneVector drops initializeMcpSession error (same as batch path)
• test/lib/request-signing-mcp-session.test.js:36 — New tests assert only export existence, not the post-signing injection invariant

[fgranata]

Cherry-picked your session-init branch onto this PR (fast-forward — thanks for pre-building it) and added one more validated fix on top: the initialize handshake now authenticates with the runner's options.auth. Without it, an unauthenticated initialize 401s on any auth-required agent, initializeMcpSession reads that as a stateless server, and every vector goes session-less into 400 Missing session ID (observed live). The signed vectors themselves remain bearer-less by design — the auth rides only on the handshake.

Live-validation state against the MCP seller: vectors now traverse routing → signature verification → signature-as-auth → session handling. The last observed blocker was the seller's own transport rate limiter mis-bucketing bearer-less signed traffic (the rate-abuse vector's 100+ requests poisoned the per-IP window) — fixed seller-side; final full-batch verification after their deploy.

[bokelley]

Thanks for the update @fgranata — the initialize auth fix makes sense; an unauthenticated handshake 401ing into a stateless-fallback chain explains the 400 Missing session ID spiral. Good catch on the rate-limiter mis-bucketing the abuse vector's bearer-less traffic too. Noted that final full-batch verification is pending their deploy. No action needed on my end — standing by.

---

Generated by Claude Code

[aao-secretariat]

Ladon verdict: Approve

Approve — 2 medium findings, no blocking issues, no escalation trigger.

This delta adds an initializeHeaders path so the auto-initialize MCP handshake can authenticate. The wiring is correctly scoped: the bearer header enters only the handshake, never the signed vectors, preserving the signature-is-auth invariant. Changesets are present for the src/lib changes.

Medium findings

• src/lib/testing/storyboard/request-signing/grader.ts:234 — Auto-init drops initializeMcpSession error on batch path; failed handshake grades session-less
• src/lib/testing/storyboard/request-signing/grader.ts:456 — gradeOneVector drops initializeMcpSession error (same as batch path)

Decision-table walkthrough: no critical/high findings (row 1 no). gated_paths is false, so row 2 does not fire despite review_decision=CHANGES_REQUESTED. high_risk is false (rows 3, 5 no). No data-loss/schema/infra medium categories (row 4 no). Prior decision was comment, not escalate (row 6 no). No no-auto-approve team match (row 7 no). Only 2 medium findings, fewer than 3 (row 8 no). Falls through to row 9 → approve.

The prior run's test-file finding is outside this delta and no longer applies; the two init.error-drop findings persist as Medium but are non-blocking.

Medium findings

• src/lib/testing/storyboard/request-signing/grader.ts:234 — Auto-init drops initializeMcpSession error on batch path; failed handshake grades session-less
• src/lib/testing/storyboard/request-signing/grader.ts:456 — gradeOneVector drops initializeMcpSession error (same as batch path)

[fgranata]

End-to-end validation of this PR (transport default + session-init + initialize-auth threading) against a live MCP seller (adcp.bidmachine.io), full vector batch via gradeRequestSigning with transport: 'mcp':

30 passed / 5 failed / 5 skipped (was 1/39 before the transport fix — every vector 404'd on the REST-binding replay).

• All 8 gradeable positive vectors pass: signature verified, session attached, tool executed, 2xx with the response read cleanly.
• Negatives return the exact expected WWW-Authenticate: Signature error="<code>" codes, including the stateful ones: replayed-nonce (10/10 pairs rejected), key-revoked, rate-abuse at the contract's grading cap.
• The 5 remaining fails are all capability-profile vectors, not transport issues: 001/027/028 assert required_for enforcement (this agent declares required_for: [] — shadow posture), 007/018 assert covers_content_digest: required/forbidden (agent declares either). Passing agentCapability correctly auto-skips them.

Three small notes from running the full batch, none blocking this PR:

1. gradeRequestSigning has its own transport: options.transport ?? 'raw' default (grader.ts), separate from resolveVectorTransport — a caller using the standalone grader against an MCP agent still gets the raw replay unless they pass transport: 'mcp' explicitly. Worth aligning with the dispatch default in a follow-up.
2. The replay-window failure diagnostic is a canned string: it reports "all N probe pairs were accepted (got 200 on second submission every time)" even when http_status in the same result is 401 — the actual condition was "rejected with a different error code than request_signature_replayed". Cosmetic, but it sent us down a false path.
3. Every vector fixture (positives included) pins verifier_capability.required_for: ["create_media_buy"], so an agent that honestly declares required_for: [] (verify-when-present posture, not yet enforcing) capability-skips the entire storyboard under agentCapability preflight — while a preflight-less run fails it on the enforcement vectors alone. A posture-aware split (only the request_signature_required vectors asserting required_for) would let shadow-posture verifiers grade the other 30+ vectors they do implement. Happy to file that separately if useful.

🤖 Generated with Claude Code

[bokelley]

30/5/5 against a live bidmachine instance is excellent confirmation — the core transport fix is working as intended for the storyboard-runner path (from 1/39 is a big jump). Thank you for running the full vector batch.

Three notes acknowledged:

Note 1 (grader.ts buildOpts default) — worth folding before merge. Confirmed: both gradeRequestSigning and gradeOneVector still carry transport: options.transport ?? 'raw' in their buildOpts, separate from the resolveVectorTransport fix in probe-dispatch.ts. A caller using the standalone grader against an MCP agent gets raw replay unless they pass transport: 'mcp' explicitly — which undercuts the PR's stated goal of making 'mcp' the default. The fix is two one-line changes (?? 'raw' → ?? 'mcp' in the buildOpts of each function in grader.ts). Since the PR's changeset description says the dispatch defaults to 'mcp', aligning grader.ts here rather than in a follow-up would keep the behavior consistent with the stated contract.

Note 2 (replay-window diagnostic string): Cosmetic but genuinely confusing — "all N probe pairs were accepted" when HTTP status is 401 is a misleading path. Worth a small follow-up issue.

Note 3 (posture-aware split for shadow-posture verifiers): Agreed on the root cause — required_for: ["create_media_buy"] pinned on every fixture means shadow-posture agents capability-skip the full storyboard under agentCapability preflight while failing only the enforcement vectors without it. Filing that separately as a storyboard fixture/preflight issue would be useful; the split between request_signature_required vectors and the rest is a clean separation.

---

Generated by Claude Code

[bokelley]

Addressing Note 1 from @bokelley's review. Confirmed: both buildOpts assignments in grader.ts still carry ?? 'raw', so a caller using gradeRequestSigning or gradeOneVector directly without an explicit transport option builds raw REST requests despite probeOpts and probe-dispatch.ts already defaulting to 'mcp'. The fix is two one-line changes:

gradeRequestSigning (~line 247):

- const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'raw' };
+ const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'mcp' };

gradeOneVector (~line 468):

- const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'raw' };
+ const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'mcp' };

Also updated the transport JSDoc in GradeOptions to reflect 'mcp' as the default (was still written as 'raw' (default)).

Notes 2 and 3 correctly scoped as separate follow-ups — not touching them here.

---

Generated by Claude Code

[bokelley]

Confirmed both ?? 'raw' instances are still present in the current head (9424efe). The PR is from a fork so I can't push directly — @fgranata, here are the three one-line changes to apply:

1. gradeRequestSigning (line ~247):

-  const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'raw' };
+  const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'mcp' };

2. gradeOneVector (line ~468):

-  const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'raw' };
+  const buildOpts: BuildOptions = { baseUrl: agentUrl, transport: options.transport ?? 'mcp' };

3. GradeOptions.transport JSDoc (line ~107) — update the default annotation:

-   * Transport shape the agent speaks. `'raw'` (default) POSTs per-operation
+   * Transport shape the agent speaks. `'mcp'` (default) wraps each

(the rest of the JSDoc already reads correctly — just the first sentence's default label needs updating)

Notes 2 and 3 from @bokelley's review are correctly scoped as separate follow-ups — not touching them here.

---

Generated by Claude Code

[bokelley]

@fgranata — applied the three changes to claude/pr-2540-grader-transport-default (one commit on top of this PR's head, 9424efe):

git fetch upstream claude/pr-2540-grader-transport-default
git cherry-pick 55e16c89d7d07274a5c82718e9a8ad9587ea14fb

Or if upstream isn't configured, replace it with the adcontextprotocol/adcp-client remote name you use.

What the commit does (diff against 9424efe):

• gradeRequestSigning buildOpts: ?? 'raw' → ?? 'mcp'
• gradeOneVector buildOpts: ?? 'raw' → ?? 'mcp'
• GradeOptions.transport JSDoc: removes `'raw'` (default) POSTs per-operation AdCP endpoints matching the vectors' URL shape. sentence, updates opening to `'mcp'` (default) wraps each

No other files touched. The only remaining 'raw' in the file is the type declaration transport?: 'raw' | 'mcp', which is correct.

---

Generated by Claude Code

[aao-secretariat]

Ladon verdict: Approve

Approve.

This is a subsequent review of a PR whose prior decision was approve. Both prior Medium findings have been resolved:

• grader.ts:234 (gradeRequestSigning) — now throws on init.error instead of grading session-less.
• grader.ts:459 (gradeOneVector) — same fix applied.

The new probe.ts helper attachMcpSessionHeader is a clean, non-mutating extraction verified by a byte-for-byte test, and initializeMcpSession now fails closed on non-2xx handshake status. Changeset is present, matching the src/lib/** change requirement. No new findings.

Decision-table walk: no critical/high findings (row 1 n/a). gated_paths is false (row 2 n/a). high_risk false, no deletions (rows 3–5 n/a). Prior decision was approve, not escalate (row 6 n/a). No no-auto-approve team match (row 7 n/a). Zero medium findings (row 8 n/a). Falls through to row 9 → approve.

Note: review_decision is REVIEW_REQUIRED, but since gated_paths is false the hard approval gate does not apply.

[fgranata]

Cherry-picked 55e16c8 onto the branch (now 57eda3d) — gradeRequestSigning/gradeOneVector buildOpts defaults aligned to 'mcp', JSDoc updated. Build clean, signing test files 6/6. Thanks for pre-building it (and for the init-failure grading commit — picked that up too).

🤖 Generated with Claude Code