fix(storyboard): align gradeRequestSigning/gradeOneVector transport defaults to 'mcp' - #2559
Conversation
Both `gradeRequestSigning` and `gradeOneVector` constructed `buildOpts` with `transport: options.transport ?? 'raw'`, bypassing the `'mcp'` default already established in `probe-dispatch.ts` via `resolveVectorTransport`. A caller using the standalone grader directly against an MCP agent without an explicit `transport` option would get raw REST-target replay, routing 404 before any verifier could run. Align both call sites to `?? 'mcp'` and update the `GradeOptions.transport` JSDoc to document `'mcp'` as the default (adcp#6548 follow-up to PR adcontextprotocol#2540). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4B1ycLynivGzCS8faZ2QS
![aao-secretariat[bot]](https://avatars.githubusercontent.com/u/221690821?s=60&v=4){kind=small}
There was a problem hiding this comment.
Ladon verdict: Request changes
Request changes — 1 blocking finding.
The transport-default flip from 'raw' to 'mcp' in gradeRequestSigning/gradeOneVector is correct and well-motivated, aligning direct API callers with resolveVectorTransport's existing 'mcp' storyboard default.
Blocking findings
.changeset— Missing changeset for a published-API default flip undersrc/lib/**. Per the repo's mandatory changeset-vs-wire-impact rule, a missing.changeset/*.mdon asrc/lib/**change (behavior-affecting default flip) is a high finding and must ship with the PR.
Medium findings
bin/adcp-grade.js:481— CLI--helptext and transport-mismatch hint still assume the old 'raw' default and now misdirect operators grading REST agents. Should be updated to match the new default.
Add the changeset (and fix the stale CLI help text) to unblock.
Blocking findings
- [
- "
- \
- .
- c
- h
- a
- n
- g
- e
- s
- e
- t
- —
- M
- i
- s
- s
- i
- n
- g
- c
- h
- a
- n
- g
- e
- s
- e
- t
- f
- o
- r
- p
- u
- b
- l
- i
- s
- h
- e
- d
-
- A
- P
- I
- d
- e
- f
- a
- u
- l
- t
- f
- l
- i
- p
- u
- n
- d
- e
- r
- s
- r
- c
- /
- l
- i
- b
- /
-
-
- (
- r
- e
- p
- o
-
- m
- a
- n
- d
- a
- t
- o
- r
- y
- h
- i
- g
- h
- )
- "
- ]
Medium findings
- bin/adcp-grade.js:481 — CLI --help text and transport-mismatch hint stale after default flip
…ort default; add changeset
|
Both findings addressed in 556668d:
🤖 Generated with Claude Code |
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — subsequent pass, both prior blocking findings resolved.
This is a follow-up to a prior request-changes. Both previously flagged findings are now addressed in the delta:
- [high] Missing changeset — resolved:
.changeset/grader-standalone-transport-default.mdis now present. - [medium] Stale CLI --help / transport-mismatch hint — resolved: help text and
detectTransportMismatchdocstring updated for the newmcpdefault.
Current-run findings: none. The transport-default flip (transport !== 'mcp' → transport === 'raw') is consistently applied across the CLI parse path and grader defaults (options.transport ?? 'mcp' at grader.ts:73,82). The patch changeset type is defensible: the affected grade functions are not on any package.json export subpath, and a default-value flip is a behavior change, not a shape break — no changeset understatement.
high_risk is true (bin/adcp-grade.js modified), but per the decision rules this is a heuristic; the reviewer found no medium-or-higher concerns on the modified file, so the modification is presumed safe. No no-auto-approve team match; gated_paths is false so row 2 does not apply. With zero findings and no escalation trigger, this falls through to row 9.
2 participants
Follow-up to #2540: @bokelley's commit aligning the standalone grader's
buildOptstransport defaults (?? 'raw'→?? 'mcp') was cherry-picked onto the PR branch after the PR had already merged, so it never landed on main — the published 13.0.0 hasresolveVectorTransportdefaulting to'mcp'(storyboard path, correct) butgradeRequestSigning/gradeOneVectorstill default'raw'. This is his commit (55e16c8) cherry-picked onto current main, unchanged.Direct
gradeRequestSigningcallers against MCP agents currently get the raw REST replay (every vector 404s) unless they passtransport: 'mcp'explicitly — we hit exactly this running the vector batch against our production seller.🤖 Generated with Claude Code