feat(runtime): add error details, method-safety mutation, search terms - #171
Conversation
Old contract: usage errors carried only code, message, and hint; agents could not learn which values a flag accepts or which required field was missing without re-reading the catalog. New contract: LatheError gains an optional detail field, filled only by validation call sites via the explicit WithUsageDetail constructor and built exclusively from spec metadata (flag names, declared value sets, required field names). Detail never echoes user-provided values, is sanitized to a single line, and is bounded to 240 runes. Human output prints a Detail: line between Error: and Hint:; JSON/YAML envelopes add detail (omitted when empty). Covered call sites: enum violations (--range accepts: 7, 30), missing required flags/params/body fields (missing required: name), and unsupported output formats. Cobra unknown command and unknown flag paths produce no detail because their causes contain user input; the machine error contract test now asserts detail stays empty there. Reason: agents should correct invalid invocations from the error alone without guessing, while keeping the no-echo leakage boundary intact. Signed-off-by: samzong <samzong.lu@gmail.com>
Old contract: every HTTP failure surfaced only as API request failed plus a status-based hint; the server's declared explanation was discarded and agents could not distinguish a revoked key from a generic 403. New contract: api_error detail is filled from the first available layer. Layer 1: when the response Content-Type is JSON and a string is extractable from a declared field (message; error as a string or error.message; detail; depth <= 2), detail is that message, stripped of control characters and bounded to 240 runes; bodies over 32KB are never parsed. Layer 2: when extraction fails and the command catalog declares a known_errors entry matching the status, detail is its cause; layers never stack. Layer 3: unchanged status plus hint. The error message stays API request failed, HTTPError now records the response Content-Type, and the raw body is never emitted in any layer; the upstream-secret negative cases keep proving non-JSON bodies do not leak. Reason: agents acting on failed API calls need the server's declared cause without lathe leaking arbitrary response bodies. Signed-off-by: samzong <samzong.lu@gmail.com>
Old contract: catalog mutation was read for GET/HEAD, GraphQL templates classified query vs mutation, and every other method surfaced as unknown, forcing agents to treat ordinary POST/PUT/PATCH/DELETE operations as undecidable. New contract: RFC 9110 safe methods (GET, HEAD, OPTIONS, TRACE) are read; every other declared method defaults to write; unknown is reserved for operations with no determinable method (and workflows with no steps). A new overlay field mutation: read|write explicitly overrides every inference, flowing overlay -> CommandSpec.Mutation -> generated literal -> catalog; invalid values are rejected at overlay merge and validation. GraphQL template classification and workflow heaviest-step aggregation are unchanged. SchemaVersion bumps 13 -> 14 and CatalogSchemaVersion bumps 20 -> 21 following the repository convention that a new CommandSpec field and a catalog behavior change require regeneration; consumers that treated unknown as dangerous now see write for the same operations and should handle it identically with more precise information. Reason: agents need an accurate read/write signal instead of a defensive unknown for every REST mutation. Signed-off-by: samzong <samzong.lu@gmail.com>
Old contract: search matched only spec-derived fields (operation id, use, aliases, shortcuts, paths, summary, description, flags); domain vocabulary like spend could not reach usage-summary without editing upstream summaries. New contract: overlay operations accept search_terms: ["..."], carried end to end: overlay parser -> CommandSpec.SearchTerms -> generated literal -> catalog search_terms field -> search index -> generated Skill module reference (Search terms: ... per operation). Search treats each curated term as an identifying synonym weighted like summary text (weight 30), so a single term surfaces the command while exact command-name and operation-id matches still outrank it. SchemaVersion bumps 14 -> 15 and CatalogSchemaVersion bumps 21 -> 22 per the repository convention for new CommandSpec and catalog fields; downstream CLIs must be regenerated to pick up search_terms, recorded in docs/contracts.md. Reason: agents search by user intent vocabulary; curated synonyms close the gap between spec wording and how operators actually ask. Signed-off-by: samzong <samzong.lu@gmail.com>
Old contract: enabling overlay body.flags on a request body containing any nested object property rejected the entire body, so commands like keys create lost every typed flag (including --name) because of one nested limits field. New contract: nested object properties (including arrays of objects) are skipped instead of fatal. Top-level scalar and scalar-array properties still expand into typed flags; skipped fields are recorded as RequestBody.SetOnlyFields, flow into the generated literal, surface in the catalog as body.set_only_fields, and are named in the --set/--set-str flag help so callers know those fields use --set, --set-str, or --file. A body whose properties are all nested objects still fails codegen with an explicit message, and the other rejection paths (maps, oneOf/anyOf/allOf, multipart, GraphQL templates, unsupported root types, flag collisions) are unchanged. Full nested flattening (e.g. --limits-max-budget-usd) is documented as a possible future extension only. SchemaVersion bumps 15 -> 16 and CatalogSchemaVersion bumps 22 -> 23 for the new RequestBody and catalog body fields. Reason: one nested field should not cost a command every typed flag agents rely on for safe body construction. Signed-off-by: samzong <samzong.lu@gmail.com>
Old contract: sanitizeErrorDetail replaced only C0 controls and DEL, so C1 control characters (CSI U+009B, OSC U+009D) and Unicode bidi overrides (U+202E) from a hostile server's declared error message passed through to the human Detail: line and the JSON/YAML envelope, contradicting the documented control-characters-stripped guarantee. New contract: every non-graphic rune (unicode.IsGraphic == false, covering C0, C1, DEL, and format characters including bidi overrides) is replaced with a space before collapsing and bounding the detail. Reason: detail is the one channel that intentionally carries server-influenced text; it must not be able to carry terminal escape or display-spoofing sequences. Signed-off-by: samzong <samzong.lu@gmail.com>
Old contract: after nested object properties were skipped into SetOnlyFields, a schema-required nested field lost local required validation entirely: typed scalar required fields in the same body were checked before the request, while an omitted required nested field was only caught by the server round-trip. New contract: validateRequiredBodyParams also fails with the same missing required: <name> usage detail when a schema-required field listed in SetOnlyFields is absent from the resolved body; providing it via --set, --set-str, or --file passes as before. Reason: required enforcement must be consistent across typed and set-only fields, or agents are back to guessing part of the body shape. Signed-off-by: samzong <samzong.lu@gmail.com>
Merging this PR will degrade performance by 49.98%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | large |
996.3 µs | 2,255.3 µs | -55.83% |
| ❌ | BenchmarkFormatTableInferredColumns |
471.3 µs | 986 µs | -52.2% |
| ❌ | yaml |
2.6 ms | 5.4 ms | -52% |
| ❌ | BenchmarkCatalogJSON |
1.5 ms | 3.1 ms | -51.82% |
| ❌ | large |
1.2 ms | 2.6 ms | -51.41% |
| ❌ | json |
658.3 µs | 1,354.4 µs | -51.4% |
| ❌ | BenchmarkFindCatalogCommand |
4.4 µs | 9 µs | -51.02% |
| ❌ | small |
99.9 µs | 203.8 µs | -50.97% |
| ❌ | large |
808.5 µs | 1,639.7 µs | -50.7% |
| ❌ | miss |
1.9 ms | 3.8 ms | -50.58% |
| ❌ | table |
379.9 µs | 767.5 µs | -50.5% |
| ❌ | hit |
1.9 ms | 3.9 ms | -50.19% |
| ❌ | json-large |
1.7 ms | 3.4 ms | -49.96% |
| ❌ | json-small |
129.6 µs | 255.4 µs | -49.24% |
| ❌ | small |
74.6 µs | 144.9 µs | -48.49% |
| ❌ | BenchmarkParseNormalize |
3.1 ms | 6 ms | -48.08% |
| ❌ | large |
707.2 µs | 1,360.1 µs | -48% |
| ❌ | yaml-large |
8.1 ms | 15.5 ms | -47.9% |
| ❌ | small |
76.7 µs | 143.3 µs | -46.51% |
| ❌ | BenchmarkBuildFlat |
1.1 ms | 2 ms | -45.88% |
| ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing fix/runtime-ux-round2 (edfa5f1) with main (7ec8e13)
Summary
Round 2 of runtime UX contracts for agent-facing generated CLIs, 7 commits:
feat(runtime): usage errors carry an optionaldetailbuilt only from spec metadata (enum sets, required flag/field names, output formats) via an explicit constructor; never echoes user input; cobra unknown command/flag paths stay detail-free.feat(runtime): API errors surface a layereddetail: declared JSON fields (message,error,error.message,detail, depth <= 2, sanitized, <= 240 runes, bodies > 32KB skipped) ->known_errorscause matching the status -> unchanged status + hint. Raw response bodies are never emitted.feat(runtime): catalogmutationfollows RFC 9110 method safety (GET/HEAD/OPTIONS/TRACE =read, other declared methods =write,unknownonly for undeterminable); overlaymutation: read|writeoverrides every inference.feat(overlay):search_termsflow overlay ->CommandSpec-> generated literal -> catalog -> search index (identity synonym at summary weight) -> generated Skill docs.feat(codegen):body.flagsskips nested object properties instead of rejecting the whole body; skipped fields land inbody.set_only_fields,--set/--set-strhelp, and keep local required validation.fix(runtime): detail sanitization strips all non-graphic runes (C1 controls, bidi overrides); required set-only body fields are validated locally.Verification
make check(fmt-check, vet, golangci-lint, tests) — passgo test -race -count=1 ./...— passgo vet ./...— passreplaceto this branch, regenerated vialathe bootstrap;__lathe verify --json20/20 checks ok; verified enum detail output, declared server message +known_errorsfallback against a local mock upstream,mutationvalues,search_termshit, and--name demo --dry-runproducing{"name":"demo"}with nestedlimitsskipped.Compatibility
SchemaVersion13 -> 16 andCatalogSchemaVersion20 -> 23: downstream CLIs must regenerate after bumping (enforced explicitly byAssertSchema).error.detail,search_terms,body.set_only_fields).mutationvalue semantics: operations previously reportedunknownfor REST write methods now reportwrite; consumer handling is unchanged, classification is more precise (documented indocs/contracts.md).docs/contracts.md,docs/cli-usage.md.Checklist
internal/generated/,.cache/, and ad-hocskills/<cli-name>/directories is not committed.