From c30ecc05fc889319f4b711c4cf2e97bbd3251e2d Mon Sep 17 00:00:00 2001 From: Manuel Traversaro Sasia Date: Fri, 11 Sep 2026 12:47:13 -0300 Subject: [PATCH 1/2] CXF-274: type the source-openapi-spec Output Contract to match the scorer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Output Contract named its field names but never their types, value shapes, or granularity, while evals/runner/stages.ts compares three of them by strict equality or exact set membership. Blind runs that sourced and judged correctly failed P3 and P4 on encoding alone. Documents the contract the scorer already enforces: - authority_rung is scored with nonEmptyString (stages.ts:489, typed at stages.ts:59), but the skill teaches a numbered ladder, so "rung 1" reads as the integer 1 and fails. Note spec_bytes in the same object genuinely is a number, so nothing signals which is which. - spec_version_checked is compared with !== (stages.ts:517), so a richer composite carrying a sha256 and a fetch timestamp fails against "1.2.0". - missing_paths is compared with Array.includes (stages.ts:518), which is exact element match, so descriptions of missing capability families fail against literal path templates. Additive only: every substring asserted by evals/runner/skills_bundle.test.ts is preserved. No behaviour change — this narrows the prose to the code. npm run eval:test exit 0 (139 pass / 0 fail); npm run typecheck exit 0. Co-Authored-By: Claude Opus 5 (1M context) --- skills/source-openapi-spec/SKILL.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/skills/source-openapi-spec/SKILL.md b/skills/source-openapi-spec/SKILL.md index e194dab..0258c31 100644 --- a/skills/source-openapi-spec/SKILL.md +++ b/skills/source-openapi-spec/SKILL.md @@ -58,7 +58,19 @@ is trimmed by selection, never by editing the vendored document. Emit the `sourcing` and `park_evidence` halves of the pre1.json artifact: - `sourcing`: `{spec_url, fetched_at, authority_rung, spec_bytes}`. + - `spec_url` (string) and `fetched_at` (string, ISO-8601). + - `authority_rung` (string, NOT a number) - the ladder rung as a string, + e.g. `"1"` for an official published spec. + - `spec_bytes` (integer) - from `wc -c`; above 0 and under 1048576. - `park_evidence`: `{spec_version_checked, missing_paths, vendor_doc, revisit_trigger}`. + - `spec_version_checked` (string) - the provider's bare version value and + nothing else, e.g. `"1.2.0"`. Not a sentence, and not a composite carrying + hashes, byte counts, or fetch timestamps; those belong in `sourcing`. + - `missing_paths` (array of strings) - the literal OpenAPI path templates + that are absent, e.g. `["/v1/users", "/v1/groups", + "/v1/groups/{groupId}/members"]`. Not prose descriptions of the missing + capability families. + - `vendor_doc` (string) and `revisit_trigger` (string), both non-empty. ## Exit criteria From 0d3979d912abfc5009e448dfde925e8956ba1efd Mon Sep 17 00:00:00 2001 From: Manuel Traversaro Sasia Date: Fri, 11 Sep 2026 16:12:05 -0300 Subject: [PATCH 2/2] CXF-274: name the Grant type and provisioning nesting in write-connector-source design-access-model routinely emits provisionable: true, but this skill never mentions grant, revoke, or provisioningResponse, and its only worked skeleton is read-only by its own README. Authoring the provisioning half means deriving it from baton/runtime.d.ts directly, and two declaration-level traps sit on that path: - Two different Grant types ship. baton/helpers.d.ts imports Grant from ./types (baton/types.d.ts:328, no "@type"), which is what provisioningResponse.grant({grants: [...]}) takes. The proto-shaped Grant at baton/sdk-types.d.ts:653 requires "@type". Writing a proto-JSON literal -- which the capabilities section trains you toward -- fails with TS2353. - ResourceTypeSpecShape (runtime.d.ts:2055-2058) accepts both a top-level grant/revoke and a nested provisioning: {grant, revoke}, and both typecheck. The runtime-facing RuntimeResourceTypeSpec (runtime.d.ts:1738) carries only provisioning, so that is the nesting to prefer. Both verified against a provisioning connector authored from this skill and typechecked clean. Kept deliberately terse: the body is at 197 of the 200-line bound that evals/runner/skills_bundle.test.ts:194 enforces, so the fuller worked guidance this skill still lacks does not fit without a references/ mechanism. Noted on CXF-274. npm run eval:test exit 0 (139 pass / 0 fail); npm run typecheck exit 0. Co-Authored-By: Claude Opus 5 (1M context) --- skills/write-connector-source/SKILL.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/skills/write-connector-source/SKILL.md b/skills/write-connector-source/SKILL.md index c91acc9..adedb7d 100644 --- a/skills/write-connector-source/SKILL.md +++ b/skills/write-connector-source/SKILL.md @@ -27,6 +27,10 @@ rule: when the served guide conflicts with any other doc, the served guide wins. 3. Slot identity is by JS reference - never re-call `slot()`. 4. The bundle targets ES5 - no `u` regex flag; goja quirks apply. 5. Import only from `@baton/runtime`, `@baton/types`, `@baton/helpers` (`@baton/*` resolution). +6. Provisioning: emit `provisioningResponse.grant({grants: [...]})` / `.revoke({})`; + that `Grant` is `@baton/types`' (NO `"@type"`), not the proto-shaped `sdk-types` + one. Attach the walkers under `provisioning: {grant, revoke}` - the runtime-facing + `RuntimeResourceTypeSpec` carries only that nesting. Worked skeletons: `examples/http/connector.ts` (transport + offset pagination + users/groups/membership grants), `examples/static/connector.ts` (zero-config), and the lifecycle doc's Okta worked example (pin in SOURCES.md). @@ -183,6 +187,8 @@ Taught as contract rules, not eval-gaming literals: - WithExternalID is DEPRECATED - never required. - Do not write plaintext secrets into `connector.ts`. - Do not claim the build schema-validates `capabilities.json`. +- Do not put a proto-JSON `"@type"` in a `provisioningResponse.grant` grant - + that `Grant` has no discriminator. ## Blocker protocol