Take the model layer to @typra/emitter 0.4.30 and green all five runtimes - #456
Open
Seth Juarez (sethjuarez) wants to merge 70 commits into
Open
Take the model layer to @typra/emitter 0.4.30 and green all five runtimes#456Seth Juarez (sethjuarez) wants to merge 70 commits into
Seth Juarez (sethjuarez) wants to merge 70 commits into
Conversation
Define the schema-owned engine ports, runtime effect metadata, deterministic acceptance vector, and legacy harness preservation gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Thread generated executor cancellation through the pipeline and provider SDK calls, with forwarding and pre-cancellation coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extend the canonical acceptance vector so PortError cannot become a generated model or wire export. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pin exact metadata, ordered parameters, wire exclusions, and complete native signatures for every configured target. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Open the Connection discriminator and pin exact forward-compatible payload preservation with shared known, unknown, and case-collision vectors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exercise generated Connection load/save APIs against the shared forward-compatibility vectors without editing generated runtime code. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Join TextPart values with newlines in Python and TypeScript and return an empty string for empty TypeScript messages, matching the canonical contract and Rust behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document ContentPart as a closed, case-sensitive union and add shared known, unknown, and wrong-case acceptance vectors. Add a non-generated Rust gate that exposes the current generated fallback-to-text defect without modifying generated code. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add native public-API gates for the shared Connection load-save-reload vectors across C#, Go, Python, Rust, and TypeScript. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add native public-API gates for the shared strict ContentPart vectors across C#, Go, Python, Rust, and TypeScript. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Make the canonical tools_function_load vector assert exact binding map keys and input values across C#, Go, Python, Rust, and TypeScript. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Define direct Property scalar coercion separately from named input shorthand and add one atomic shared vector across C#, Go, Python, Rust, and TypeScript. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Define collection-context scalar shorthand precedence and add focused string, integer, float, and boolean vectors with a Rust no-degradation gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Consume the shared Record<unknown> vectors in C# and assert exact model-field coverage plus load/save/reload preservation of direct, nested, and list nulls. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assert the canonical IDictionary<string, object?> mapping and outer optionality for all nine shared Record<unknown> vector surfaces. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…iminator
Connection is an open discriminator: `.prompty` files may carry connection
kinds this schema has never heard of, and load -> save -> reload must return
the payload byte-for-byte, including the exact `kind` string and every
undeclared top-level key.
The previous shape could not do that:
alias ConnectionType = "remote" | "reference" | ... ;
model Connection { kind: ConnectionType; }
Unknown kinds silently collapsed to "reference" (the first variant), failing
spec/vectors/model/connection_roundtrip_vectors.json 1 pass / 2 fail.
This commit changes `kind` to a bare `string` and keeps @abstract and
@Discriminator("kind"). That is the whole fix.
Why not a wildcard subtype
--------------------------
The obvious alternative - mirroring Tool's `union ToolTypes { ..., string }`
plus `model CustomTool extends Tool { kind: "*" }` - was implemented and
measured, and it is wrong here. It preserves the discriminator exactly
(including case), but a wildcard subtype serializes only *declared* fields:
to_value() re-emits `kind` plus the base fields and structurally drops every
undeclared key. Connection vectors carry arbitrary extras (endpoint, tenant,
priority, regions, providerOptions, ...), so that shape still failed 2 of 3.
Bare `kind: string` on an @abstract base gives full raw-payload passthrough,
which is what the vector's exact-equality requirement needs. This matches the
emitter's own fixtures/rust-unknown/main.tsp, which is a near-exact replica of
this shape.
Also note @abstract does NOT need to be dropped - an earlier proposal paired
"kind: string" with removing @abstract. Only the first half was necessary.
Measured (Rust, connection_roundtrip_vectors):
alias + no wildcard -> 1 pass / 2 fail (kinds collapse to "reference")
union + kind: "*" -> 1 pass / 2 fail (kind exact, extras dropped)
bare kind: string -> 3 pass / 0 fail
This commit is schema-only and does not include regenerated models.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
*** THIS COMMIT IS NOT REPRODUCIBLE FROM package.json. DO NOT MERGE AS-IS. ***
package.json here pins @typra/emitter 0.4.15, but this output was NOT produced
by published 0.4.15. It was produced by a locally built, unpublished 0.4.18
carrying the two-line IR fix on the typra branch:
wip/nonabstract-base-discriminator-fallback (typra, commit d870296)
It was installed with `npm install --no-save`, so schema/node_modules holds
bytes that no version number describes. Anyone who runs `npm ci` and
regenerates WILL get materially different output than what is committed here.
To reproduce: build that typra branch, `npm pack` it, install the tarball into
schema/, then regenerate. Or, better, wait for a real published release and
regenerate from a clean `npm ci`.
Why the emitter needed patching at all
--------------------------------------
Bumping 0.4.2 -> 0.4.15 to pick up open-discriminator handling also picked up a
regression in *concrete* polymorphic bases. Prompty's `Property` is a
non-@abstract model with @Discriminator("kind") whose union permits scalar
kinds (string, integer, float, boolean, thread, audio) that no subtype claims.
0.4.2 emitted a self-referencing fallback for those; 0.4.15+ emits a panic!
plus a validate_discriminator() that rejects anything outside
array|object|union. Every scalar-kind Property therefore panics on load.
This is the same defect class reported earlier against Go's
model/property.go:LoadProperty, which returned a zero Property for exactly
these kinds. It is a shared emitter bug, not a per-language one.
Verification status - read this before trusting anything here
--------------------------------------------------------------
Rust: 296/0 at 0.4.2 (clean baseline)
289/7 at published 0.4.15
286/10 at published 0.4.18
293/3 with the patched emitter used for this commit
The 3 remaining failures (model_invocation_request.rs:45,
tool.rs:115 x2) are NOT investigated.
Go / C# / Python / TypeScript / Java / Swift: NOT BUILT, NOT TESTED against
this regeneration. 0.4.15 is a breaking change - optional collections
became Option<Vec<T>> / equivalent - so handwritten seams in every
runtime are expected to break the same way Rust's did.
Typra's own test suite: NOT RUN against the patch.
Committed to preserve the work for manual triage, not because it is ready.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…>> change
Emitter 0.4.15 is a breaking change for optional collections: fields that were
`Vec<T>` are now `Option<Vec<T>>`. Prompty's `inputs` / `outputs` moved;
`tools` did not, because it carries a `#[]` default.
Two handwritten sites had to follow:
src/model_ext.rs
as_inputs() / as_outputs() now go through
self.inputs.as_ref().filter(|items| !items.is_empty())
preserving the previous "empty collection reads as absent" semantics.
tests/named_collection_vectors.rs
two assertions rebound loaded_inputs / reloaded_inputs via .as_ref().
This is exactly the class of break to expect in the other six runtimes, none
of which have been built against this regeneration. Anyone picking this up
should grep each runtime for direct field access on inputs/outputs first.
Scope note: this repairs compilation and the collection semantics only. It does
not address the 3 remaining Rust test failures (model_invocation_request.rs:45,
tool.rs:115 x2), which are uninvestigated.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…curring Prepends a mandatory "Operating Rules" section to .github/copilot-instructions.md. Placed at the top rather than appended because the file is ~68KB of rebuild plan and anything at the end is not read in practice. Each rule is derived from a specific failure in the seven-runtime parity effort, which deadlocked for roughly 24 hours and had to be abandoned: 1. Vectors are the only authority - a "must load as the base type" constraint circulated for a day and drove an emitter-release escalation. It exists in no vector. Constraints must be quoted from spec/vectors/** or labeled an opinion. 2. Green baseline before bumping codegen - the 0.4.2 -> 0.4.15 bump regressed Rust 296/0 to 289/7, but no baseline had been recorded, so the regression was initially misread as pre-existing. 3. Measure before escalating - the entire deadlock was an argument about which emitter version could be published. No publish was needed. Three schema shapes A/B'd afterward settled it in under an hour. 4. Report numbers, not impressions, and name what was NOT verified. 5. Orchestration budget - two exchanges on one blocker, then pull it in-house. Also records that archive_session is parent-only and idle != stopped. 6. On "stop", commit to wip/ branches, split for cherry-picking, and flag any state not reproducible from the manifest. 7. Repo facts that cost hours to rediscover: origin fails SAML 403 (use ssh), origin/* refs go stale, npm versions are immutable, the Connection open-discriminator shape, and the non-abstract-base fallback rule. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bumps the emitter pin from 0.4.15 to 0.4.20 and regenerates all backends.
0.4.20 is the first release cut from typra `main` -- every version from
0.4.3 through 0.4.18 was published from an unmerged branch, which is why
`main` sat at 0.4.2 while npm `latest` reported 0.4.18.
Emitter fixes included in this bump:
- non-abstract polymorphic base now absorbs discriminator values that no
subtype claims, instead of returning a zero value (Go) or panicking
(Rust). This is what `Property` relies on: it is concrete and its union
permits string/integer/float/boolean/thread/audio, none of which have
subtypes.
- open discriminators are no longer pre-validated against their declared
union before dispatch, which had made the open fallback unreachable.
- Go scalar coercions now bridge decoder-native numeric types.
`encoding/json` yields float64 for *every* JSON number, so the emitted
`case int:` / `case float32:` arms matched nothing and fell through to a
zero value.
- generated tests now synthesize payloads for required complex fields that
carry no @sample, so a generated test can pass its own generated
validation.
- Go abstract open-discriminator bases preserve unknown kinds and their
payloads rather than erroring.
- load diagnostics now report array element indices.
VERIFIED
typra npm test 290 passed / 0 failed
typra validate:fixtures pass
npm run generate "Compilation completed successfully."
runtime/go/prompty go test ./... 62 failures -> 1
runtime/rust/prompty cargo test 293 passed / 3 failed
runtime/typescript (core) vitest run 67 failures -> 14
NOT VERIFIED
Python runtime does not build (`uv pip install -e ".[dev,all]"` fails with
ModuleNotFoundError: No module named 'prompty' inside its own build
isolation), so its suite was never run. C# was not run either.
KNOWN REMAINING FAILURES, all pre-existing and attributed
TypeScript, 6 of 14 -- emitter defect, tracked as sethjuarez/typra#59:
TypeScript/C#/Python still conflate abstract with closed in polymorphic
dispatch, so Connection rejects unknown kinds:
Error: Unknown Connection discriminator field 'kind' value: future-auth
This violates spec/vectors/model/connection_roundtrip_vectors.json. The Go
half of this landed as typra#58; the other three backends did not.
Go 1 of 1, and TypeScript 1 of 14 -- prompty handwritten seam, not emitter:
turn_runner's permission-denied branch returns HostToolResult without first
recording TurnEventTypeToolResult, so the golden replay journal is missing
turn:tool_result:0:add:false:permission_denied
violating spec/vectors/harness/replay_vectors.json.
Rust 3 of 3 -- prompty test data, not emitter: tool_dispatch.rs:1044 and
:1065 omit `connection`; a live_turn test omits `context`.
TypeScript, remaining 7 -- not yet attributed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`ReferenceTurnRunner`'s permission-denied branch built a `HostToolResult`
and returned it without ever recording a `tool_result` turn event, so the
denial never reached the event sink or the journal. Any consumer replaying
the journal saw a permission decision with no corresponding outcome.
`spec/vectors/harness/replay_vectors.json` is explicit about the expected
sequence for the `permission_denied` scenario:
turn:permission_requested:0:exec-1-permission
turn:permission_completed:0:false
turn:tool_result:0:add:false:permission_denied
turn:messages_updated:0
Note there is deliberately no `tool_execution_start` / `tool_execution_complete`
pair -- the tool is never executed -- but the result is still recorded. The
approved path already did this (Go `turn_runner.go:245`, TypeScript
`turn-runner.ts:201`); only the denied path was missing it.
Fixed in both runtimes by binding the result to a local, recording
`tool_result`, and then returning it. Go additionally propagates a recording
error rather than swallowing it, matching every other `recordTurn` call in
that function.
Rust already recorded this event correctly and needed no change.
VERIFIED
runtime/go/prompty go test ./... 1 failure -> 0
runtime/typescript core vitest tests/harness/turn-runner 6 passed / 0 failed
runtime/typescript core vitest run 14 failures -> 13
Both suites were failing on this exact assertion before the change and pass
after it, with no other test changing state in either direction.
NOT VERIFIED
Python and C# runtimes were not run. Python does not currently build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
spec/vectors/load/load_vectors.json, vector `empty_frontmatter_body_only`,
requires that a .prompty carrying only a name loads successfully:
"input": { "frontmatter": { "name": "empty-fm" } },
"expected": { "kind": "prompt", "name": "empty-fm",
"model": null, "inputs": null, "tools": null }
The vector asks for a null model -- not a defaulted one -- but agent.tsp
declared `model: Model | string` as required, so any model-less prompt was
rejected at load with "model: missing required field". Every sibling field
(inputs, outputs, tools) is already optional; model was the lone outlier.
This is a defect in prompty's own TypeSpec, not in the emitter. The emitter
was correctly enforcing what the schema declared.
Schema edit only -- regeneration follows in the next commit so the source
change and the generated churn stay independently reviewable and revertable.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Regeneration only -- no hand edits. Produced by `npm run generate` from schema/ against @typra/emitter 0.4.20, picking up the previous commit's change making `Prompty.model` optional. Every backend now emits model as a nullable/optional field: rust model: Option<Model> go Model *Model ts model?: Model | string csharp Model? python model: Model | None The JSON AST and the VS Code schema follow. Handwritten call sites that assumed a non-optional model break as a result; those repairs are in the following commit, kept separate so the generated churn stays independently reviewable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`Prompty.model` became `Option<Model>`, so seven handwritten sites that
reached through it no longer compile. Adapted rather than unwrapped blindly:
pipeline.rs resolve_provider / is_streaming
thread the Option through with as_ref().and_then(..); a model-less
prompt now falls back to DEFAULT_PROVIDER and non-streaming instead
of being unrepresentable.
pipeline.rs trace payload
model.id becomes null in the trace when absent rather than "".
live_turn.rs model_id
map-then-filter, preserving the previous "empty id means None" rule.
tests/load_vectors.rs
This one is a genuine strengthening. It previously inferred a null
model from an empty id:
// expected null model -> id should be empty (default)
assert!(agent.model.id.is_empty(), ..)
which is not what the vector asks for. load_vectors.json's
`empty_frontmatter_body_only` states "model": null, and that is now
directly assertable, so it asserts agent.model.is_none().
tests/loader_test.rs, tests/live_turn_execution.rs
Fixtures here all declare a model; added a single model_of() helper
instead of scattering .as_ref().unwrap() across thirteen assertions.
MEASURED, cargo test --no-fail-fast, whole target set
before 39 failed
after 36 failed
Nothing regressed; three tests newly pass (loader_test +1,
named_collection_vectors +1, record_unknown_nullability_vectors 0/1 -> 1/0).
A NOTE ON THE BASELINE
Earlier runs in this effort reported Rust as "293 passed / 3 failed". That
number was wrong. Plain `cargo test` stops at the first failing target, so
it only ever reported the lib target and silently skipped every integration
test binary. The true figure is above. Use --no-fail-fast for any Rust
measurement in this repo.
The 36 remaining failures are NOT caused by this change and are unrelated to
Prompty.model. The largest block is tests/agent_vectors.rs, 0 passed / 28
failed, all panicking identically:
tools.parameters.properties: invalid named collection entry category array
spec/vectors/agent/agent_vectors.json supplies properties in list form:
"parameters": { "properties": [ { "name": "city", "kind": "string" } ] }
which spec/vectors/model/named_collection_vectors.json declares canonical --
its own first vector loads "inputs" in exactly that shape, and its header says
array-valued entries are rejected only "in name-keyed object form", while
"arrays in declared entry fields remain valid". The emitted validator is
rejecting the legal collection-level list form. Tracked separately; this is an
emitter defect, not a prompty one.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
structured.prompty declared an array output with no items:
- name: keyPoints
kind: array
required: true
schema/model/core/properties.tsp makes items required on ArrayProperty --
`items: Property | Named<..>` with no `?` -- so the loader correctly
rejected the fixture with "outputs[1].items: missing required field". The
fixture was violating the schema it exists to exercise; the emitter was right.
(The indexed `outputs[1]` path in that message is the array-index reporting
from typra #47 working as intended.)
MEASURED, packages/core, npx vitest run
tests/loader.test.ts structured output 4 failed -> 0 failed
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extends commit 233ef47 (which legislated per-provider ModelOptions omission for Anthropic in Rust and TypeScript) to the Python and C# wire runners. Test-only; no runtime source changed in any language. I reported to the typra session that "Python and C# run only the OpenAI side of wire_vectors.json". That claim was WRONG. Both dispatch the Anthropic vectors: runtime/python/prompty/tests/test_spec_vectors.py:671 provider == "anthropic" -> _anthropic_build_chat_args runtime/csharp/Prompty.OpenAI.Tests/SpecVectorWireTests.cs:393 AnthropicChatVectors() So both already picked up the new anthropic_unmapped_options vector on the previous commit -- and both PASSED it while measurably blind. Measured blindness. Both are pure delegations to the generated to_wire/ToWire with no allow-list, i.e. the most exposed consumers of the omission rule: python providers/anthropic/executor.py:133 opts.to_wire("anthropic") csharp Prompty.Anthropic/AnthropicExecutor.cs:122 opts.ToWire("anthropic") Simulating typra #84 at each of those seams (leak fields that carry no provider mapping, which is the defect's actual trigger): python 28 passed / 0 failed <- fully blind csharp 6 passed / 0 failed <- fully blind, no extra-key check at all Root cause per runner: python three hand-enumerated absence checks (tools, output_config, system) -- whack-a-mole, one line per key, so any new leaked key is invisible. Replaced with one general set(actual) - set(expected) check. csharp Anthropic_Chat_WireFormat iterated expected keys only and had no extra-key assertion whatsoever. Added one. Sensitivity after the fix, same simulations: python 1 failed / 27 passed <- only anthropic_unmapped_options csharp 1 failed / 5 passed <- error names frequencyPenalty, presencePenalty, seed exactly Cost of the tightening on clean trees: zero. python 28 / 0 csharp 6 / 0 Full suites, after simulations removed and git diff on every src/ confirmed empty: cd runtime/python/prompty && uv run pytest tests/ -q 1483 passed / 0 failed (17 skipped) baseline 1482, +1 = new vector cd runtime/csharp && dotnet test prompty.sln --nologo 1355 passed / 0 failed baseline 1354, +1 = new vector (Core 1112, OpenAI 209, Anthropic 21, Foundry 13) Consumer-side enforcement of the omission rule now exists in four runtimes: Rust, TypeScript, Python, C#. Not verified / known gaps: - Go remains structurally uncovered. runtime/go/prompty contains only model/ -- there are no provider crates and no wire runner at all. This is not fixable by a test; it needs providers to exist. - Only the Anthropic path is tightened. The OpenAI wire runners in both languages were not audited for the same blindness in this commit. - The Rust wire_test! list in prompty-anthropic/tests/vectors.rs is still hand-maintained, so new vectors there are opt-in rather than discovered. Unchanged by this commit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Follow-up to f07df49, which legislated typra #84's rule (a ModelOptions field with no wire mapping for a provider must be omitted, never emitted under its schema field name) on the Anthropic surface. That commit's "not verified" note said only the Anthropic path had been tightened. This attacks that note rather than leaving it. The wire map (model/_ModelOptions.py:149-163, mirrored in every backend) determines exposure, and it is not uniform: openai maps all 9 options -> structurally immune responses maps 3 of 9 -> 6 unmapped, fully exposed anthropic maps 5 of 9 -> 4 unmapped, exposed The Responses surface was a second, wholly unlegislated exposure. All three pre-existing responses vectors set ZERO options, so the surface had no option coverage of any kind -- despite pure-delegation builders that call the generated to_wire directly (Rust prompty-openai/src/wire.rs:661, Python providers/openai/executor.py:329, C# WireFormat). New vector `responses_unmapped_options` sets the three mapped options plus all six unmapped ones and expects only model, input, temperature, max_output_tokens, top_p -- a positive and a negative assertion in one case. Registered in Rust's hand-maintained wire_test! list; TS guard counts bumped. Also generalises the Python checker. f07df49 fixed only the Anthropic checker; auditing the other four found the same defect class in all of them -- _check_wire_chat and _check_wire_responses hand-enumerated two keys each, and _check_wire_embedding and _check_wire_image had no extra-key check at all. All five now share _extra_key_errors(). Sensitivity, measured by simulating #84 in the *generated* to_wire, scoped to the responses provider string (a blanket leak breaks several vectors and proves nothing about incremental coverage): Rust 1 failed / 26 passed, sole failure responses_unmapped_options; all three pre-existing responses vectors pass under the defect Python A tightened checkers + defect -> 1 failed / 28 Python B HEAD checkers + defect + new vector -> 29 / 0, FULLY BLIND So the vector and the checker tightening are jointly necessary; neither alone catches this. Tightening the Python OpenAI-chat checker also surfaced a live gap: under a blanket defect the tightened suite reports 2 failures against HEAD's 1, and the delta is options_additional_properties -- the chat path was blind to additionalProperties leaking into every request. C# is structurally immune on the OpenAI and Responses surfaces, and this is documented in the source (WireFormat.cs:137-138): those paths never call ToWire, they assign each option onto typed SDK properties, so there is no dictionary for an unmapped key to leak into. Confirmed by measurement -- with the defect applied, Responses reported 4/0 before the check and 53/0 after. The assertion added here is therefore a forward guard against a refactor onto the ToWire path, not a live catch. C# does delegate for Anthropic, which f07df49 already covered. Cost: zero. Python 28/0 -> 29/0, C# OpenAI 209/0 -> 210/0. Measured after: rust cargo test --workspace --no-fail-fast 1147 / 0 typescript npm test 1734 / 0 csharp dotnet test prompty.sln 1356 / 0 go go test ./... ok docs lint 2 pre-existing failures (unchanged) Not verified: - Go is structurally uncovered: runtime/go/prompty contains only model/, no providers and no wire runner. Not fixable by a test. - TypeScript was not simulated for the responses defect. Its toEqual-based comparison suggests it would catch it, but that is reasoning, not a measurement. - C# chat/embedding/image extra-key checks: chat now carries the same forward guard, embedding and image do not. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A clean regeneration against the pinned emitter reported 291 modified files. Investigated before acting, per the rule about attributing a diff before writing a fix. Finding: not staleness. The Rust emitter does not format its output, but the committed tree is rustfmt-formatted. The entire diff was line-joining and trailing commas -- 13,329 deletions against 4,038 insertions, all of it collapsing multi-line expressions onto single lines. Every other language regenerated byte-identically; 157 of the 157 files that differed under a whitespace-insensitive comparison were `runtime/rust/prompty`. Proved it by running `cargo fmt -p prompty` on the regenerated output: `git status` went clean. So the committed generated code is exactly `rustfmt(emitter 0.4.27 output)` -- fully reproducible, nothing stale. This is a trap worth removing rather than documenting. A 291-file phantom diff on a clean regeneration hides real diffs, and the next person to bump the emitter has to rediscover that none of it is theirs. normalize-typra-output.mjs already exists for exactly this purpose and already handles two cases -- empty Python test files, Go trailing whitespace. Rust formatting is the missing third, and by far the largest. Scoped to `-p prompty` deliberately: the Rust workspace also contains the handwritten provider crates (prompty-openai, prompty-anthropic, and the harness), which this script has no business reformatting. Degrades with a warning rather than failing the build when no Rust toolchain is present, since the other emitted languages should still generate. Verified: `npm run generate` on a clean tree now leaves zero modified files. Before this change the same command produced 291. The working tree is byte-identical to HEAD, which measured `cargo test --workspace --no-fail-fast` -> 1147 / 0, so no re-run is implied by this commit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Nothing enforced this, and the gap is demonstrable in this branch's own
history. Two commits bumped the emitter pin without regenerating:
00a040f chore(model): bump @typra/emitter to 0.4.26 3 files
38c7a5a chore(model): bump @typra/emitter to 0.4.27 3 files
Neither touched generated source; the last full regeneration was 4494d96
at 0.4.25. Both ran **zero CI**, because every runtime workflow is
paths-filtered to `runtime/<lang>/**` and there is no workflow for
`schema/**` at all. A schema-only commit triggers nothing.
Both turned out to be no-ops -- I verified by regenerating at the pinned
0.4.27, which produced 0 modified and 0 deleted files, proving 0.4.25 and
0.4.27 output are byte-identical for prompty's target set (and, since
there were no deletions, that 0.4.26's pruning had no orphans to remove).
But that was luck, not a gate. The same pattern with a release that did
change output would have shipped stale generated code silently, and the
only symptom would be runtime failures attributed to the wrong cause.
The check is now trivially expressible because the preceding commit made
`npm run generate` idempotent: regenerate, assert `git status` is clean.
Before that it reported ~291 phantom files and could not have been a gate.
rustfmt is installed deliberately. The Rust emitter does not format its
output while the committed tree is formatted, so without that component
normalize-typra-output.mjs degrades to a warning and this job fails on a
pure line-joining diff. Same reason the job pins a real Rust toolchain
rather than only Node.
Verified:
- YAML parses; 3 triggers, 6 steps
- On the current tree the assertion passes (measured: `npm run generate`
-> 0 modified files)
- Revert-check: appending one line to a generated file makes
`git status --porcelain` non-empty, so the gate fires. Reverted clean.
Not verified: this has not executed on a GitHub runner. `npm ci` in
schema/ and the tsp compile are assumed to work there; both run clean
locally, and node_modules is gitignored (confirmed -- the local zero-diff
run had it present).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Ships typra #88 (issue #87): array-element diagnostics lost the element index in the Rust, C# and Swift backends. Loading a collection whose Nth entry fails validation reported the bare collection path, so with ten entries every failure produced an identical diagnostic: before inputs.detail after inputs[1].detail TypeScript, Go, Python and Java already threaded the index; the contract from typra #47 had reached only four of seven backends. Regeneration touches Rust (30 files) and C# (33 files) only, +303 -147. The generated change is uniform: collection loops become `for (index, entry) in entries.iter().enumerate()` with the index folded into the path passed to the element validator. Swift is not an emit target here, so it is not represented. Bump and regeneration are committed together deliberately. The schema reproducibility gate added in 9ce13c1 fails any commit where generated output does not match the pinned emitter, so a bump-only commit -- the shape of 00a040f and 38c7a5a, both of which silently shipped stale output -- is no longer a valid intermediate state. Verified, full suites, before -> after: rust cargo test --workspace --no-fail-fast 1147/0 -> 1147/0 csharp dotnet test prompty.sln 1356/0 -> 1356/0 typescript npm test 1734/0 -> 1734/0 go go test ./... ok -> ok python uv run pytest tests/ -q 1484/0 -> 1484/0 typra flagged one risk: any test asserting an exact error message containing a collection path would now see `field[i].sub`. Searched the Rust, C#, Python, TypeScript and Go trees for diagnostic-string assertions before bumping and found none -- the only message assertion matching the shape checks a filename (`resolve.rs:318`). The zero delta across all five runtimes confirms it: prompty's vectors assert that a load fails, not the path it reports. That is also why no consumer could have caught this defect, and why the fix is worth taking despite moving no numbers. It was found by asserting the #47 contract uniformly across typra's own conformance runners. Regeneration remains idempotent at 0.4.28: a second `npm run generate` leaves the tree unchanged (66 files both runs), so the reproducibility gate stays a reliable staleness signal. Not verified: Swift and Java emitters, which prompty does not generate. The diagnostic path change itself is exercised only by typra's fixtures; no prompty vector asserts a diagnostic string, by design. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Upstream fix: typra #89 — the Rust backend honoured @entryShorthand on load but never on save, writing a name-keyed collection entry in expanded form where the other six backends wrote the shorthand scalar. Generated delta is small and Rust-only: 3 source files plus the export surface manifest, +30 lines. The new code is a guarded save-side branch: if ctx.use_shorthand && item_data.len() == 1 { ... } Measured — every runtime matches the 0.4.28 baseline exactly, zero delta: Rust cd runtime/rust && cargo test --workspace --no-fail-fast 1147 passed / 0 failed C# cd runtime/csharp && dotnet test prompty.sln --nologo 1356 passed / 0 failed (46 skipped) TypeScript cd runtime/typescript && npm test 1734 passed / 0 failed (17 skipped) Python cd runtime/python/prompty && uv run pytest tests/ -q 1484 passed / 0 failed (17 skipped) Go cd runtime/go/prompty && go test -count=1 ./... ok Pin and regeneration travel together in one commit, as with 0.4.28. The schema-repro-check workflow regenerates from the pinned emitter and fails on a dirty tree, so a bump without regeneration is not a valid intermediate state. This is a deliberate deviation from the separate-commits rule. Verified but NOT fixed here — recorded so it is not rediscovered: The new save-side shorthand is unreachable in prompty, and its target field disagrees with the load side. 1. Reachability. The gate requires item_data.len() == 1. Property always serialises `kind` and `enumValues` (as "" and [] when unset), so the length is never 1. A probe over four input shapes produced no shorthand save in either Rust or Python; both emit byte-identical expanded form: {"city": "Seattle"} -> {"kind":"string","default":"Seattle","enumValues":[]} {"city": {"example": "..."}} -> {"kind":"","example":"Seattle","enumValues":[]} So prompty cannot serve as the regression lock for typra #89. 2. Target mismatch. schema/model/core/properties.tsp:15 declares @entryShorthand("default"), and loading a shorthand scalar does place the value in `default` — asserted by the string_scalar_in_name_keyed_inputs_infers_property vector. But every backend's save-side shorthand reads `example`, which is the @Coerce target from properties.tsp:18/24/31/37, not the @entryShorthand target: Rust model/core/property.rs item_data.get("example") Go model/property.go copy["example"] TS core/property.ts:8 shorthandProperty = "example" C# core/Property.cs:29 ShorthandProperty => "example" Uniform across backends, so this is not a cross-runtime divergence — but load and save name different fields. Were the length gate ever satisfied, a Property carrying only `example` would save as a bare scalar and reload with that value in `default`. Reported upstream; not actionable here while unreachable. Also verified: regeneration is idempotent at this pin (a second `npm run generate` leaves the tree clean), and the pin is exact rather than a caret so `npm ci` cannot silently absorb a future publish. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The Python package could not be installed from a clean environment. This
was not a CI flake; `uv pip install -e .` failed outright, and every green
Python number this branch has reported was measured in a virtualenv that
predated the generated pipeline ports.
Reproduced before changing anything, in a fresh venv:
uv venv <tmp> && uv pip install --python <tmp>/Scripts/python -e .
-> File ".../prompty/model/pipeline/_EnginePermissionPort.py", line 10
from prompty.core.cancellation import CancellationToken
ModuleNotFoundError: No module named 'prompty'
Mechanism: pyproject declares `dynamic = ["version"]`, so flit resolves
__version__ by loading prompty/__init__.py through spec_from_file_location.
That executes the package without putting its parent directory on sys.path,
so relative imports resolve and absolute self-imports do not. The four
generated runtime-cancellable ports were the only absolute self-imports in
the package:
model/pipeline/_EnginePermissionPort.py
model/pipeline/_EnginePostCommitPort.py
model/pipeline/_EngineToolPort.py
model/pipeline/_Executor.py
Each already imported its generated siblings relatively (`from ..agent._Prompty
import Prompty`) and only the cancellation token absolutely, so the file was
internally inconsistent — the same two-halves-disagreeing shape as typra #78,
#87 and #89.
Fix is configuration, not an emitter change. `cancellationTokenPath` is an
existing emit-target option; the emitter only splits it on the last dot
(python/emitter.js:107-118), so a relative dotted path passes through intact.
Generated ports live at prompty/model/pipeline/, so `...` is the package root.
Not escalated to typra, because the cheap experiment settled it: one config
line and a regeneration, versus an emitter release. Worth recording for them
anyway that the default is absolute and that the adjacent context import in
the same function is already depth-aware (`group ? ".." : "."`) while the
cancellation import is not — so the default will keep breaking any consumer
whose packaging imports the module. That is an upstream defect; it is simply
not one we need a release for.
Verified in a clean venv, which is the environment that previously failed:
uv pip install -e ".[dev,all]" ok
python -c "import prompty" ok (2.0.0b3)
ruff check . All checks passed
python -m pytest tests/ -q 1484 passed, 17 skipped, 0 failed
Same 1484 as before, but now reproducible from the manifest rather than from
a pre-existing local virtualenv.
Regeneration remains idempotent: a second `npm run generate` leaves the tree
clean, so the reproducibility gate still holds. Schema config and its
regeneration travel together for that reason.
Not verified: Rust, C#, TypeScript and Go were not re-run. This change is
scoped to the Python emit target and touches no other runtime's output.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`ruff format --check .` reported two files, which fails the Python CI's
formatting step. The failure was previously masked: the install step ran
first and died on the absolute cancellation import, so this step never
executed and the drift accumulated unnoticed.
prompty/providers/openai/executor.py two call sites re-joined onto one line
README.md python blocks in fenced examples
Both changes are cosmetic. The executor diff is line-joining only, no
semantic change:
props[p.name] = _property_to_json_schema(p, optional=..., strict=strict)
tool_def["parameters"] = _schema_to_wire(tool.parameters, strict=...)
Neither file is generated, so this is a handwritten-seam repair and is kept
in its own commit, separate from the schema change and its regeneration.
Verified in the same clean venv used for the install fix:
ruff check . All checks passed
ruff format --check . 368 files already formatted
Not verified: tests were not re-run for this commit. `ruff format` makes no
semantic change, and the full suite (1484 passed / 0 failed) was measured on
the parent commit with these same two files already formatted.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The schema reproducibility gate has never passed. It failed on all three commits it has ever run against, including 9ce13c1, the commit that added it. I added a gate and never confirmed it went green — the exact baseline failure the operating rules warn about. gh run list --workflow schema-repro-check.yml --branch wip/typra-0.4.20-regen bf0bb00 failure 6c78733 failure 9ce13c1 failure <- introduced the gate CI reported 547 files changed / 3351 insertions / 6576 deletions while local regeneration was clean. Root cause is in the job log, not inferred: Warning: prettier not found for .../typescript/packages/core/src/model Warning: prettier not found for .../typescript/packages/core/tests/model Warning: ruff check failed for .../python/prompty/prompty/model Warning: ruff format failed for .../python/prompty/prompty/model Generation shells out to three formatters, and each degrades to a warning when absent, producing unformatted output that is indistinguishable from stale generated code in a diff: rustfmt schema/scripts/normalize-typra-output.mjs:27 -> cargo fmt -p prompty prettier emitter typescript/driver.js:125 walks up for node_modules/prettier/bin/prettier.cjs, then npx eslint --fix ruff emitter python/driver.js:137 -> uv run ruff check --fix and uv run ruff format, from the Python project root Only rustfmt was installed. Locally prettier resolves from runtime/typescript/node_modules and ruff from runtime/python/prompty/.venv, which is why nothing reproduced on a developer machine. Changes: - npm ci in runtime/typescript, providing prettier and eslint - astral-sh/setup-uv, plus uv venv and an editable [dev] install in runtime/python/prompty, providing ruff - the regenerate step tees to $RUNNER_TEMP/generate.log and greps for the formatter-missing warnings, failing with the real cause rather than letting the next step report false staleness. The log lives outside the repo deliberately: a log file under schema/ would itself appear in git status --porcelain and fail the assertion it exists to explain. - UV_FROZEN=1 at job level. runtime/python/prompty/uv.lock is tracked and the emitter invokes `uv run`, which would otherwise re-lock and dirty the tree — again reporting as generated-code staleness. Frozen turns that into an explicit lockfile error instead. Both installed trees are ignored, so neither can trip the assertion: runtime/python/prompty/.gitignore:126 (.venv) and .gitignore:4 (node_modules/). NOT VERIFIED, and this is the important caveat: nothing local can confirm this fix. The gate's failure mode is specific to the CI environment, so the only proof is a green run. If it still fails, read the log for which formatter is still missing rather than guessing again — that is how this was found, and guessing is what let it sit red for three commits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Regeneration produces no change to any generated source file. The only
delta is the version stamp in the generated manifest:
schema/tsp-output/.typra-generated/export-surfaces.json 0.4.29 -> 0.4.30
This is the expected outcome, not a skipped regeneration. 0.4.30 fixes
three C# backend defects that prompty cannot reach:
- `float` and `numeric` had no entry in the C# type map, and `number`
mapped to 32-bit `float`. prompty declares neither bare `float` nor
`numeric` anywhere in schema/model -- only explicit float32/float64
(verified by grep; the sole `numeric` occurrence is in a comment at
core/properties.tsp:130). So no prompty field changes type.
- Generated C# conversion tests asserted members that do not exist on
polymorphic bases. prompty's generated conversion tests were already
passing and none of them changed.
- Generated C# factory tests asserted raw templates. Same -- no diff.
Worth recording, because a file count was previously read as evidence
that a bump had skipped regeneration: a fully-executed `npm run generate`
against a release that touches none of our emit targets modifies exactly
three files -- package.json, package-lock.json, and the two-line version
stamp in export-surfaces.json. That signature means "regenerated, and the
release was a no-op for us". It does not distinguish itself from a
skipped regeneration by file count alone; only by running the generator
and observing a clean tree afterwards, which is what the schema
reproducibility gate now enforces on every commit.
Verified at this commit:
cd runtime/csharp && dotnet test prompty.sln
-> 1356 passed / 0 failed (46 skipped), identical to 0.4.29
npm run generate -> clean, no formatter-missing warnings
second `npm run generate` -> idempotent, no further diff
Not verified at this commit: Rust, TypeScript, Python and Go were not
re-run. No generated source changed, so no behavioural change is
possible in any runtime; the schema reproducibility gate and the
per-runtime CI jobs cover this on the PR.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| } | ||
|
|
||
| // LoadCompactionConfig creates a CompactionConfig from a map[string]interface{} | ||
| func LoadCompactionConfig(data interface{}, ctx *LoadContext) (CompactionConfig, error) { | ||
| if ctx == nil { | ||
| ctx = NewLoadContext() |
Seth Juarez (sethjuarez)
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the generated model layer from
@typra/emitter0.4.19 to 0.4.30 and takes all five runtimes to zero failures.Measured at
752d80c7cd runtime/rust && cargo test --workspace --no-fail-fastcd runtime/csharp && dotnet test prompty.slncd runtime/typescript && npm testpytest tests/ -qin a clean venvgo test -count=1 ./...5721 passing, 0 failing. CI: 34/34 checks green.
Notable fixes in this branch
The Python package was uninstallable from a clean environment.
pyproject.tomlusesdynamic = ["version"], so flit resolves__version__by loading__init__.pyviaspec_from_file_location— which executes the package without putting its parent onsys.path. Four generated runtime-cancellable ports carried an absolute self-import alongside relative sibling imports, souv pip install -e .died withModuleNotFoundError: No module named 'prompty'. Any Python measurement taken before this was made in a stale virtualenv. Fixed with one line of emit config (cancellation-token-path), no emitter release required.A second CI blocker was masked behind the first. The install failure always killed the job before
ruff format --checkran, so formatting drift had accumulated invisibly.The schema reproducibility gate had never passed — including on the commit that introduced it. Generation shells out to rustfmt, prettier/eslint and ruff, and each degrades to a warning when absent, producing unformatted output indistinguishable from stale generated code. The gate now installs all three, fails fast on formatter-missing warnings, and pins
UV_FROZEN=1so the trackeduv.lockcannot be silently rewritten and reported as staleness.On the emitter bumps
0.4.28, 0.4.29 and 0.4.30 each changed zero generated source files here — their fixes target constructs prompty's schema does not express (bare
float/numeric, Swift/JavatoWire, C# fixture generation). The 0.4.30 bump commit touches only 3 files, and that is the expected signature of a fully-executed regeneration against a release that is a no-op for our emit targets — not a skipped regeneration. The repro-check gate now distinguishes the two automatically.Not verified
4.0classifiedintegerby Go andfloatby Rust; Python subtype-load inconsistency (_Property.py:136); Anthropic bare{"type":"array"}divergence.