Repository navigation
Typed encode/decode: inferred types, the asn DSL, and ASN.1 code generation - #43
Merged
Merged
Conversation
decode() returned unknown and encode() took unknown, so every caller had to
write an interface by hand and cast. The schema already describes the shape;
this makes TypeScript read it.
SchemaCodec, SchemaBuilder.build and the new createCodec/createCodecs take the
schema as a `const` type parameter, so an inline schema keeps its literal types.
Three conditional types in src/schema/Infer.ts walk those literals:
- Infer<S> what decode() returns
- InferInput<S> what encode() accepts (DEFAULT fields omittable, RawBytes
allowed at any node)
- InferMetadata<S> the DecodedNode tree, and stripMetadata() now derives its
return type from the node type it is given
ENUMERATED narrows to a literal union, CHOICE becomes a discriminated union on
`key`, OPTIONAL fields become optional properties while DEFAULT fields stay
required after decoding, and $ref resolves through the registry passed to
createCodecs (bounded expansion, so recursive schemas terminate).
This is compile-time only: no validation runs, no dependency is added, and the
emitted JavaScript is unchanged. A schema typed as the wide SchemaNode union —
parsed from JSON or from ASN.1 text — degrades to exactly the unknown-typed API
that existed before.
Supporting changes:
- SchemaNode moves to src/schema/SchemaNode.ts with readonly collections, so
inline and `as const` schemas are assignable without widening
- Codec gains a TInput parameter, defaulting to T
- SchemaBuilder's two near-identical builders collapse into one buildNode()
parameterised by how $ref is resolved
- examples/typed-api.md documents the mapping and the pitfalls; the CHOICE
examples in encoding.md/decoding.md were describing a `{ name: value }` shape
the codecs stopped producing, and now show `{ key, value }` with working hex
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
The literal-inference API shipped in the previous commit has two weaknesses
that are structural rather than polish: it depends on schemas and values
keeping their literal types, and its errors print deferred conditional-type
aliases (`InferInputChoice<{ readonly type: "CHOICE"; ... }, Record<...>, 12>`)
instead of resolved types. It also gives nothing to a schema parsed from a
.asn file at runtime, which is the library's headline workflow.
Rather than replace it, add two more front-ends over the same interchange
format. SchemaNode is that format: the ASN.1 parser, the DSL and hand-written
literals all produce it; SchemaBuilder and the generator both consume it. All
three now hand back the same TypedCodec<TOut, TIn, TNode>.
asn DSL (src/dsl/) — schemas built from functions, so the types travel in the
value's type parameters instead of being recomputed from literals:
const Ticket = asn.sequence({
id: asn.integer({ min: 0, max: 255 }),
status: asn.enumerated(['pending', 'approved']),
version: asn.integer({ min: 0, max: 10 }).default(1),
});
A plain `const` keeps its type, schemas are reusable, and errors name the
offending field. Recursion goes through asn.ref<T>(name) + asn.compile, and
metadata trees stay typed. SEQUENCE field order is object key order, so the
builder rejects integer-like keys — the engine hoists those, which would
silently reorder the encoding rather than fail.
Codegen (src/codegen/, cli/generate-types.ts) — .asn or SchemaNode JSON to a
TypeScript module of named interfaces plus typed codecs. The parser resolves
type references by inlining them, so the generator matches those expanded
structures back to the types they came from; without that the output repeats
whole structures and loses the point of naming them. ASN.1 hyphens are handled
both ways: type names become PascalCase (with collisions numbered and shadowing
avoided), field names stay verbatim and quoted. Each type gets an Input variant
where a DEFAULT is reachable, computed as a fixpoint so $ref cycles terminate.
On a 150-type schema, generated code costs ~20k type instantiations against
~109k for the equivalent inline literal, and errors name the type.
RawBytes: InferInput no longer folds `RawBytes |` into every node. Passthrough
moves to `codec.raw`, which exposes the three encode entry points with every
node widened. The old shape doubled the length of every encode error message
for a feature most values never use.
tests/fixtures/generated/sampleModule.ts is checked-in generator output: a test
asserts the generator still reproduces it byte for byte and round-trips values
through it, and ts-jest compiles it, so generated code cannot silently stop
compiling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
The two scripts under cli/ were only runnable from a checkout via `npx tsx`, because the build compiles src/ alone and the package ships dist/. Move them into src/cli/ so the existing build emits them, and expose one binary with a subcommand each: npx asn1-per-ts types ticket.asn src/generated/ticket.ts npx asn1-per-ts schema ticket.asn ticket.schema.json The argument handling lives in src/cli/cli.ts behind a CliIo interface, so the commands are driven by an in-memory IO in tests; src/cli/main.ts is the only part that touches fs and process. CliIo deliberately uses no Node types, to keep them out of the published declarations. Behaviour the old scripts did not have: data goes to stdout and progress to stderr, so redirecting produces a clean file; failures print a message and exit 1 rather than throwing a stack trace; `--no-runtime` emits types without the schemas and codecs values. That last flag exposed a bug worth fixing anyway — the generator emitted its type imports unconditionally, so a types-only module imported SchemaRegistry and TypedCodec without using them and failed under noUnusedLocals. Imports are now emitted only for what the output references. sideEffects narrows from false to the CLI entry, which does run on import; the library stays tree-shakeable. Verified against a real install rather than the checkout: npm pack, install the tarball into an empty project, run the linked binary, then compile and run the generated module against the packaged runtime under strict + noUnusedLocals + NodeNext. There is no --version flag: reading package.json at runtime needs import.meta or createRequire, and the root tsconfig type-checks as commonjs where import.meta is an error. Not worth restructuring for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
Answering "are the docs current": the README had drifted and there was no changelog at all — releases relied on GitHub's generated notes, which do not distinguish a new export from a change that stops existing code compiling. The last three commits contain three such type-level breaks, so they need somewhere to be stated. CHANGELOG.md starts at Unreleased and points at the GitHub releases for v1.0.2 to v1.4.0; reconstructing those from git history would have been guesswork. It records the typed API, the DSL, the code generator and the bin, and calls out the breaking items: RawBytes moving to codec.raw, encode() now rejecting values that do not fit, SchemaNode's collections becoming readonly, and the removal of the cli/ scripts. PUBLISH.md gains a first step to fold Unreleased into a version, and notes that the Breaking entries decide major versus minor. README corrections, the first two of which predate this branch: - OBJECT IDENTIFIER was missing from the feature list and the supported-types table, though the codec has always existed. - RawBytes passthrough was never mentioned at all. - The metadata example cast node.value to Record<string, DecodedNode>. That cast became unnecessary when the tree became typed, and following it throws the types away; it now shows node.value.id.value directly. - The three-front-ends snippet used `asn` without importing it. - The Examples section listed only the three original guides, not typed-api, dsl or codegen. - Development commands omitted the type-check and running the CLI from source. CLAUDE.md: src/cli/ was nested under the examples/ sub-list by a bad search-and-replace in the previous commit, and the schemas/ entry described a directory deleted in 5cc15dd. Every runnable README snippet is now compiled in a scratch file, including the @ts-expect-error on the ENUMERATED example, so the claims are checked rather than asserted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
The guide is the entry point for anyone with a .asn file, and it dead-ended at an untyped SchemaCodec without ever saying the types were missing or that codegen exists. It also documented the converter's reference handling backwards. - A "which path do you want" table up front: parse at runtime for schemas chosen at runtime, generate TypeScript when the .asn file is checked in. The encode/decode example now says plainly that `decoded` is `unknown` to TypeScript, and gives the three ways to get a type back. - "Schemas with $ref (recursive or cross-referenced types)" was wrong: a cross-reference is *inlined*, and only a cycle produces a $ref. Verified against the parser — `Status` referenced from `Request` comes back fully expanded with no $ref anywhere. Split into two sections, one for inlining (with its consequences: a registry larger than its source, and why the generator de-inlines) and one for genuine recursion. - The recursion example used SchemaBuilder.buildAll and the low-level Codec/BitBuffer API. It now uses createCodecs, notes that `new SchemaCodec(schemas.Tree)` throws for a $ref schema, and mentions buildAll as the lower-level alternative. - Noted that constraints are runtime-only and never reach the types, and that OPTIONAL and DEFAULT are asymmetric once types are involved. - "Building SchemaNode Manually" became "Writing schemas without the parser", making the point that an inline schema does carry types — and warning that annotating it `: SchemaNode` throws them away again. - Added the limits of the parser subset with a pointer to TODO.md; corrected the SchemaNode location to src/schema/SchemaNode.ts; added SchemaNode.ts and the generator to the related-files table. Claims checked rather than asserted: every JSON output and the '002a40' hex come from running the parser, and every TypeScript snippet compiles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
decode()returnedunknownandencode()tookunknown, so every caller wrote an interface by hand and cast. The schema already describes the shape; this makes TypeScript read it — three ways, over one interchange format.SchemaNodeis the interchange format. The ASN.1 parser, the DSL and hand-written literals all produce it;SchemaBuilderand the code generator both consume it. All three front-ends hand back the sameTypedCodec<TOut, TIn, TNode>.None of this runs at runtime: no validation was added, no dependency was added, and the emitted JavaScript is unchanged.
1. Types inferred from an inline schema
SchemaCodec,SchemaBuilder.buildand the newcreateCodec/createCodecstake the schema as aconsttype parameter, so an inline schema keeps its literal types. Conditional types insrc/schema/Infer.tswalk them:Infer<S>decode()returnsInferInput<S>encode()acceptsInferInputRaw<S>RawBytesat any nodeInferMetadata<S>DecodedNodetreeENUMERATED narrows to a literal union, CHOICE becomes a discriminated union on
key, OPTIONAL fields become optional properties while DEFAULT fields stay required after decoding, and$refresolves through the registry passed tocreateCodecs(bounded expansion, so recursive schemas terminate).stripMetadata()derives its return type from the node it is given.2. The
asnbuilder DSLTypes ride in the value's type parameters instead of being recomputed from literals, so a plain
constkeeps its type and schemas are reusable. Error messages name the offending field rather than printing a deferred conditional-type alias. Recursion goes throughasn.ref<T>(name)+asn.compile; metadata trees stay typed.SEQUENCE field order is object key order, so the builder rejects integer-like keys — engines hoist those, which would silently reorder the encoding rather than fail.
3. Code generation
.asnorSchemaNodeJSON to a module of named interfaces plus typed codecs. The parser resolves type references by inlining them, so the generator matches those expanded structures back to the types they came from — without that the output repeats whole structures and loses the point of naming them. ASN.1 hyphens are handled both ways: type names become PascalCase (collisions numbered, shadowing avoided), field names stay verbatim and quoted. Each type gets anInputvariant where a DEFAULT is reachable, computed as a fixpoint so$refcycles terminate.On a 150-type schema, generated code costs ~20k type instantiations against ~109k for the equivalent inline literal.
The package now installs an
asn1-per-tsbinary withtypesandschemasubcommands, replacing thecli/*.tsscripts that could only be run from a checkout. Argument handling sits behind aCliIointerface so the commands are tested with an in-memory filesystem.Breaking changes (types)
Recorded in the new
CHANGELOG.mdunder[Unreleased]. These warrant at least a minor bump, arguably major:RawBytesmoved tocodec.raw. The default encode signatures no longer accept it. FoldingRawBytesinto every node doubled the length of every type printed in an encode error, for a feature most values never use. The runtime is unchanged and still acceptsRawBytesanywhere.encode()now rejects values that do not fit. The point of the change, but code that compiled against the oldunknownsignature may need fixing — usually an intermediateconstwidened a literal, which annotating withInfer<typeof schema>solves.SchemaNode's collections arereadonly, so inline schemas keep their literal types. Code that mutatedfields,alternativesorvaluesafter construction no longer compiles.cli/generate-schema.tsandcli/generate-types.tsremoved in favour of the bin; from a checkout,npx tsx src/cli/main.ts <subcommand>.A schema known only at runtime — parsed from JSON or ASN.1 text — stays the wide
SchemaNodeunion and keeps theunknown-typed API it had before.Docs
New guides
examples/typed-api.md,examples/dsl.mdandexamples/codegen.md;examples/schema-parser.mdreworked. A newCHANGELOG.md(there was none) withPUBLISH.mdwired to it.Several doc errors fixed along the way, some predating this branch: the CHOICE examples in
encoding.md/decoding.mddocumented a{ name: value }shape the codecs stopped producing, with hex that threw when decoded;schema-parser.mdclaimed cross-referenced types produce$refwhen in fact only cycles do; the README omittedOBJECT IDENTIFIERentirely and its metadata example cast away the typing this PR adds.Verification
587 tests across 33 suites.
tsc --noEmiton library, CLI and website; build clean.Equal<X, Y>, notextends, soany/unknowncannot make one pass vacuously;@ts-expect-errorcases prove wrong values are rejected.tests/fixtures/generated/sampleModule.tsis checked-in generator output, imported by a test so ts-jest compiles it, with an assertion that the generator still reproduces it byte for byte.npm pack, install the tarball into an empty project, run the linked binary, then compile and run the generated module against the packaged runtime understrict+noUnusedLocals+ NodeNext.schema-parser.mdcome from running the parser.🤖 Generated with Claude Code
https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS