Skip to content

Typed encode/decode: inferred types, the asn DSL, and ASN.1 code generation - #43

Merged
chtitux merged 5 commits into
mainfrom
claude/typed-encode-decode-api-a0zy2g
Sep 29, 2026
Merged

chtitux merged 5 commits into
mainfrom
claude/typed-encode-decode-api-a0zy2g

Conversation

@chtitux

@chtitux chtitux commented Aug 23, 2026 •

Copy link
Copy Markdown
Contributor

decode() returned unknown and encode() took unknown, so every caller wrote an interface by hand and cast. The schema already describes the shape; this makes TypeScript read it — three ways, over one interchange format.

SchemaNode is the interchange format. The ASN.1 parser, the DSL and hand-written literals all produce it; SchemaBuilder and the code generator both consume it. All three front-ends hand back the same TypedCodec<TOut, TIn, TNode>.

  ASN.1 text ──parseAsn1Module──┐
                                │
  asn DSL ─────asn.toSchemaNode─┼──▶  SchemaNode  ──┬── SchemaBuilder ──▶ codecs
                                │    (interchange)  │
  SchemaNode literal ───────────┘                   └── generateTypeScript ──▶ .ts

None of this runs at runtime: no validation was added, no dependency was added, and the emitted JavaScript is unchanged.

1. Types inferred from an inline schema

SchemaCodec, SchemaBuilder.build and the new createCodec/createCodecs take the schema as a const type parameter, so an inline schema keeps its literal types. Conditional types in src/schema/Infer.ts walk them:

Helper Describes
Infer<S> what decode() returns
InferInput<S> what encode() accepts
InferInputRaw<S> the same, allowing RawBytes at any node
InferMetadata<S> the DecodedNode tree

ENUMERATED narrows to a literal union, CHOICE becomes a discriminated union on key, OPTIONAL fields become optional properties while DEFAULT fields stay required after decoding, and $ref resolves through the registry passed to createCodecs (bounded expansion, so recursive schemas terminate). stripMetadata() derives its return type from the node it is given.

2. The asn builder DSL

const Ticket = asn.sequence({
  id: asn.integer({ min: 0, max: 255 }),
  status: asn.enumerated(['pending', 'approved']),
  version: asn.integer({ min: 0, max: 10 }).default(1),
});

Types ride in the value's type parameters instead of being recomputed from literals, so a plain const keeps its type and schemas are reusable. Error messages name the offending field rather than printing a deferred conditional-type alias. Recursion goes through asn.ref<T>(name) + asn.compile; metadata trees stay typed.

SEQUENCE field order is object key order, so the builder rejects integer-like keys — engines hoist those, which would silently reorder the encoding rather than fail.

3. Code generation

npx asn1-per-ts types ticket.asn src/generated/ticket.ts

.asn or SchemaNode JSON to a module of named interfaces plus typed codecs. The parser resolves type references by inlining them, so the generator matches those expanded structures back to the types they came from — without that the output repeats whole structures and loses the point of naming them. ASN.1 hyphens are handled both ways: type names become PascalCase (collisions numbered, shadowing avoided), field names stay verbatim and quoted. Each type gets an Input variant where a DEFAULT is reachable, computed as a fixpoint so $ref cycles terminate.

On a 150-type schema, generated code costs ~20k type instantiations against ~109k for the equivalent inline literal.

The package now installs an asn1-per-ts binary with types and schema subcommands, replacing the cli/*.ts scripts that could only be run from a checkout. Argument handling sits behind a CliIo interface so the commands are tested with an in-memory filesystem.

Breaking changes (types)

Recorded in the new CHANGELOG.md under [Unreleased]. These warrant at least a minor bump, arguably major:

  • RawBytes moved to codec.raw. The default encode signatures no longer accept it. Folding RawBytes into every node doubled the length of every type printed in an encode error, for a feature most values never use. The runtime is unchanged and still accepts RawBytes anywhere.
  • encode() now rejects values that do not fit. The point of the change, but code that compiled against the old unknown signature may need fixing — usually an intermediate const widened a literal, which annotating with Infer<typeof schema> solves.
  • SchemaNode's collections are readonly, so inline schemas keep their literal types. Code that mutated fields, alternatives or values after construction no longer compiles.
  • cli/generate-schema.ts and cli/generate-types.ts removed in favour of the bin; from a checkout, npx tsx src/cli/main.ts <subcommand>.

A schema known only at runtime — parsed from JSON or ASN.1 text — stays the wide SchemaNode union and keeps the unknown-typed API it had before.

Docs

New guides examples/typed-api.md, examples/dsl.md and examples/codegen.md; examples/schema-parser.md reworked. A new CHANGELOG.md (there was none) with PUBLISH.md wired to it.

Several doc errors fixed along the way, some predating this branch: the CHOICE examples in encoding.md/decoding.md documented a { name: value } shape the codecs stopped producing, with hex that threw when decoded; schema-parser.md claimed cross-referenced types produce $ref when in fact only cycles do; the README omitted OBJECT IDENTIFIER entirely and its metadata example cast away the typing this PR adds.

Verification

587 tests across 33 suites. tsc --noEmit on library, CLI and website; build clean.

  • Type-level assertions use a strict Equal<X, Y>, not extends, so any/unknown cannot make one pass vacuously; @ts-expect-error cases prove wrong values are rejected.
  • tests/fixtures/generated/sampleModule.ts is checked-in generator output, imported by a test so ts-jest compiles it, with an assertion that the generator still reproduces it byte for byte.
  • The bin was verified against a real install — npm pack, install the tarball into an empty project, run the linked binary, then compile and run the generated module against the packaged runtime under strict + noUnusedLocals + NodeNext.
  • Every runnable snippet in the new and edited guides was compiled; the parser outputs and hex strings quoted in schema-parser.md come from running the parser.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS

claude added 5 commits August 22, 2026 20:08
decode() returned unknown and encode() took unknown, so every caller had to
write an interface by hand and cast. The schema already describes the shape;
this makes TypeScript read it.

SchemaCodec, SchemaBuilder.build and the new createCodec/createCodecs take the
schema as a `const` type parameter, so an inline schema keeps its literal types.
Three conditional types in src/schema/Infer.ts walk those literals:

- Infer<S>        what decode() returns
- InferInput<S>   what encode() accepts (DEFAULT fields omittable, RawBytes
                  allowed at any node)
- InferMetadata<S> the DecodedNode tree, and stripMetadata() now derives its
                  return type from the node type it is given

ENUMERATED narrows to a literal union, CHOICE becomes a discriminated union on
`key`, OPTIONAL fields become optional properties while DEFAULT fields stay
required after decoding, and $ref resolves through the registry passed to
createCodecs (bounded expansion, so recursive schemas terminate).

This is compile-time only: no validation runs, no dependency is added, and the
emitted JavaScript is unchanged. A schema typed as the wide SchemaNode union —
parsed from JSON or from ASN.1 text — degrades to exactly the unknown-typed API
that existed before.

Supporting changes:
- SchemaNode moves to src/schema/SchemaNode.ts with readonly collections, so
  inline and `as const` schemas are assignable without widening
- Codec gains a TInput parameter, defaulting to T
- SchemaBuilder's two near-identical builders collapse into one buildNode()
  parameterised by how $ref is resolved
- examples/typed-api.md documents the mapping and the pitfalls; the CHOICE
  examples in encoding.md/decoding.md were describing a `{ name: value }` shape
  the codecs stopped producing, and now show `{ key, value }` with working hex

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
The literal-inference API shipped in the previous commit has two weaknesses
that are structural rather than polish: it depends on schemas and values
keeping their literal types, and its errors print deferred conditional-type
aliases (`InferInputChoice<{ readonly type: "CHOICE"; ... }, Record<...>, 12>`)
instead of resolved types. It also gives nothing to a schema parsed from a
.asn file at runtime, which is the library's headline workflow.

Rather than replace it, add two more front-ends over the same interchange
format. SchemaNode is that format: the ASN.1 parser, the DSL and hand-written
literals all produce it; SchemaBuilder and the generator both consume it. All
three now hand back the same TypedCodec<TOut, TIn, TNode>.

asn DSL (src/dsl/) — schemas built from functions, so the types travel in the
value's type parameters instead of being recomputed from literals:

  const Ticket = asn.sequence({
    id: asn.integer({ min: 0, max: 255 }),
    status: asn.enumerated(['pending', 'approved']),
    version: asn.integer({ min: 0, max: 10 }).default(1),
  });

A plain `const` keeps its type, schemas are reusable, and errors name the
offending field. Recursion goes through asn.ref<T>(name) + asn.compile, and
metadata trees stay typed. SEQUENCE field order is object key order, so the
builder rejects integer-like keys — the engine hoists those, which would
silently reorder the encoding rather than fail.

Codegen (src/codegen/, cli/generate-types.ts) — .asn or SchemaNode JSON to a
TypeScript module of named interfaces plus typed codecs. The parser resolves
type references by inlining them, so the generator matches those expanded
structures back to the types they came from; without that the output repeats
whole structures and loses the point of naming them. ASN.1 hyphens are handled
both ways: type names become PascalCase (with collisions numbered and shadowing
avoided), field names stay verbatim and quoted. Each type gets an Input variant
where a DEFAULT is reachable, computed as a fixpoint so $ref cycles terminate.

On a 150-type schema, generated code costs ~20k type instantiations against
~109k for the equivalent inline literal, and errors name the type.

RawBytes: InferInput no longer folds `RawBytes |` into every node. Passthrough
moves to `codec.raw`, which exposes the three encode entry points with every
node widened. The old shape doubled the length of every encode error message
for a feature most values never use.

tests/fixtures/generated/sampleModule.ts is checked-in generator output: a test
asserts the generator still reproduces it byte for byte and round-trips values
through it, and ts-jest compiles it, so generated code cannot silently stop
compiling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
The two scripts under cli/ were only runnable from a checkout via `npx tsx`,
because the build compiles src/ alone and the package ships dist/. Move them
into src/cli/ so the existing build emits them, and expose one binary with a
subcommand each:

  npx asn1-per-ts types  ticket.asn src/generated/ticket.ts
  npx asn1-per-ts schema ticket.asn ticket.schema.json

The argument handling lives in src/cli/cli.ts behind a CliIo interface, so the
commands are driven by an in-memory IO in tests; src/cli/main.ts is the only
part that touches fs and process. CliIo deliberately uses no Node types, to
keep them out of the published declarations.

Behaviour the old scripts did not have: data goes to stdout and progress to
stderr, so redirecting produces a clean file; failures print a message and exit
1 rather than throwing a stack trace; `--no-runtime` emits types without the
schemas and codecs values.

That last flag exposed a bug worth fixing anyway — the generator emitted its
type imports unconditionally, so a types-only module imported SchemaRegistry
and TypedCodec without using them and failed under noUnusedLocals. Imports are
now emitted only for what the output references.

sideEffects narrows from false to the CLI entry, which does run on import; the
library stays tree-shakeable.

Verified against a real install rather than the checkout: npm pack, install the
tarball into an empty project, run the linked binary, then compile and run the
generated module against the packaged runtime under strict + noUnusedLocals +
NodeNext.

There is no --version flag: reading package.json at runtime needs import.meta
or createRequire, and the root tsconfig type-checks as commonjs where
import.meta is an error. Not worth restructuring for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
Answering "are the docs current": the README had drifted and there was no
changelog at all — releases relied on GitHub's generated notes, which do not
distinguish a new export from a change that stops existing code compiling. The
last three commits contain three such type-level breaks, so they need somewhere
to be stated.

CHANGELOG.md starts at Unreleased and points at the GitHub releases for v1.0.2
to v1.4.0; reconstructing those from git history would have been guesswork. It
records the typed API, the DSL, the code generator and the bin, and calls out the
breaking items: RawBytes moving to codec.raw, encode() now rejecting values that
do not fit, SchemaNode's collections becoming readonly, and the removal of the
cli/ scripts. PUBLISH.md gains a first step to fold Unreleased into a version,
and notes that the Breaking entries decide major versus minor.

README corrections, the first two of which predate this branch:

- OBJECT IDENTIFIER was missing from the feature list and the supported-types
  table, though the codec has always existed.
- RawBytes passthrough was never mentioned at all.
- The metadata example cast node.value to Record<string, DecodedNode>. That cast
  became unnecessary when the tree became typed, and following it throws the
  types away; it now shows node.value.id.value directly.
- The three-front-ends snippet used `asn` without importing it.
- The Examples section listed only the three original guides, not typed-api,
  dsl or codegen.
- Development commands omitted the type-check and running the CLI from source.

CLAUDE.md: src/cli/ was nested under the examples/ sub-list by a bad
search-and-replace in the previous commit, and the schemas/ entry described a
directory deleted in 5cc15dd.

Every runnable README snippet is now compiled in a scratch file, including the
@ts-expect-error on the ENUMERATED example, so the claims are checked rather
than asserted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
The guide is the entry point for anyone with a .asn file, and it dead-ended at
an untyped SchemaCodec without ever saying the types were missing or that
codegen exists. It also documented the converter's reference handling
backwards.

- A "which path do you want" table up front: parse at runtime for schemas
  chosen at runtime, generate TypeScript when the .asn file is checked in. The
  encode/decode example now says plainly that `decoded` is `unknown` to
  TypeScript, and gives the three ways to get a type back.
- "Schemas with $ref (recursive or cross-referenced types)" was wrong: a
  cross-reference is *inlined*, and only a cycle produces a $ref. Verified
  against the parser — `Status` referenced from `Request` comes back fully
  expanded with no $ref anywhere. Split into two sections, one for inlining
  (with its consequences: a registry larger than its source, and why the
  generator de-inlines) and one for genuine recursion.
- The recursion example used SchemaBuilder.buildAll and the low-level
  Codec/BitBuffer API. It now uses createCodecs, notes that
  `new SchemaCodec(schemas.Tree)` throws for a $ref schema, and mentions
  buildAll as the lower-level alternative.
- Noted that constraints are runtime-only and never reach the types, and that
  OPTIONAL and DEFAULT are asymmetric once types are involved.
- "Building SchemaNode Manually" became "Writing schemas without the parser",
  making the point that an inline schema does carry types — and warning that
  annotating it `: SchemaNode` throws them away again.
- Added the limits of the parser subset with a pointer to TODO.md; corrected
  the SchemaNode location to src/schema/SchemaNode.ts; added SchemaNode.ts and
  the generator to the related-files table.

Claims checked rather than asserted: every JSON output and the '002a40' hex
come from running the parser, and every TypeScript snippet compiles.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPw4YEzoKtskJR9n33a1yS
@chtitux chtitux changed the title Infer encode/decode types from the schema Typed encode/decode: inferred types, the asn DSL, and ASN.1 code generation Sep 29, 2026
@chtitux
chtitux merged commit 96fc076 into main Sep 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants