Skip to content

Extract shared response signing conformance fixtures - #64

Open
leoromanovsky wants to merge 3 commits into
mainfrom
leo.romanovsky/response-signing-fixtures-20261102
Open

leoromanovsky wants to merge 3 commits into
mainfrom
leo.romanovsky/response-signing-fixtures-20261102

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Response signers and client SDKs need to agree on the same bytes, certificate rules and acceptance decisions. Keeping separate copies makes it easy for one implementation to accept a response another rejects, especially during leaf or intermediate rotation.

Changes

Extract the shared public protocol v1 fixtures into response-signing/v1, with a JSON Schema, SHA-256 manifest, consumer guide and structural-validation workflow. The main corpus contains 46 response cases (six accepted, 40 rejected) and two signer key-matching cases. A separate two-case supplement covers canonical and noncanonical nested DER.

The comma-combined header case uses 1, 1, and the trailing-DER case has a matching certificate hash and valid response signature so each rejection exercises its intended rule. The validator rejects duplicate JSON object keys, with CI regressions for the schema, manifest and both corpora. HTTP header duplicates remain represented as arrays. The consumer guide specifies application bytes after HTTP transfer/content decoding and a Unix-seconds clock.

flowchart TD
  Fixtures[Shared v1 JSON fixtures and expected results] --> Structure[Schema and manifest validation]
  Fixtures --> Revision[Exact repository revision]
  Revision --> Signer[Native signer tests]
  Revision --> Browser[Browser verifier tests]
  Revision --> Mobile[iOS and Android verifier tests]
  Signer --> Agreement[Assert shared acceptance decisions]
  Browser --> Agreement
  Mobile --> Agreement
Loading

The diagram describes test-time consumption. Browser tests pin a submodule; native consumers can pin an archive or import an exact snapshot with provenance. Cryptography, transport and lifecycle adapters remain in their implementation repositories. No fixture download is added to assignment requests.

Decisions

Version 1 is the first public wire contract. Earlier prototype numbers are not compatibility promises; X.509 v3 certificate encoding remains unchanged. Synthetic chains and signatures were regenerated for v1, preserving the original acceptance decisions and adding rejection cases for prototype version headers and the abandoned transcript domain.

All certificates, identities, tokens and included leaf private keys are intentionally public synthetic test material. They must never become deployed credentials or runtime trust anchors. Accepted fixture responses carry the complete leaf/intermediate chain, allowing an unchanged client root to verify later rotations without another request. The corpus covers buffered response proofs; signed NDJSON checkpoints are outside this change.

Consumers must preserve exact body bytes, header order/duplicates and the fixture clock in Unix seconds. Schema validation checks structure and integrity; native tests establish cryptographic behavior. Implementation-specific diagnostics and the additional SDK KeyUsage tests remain downstream.

Merge this PR before consumers finalize their pins. Browser verifier #435 already pins 7caa9706dcab035854a34ea41311f5eeb6ce6743; iOS #3357 and Android #4049 carry matching copies of the main corpus. If merging changes the commit identity, refresh consumer pins and provenance.

Checklist

  • JSON files are valid (no syntax errors); the signing validator checks 48 response cases and two signer cases; all 19 repository validator tests pass, including five duplicate-key mutation subcases.
  • New test cases reference existing flags in config/ufc-config.json — not applicable to the separate signing corpus.
  • New flags have corresponding test cases in evaluation-cases/ — no evaluator flags added.
  • Expected results match the flag configuration logic — evaluator expectations are unchanged; signing expectations were checked against native implementations. Structural validation alone does not verify signatures.

Preserve the existing transcript and certificate fixture bytes for native
signer and SDK consumers, with a schema, integrity manifest and validation.
Regenerate synthetic signatures for the public v1 transcript and reject
abandoned prototype versions without changing certificate encoding.
@leoromanovsky leoromanovsky added the crucible: reviewing A Crucible review has started; this PR has reviewer attention. label Nov 2, 2026
@leoromanovsky
leoromanovsky marked this pull request as ready for review November 3, 2026 02:42
@leoromanovsky
leoromanovsky requested a review from a team as a code owner November 3, 2026 02:42
Use public v1 in the combined-header case and bind the trailing-DER proof to its actual certificate bytes. Reject duplicate JSON keys with CI regression coverage and clarify application bytes, transport bytes, and clock units.

Environment: Datadog workspace
],
[
"x-datadog-feature-flags-signing-certificate",
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Comment from Codex working with @vjfridge:

[P2] Make the certificate-size and CA-flag cases isolate the rule they test

These three rejection cases also break other certificate rules, so they still pass when the intended check is missing. In the browser verifier, all 48 existing response cases passed after raising the certificate-size limit or removing either CA-flag check individually. Could we add otherwise-valid, correctly signed cases that fail only the intended rule? This is a coverage improvement; the tests did not demonstrate a bug in the unchanged verifier.

Affected cases, reproduction results, and suggested fix

Why the current cases do not isolate the check

Case Other reasons it can still be rejected
oversized-certificate The certificate is 4,097 zero bytes, so it is invalid DER, and its certificate ID does not match those bytes. Relaxing the size limit still leaves these rejection paths.
leaf-ca-true It changes the CA flag, path-length constraint, and KeyUsage together. The latter two still reject it when the leaf CA check is removed.
intermediate-ca-false It also changes the CA flag, path-length constraint, and KeyUsage together, masking removal of the intermediate CA check.

Reproduction

The fixture revision was 7caa9706dcab035854a34ea41311f5eeb6ce6743. The executed consumer was the browser verifier from the browser response-signing PR at 0509003f57fc5085c167e9023fdf92abe1c25118, using a focused Node/WebCrypto harness with the pinned ASN.1 dependencies. This was not a run of the full browser package suite.

The harness loaded all 46 cases from corpus.json and both cases from nested-der.json, preserving the fixture clock, body bytes, header arrays, and trust anchor. It compared actual acceptance with expected_accept and checked the transcript bytes for accepted cases. The unmodified verifier passed 48/48 cases and all seven accepted transcript comparisons.

Each mutation below was applied separately to the original verifier, with every other check retained:

Mutation Existing 48 cases Independently generated four-case supplement
Raise MAX_CERTIFICATE from 4096 to 8192 48/48 pass 3/4 pass: the isolated oversized-certificate case is incorrectly accepted
Change !constraints.cA && constraints.pathLenConstraint === undefined to constraints.pathLenConstraint === undefined 48/48 pass 3/4 pass: the isolated leaf-CA case is incorrectly accepted
Change constraints.cA && constraints.pathLenConstraint === 0 to constraints.pathLenConstraint === 0 48/48 pass 3/4 pass: the isolated intermediate-CA case is incorrectly accepted

As a harness control, disabling response-signature verification failed ten existing cases; removing the intermediate path-length check failed its named case. The harness therefore detected other relevant regressions.

Suggested fixture change

Add one accepted control and three rejected cases from a synthetic chain:

  1. Valid control: A valid leaf/intermediate chain and correctly signed response. This confirms that the added chain and transcript are usable.
  2. Oversized certificate: A correctly signed, well-formed leaf above the 4,096-byte limit, with all other profile constraints valid. The tested candidate was 4,169 bytes, using a noncritical extension for padding.
  3. Leaf CA flag: Set only the leaf's CA flag to true. Keep its path-length constraint absent and retain its leaf KeyUsage.
  4. Intermediate CA flag: Set only the intermediate's CA flag to false. Keep its path-length constraint at zero and retain its intermediate KeyUsage. This deliberately violates the CA profile; preserve valid DER and valid signatures rather than substituting a leaf-profile certificate.

For each changed certificate, regenerate its certificate ID, transcript, and response signature from the actual bytes. Keep expected outcomes independent of the implementation being tested. The tested generator independently verified the certificate and response signatures, including those on the deliberately invalid profiles.

Either regenerate the existing cases with the original fixture generator, or add these as a supplement. For a supplement, update the manifest/hash/count validation and explicitly load it in the consumer adapters: the current browser adapter names the two existing fixture files, so adding a manifest entry alone does not execute the new cases. Refresh consumer fixture pins or snapshots after the fixture change.

Validation of the proposed approach

The four generated cases passed 4/4 with the original browser verifier. Each targeted mutation failed exactly its corresponding new negative case. Restoring the verifier passed all 52 combined cases and all eight accepted transcript comparisons. The fixture-side integration also passed structural validation and all 19 repository validator tests.

This execution evidence is specific to the browser verifier. The proposed supplement still needs to be run through the native signer and mobile adapters; no equivalent regression behavior is claimed for those implementations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

crucible: reviewing A Crucible review has started; this PR has reviewer attention.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants