Repository navigation
Extract shared response signing conformance fixtures - #64
leoromanovsky wants to merge 3 commits into
Conversation
Preserve the existing transcript and certificate fixture bytes for native signer and SDK consumers, with a schema, integrity manifest and validation.
Regenerate synthetic signatures for the public v1 transcript and reject abandoned prototype versions without changing certificate encoding.
Use public v1 in the combined-header case and bind the trailing-DER proof to its actual certificate bytes. Reject duplicate JSON keys with CI regression coverage and clarify application bytes, transport bytes, and clock units. Environment: Datadog workspace
| ], | ||
| [ | ||
| "x-datadog-feature-flags-signing-certificate", | ||
| "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=" |
There was a problem hiding this comment.
🤖 Comment from Codex working with @vjfridge:
[P2] Make the certificate-size and CA-flag cases isolate the rule they test
These three rejection cases also break other certificate rules, so they still pass when the intended check is missing. In the browser verifier, all 48 existing response cases passed after raising the certificate-size limit or removing either CA-flag check individually. Could we add otherwise-valid, correctly signed cases that fail only the intended rule? This is a coverage improvement; the tests did not demonstrate a bug in the unchanged verifier.
Affected cases, reproduction results, and suggested fix
Why the current cases do not isolate the check
| Case | Other reasons it can still be rejected |
|---|---|
oversized-certificate |
The certificate is 4,097 zero bytes, so it is invalid DER, and its certificate ID does not match those bytes. Relaxing the size limit still leaves these rejection paths. |
leaf-ca-true |
It changes the CA flag, path-length constraint, and KeyUsage together. The latter two still reject it when the leaf CA check is removed. |
intermediate-ca-false |
It also changes the CA flag, path-length constraint, and KeyUsage together, masking removal of the intermediate CA check. |
Reproduction
The fixture revision was 7caa9706dcab035854a34ea41311f5eeb6ce6743. The executed consumer was the browser verifier from the browser response-signing PR at 0509003f57fc5085c167e9023fdf92abe1c25118, using a focused Node/WebCrypto harness with the pinned ASN.1 dependencies. This was not a run of the full browser package suite.
The harness loaded all 46 cases from corpus.json and both cases from nested-der.json, preserving the fixture clock, body bytes, header arrays, and trust anchor. It compared actual acceptance with expected_accept and checked the transcript bytes for accepted cases. The unmodified verifier passed 48/48 cases and all seven accepted transcript comparisons.
Each mutation below was applied separately to the original verifier, with every other check retained:
| Mutation | Existing 48 cases | Independently generated four-case supplement |
|---|---|---|
Raise MAX_CERTIFICATE from 4096 to 8192 |
48/48 pass | 3/4 pass: the isolated oversized-certificate case is incorrectly accepted |
Change !constraints.cA && constraints.pathLenConstraint === undefined to constraints.pathLenConstraint === undefined |
48/48 pass | 3/4 pass: the isolated leaf-CA case is incorrectly accepted |
Change constraints.cA && constraints.pathLenConstraint === 0 to constraints.pathLenConstraint === 0 |
48/48 pass | 3/4 pass: the isolated intermediate-CA case is incorrectly accepted |
As a harness control, disabling response-signature verification failed ten existing cases; removing the intermediate path-length check failed its named case. The harness therefore detected other relevant regressions.
Suggested fixture change
Add one accepted control and three rejected cases from a synthetic chain:
- Valid control: A valid leaf/intermediate chain and correctly signed response. This confirms that the added chain and transcript are usable.
- Oversized certificate: A correctly signed, well-formed leaf above the 4,096-byte limit, with all other profile constraints valid. The tested candidate was 4,169 bytes, using a noncritical extension for padding.
- Leaf CA flag: Set only the leaf's CA flag to true. Keep its path-length constraint absent and retain its leaf KeyUsage.
- Intermediate CA flag: Set only the intermediate's CA flag to false. Keep its path-length constraint at zero and retain its intermediate KeyUsage. This deliberately violates the CA profile; preserve valid DER and valid signatures rather than substituting a leaf-profile certificate.
For each changed certificate, regenerate its certificate ID, transcript, and response signature from the actual bytes. Keep expected outcomes independent of the implementation being tested. The tested generator independently verified the certificate and response signatures, including those on the deliberately invalid profiles.
Either regenerate the existing cases with the original fixture generator, or add these as a supplement. For a supplement, update the manifest/hash/count validation and explicitly load it in the consumer adapters: the current browser adapter names the two existing fixture files, so adding a manifest entry alone does not execute the new cases. Refresh consumer fixture pins or snapshots after the fixture change.
Validation of the proposed approach
The four generated cases passed 4/4 with the original browser verifier. Each targeted mutation failed exactly its corresponding new negative case. Restoring the verifier passed all 52 combined cases and all eight accepted transcript comparisons. The fixture-side integration also passed structural validation and all 19 repository validator tests.
This execution evidence is specific to the browser verifier. The proposed supplement still needs to be run through the native signer and mobile adapters; no equivalent regression behavior is claimed for those implementations.
Motivation
Response signers and client SDKs need to agree on the same bytes, certificate rules and acceptance decisions. Keeping separate copies makes it easy for one implementation to accept a response another rejects, especially during leaf or intermediate rotation.
Changes
Extract the shared public protocol v1 fixtures into
response-signing/v1, with a JSON Schema, SHA-256 manifest, consumer guide and structural-validation workflow. The main corpus contains 46 response cases (six accepted, 40 rejected) and two signer key-matching cases. A separate two-case supplement covers canonical and noncanonical nested DER.The comma-combined header case uses
1, 1, and the trailing-DER case has a matching certificate hash and valid response signature so each rejection exercises its intended rule. The validator rejects duplicate JSON object keys, with CI regressions for the schema, manifest and both corpora. HTTP header duplicates remain represented as arrays. The consumer guide specifies application bytes after HTTP transfer/content decoding and a Unix-seconds clock.The diagram describes test-time consumption. Browser tests pin a submodule; native consumers can pin an archive or import an exact snapshot with provenance. Cryptography, transport and lifecycle adapters remain in their implementation repositories. No fixture download is added to assignment requests.
Decisions
Version 1 is the first public wire contract. Earlier prototype numbers are not compatibility promises; X.509 v3 certificate encoding remains unchanged. Synthetic chains and signatures were regenerated for v1, preserving the original acceptance decisions and adding rejection cases for prototype version headers and the abandoned transcript domain.
All certificates, identities, tokens and included leaf private keys are intentionally public synthetic test material. They must never become deployed credentials or runtime trust anchors. Accepted fixture responses carry the complete leaf/intermediate chain, allowing an unchanged client root to verify later rotations without another request. The corpus covers buffered response proofs; signed NDJSON checkpoints are outside this change.
Consumers must preserve exact body bytes, header order/duplicates and the fixture clock in Unix seconds. Schema validation checks structure and integrity; native tests establish cryptographic behavior. Implementation-specific diagnostics and the additional SDK KeyUsage tests remain downstream.
Merge this PR before consumers finalize their pins. Browser verifier #435 already pins
7caa9706dcab035854a34ea41311f5eeb6ce6743; iOS #3357 and Android #4049 carry matching copies of the main corpus. If merging changes the commit identity, refresh consumer pins and provenance.Checklist
config/ufc-config.json— not applicable to the separate signing corpus.evaluation-cases/— no evaluator flags added.