feat: add relay conform --manifest (relay-conform/1), spec §17.2, Requirement 15 - #180
Merged
Merged
Conversation
…uirement 15 - New §17 Requirement 15: implementations MUST commit a relay-conform/1 manifest generated by `relay conform --manifest`; CI MUST regenerate and fail the build on any diff or any FAIL requirement entry. - New §17.2 defines the manifest JSON shape and a 4-value status vocabulary (PASS/FAIL/SHAPE_ONLY/NOT_OBSERVABLE), deliberately distinct from relay conform's existing PASS/WARN/FAIL finding severities: a manifest MUST NOT report PASS for a requirement relay conform cannot actually observe. - New optional `commit` field on the §12.1 version document (backward compatible) so a manifest can populate its own git_sha. - New §20.1 CI gate for manifest regeneration/diffing. - New embedded schema spec/schemas/relay-conform-manifest.json. - Reference implementation: --manifest flag + buildManifest() in cmd/relay/conform.go. Only Requirements 1, 6, 7, 12 get real PASS/FAIL/SHAPE_ONLY statuses (mirroring what relay conform can actually observe black-box); everything else, including Requirement 15's own entry, is honestly NOT_OBSERVABLE. - New REQ-RELAY-095 (not a reuse of REQ-RELAY-052 — this is materially new behavior: new output format, new status vocabulary, new artifact). 7 new tests cover schema validity, capabilities_sha256 correctness, NOT_OBSERVABLE defaults, FAIL propagation, and CLI exit codes. - SpecVersion bumped 2.2 -> 2.3 (MINOR, matching the Requirement 13/14 precedent: a new numbered §17 requirement is MINOR-worthy even where relay conform cannot fully observe it). Closes #139. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements issue #139 (NEW-SPEC-1): turns
relay conform's self-reported conformance into a machine-checkable, committable artifact instead of only an on-demand CLI report.What's added
relay-conform/1manifest (§17.2), generated byrelay conform --manifestagainst its own built binary; CI MUST regenerate the manifest on every change and fail the build on any diff from the committed copy, or on any requirement entry whosestatusisFAIL.kind,manifest_version,tool,binary_version,spec_version,git_sha,capabilities_sha256,requirements[],overall) and a 4-value status vocabulary —PASS/FAIL/SHAPE_ONLY/NOT_OBSERVABLE— deliberately distinct fromrelay conform's existingPASS/WARN/FAILfinding severities. Core rule: a manifest MUST NOT reportPASSfor a requirementrelay conformcannot actually observe.commitfield on the §12.1 version document (backward-compatible — not in the schema'srequiredarray) so an implementation can populate the manifest'sgit_sha.FAILentry.spec/schemas/relay-conform-manifest.json.--manifestflag +buildManifest()incmd/relay/conform.go. Only Requirements 1, 6, 7, 12 get realPASS/FAIL/SHAPE_ONLYstatuses, mirroring §17's existing narrative of whatrelay conformcan observe black-box. Everything else — including Requirement 15's own manifest-generation entry — is honestlyNOT_OBSERVABLE, per this PR's own new rule (no self-congratulatory PASS).REQ-RELAY-095in.fusa-reqs.json— a fresh ID rather than reusingREQ-RELAY-052, since--manifestis materially new behavior (new output format, new status vocabulary, new artifact type), not the base findings-report path.Version
SpecVersionbumped2.2→2.3(MINOR), matching the established precedent from Requirements 13 (v2.1) and 14 (v2.2): a new numbered §17 requirement is MINOR-worthy even whererelay conformcannot fully observe it.Verification
cmd/relay/conform_test.go: manifest schema validity (against the new embedded schema),capabilities_sha256independently recomputed and compared, all-15-requirements-present with correctNOT_OBSERVABLEdefaults,FAILpropagation tooverall, and CLI-level exit codes for both--manifestsuccess and failure paths.go build ./... && go vet ./... && go test ./...— all packages green.gofusa check(pinned v0.48.0, matching CI): 0 errors.gofusa trace -req-coverage 100: 100% (95/95 requirements traced).capabilities_sha256byte-for-byte via independentsha256sumcomparison against a locally built binary.Closes #139.