Emit 0.2 build manifests with supplementary file support, plan-driven archive assembly, and filesystem-identity validation

openspec/changes/support-non-asset-files/tasks.md

@@ -99,20 +99,20 @@
 
 ## 10. Current Producer and Build Pipeline — Research
 
-- [ ] 10.1 Explore: Trace source-file loading, build validation stages, archive assembly, output cleanup, and build-result rendering
-- [ ] 10.2 Explore: Inspect source filesystem APIs needed to reject missing files, links, resolved aliases, and non-regular declarations before output mutation
-- [ ] 10.3 Explore: Inspect Changesets and CLI packaging to confirm the complete `0.2` producer may be implemented and merged without publishing `agent-facets`, while protocol, registry, adapter, and final CLI activation remain independently controlled release gates
-- [ ] 10.4 Propose: Define the producer implementation that reuses the archive plan, preserves deterministic bytes, validates before cleanup, emits only current-format output in the unreleased candidate, and requires no long-lived runtime dual-format flag
+- [x] 10.1 Explore: Trace source-file loading, build validation stages, archive assembly, output cleanup, and build-result rendering
+- [x] 10.2 Explore: Inspect source filesystem APIs needed to reject missing files, links, resolved aliases, and non-regular declarations before output mutation
+- [x] 10.3 Explore: Inspect Changesets and CLI packaging to confirm the complete `0.2` producer may be implemented and merged without publishing `agent-facets`, while protocol, registry, adapter, and final CLI activation remain independently controlled release gates
+- [x] 10.4 Propose: Define the producer implementation that reuses the archive plan, preserves deterministic bytes, validates before cleanup, emits only current-format output in the unreleased candidate, and requires no long-lived runtime dual-format flag
 
 ## 11. Current Producer and Build Pipeline — Implementation
 
-- [ ] 11.1 Implement: Load declared supplementary files as exact bytes, validate their resolved regular-file identities, and preserve previous `dist/` output on every input failure
-- [ ] 11.2 Implement: Drive archive collection and all-entry hashing from the shared archive plan, preserving deterministic ordering and opaque binary or empty supplementary content
-- [ ] 11.3 Implement: Switch every build in the unreleased source candidate, including asset-only facets, to flat build-manifest `0.2` output with a complete `files` map while retaining legacy consumer support
-- [ ] 11.4 Implement: Update build results and CLI output to show the emitted format, complete entry listing, integrity, and archive-assembly stage
-- [ ] 11.5 Implement: Add the build failure-class matrix for traversal, absolute/drive/URL prefixes, backslashes, NUL, empty/`.`/`..` segments, Unicode-normalization and portable-case aliases, Windows-reserved device names, forbidden portable characters, trailing dot/space segments, file/directory prefix collisions, symlinks, hard links, duplicate paths, reserved root `facet.json`, conventional-primary-path collisions, missing declarations, undeclared entries, and tampered bytes, plus success tests for top-level files, nested companions, binary/empty bytes, exact manifest-byte hashing, canonical-tar determinism, scoped output paths, and validation-before-cleanup
-- [ ] 11.6 Implement: Add a reproducible candidate archive/interop path that can produce a representative `0.2` artifact for registry stage acceptance without publishing or releasing the CLI
-- [ ] 11.7 Verify: Run focused build pipeline and CLI build tests and inspect representative `0.2` archives for exact deterministic membership while confirming no `agent-facets` release changeset is present
+- [x] 11.1 Implement: Load declared supplementary files as exact bytes, validate their resolved regular-file identities, and preserve previous `dist/` output on every input failure
+- [x] 11.2 Implement: Drive archive collection and all-entry hashing from the shared archive plan, preserving deterministic ordering and opaque binary or empty supplementary content
+- [x] 11.3 Implement: Switch every build in the unreleased source candidate, including asset-only facets, to flat build-manifest `0.2` output with a complete `files` map while retaining legacy consumer support
+- [x] 11.4 Implement: Update build results and CLI output to show the emitted format, complete entry listing, integrity, and archive-assembly stage
+- [x] 11.5 Implement: Add the build failure-class matrix for traversal, absolute/drive/URL prefixes, backslashes, NUL, empty/`.`/`..` segments, Unicode-normalization and portable-case aliases, Windows-reserved device names, forbidden portable characters, trailing dot/space segments, file/directory prefix collisions, symlinks, hard links, duplicate paths, reserved root `facet.json`, conventional-primary-path collisions, missing declarations, undeclared entries, and tampered bytes, plus success tests for top-level files, nested companions, binary/empty bytes, exact manifest-byte hashing, canonical-tar determinism, scoped output paths, and validation-before-cleanup
+- [x] 11.6 Implement: Add a reproducible candidate archive/interop path that can produce a representative `0.2` artifact for registry stage acceptance without publishing or releasing the CLI
+- [x] 11.7 Verify: Run focused build pipeline and CLI build tests and inspect representative `0.2` archives for exact deterministic membership while confirming no `agent-facets` release changeset is present
 
 ## 12. Create and Edit Authoring — Research
 

packages/cli/src/__tests__/candidate-archive.e2e.test.ts

@@ -0,0 +1,148 @@
+import { afterAll, beforeAll, describe, expect, test } from 'bun:test'
+import { existsSync } from 'node:fs'
+import { mkdtemp, rm } from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import { join, resolve } from 'node:path'
+import { gunzipSync } from 'node:zlib'
+import { type GunzipFn, parseFacetArchive, validateFacetArchive } from '@agent-facets/protocol'
+
+/**
+ * Reproducible candidate `0.2` archive / interop path (task 11.6).
+ *
+ * Builds a representative facet — a skill with a text companion and a binary
+ * companion, plus an archive-only README — using the freshly compiled
+ * candidate CLI (`dist/facet`, produced by `test:e2e`), then verifies the
+ * emitted `.facet` through the SAME protocol verifier a registry stage uses
+ * (`validateFacetArchive`). This proves the candidate producer emits a
+ * verifiable `0.2` artifact with exact deterministic membership, WITHOUT
+ * publishing or releasing the CLI — no Changeset is involved, and the binary
+ * is the local compile.
+ *
+ * A registry stage acceptance run can reproduce this exact archive by running
+ * `facet build` on the same source tree; the deterministic tar layout makes
+ * the bytes stable across machines.
+ */
+
+let testDir: string
+
+beforeAll(async () => {
+  testDir = await mkdtemp(join(tmpdir(), 'cli-candidate-archive-'))
+})
+
+afterAll(async () => {
+  await rm(testDir, { recursive: true, force: true })
+})
+
+const CLI_PATH = resolve(import.meta.dir, '../../dist/facet')
+
+if (!existsSync(CLI_PATH)) {
+  throw new Error(`[e2e] dist/facet not found at ${CLI_PATH}.\nBuild the CLI first: bun run --cwd packages/cli build`)
+}
+
+async function runCli(cwd: string, ...args: string[]) {
+  const facetDir = await mkdtemp(join(testDir, 'facet-dir-'))
+  const proc = Bun.spawn([CLI_PATH, ...args], {
+    cwd,
+    stdout: 'pipe',
+    stderr: 'pipe',
+    env: { ...process.env, NO_COLOR: '1', FACET_DIR: facetDir },
+  })
+  const stdout = await new Response(proc.stdout).text()
+  const stderr = await new Response(proc.stderr).text()
+  const exitCode = await proc.exited
+  return { stdout: stdout.trim(), stderr: stderr.trim(), exitCode }
+}
+
+const REPRESENTATIVE_BINARY = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x00, 0x01, 0xff, 0xfe])
+
+/** Write the representative facet source tree into `dir`. */
+async function writeRepresentativeFacet(dir: string): Promise<void> {
+  await Bun.write(join(dir, 'skills/planning/SKILL.md'), '# planning\n\nPlan things.\n')
+  await Bun.write(join(dir, 'skills/planning/references/api.md'), '# API reference\n')
+  await Bun.write(join(dir, 'skills/planning/assets/logo.bin'), REPRESENTATIVE_BINARY)
+  await Bun.write(join(dir, 'README.md'), '# Representative facet\n\nShips a README.\n')
+  await Bun.write(
+    join(dir, 'facet.json'),
+    JSON.stringify(
+      {
+        name: 'representative',
+        version: '1.0.0',
+        description: 'A representative 0.2 facet for stage interop',
+        files: ['README.md'],
+        skills: {
+          planning: { description: 'Planning skill', files: ['references/api.md', 'assets/logo.bin'] },
+        },
+      },
+      null,
+      2,
+    ),
+  )
+}
+
+const EXPECTED_MEMBERSHIP = [
+  'README.md',
+  'facet.json',
+  'skills/planning/SKILL.md',
+  'skills/planning/assets/logo.bin',
+  'skills/planning/references/api.md',
+].sort()
+
+const gunzip: GunzipFn = async (bytes) => {
+  try {
+    return { ok: true, bytes: new Uint8Array(gunzipSync(bytes)) }
+  } catch {
+    return { ok: false, reason: 'corrupt' }
+  }
+}
+
+describe('candidate 0.2 archive interop', () => {
+  test('the candidate CLI builds a verifiable 0.2 archive with exact membership', async () => {
+    const dir = await mkdtemp(join(testDir, 'build-'))
+    await writeRepresentativeFacet(dir)
+
+    const built = await runCli(dir, 'build')
+    expect(built.exitCode).toBe(0)
+
+    const archivePath = join(dir, 'dist/representative-1.0.0.facet')
+    expect(existsSync(archivePath)).toBe(true)
+    const outerBytes = new Uint8Array(await Bun.file(archivePath).arrayBuffer())
+
+    // Parse the outer container: the build manifest must be current 0.2.
+    const parsed = parseFacetArchive(outerBytes)
+    if (!parsed.ok) expect.unreachable()
+    expect(parsed.data.manifest.facetVersion).toBe(0.2)
+
+    // Verify through the shared registry-grade verifier.
+    const verified = await validateFacetArchive(outerBytes, { gunzip })
+    if (!verified.ok) expect.unreachable()
+    if (verified.data.archiveVersion !== 0.2) expect.unreachable()
+
+    // Exact 0.2 membership, including the archive-only README and both
+    // skill companions (text + binary).
+    const observed = verified.data.entries.map((e) => e.path).sort()
+    expect(observed).toEqual(EXPECTED_MEMBERSHIP)
+
+    // The archive-only README is classified as such (never a primary asset),
+    // and the binary companion is grouped with its owning skill.
+    const readme = verified.data.entries.find((e) => e.path === 'README.md')
+    expect(readme?.kind).toBe('archive-only')
+    const logo = verified.data.entries.find((e) => e.path === 'skills/planning/assets/logo.bin')
+    if (logo?.kind !== 'skill-companion') expect.unreachable()
+    expect(logo.skill).toBe('planning')
+    expect(logo.bytes).toEqual(REPRESENTATIVE_BINARY)
+  })
+
+  test('two candidate builds of the same source are byte-identical (deterministic)', async () => {
+    const build = async (name: string): Promise<Uint8Array> => {
+      const dir = await mkdtemp(join(testDir, `${name}-`))
+      await writeRepresentativeFacet(dir)
+      const built = await runCli(dir, 'build')
+      expect(built.exitCode).toBe(0)
+      const archivePath = join(dir, 'dist/representative-1.0.0.facet')
+      return new Uint8Array(await Bun.file(archivePath).arrayBuffer())
+    }
+    const a = await build('det-a')
+    const b = await build('det-b')
+    expect(Array.from(a)).toEqual(Array.from(b))
+  })
+})

packages/cli/src/__tests__/create-build.e2e.test.ts

@@ -348,11 +348,14 @@ describe('facet build --verify', () => {
     const result = await runCli('build', dir, '--verify', '--json')
     expect(result.exitCode).toBe(0)
     const doc = JSON.parse(result.stdout)
-    expect(doc.schemaVersion).toBe('1')
+    expect(doc.schemaVersion).toBe('2')
     expect(doc.ok).toBe(true)
     expect(doc.verified).toBe(true)
     expect(doc.name).toBe('verifiable')
-    expect(Array.isArray(doc.assets)).toBe(true)
+    expect(doc.facetVersion).toBe(0.2)
+    // Complete inner-archive entry listing (includes facet.json + primaries).
+    expect(Array.isArray(doc.files)).toBe(true)
+    expect(doc.files).toContain('facet.json')
     expect(existsSync(join(dir, 'dist'))).toBe(false)
   })
 

packages/cli/src/commands/build.ts

@@ -17,8 +17,15 @@ import {
 import { writeCliError } from '../util/errors.ts'
 import { resolveTargetDir } from './resolve-dir.ts'
 
-/** Version tag for the machine-readable `--json` output document. */
-const BUILD_JSON_SCHEMA_VERSION = '1'
+/**
+ * Version tag for the machine-readable `--json` output document.
+ *
+ * `2` replaces the `assets` array (primary-asset paths only) with a complete
+ * `files` array (every inner-archive entry — manifest, primaries, and
+ * supplementary files) and adds `facetVersion`. Consumers pinned to schema
+ * `1` must migrate rather than silently receive a different `assets` set.
+ */
+const BUILD_JSON_SCHEMA_VERSION = '2'
 
 export const buildCommand: Command = {
   name: 'build',
@@ -108,9 +115,8 @@ export const buildCommand: Command = {
       await instance.waitUntilExit()
       // Ink has unmounted — print stdout summary for scroll-back
       const shortHash = integrity.length > 20 ? `${integrity.slice(0, 20)}...` : integrity
-      process.stdout.write(
-        `✓ Built ${buildName} v${buildVersion} → ${displayDir}/dist/ (${artifactCount} assets, ${shortHash})\n`,
-      )
+      const entries = `${artifactCount} entr${artifactCount !== 1 ? 'ies' : 'y'}`
+      process.stdout.write(`✓ Built ${buildName} v${buildVersion} → ${displayDir}/dist/ (${entries}, ${shortHash})\n`)
       return 0
     } catch {
       process.stdout.write(
@@ -133,7 +139,10 @@ function printBuildJson(result: BuildResult | BuildFailure, verified: boolean):
         verified,
         name: result.data.name,
         version: result.data.version,
-        assets: Object.keys(result.assetHashes).sort(),
+        facetVersion: result.facetVersion,
+        // Complete inner-archive entry listing: facet.json, every primary
+        // asset, and every supplementary file (skill companions + archive-only).
+        files: Object.keys(result.fileHashes).sort(),
         integrity: result.integrity,
         warnings: result.warnings,
       }
@@ -159,14 +168,16 @@ function printBuildPlain(result: BuildResult | BuildFailure, verified: boolean,
     process.stderr.write(`⚠ ${warning}\n`)
   }
   if (result.ok) {
-    const assetCount = Object.keys(result.assetHashes).length
+    // Count every inner-archive entry, not just primary assets.
+    const entryCount = Object.keys(result.fileHashes).length
+    const entries = `${entryCount} entr${entryCount !== 1 ? 'ies' : 'y'}`
     if (verified) {
       process.stdout.write(
-        `✓ Verified ${result.data.name} v${result.data.version} (${assetCount} asset${assetCount !== 1 ? 's' : ''}, no output written)\n`,
+        `✓ Verified ${result.data.name} v${result.data.version} (facetVersion ${result.facetVersion}, ${entries}, no output written)\n`,
       )
     } else {
       process.stdout.write(
-        `✓ Built ${result.data.name} v${result.data.version} → ${displayDir}/dist/ (${assetCount} asset${assetCount !== 1 ? 's' : ''}, ${result.integrity})\n`,
+        `✓ Built ${result.data.name} v${result.data.version} → ${displayDir}/dist/ (facetVersion ${result.facetVersion}, ${entries}, ${result.integrity})\n`,
       )
     }
     return

packages/cli/src/tui/views/build/build-view.tsx

@@ -17,6 +17,8 @@ import { THEME } from '../../theme.ts'
 interface BuildViewResult {
   name: string
   version: string
+  facetVersion: number
+  /** Complete inner-archive entry listing (manifest, primaries, supplementary). */
   files: string[]
   archiveFilename: string
   integrity: string
@@ -98,12 +100,13 @@ export function BuildView({
     try {
       await writeBuildOutput(pipelineResult, rootDir, { emitManifest })
 
-      const files = Object.keys(pipelineResult.assetHashes).sort()
+      const files = Object.keys(pipelineResult.fileHashes).sort()
 
       updateStage('Writing output', { status: 'done' })
       setResult({
         name: pipelineResult.data.name,
         version: pipelineResult.data.version,
+        facetVersion: pipelineResult.facetVersion,
         files,
         archiveFilename: pipelineResult.archiveFilename,
         integrity: pipelineResult.integrity,
@@ -183,13 +186,14 @@ export function BuildView({
             Built successfully → dist/
           </Text>
           <Text> {result.archiveFilename}</Text>
+          <Text color={THEME.hint}> facetVersion {result.facetVersion}</Text>
           <Text color={THEME.hint}> Archive contents:</Text>
           {result.files.map((f) => (
             <Text key={f}> {f}</Text>
           ))}
           <Box marginTop={1}>
             <Text color={THEME.hint}>
-              {result.files.length} asset{result.files.length !== 1 ? 's' : ''} · {result.integrity}
+              {result.files.length} entr{result.files.length !== 1 ? 'ies' : 'y'} · {result.integrity}
             </Text>
           </Box>
           <Box marginTop={1}>