diff --git a/CHANGELOG.md b/CHANGELOG.md index ca224d1..b97191e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,22 @@ # Changelog +## 0.1.7 source candidate - 2026-10-10 + +- Add `artifacts --format html`: a self-contained offline delivery review with + declared-check status, file/Receipt binding, record-identity defects, incomplete + checks, observed metadata and concrete review actions. +- Preserve JSON/text and exit semantics. HTML uses the same local-path redaction, + escapes every metadata field, includes no script or external request, and keeps + source truth, author authentication and business approval separate. +- Locate affected CSV data records in preserved expected/delivered files, including + split groups, without reporting IDs or amounts. Keep 8 row references per group, + 32 examples per check and 64 total; unknown positions are never invented. + Default JSON/text observation and report shapes stay intact. +- Show at most 100 file details and 64 reconciliation cards, prioritizing failed + and incomplete checks. Keep full-scope counts and explicit shortening notices; + raw IDs, values and file contents never enter the report. +- This is a source update. GitHub0.1.4/npm0.1.0 remain the published packages. + ## 0.1.6 source candidate - 2026-10-10 - Compare ordered two-to-four-column string identities as well as existing diff --git a/README.md b/README.md index 82c1324..62d25f2 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,10 @@ **Verify Agent Skills before you trust them.** SkillSync checks a local Skill's provenance, compatibility, and changes without executing it. -> **Alpha · source candidate v0.1.6 · Node.js 20+** +> **Alpha · source candidate v0.1.7 · Node.js 20+** > SkillSync performs offline checks of local Skill content. It does not execute Skill scripts, does not read credentials, and does not enable live provider, remote-worker, or runtime capabilities. -Source v0.1.6 adds bounded ordered composite keys to per-key integer reconciliation. The published GitHub package +Source v0.1.7 adds an offline human acceptance report on top of bounded composite-key reconciliation. The published GitHub package is still v0.1.4 and npm is still 0.1.0; neither includes the new rule. [![Terminal demo](https://raw.githubusercontent.com/Chumaniac/skillsync/main/docs/assets/verify-demo.svg)](https://github.com/Chumaniac/skillsync/blob/main/docs/assets/verify-demo.svg) @@ -21,7 +21,7 @@ npm install -g @chumanic/skillsync@0.1.0 skillsync verify --path . --target codex ``` -**From source (0.1.6 candidate):** +**From source (0.1.7 candidate):** ```bash git clone https://github.com/Chumaniac/skillsync.git @@ -62,6 +62,10 @@ The command above verifies the included sample Skill. Replace the fixture path w ## What you get +- Source0.1.7 produces an offline acceptance report with `artifacts --format html`. + Reviewers see file binding, failed or incomplete identity checks, affected + metadata and concrete review actions in a browser. The report includes no raw + record values, scripts or network requests; see [the visible review guide](docs/artifact-delivery.md#hand-a-visible-report-to-a-reviewer). - Source0.1.6 supports ordered two-to-four-column identity tuples for integer reconciliation. The [multi-organization example](fixtures/product/tenant-reconciliation/README.md) separates repeated local IDs, without delimiter collisions or raw ID reporting. diff --git a/docs/artifact-delivery.md b/docs/artifact-delivery.md index f596087..6329f49 100644 --- a/docs/artifact-delivery.md +++ b/docs/artifact-delivery.md @@ -21,6 +21,54 @@ Receipt hash, artifact hashes/sizes, and deterministic artifact-set digest befor accepting a complete result. A consistently rewritten manifest is not signed or authenticated evidence; the report marks provenance as not authenticated. +## Hand a visible report to a reviewer + +Source0.1.7 adds `--format html`. The report opens directly in a browser, with no +server, script, login, provider or network request. A reviewer sees the declared +result, supplied file/Receipt binding, failed or incomplete reconciliation, +affected file/column metadata and a concrete next review step. + +```bash +# Build the current source first. Use a new destination; keep shell no-clobber. +(set -C; node dist/cli/index.js artifacts \ + --contract fixtures/product/tenant-reconciliation/contract.json \ + --path fixtures/product/tenant-reconciliation/artifacts \ + --format html > acceptance.html) +``` + +For a retained physical delivery, select `--delivery` instead. Exit0 means the +declared local checks passed, exit1 retains findings/incomplete checks, and exit2 +reports invalid setup. A failed inspection still produces a reviewable HTML +report; caller-side redirect errors remain the caller's responsibility. + +Reconciliation cards separate expected/delivered identity counts, missing or +unexpected identities, and mismatched values. Unknown checks never display zero +mismatches as accepted evidence. The review actions point to saved originals, +declared identity columns and reviewed expectations; they do not execute a +correction or establish business approval. + +HTML also requests bounded CSV record positions for affected identity groups. +Expand "Locate affected CSV records" to find the expected and delivered data +records in the original local files. Positions are 1-based, exclude the header, +and count a quoted multiline record once; they are not physical text-line numbers. +Up to8 positions per group, 32 examples per check and 64 examples across the +invocation are retained. Truncated or unavailable positions are explicit, and +unknown checks carry no position evidence. No IDs, fingerprints or amounts are +returned. Ordinary JSON/text inspections do not request or retain this metadata. + +HTML shows at most 100 file details, 64 reconciliation cards and the existing +256 findings. Failed/incomplete cards appear first, full-scope counts remain +visible, and metadata longer than160 characters is shortened. Use the existing +JSON output when complete metadata is required. HTML retains no raw record IDs, +integer values, contents, key fingerprints or absolute host paths. Every visible +field is escaped after the existing path redaction. CSP disables scripts and +external resources. The output is bounded by the report ceilings and has no +client timers or caches. + +The report is a captured local snapshot. Keep the successful original delivery; +rerun read-only checks on its files to capture a later state. Source0.1.7 HTML is +absent from the published GitHub0.1.4/npm0.1.0 packages. + ## Per-key integer reconciliation Source candidate 0.1.6 supports `keyed_integer_sum_equals`. It is absent from diff --git a/docs/deep-optimization-roadmap.md b/docs/deep-optimization-roadmap.md index 9c1fd63..9dc7047 100644 --- a/docs/deep-optimization-roadmap.md +++ b/docs/deep-optimization-roadmap.md @@ -49,6 +49,16 @@ Keep the existing 10,000-file/16 MiB-file/64 MiB-total and 256-finding ceilings. ## Domain expansion +### Visible business continuation — 2026-10-10 + +Source0.1.7 converts physical-file and composite-identity results into an offline +acceptance report. Operations reviewers can see which declared relation needs +attention, whether counts are incomplete, and which original files/columns to +review without exposing the underlying record identities or values. The local +CLI generates the HTML from the real inspection result; it does not embed a +fabricated dashboard or claim that an automated check supplies business approval. +The source-only HTML and current published packages remain separate. + ### Continuation cycle — 2026-10-10 Source0.1.6 implements ordered two-to-four-column string keys for integer diff --git a/package-lock.json b/package-lock.json index f120d20..bbf8a84 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@chumanic/skillsync", - "version": "0.1.6", + "version": "0.1.7", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@chumanic/skillsync", - "version": "0.1.6", + "version": "0.1.7", "license": "MIT", "dependencies": { "commander": "^13.1.0", diff --git a/package.json b/package.json index b1c27ff..818ffaa 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@chumanic/skillsync", - "version": "0.1.6", + "version": "0.1.7", "private": false, "publishConfig": { "access": "public" diff --git a/src/cli/commands/artifacts.ts b/src/cli/commands/artifacts.ts index 76192a6..a61350d 100644 --- a/src/cli/commands/artifacts.ts +++ b/src/cli/commands/artifacts.ts @@ -10,6 +10,7 @@ import { inspectArtifactContent, type ArtifactFacts, type ArtifactFinding } from import { createArtifactReferenceIndex } from "../../domain/artifact-references.js"; import { createArtifactReconciliationIndex, type ReconciliationSummary } from "../../domain/artifact-reconciliation.js"; import { redactLocalPaths } from "../../reporters/local-paths.js"; +import { renderArtifactsHtml } from "../../reporters/artifacts-html.js"; import { readWorkspaceFile, scanStagedWorkspace, WorkspaceTreeError } from "../../sandbox/workspace-tree.js"; const hash = (bytes: Uint8Array | string) => createHash("sha256").update(bytes).digest("hex"); @@ -19,7 +20,7 @@ const manifestSchema = z.object({ schema: z.literal("skilltape.dev/delivery/v1") files: z.array(z.object({ path: z.string().min(1).max(1024), bytes: z.number().int().min(0).max(16 * 1024 * 1024), sha256: sha }).strict()).max(10_000), }).strict(); -export type ArtifactsOptions = { contract: string; delivery?: string; path?: string }; +export type ArtifactsOptions = { contract: string; delivery?: string; path?: string; includeRowEvidence?: boolean }; export type ArtifactReport = { schema: "skillsync.artifacts-report/v1"; status: "passed" | "failed"; exitCode: number; evidence: "physical-files"; execution: "not-run"; provenance: "not-authenticated"; @@ -89,7 +90,7 @@ export async function runArtifacts(options: ArtifactsOptions): Promise(); const references = createArtifactReferenceIndex(contract); - const reconciliation = createArtifactReconciliationIndex(contract); + const reconciliation = createArtifactReconciliationIndex(contract, { rowEvidence: options.includeRowEvidence }); try { const rules = new Map(contract.files.map(file => [file.path, file])); const tree = await scanStagedWorkspace(root, { @@ -146,6 +147,7 @@ export async function runArtifacts(options: ArtifactsOptions): Promise", "JSON or YAML artifact contract") .option("--delivery ", "SkillTape delivery bundle directory") .option("--path ", "Standalone artifact directory; choose this or --delivery") - .option("--format ", "Output format: text or json", "text") + .option("--format ", "Output format: text, json or offline html", "text") .action(async (options: ArtifactsOptions & { format?: string }) => { - if (options.format !== "text" && options.format !== "json") throw new Error("artifacts format must be text or json"); - const report = await runArtifacts(options); + if (options.format !== "text" && options.format !== "json" && options.format !== "html") throw new Error("artifacts format must be text, json or html"); + const report = await runArtifacts({ ...options, includeRowEvidence: options.format === "html" }); io.writeOut(renderArtifacts(report, options.format)); if (report.exitCode) io.setExitCode?.(report.exitCode); }); diff --git a/src/domain/artifact-content.ts b/src/domain/artifact-content.ts index 3ead466..6c51ff7 100644 --- a/src/domain/artifact-content.ts +++ b/src/domain/artifact-content.ts @@ -4,7 +4,7 @@ import { parseStrictJson } from "./strict-json.js"; export type ArtifactFinding = { code: string; path?: string; field?: string | string[]; row?: number; message?: string }; export type ArtifactFacts = { rows?: number; integers: Map }; export type ArtifactCellObserver = (field: string, value: string, row: number) => void; -export type ArtifactRowObserver = (values: ReadonlyMap) => void; +export type ArtifactRowObserver = (values: ReadonlyMap, dataRecord?: number) => void; function* csvRows(text: string, maxFields: number): Generator { let field = "", row: string[] = [], quoted = false, afterQuote = false, active = false; @@ -141,7 +141,7 @@ export function inspectArtifactContent(rule: ArtifactFileRule, bytes: Uint8Array else facts.integers.set(column.name, total); } } - if (values) observeRow?.(values); + if (values) observeRow?.(values, count); } facts.rows = count; if (count < rule.min_rows) issue("artifact.csv-too-few-rows"); diff --git a/src/domain/artifact-reconciliation.ts b/src/domain/artifact-reconciliation.ts index 2404b79..4e7a50b 100644 --- a/src/domain/artifact-reconciliation.ts +++ b/src/domain/artifact-reconciliation.ts @@ -5,15 +5,19 @@ import type { ArtifactFinding, ArtifactRowObserver } from "./artifact-content.js export const RECONCILIATION_CELL_LIMIT = 100_000; type Check = Extract; type Side = Check["source"]; -type Group = { sums: Map; overflow: boolean; invalid: boolean }; +type Locations = { rows: number[]; total: number; unavailable: boolean }; +type Group = { sums: Map; overflow: boolean; invalid: boolean; rows?: Map }; type Selection = { side: Side; group: Group; keys: string[]; keyIdentity: string }; +export type RecordEvidence = { kind: "value_mismatch" | "missing" | "unexpected"; + sourceRows: number[]; targetRows: number[]; sourceRowsTruncated: boolean; targetRowsTruncated: boolean }; export type ReconciliationSummary = Check & { status: "passed" | "failed" | "unknown"; sourceKeys?: number; targetKeys?: number; missingKeys?: number; unexpectedKeys?: number; valueMismatches?: number; + recordEvidence?: { items: RecordEvidence[]; total: number; truncated: boolean }; }; -export function createArtifactReconciliationIndex(contract: ArtifactContract) { +export function createArtifactReconciliationIndex(contract: ArtifactContract, options: { rowEvidence?: boolean } = {}) { const checks = contract.checks.filter((check): check is Check => check.type === "keyed_integer_sum_equals"); const groups = new Map(); const byPath = new Map(); @@ -21,7 +25,7 @@ export function createArtifactReconciliationIndex(contract: ArtifactContract) { for (const check of checks) for (const side of [check.source, check.target]) { const id = identity(side); if (groups.has(id)) continue; - const group: Group = { sums: new Map(), overflow: false, invalid: false }; + const group: Group = { sums: new Map(), overflow: false, invalid: false, ...(options.rowEvidence ? { rows: new Map() } : {}) }; groups.set(id, group); const entries = byPath.get(side.path) ?? []; const keys = typeof side.key === "string" ? [side.key] : side.key; @@ -29,12 +33,13 @@ export function createArtifactReconciliationIndex(contract: ArtifactContract) { } let observedCells = 0, exhausted = false; let completed: { summaries: ReconciliationSummary[]; findings: ArtifactFinding[] } | undefined; - const release = () => { for (const group of groups.values()) group.sums.clear(); }; + const clear = (group: Group) => { group.sums.clear(); group.rows?.clear(); }; + const release = () => { for (const group of groups.values()) clear(group); }; return { observer(path: string): ArtifactRowObserver | undefined { const entries = byPath.get(path); if (!entries) return undefined; - return values => { + return (values, dataRecord) => { if (completed) throw new Error("reconciliation index is closed"); if (exhausted) return; // Hash each selected key tuple once per row; raw IDs never enter the index. @@ -45,7 +50,7 @@ export function createArtifactReconciliationIndex(contract: ArtifactContract) { const value = values.get(side.value); if (!components.every((key): key is string => typeof key === "string" && key.length > 0) || typeof value !== "number" || !Number.isSafeInteger(value)) { - group.invalid = true; group.sums.clear(); continue; + group.invalid = true; clear(group); continue; } observedCells += keys.length + 1; if (observedCells > RECONCILIATION_CELL_LIMIT) { exhausted = true; release(); return; } @@ -67,18 +72,28 @@ export function createArtifactReconciliationIndex(contract: ArtifactContract) { fingerprints.set(keyIdentity, fingerprint); } const sum = (group.sums.get(fingerprint) ?? 0) + value; - if (!Number.isSafeInteger(sum)) { group.overflow = true; group.sums.clear(); } - else group.sums.set(fingerprint, sum); + if (!Number.isSafeInteger(sum)) { group.overflow = true; clear(group); } + else { + group.sums.set(fingerprint, sum); + if (group.rows) { + const locations = group.rows.get(fingerprint) ?? { rows: [], total: 0, unavailable: false }; + locations.total += 1; + if (dataRecord === undefined || !Number.isSafeInteger(dataRecord) || dataRecord < 1 || dataRecord > 100_000) locations.unavailable = true; + else if (locations.rows.length < 8) locations.rows.push(dataRecord); + group.rows.set(fingerprint, locations); + } + } } }; }, invalidate(path: string) { if (completed) throw new Error("reconciliation index is closed"); - for (const { group } of byPath.get(path) ?? []) { group.invalid = true; group.sums.clear(); } + for (const { group } of byPath.get(path) ?? []) { group.invalid = true; clear(group); } }, finish(): { summaries: ReconciliationSummary[]; findings: ArtifactFinding[] } { if (completed) return completed; const summaries: ReconciliationSummary[] = [], findings: ArtifactFinding[] = []; + let evidenceBudget = 64; if (exhausted) findings.push({ code: "artifact.reconciliation-capacity" }); for (const check of checks) { const source = groups.get(identity(check.source))!, target = groups.get(identity(check.target))!; @@ -90,15 +105,28 @@ export function createArtifactReconciliationIndex(contract: ArtifactContract) { continue; } let missingKeys = 0, unexpectedKeys = 0, valueMismatches = 0; + const examples: RecordEvidence[] = []; + const example = (kind: RecordEvidence["kind"], key: string) => { + if (!options.rowEvidence || examples.length >= 32 || evidenceBudget === 0) return; + const expected = source.rows?.get(key), delivered = target.rows?.get(key); + if ((source.sums.has(key) && (!expected || expected.unavailable)) || + (target.sums.has(key) && (!delivered || delivered.unavailable))) return; + examples.push({ kind, sourceRows: [...expected?.rows ?? []], targetRows: [...delivered?.rows ?? []], + sourceRowsTruncated: Boolean(expected && expected.total > expected.rows.length), + targetRowsTruncated: Boolean(delivered && delivered.total > delivered.rows.length) }); + evidenceBudget -= 1; + }; for (const [key, value] of source.sums) { - if (!target.sums.has(key)) missingKeys += 1; - else if (target.sums.get(key) !== value) valueMismatches += 1; + if (!target.sums.has(key)) { missingKeys += 1; example("missing", key); } + else if (target.sums.get(key) !== value) { valueMismatches += 1; example("value_mismatch", key); } } - for (const key of target.sums.keys()) if (!source.sums.has(key)) unexpectedKeys += 1; + for (const key of target.sums.keys()) if (!source.sums.has(key)) { unexpectedKeys += 1; example("unexpected", key); } if (missingKeys || unexpectedKeys) findings.push({ code: "artifact.reconciliation-keys", path: check.target.path, field: check.target.key }); if (valueMismatches) findings.push({ code: "artifact.reconciliation-sum", path: check.target.path, field: check.target.value }); summaries.push({ ...check, status: missingKeys || unexpectedKeys || valueMismatches ? "failed" : "passed", - sourceKeys: source.sums.size, targetKeys: target.sums.size, missingKeys, unexpectedKeys, valueMismatches }); + sourceKeys: source.sums.size, targetKeys: target.sums.size, missingKeys, unexpectedKeys, valueMismatches, + ...(options.rowEvidence ? { recordEvidence: { items: examples, total: missingKeys + unexpectedKeys + valueMismatches, + truncated: examples.length < missingKeys + unexpectedKeys + valueMismatches } } : {}) }); } // Reports retain only declared metadata and counts, never the per-key index. release(); diff --git a/src/reporters/artifacts-html.ts b/src/reporters/artifacts-html.ts new file mode 100644 index 0000000..cb21268 --- /dev/null +++ b/src/reporters/artifacts-html.ts @@ -0,0 +1,95 @@ +import type { ArtifactReport } from "../cli/commands/artifacts.js"; +import type { ArtifactFinding } from "../domain/artifact-content.js"; +import type { ReconciliationSummary } from "../domain/artifact-reconciliation.js"; +import { VERSION } from "../version.js"; +import { redactLocalPaths } from "./local-paths.js"; + +const FILE_DETAILS = 100; +const CHECK_DETAILS = 64; +const TEXT_LENGTH = 160; + +// Only metadata supplied by the existing inspector reaches this offline view. +function text(value: unknown): string { + const original = String(value ?? "Not reported"); + const clipped = original.length > TEXT_LENGTH ? original.slice(0, TEXT_LENGTH) + "…" : original; + return clipped.replaceAll("&", "&").replaceAll("<", "<").replaceAll(">", ">") + .replaceAll('"', """).replaceAll("'", "'"); +} + +function columns(value: string | string[]): string { + return text(Array.isArray(value) ? value.join(" + ") : value); +} + +function action(code: string): string { + if (code === "artifact.reconciliation-keys" || code === "artifact.reconciliation-sum") return "Compare expected and delivered records using the declared key columns. Preserve the original delivery while reviewing a corrected copy."; + if (code.startsWith("artifact.reconciliation-")) return "No partial result is accepted. Review the declared types, integer range and selected-cell capacity before checking again."; + if (code === "artifact.contract-invalid") return "Review the contract declaration and directory selection before inspecting files. No file acceptance is established."; + if (code.includes("digest") || code.includes("changed") || code.includes("snapshot")) return "Retain this copy for comparison. Recover known successful originals and recheck the same reviewed expectations."; + if (code.includes("missing")) return "Recover missing files from the saved successful delivery, then recheck the complete required scope."; + if (code.includes("capacity")) return "Review the documented capacity bounds. A partial check cannot accept the whole delivery."; + return "Review the declared file, field and requirement below. Keep original evidence and check again after an approved correction."; +} + +function findingRow(finding: ArtifactFinding): string { + return `${text(finding.code)}${text(finding.path ?? "Whole delivery")} + ${finding.field ? `Field: ${columns(finding.field)}` : ""} + ${finding.row !== undefined ? `Row: ${text(finding.row)}` : ""} + ${finding.message ? `

${text(finding.message)}

` : ""}${action(finding.code)}`; +} + +function reconciliationCard(check: ReconciliationSummary, rowBudget: { remaining: number }): string { + const label = check.status === "passed" ? "Passed" : check.status === "unknown" ? "Unknown — check incomplete" : "Failed"; + const tone = check.status === "passed" ? "pass" : check.status === "unknown" ? "unknown" : "fail"; + const positions = check.recordEvidence?.items.slice(0, Math.min(32, rowBudget.remaining)) ?? []; + rowBudget.remaining -= positions.length; + return `

Integer allocation by identity

${label}
+

Expected${text(check.source.path)}${columns(check.source.key)}Value column: ${text(check.source.value)}

+

Delivered${text(check.target.path)}${columns(check.target.key)}Value column: ${text(check.target.value)}

+ ${check.status === "unknown" ? `

No partial result is accepted. Counts are not reported for this incomplete check.

` : `
+
Expected identities
${text(check.sourceKeys)}
Delivered identities
${text(check.targetKeys)}
+
Missing identities
${text(check.missingKeys)}
Unexpected identities
${text(check.unexpectedKeys)}
+
Mismatched values
${text(check.valueMismatches)}
`} + ${check.recordEvidence && check.recordEvidence.total > 0 ? `
Locate affected CSV records (${positions.length} shown) +

Data-record positions start at 1 and exclude the header. Quoted multiline records still count as one record. No record identities or amounts are included.

+

Showing ${positions.length} of ${check.recordEvidence.total} affected identity groups.${check.recordEvidence.truncated || positions.length < check.recordEvidence.total ? " Position details are incomplete or limited; all defect counts remain above." : ""}

+
+ ${positions.map(item => ` + + `).join("") || ''} +
DefectExpected data recordsDelivered data records
${item.kind === "value_mismatch" ? "Value mismatch" : item.kind === "missing" ? "Missing expected identity" : "Unexpected delivered identity"}${text(item.sourceRows.slice(0, 8).join(", ") || "No matching record")}${item.sourceRowsTruncated || item.sourceRows.length > 8 ? " (first 8 shown)" : ""}${text(item.targetRows.slice(0, 8).join(", ") || "No matching record")}${item.targetRowsTruncated || item.targetRows.length > 8 ? " (first 8 shown)" : ""}
Record positions are unavailable in this view. No coordinates are fabricated.
` : ""} +
`; +} + +/** Use the same path policy as text/JSON, including when called directly. */ +export function renderArtifactsHtml(rawReport: ArtifactReport): string { + const report = redactLocalPaths(rawReport); + const checks = report.reconciliations ?? []; + const incomplete = checks.filter(check => check.status === "unknown").length; + const failedChecks = checks.filter(check => check.status === "failed").length; + const passed = report.status === "passed"; + const title = passed ? "Declared checks passed" : report.exitCode === 2 ? "Check setup required" : "Review required"; + const csp = "default-src 'none'; style-src 'unsafe-inline'; base-uri 'none'; form-action 'none'"; + const files = report.files.slice(0, FILE_DETAILS); + // Failed and incomplete checks stay visible before successful checks. + const ordered = [...checks.filter(check => check.status !== "passed"), ...checks.filter(check => check.status === "passed")]; + const rowBudget = { remaining: 64 }; + return ` + +SkillSync delivery review — ${title} + +
SkillSync
Offline delivery review
+
${passed ? "Passed" : "Not accepted"}

${title}

+

${passed ? "The observed files satisfy the declared local checks. Review the expectations and responsible approval before using this result." : "This delivery has findings or incomplete checks. Use the file and identity metadata below to review the affected requirement."}

+
${report.files.length}Observed files
${report.findings.length}Review findings
${failedChecks}Failed record checks
${incomplete}Incomplete record checks
+
Receipt / file binding${text(report.receipt_binding)}Hash consistency with supplied local evidence; author identity is not authenticated.
+

Allocation and identity checks

Showing ${Math.min(CHECK_DETAILS, checks.length)} of ${checks.length} reconciliation checks. Failed and incomplete checks appear first.

+${ordered.slice(0, CHECK_DETAILS).map(check => reconciliationCard(check, rowBudget)).join("") || "

No record reconciliation was declared. Other declared file checks are reflected in the result.

"}
+

Review actions

${report.findings.length ? `
${report.findings.slice(0, 256).map(findingRow).join("")}
FindingAffected metadataNext review step
` : "

No finding was reported for the declared scope. This does not authenticate source truth or establish business approval.

"}
+

Observed files

Showing ${files.length} of ${report.files.length} files. Metadata values longer than 160 characters are shortened; the original JSON report retains full metadata.

+
${files.map(file => ``).join("") || ''}
Declared pathBytesRowsSHA-256
${text(file.path)}${text(file.bytes)}${text(file.rows)}${text(file.sha256)}
No accepted file observation is available.
+
Evidence boundaries and snapshot identity

This is a local inspection snapshot. It does not execute a Skill, authenticate an author, or replace reviewed expectations and business approval.

+
Evidence
physical-files
Execution
execution: not-run
Provenance
provenance: not-authenticated
Contract SHA-256
${text(report.contract_sha256)}
Artifact set SHA-256
${text(report.artifact_set_sha256)}
+
SkillSync source ${text(VERSION)} · Standalone HTML, no scripts or external requests.${text(report.schema)}
\n`; +} diff --git a/tests/cli/help.test.ts b/tests/cli/help.test.ts index 58ec45f..3884df1 100644 --- a/tests/cli/help.test.ts +++ b/tests/cli/help.test.ts @@ -12,7 +12,7 @@ describe("skillsync CLI", () => { it("reports the installed source version and lists the verification commands", async () => { const version = await runCli(["--version"]); expect(version.exitCode).toBe(0); - expect(version.stdout).toBe("0.1.6\n"); + expect(version.stdout).toBe("0.1.7\n"); const result = await runCli(["--help"]); diff --git a/tests/cli/report.test.ts b/tests/cli/report.test.ts index 3612554..025923b 100644 --- a/tests/cli/report.test.ts +++ b/tests/cli/report.test.ts @@ -218,7 +218,7 @@ describe("skillsync report", () => { const json = await runCli(["report", "--before", beforePath, "--after", afterPath, "--format", "json"]); expect(JSON.parse(json.stdout).conclusion).toBe("verified"); - expect(JSON.parse(json.stdout).toolVersion).toBe("0.1.6"); + expect(JSON.parse(json.stdout).toolVersion).toBe("0.1.7"); const sarif = await runCli(["report", "--before", beforePath, "--after", afterPath, "--format", "sarif"]); const parsedSarif = JSON.parse(sarif.stdout) as { @@ -229,7 +229,7 @@ describe("skillsync report", () => { }>; }; expect(parsedSarif.version).toBe("2.1.0"); - expect(parsedSarif.runs[0]?.tool.driver.version).toBe("0.1.6"); + expect(parsedSarif.runs[0]?.tool.driver.version).toBe("0.1.7"); expect(parsedSarif.runs[0]?.results[0]?.properties.issueId).toMatch(/^iss_/); }); diff --git a/tests/docs/documentation.test.ts b/tests/docs/documentation.test.ts index ee12c13..290ac66 100644 --- a/tests/docs/documentation.test.ts +++ b/tests/docs/documentation.test.ts @@ -39,7 +39,7 @@ describe("release documentation", () => { }; expect(readme).toContain("Verify Agent Skills before you trust them."); - expect(readme).toContain("Alpha · source candidate v0.1.6 · Node.js 20+"); + expect(readme).toContain("Alpha · source candidate v0.1.7 · Node.js 20+"); expect(readme).toContain("published GitHub package"); expect(readme).toContain("still v0.1.4"); expect(readme).toContain("## Install"); diff --git a/tests/domain/artifact-row-evidence.test.ts b/tests/domain/artifact-row-evidence.test.ts new file mode 100644 index 0000000..4313101 --- /dev/null +++ b/tests/domain/artifact-row-evidence.test.ts @@ -0,0 +1,96 @@ +import { readFileSync } from "node:fs"; +import { describe, expect, it } from "vitest"; +import { parseArtifactContract } from "../../src/domain/artifact-contract.js"; +import { createArtifactReconciliationIndex } from "../../src/domain/artifact-reconciliation.js"; + +function contract() { + return parseArtifactContract(JSON.parse(readFileSync("fixtures/product/tenant-reconciliation/contract.json", "utf8"))); +} + +const row = (tenant: string, order: string, cents: number) => new Map([["tenant_id", tenant], ["order_id", order], ["amount_cents", cents]]); + +describe("opt-in private-value-free CSV record evidence", () => { + it("locates split-group defects using actual record positions without exposing identities or amounts", () => { + const index = createArtifactReconciliationIndex(contract(), { rowEvidence: true }); + const expected = index.observer("expected.csv")!, actual = index.observer("allocations.csv")!; + expected(row("PRIVATE_ORG", "PRIVATE_ID", 1000), 2); + actual(row("PRIVATE_ORG", "PRIVATE_ID", 500), 2); + actual(row("PRIVATE_ORG", "PRIVATE_ID", 600), 4); + const result = index.finish(); + expect(result.summaries[0]).toMatchObject({ status: "failed", valueMismatches: 1, + recordEvidence: { items: [{ kind: "value_mismatch", sourceRows: [2], targetRows: [2, 4], + sourceRowsTruncated: false, targetRowsTruncated: false }], total: 1, truncated: false } }); + expect(JSON.stringify(result).includes("PRIVATE_ORG")).toBe(false); + expect(JSON.stringify(result).includes("PRIVATE_ID")).toBe(false); + expect(JSON.stringify(result).includes("1000")).toBe(false); + expect(index.finish()).toEqual(result); + }); + + it("does not change default report shapes or fabricate positions", () => { + const original = createArtifactReconciliationIndex(contract()); + original.observer("expected.csv")!(row("A", "B", 1), 2); + original.observer("allocations.csv")!(row("A", "B", 2), 2); + expect(original.finish().summaries[0]).not.toHaveProperty("recordEvidence"); + const index = createArtifactReconciliationIndex(contract(), { rowEvidence: true }); + index.observer("expected.csv")!(row("A", "B", 1), 0); + index.observer("allocations.csv")!(row("A", "B", 2), 0); + expect(index.finish().summaries[0]).toMatchObject({ recordEvidence: { items: [], total: 1, truncated: true } }); + }); + + it("caps row references and defect examples while preserving full counts", () => { + const index = createArtifactReconciliationIndex(contract(), { rowEvidence: true }); + const expected = index.observer("expected.csv")!, actual = index.observer("allocations.csv")!; + for (let i = 0; i < 70; i++) { + expected(row("T", String(i), 1), i + 2); + for (let n = 0; n < 10; n++) actual(row("T", String(i), 1), i * 10 + n + 2); + } + const summary = index.finish().summaries[0]; + expect(summary.valueMismatches).toBe(70); + expect(summary.recordEvidence?.items).toHaveLength(32); + expect(summary.recordEvidence?.items[0].targetRows).toHaveLength(8); + expect(summary.recordEvidence).toMatchObject({ total: 70, truncated: true }); + expect(summary.recordEvidence?.items[0].targetRowsTruncated).toBe(true); + }); + + it("drops record evidence for invalid or exhausted observations", () => { + const index = createArtifactReconciliationIndex(contract(), { rowEvidence: true }); + index.observer("expected.csv")!(row("T", "O", 1), 2); + index.invalidate("allocations.csv"); + const result = index.finish(); + expect(result.summaries[0].status).toBe("unknown"); + expect(result.summaries[0]).not.toHaveProperty("recordEvidence"); + const exhausted = createArtifactReconciliationIndex(contract(), { rowEvidence: true }); + const observe = exhausted.observer("expected.csv")!; + for (let i = 1; i < 33_335; i++) observe(row("T", "O", 0), i); + expect(exhausted.finish().findings).toContainEqual({ code: "artifact.reconciliation-capacity" }); + expect(exhausted.finish().summaries[0]).not.toHaveProperty("recordEvidence"); + }); + + it("locates missing and unexpected groups without exposing their keys", () => { + const index = createArtifactReconciliationIndex(contract(), { rowEvidence: true }); + index.observer("expected.csv")!(row("T", "REQUIRED_PRIVATE", 1), 2); + index.observer("allocations.csv")!(row("T", "EXTRA_PRIVATE", 1), 3); + const result = index.finish(); + expect(result.summaries[0].recordEvidence?.items).toMatchObject([ + { kind: "missing", sourceRows: [2], targetRows: [] }, + { kind: "unexpected", sourceRows: [], targetRows: [3] }, + ]); + expect(JSON.stringify(result).includes("REQUIRED_PRIVATE")).toBe(false); + expect(JSON.stringify(result).includes("EXTRA_PRIVATE")).toBe(false); + }); + + it("bounds total retained examples across multiple declared checks", () => { + const value = contract(); + const check = value.checks.find(item => item.type === "keyed_integer_sum_equals")!; + value.checks = [check, check, check]; + const index = createArtifactReconciliationIndex(value, { rowEvidence: true }); + for (let i = 1; i <= 40; i++) { + index.observer("expected.csv")!(row("T", String(i), 1), i); + index.observer("allocations.csv")!(row("T", String(i), 2), i); + } + const results = index.finish().summaries; + expect(results.reduce((total, item) => total + (item.recordEvidence?.items.length ?? 0), 0)).toBe(64); + expect(results.every(item => item.valueMismatches === 40 && item.recordEvidence?.truncated)).toBe(true); + expect(results[2].recordEvidence?.items).toEqual([]); + }); +}); diff --git a/tests/integration/artifacts-html.test.ts b/tests/integration/artifacts-html.test.ts new file mode 100644 index 0000000..c363aac --- /dev/null +++ b/tests/integration/artifacts-html.test.ts @@ -0,0 +1,73 @@ +import { cp, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { describe, expect, it } from "vitest"; +import { runCli } from "../../src/cli/index.js"; + +const source = resolve("fixtures/product/tenant-reconciliation"); + +async function fixture(check: (root: string) => Promise) { + const root = await mkdtemp(join(tmpdir(), "skillsync-html-")); + try { await cp(source, root, { recursive: true }); await check(root); } + finally { await rm(root, { recursive: true, force: true }); } +} + +const command = (root: string) => ["artifacts", "--contract", join(root, "contract.json"), + "--path", join(root, "artifacts"), "--format", "html"]; + +describe("human acceptance report CLI", () => { + it("renders actual correct files without exposing their organization/order data", async () => fixture(async root => { + const result = await runCli(command(root)); + expect(result.exitCode).toBe(0); + expect(result.stdout.startsWith("")).toBe(true); + expect(result.stdout.includes("Declared checks passed")).toBe(true); + expect(result.stdout.includes("not-provided")).toBe(true); + expect(result.stdout.includes("tenant_id + order_id")).toBe(true); + for (const secret of [root, "ORG_A", "ORG_B", "ORDER_1"]) expect(result.stdout.includes(secret)).toBe(false); + })); + + it("reports two actual allocation defects with file/column/action metadata", async () => fixture(async root => { + await cp(join(root, "faults/shifted-allocations.csv"), join(root, "artifacts/allocations.csv")); + const result = await runCli(command(root)); + expect(result.exitCode).toBe(1); + expect(result.stdout.includes('data-value-mismatches="2"')).toBe(true); + expect(result.stdout.includes("Compare expected and delivered records using the declared key columns")).toBe(true); + expect(result.stdout.includes("artifact.reconciliation-sum")).toBe(true); + expect(result.stdout.includes("Review required")).toBe(true); + expect(result.stdout.includes('data-source-records="1"')).toBe(true); + expect(result.stdout.includes('data-target-records="1,3"')).toBe(true); + expect(result.stdout.includes('data-source-records="2"')).toBe(true); + expect(result.stdout.includes('data-target-records="2"')).toBe(true); + expect(result.stdout.includes("exclude the header")).toBe(true); + expect(result.stdout.includes("ORG_A")).toBe(false); + })); + + it("keeps actual invalid sums incomplete instead of declaring zero mismatches", async () => fixture(async root => { + const path = join(root, "artifacts/allocations.csv"); + const csv = await readFile(path, "utf8"); + await writeFile(path, csv.replace(",400", ",9007199254740991")); + const result = await runCli(command(root)); + expect(result.exitCode).toBe(1); + expect(result.stdout.includes("Unknown — check incomplete")).toBe(true); + expect(result.stdout.includes("No partial result is accepted")).toBe(true); + expect(result.stdout.includes('data-value-mismatches="0"')).toBe(false); + })); + + it("renders setup failure as a report without reading or revealing paths", async () => { + const result = await runCli(["artifacts", "--contract", "/private/tmp/ABSENT_CONTRACT_SENTINEL", + "--path", "/private/tmp/ABSENT_PAYLOAD_SENTINEL", "--format", "html"]); + expect(result.exitCode).toBe(2); + expect(result.stdout.includes("Check setup required")).toBe(true); + expect(result.stdout.includes("artifact.contract-invalid")).toBe(true); + expect(result.stdout.includes("ABSENT_CONTRACT_SENTINEL")).toBe(false); + }); + + it("advertises the offline view and rejects unsupported formats", async () => { + const help = await runCli(["artifacts", "--help"]); + expect(help.stdout.includes("offline html")).toBe(true); + const invalid = await runCli(["artifacts", "--contract", "absent.json", "--path", "absent", + "--format", "csv"]); + expect(invalid.exitCode).not.toBe(0); + expect(invalid.stdout.includes("")).toBe(false); + }); +}); diff --git a/tests/reporters/artifacts-html.test.ts b/tests/reporters/artifacts-html.test.ts new file mode 100644 index 0000000..94df369 --- /dev/null +++ b/tests/reporters/artifacts-html.test.ts @@ -0,0 +1,84 @@ +import { describe, expect, it } from "vitest"; +import { renderArtifacts, type ArtifactReport } from "../../src/cli/commands/artifacts.js"; +import { VERSION } from "../../src/version.js"; + +function report(): ArtifactReport { + return { schema: "skillsync.artifacts-report/v1", status: "passed", exitCode: 0, + evidence: "physical-files", execution: "not-run", provenance: "not-authenticated", + receipt_binding: "verified", files: [{ path: "expected.csv", bytes: 70, rows: 2, sha256: "a".repeat(64) }], + findings: [], reconciliations: [{ type: "keyed_integer_sum_equals", status: "passed", + source: { path: "expected.csv", key: ["tenant_id", "order_id"], value: "cents" }, + target: { path: "allocations.csv", key: ["tenant_id", "order_id"], value: "cents" }, + sourceKeys: 2, targetKeys: 2, missingKeys: 0, unexpectedKeys: 0, valueMismatches: 0 }] }; +} + +describe("offline artifact acceptance HTML", () => { + it("turns physical evidence into a human decision surface without stronger claims", () => { + const html = renderArtifacts(report(), "html"); + expect(html).toMatch(/^/iu); + expect(html.includes("Declared checks passed")).toBe(true); + expect(html.includes("tenant_id + order_id")).toBe(true); + expect(html.includes("expected.csv")).toBe(true); + expect(html.includes("execution: not-run")).toBe(true); + expect(html.includes("provenance: not-authenticated")).toBe(true); + expect(html.includes("Receipt / file binding")).toBe(true); + expect(html.includes(VERSION)).toBe(true); + expect(html.includes("default-src 'none'")).toBe(true); + expect(html).not.toMatch(/ { + const value = report(); value.status = "failed"; value.exitCode = 1; + value.reconciliations![0].status = "failed"; value.reconciliations![0].valueMismatches = 2; + value.findings = [{ code: "artifact.reconciliation-sum", path: "allocations.csv", field: "cents" }]; + const html = renderArtifacts(value, "html"); + expect(html.includes("Review required")).toBe(true); + expect(html.includes("Mismatched values")).toBe(true); + expect(html.includes('data-value-mismatches="2"')).toBe(true); + expect(html.includes("Compare expected and delivered records using the declared key columns")).toBe(true); + expect(html.includes("Declared checks passed")).toBe(false); + }); + + it("keeps incomplete reconciliation distinct from a zero mismatch result", () => { + const value = report(); value.status = "failed"; value.exitCode = 1; + const { type, source, target } = value.reconciliations![0]; + value.reconciliations = [{ type, source, target, status: "unknown" }]; + value.findings = [{ code: "artifact.reconciliation-capacity" }]; + const html = renderArtifacts(value, "html"); + expect(html.includes("Unknown — check incomplete")).toBe(true); + expect(html.includes("No partial result is accepted")).toBe(true); + expect(html.includes('data-value-mismatches="0"')).toBe(false); + }); + + it("redacts local paths first and escapes every caller-controlled metadata surface", () => { + const value = report(); + value.files[0].path = '/private/tmp/LOCAL_PATH_SENTINEL'; + value.findings = [{ code: 'CODE_SENTINEL', + path: "relative/", field: ['KEY<&"', 'NEXTFIELD_SENTINEL'], + message: 'MESSAGE_SENTINEL at /private/tmp/LOCAL_MESSAGE_SENTINEL', row: 7 }]; + value.reconciliations![0].target.path = "nested/DESTINATION_SENTINEL.csv"; + value.reconciliations![0].source.key = ['KEY_SENTINEL', 'a&b']; + const html = renderArtifacts(value, "html"); + expect(html).not.toMatch(/KEY_SENTINEL/iu); + expect(html.includes("LOCAL_PATH_SENTINEL")).toBe(false); + expect(html.includes("LOCAL_MESSAGE_SENTINEL")).toBe(false); + expect(html.includes("<local-path>")).toBe(true); + expect(html.includes("<script>PATH_SENTINEL</script>")).toBe(true); + expect(html.includes("KEY<&"")).toBe(true); + expect(html.includes("<i>DESTINATION_SENTINEL</i>")).toBe(true); + }); + + it("bounds human details while retaining truthful full-scope counts", () => { + const value = report(); + value.files = Array.from({ length: 1000 }, (_, i) => ({ path: `file-${i}.csv`, bytes: 1, sha256: "a".repeat(64) })); + value.reconciliations = Array.from({ length: 128 }, () => ({ ...report().reconciliations![0] })); + value.findings = Array.from({ length: 256 }, () => ({ code: "artifact.json-value", message: "x".repeat(10_000) })); + const html = renderArtifacts(value, "html"); + expect(html.includes("Showing 100 of 1000 files")).toBe(true); + expect(html.includes("Showing 64 of 128 reconciliation checks")).toBe(true); + expect(html.includes("Metadata values longer than 160 characters are shortened")).toBe(true); + expect(html.includes("file-100.csv")).toBe(false); + expect(html.includes("x".repeat(161))).toBe(false); + expect(Buffer.byteLength(html)).toBeLessThan(2 * 1024 * 1024); + }); +}); diff --git a/tests/reporters/sarif.test.ts b/tests/reporters/sarif.test.ts index 150452f..096c2ab 100644 --- a/tests/reporters/sarif.test.ts +++ b/tests/reporters/sarif.test.ts @@ -44,7 +44,7 @@ describe("renderSarif", () => { expect(sarif.version).toBe("2.1.0"); expect(sarif.runs[0]?.tool.driver.name).toBe("skillsync"); - expect(sarif.runs[0]?.tool.driver.version).toBe("0.1.6"); + expect(sarif.runs[0]?.tool.driver.version).toBe("0.1.7"); expect(sarif.runs[0]?.tool.driver.rules[0]?.id).toBe("structure.missing-reference"); expect(sarif.runs[0]?.tool.driver.rules[0]?.help?.text).toBe("Add the referenced file."); expect(sarif.runs[0]?.results[0]?.ruleId).toBe("structure.missing-reference");