diff --git a/CHANGELOG.md b/CHANGELOG.md index a99ec23..e0436b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +- Source `https_url` artifact fields validate bounded HTTPS addresses + against exact declared DNS hosts, with an offline knowledge-source inventory. + The checker never fetches sources, retains URL values in reports or proves truth. - Candidate `artifacts` command validates actual local files with strict capacity, CSV/JSON contracts, digest binding and cross-file summaries. Reports omit values and local paths. Shared scans use bounded reads, metadata allowlists, streamed diff --git a/README.md b/README.md index 38e21ed..e1e4934 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,8 @@ The command above verifies the included sample Skill. Replace the fixture path w - The local source candidate `artifacts` command independently checks actual CSV/JSON delivery files, Receipt/file hashes and cross-file count/integer sums. See [actual artifact checks](docs/artifact-delivery.md); it runs no Skill code. + The [knowledge source inventory](fixtures/product/source-index/README.md) adds + exact declared HTTPS hosts to physical CSV/JSON checks without fetching sources. - `verify` reviews one local Skill for provenance, target compatibility, and changes without running its scripts. - `scan` inventories local Skills, while `compat` checks their declared features against agent profiles. - `diff` shows the meaningful changes between two Skill versions before you accept them. diff --git a/docs/artifact-delivery.md b/docs/artifact-delivery.md index 74d8c89..b29ee32 100644 --- a/docs/artifact-delivery.md +++ b/docs/artifact-delivery.md @@ -32,6 +32,22 @@ The [synthetic contract](../fixtures/product/order-summary/contract.json) declar - independently recomputed CSV record counts and safe integer sums compared with declared JSON fields. Integer sums avoid unspecified floating-point tolerances. +Source builds support `https_url` scalar fields in CSV +columns and JSON top-level fields. Every such field requires `allowed_hosts`: +1–16 unique, exact lowercase ASCII DNS names. Wildcards, bare localhost and IP +literals are not host declarations. URL values are limited to 2,048 characters; +literal whitespace/control/format characters and backslashes are rejected. +The native URL parser must return HTTPS, the exact declared hostname, no userinfo +and the default port (explicit443 is accepted). Host suffixes and undeclared +subdomains fail. Other scalar types cannot carry `allowed_hosts`. + +The [synthetic knowledge source inventory](../fixtures/product/source-index/README.md) +uses real CSV/JSON files plus existing unique-ID, date, capacity and summary rules. +This extension is not included in published npm0.1.0. It does not fetch URLs, resolve DNS, +follow redirects, assess the source text or prove that a named host is public or +trustworthy. Exact URL spelling remains unchanged for uniqueness comparisons. +Optional JSON fields may be omitted; an empty URL does not represent a source. + `bytes` rules check integrity and capacity without interpreting content. JSON checks cover declared top-level fields, not full JSON Schema or arbitrary expressions. Contract files are limited to 64 KiB. Limits must be explicit and diff --git a/docs/domain-adaptation.md b/docs/domain-adaptation.md index 943c2d2..1f5a6d3 100644 --- a/docs/domain-adaptation.md +++ b/docs/domain-adaptation.md @@ -43,7 +43,10 @@ not vendor certification, and unknown/runtime-dependent features stay explicit. ## Next slices -- Add versioned input-schema checks and bounded fixture resources. +- The source [knowledge source inventory](../fixtures/product/source-index/README.md) + checks actual bounded CSV/JSON, exact HTTPS host declarations and independent + reference counts. It is not in npm0.1.0 and does not verify external source truth. +- Add further versioned input-schema checks and bounded fixture resources. - Add references and output-evidence requirements for additional domains. - Validate a separately approved local execution backend using synthetic inputs before describing it as an actual integration. diff --git a/fixtures/product/source-index/README.md b/fixtures/product/source-index/README.md new file mode 100644 index 0000000..deb8e6f --- /dev/null +++ b/fixtures/product/source-index/README.md @@ -0,0 +1,21 @@ +# Synthetic knowledge source inventory + +This source example checks two fictional document references in actual CSV/JSON +files. It reuses the physical artifact command and independent count/sum checks. +The `https_url` field adds an exact, explicitly declared DNS-host requirement. +The example.test domains and SYN-NOTE identifiers are synthetic; no website is +visited, source content verified, vault opened or provider executed. + +After building the source checkout: + +```bash +npm run build +node dist/cli/index.js artifacts \ + --contract fixtures/product/source-index/contract.json \ + --path fixtures/product/source-index/artifacts --format json +``` + +Declared bounds: two files, 64 KiB per file, 128 KiB total, 1,000 CSV records. +URL fields are at most 2,048 characters with at most 16 exact declared hosts. +The fixture remains unchanged during negative checks; tests operate on copies. +The new URL type is available from source and is not in published npm0.1.0. diff --git a/fixtures/product/source-index/artifacts/sources.csv b/fixtures/product/source-index/artifacts/sources.csv new file mode 100644 index 0000000..81ebb9b --- /dev/null +++ b/fixtures/product/source-index/artifacts/sources.csv @@ -0,0 +1,3 @@ +document_id,source_url,checked_on,reference_count +SYN-NOTE-001,https://docs.example.test/reference-one,2026-10-07,1 +SYN-NOTE-002,https://papers.example.test/reference-two,2026-10-07,2 diff --git a/fixtures/product/source-index/artifacts/summary.json b/fixtures/product/source-index/artifacts/summary.json new file mode 100644 index 0000000..c9ace3c --- /dev/null +++ b/fixtures/product/source-index/artifacts/summary.json @@ -0,0 +1,5 @@ +{ + "schema_version": 1, + "row_count": 2, + "total_references": 3 +} diff --git a/fixtures/product/source-index/contract.json b/fixtures/product/source-index/contract.json new file mode 100644 index 0000000..1deb09a --- /dev/null +++ b/fixtures/product/source-index/contract.json @@ -0,0 +1,37 @@ +{ + "schema": "skillsync.artifacts/v1", + "limits": { "max_files": 2, "max_file_bytes": 65536, "max_total_bytes": 131072 }, + "files": [ + { + "path": "sources.csv", + "format": "csv", + "columns": [ + { "name": "document_id", "type": "string" }, + { "name": "source_url", "type": "https_url", "allowed_hosts": ["docs.example.test", "papers.example.test"] }, + { "name": "checked_on", "type": "date" }, + { "name": "reference_count", "type": "integer", "min": 0 } + ], + "min_rows": 1, + "max_rows": 1000, + "unique_by": ["document_id"] + }, + { + "path": "summary.json", + "format": "json", + "fields": [ + { "name": "schema_version", "type": "integer", "equals": 1 }, + { "name": "row_count", "type": "integer", "min": 0 }, + { "name": "total_references", "type": "integer", "min": 0 } + ] + } + ], + "checks": [ + { + "type": "csv_summary", + "csv": "sources.csv", + "json": "summary.json", + "row_count_field": "row_count", + "sums": [{ "column": "reference_count", "field": "total_references" }] + } + ] +} diff --git a/src/domain/artifact-content.ts b/src/domain/artifact-content.ts index dc09631..6c3d77b 100644 --- a/src/domain/artifact-content.ts +++ b/src/domain/artifact-content.ts @@ -43,11 +43,23 @@ function validDate(value: string): boolean { return Number.isFinite(time) && new Date(time).toISOString().slice(0, 10) === value; } +function validSourceUrl(value: unknown, rule: ArtifactScalarRule): boolean { + // Parse one bounded scalar; never resolve DNS, fetch a source or retain its value. + if (typeof value !== "string" || value.length > 2048 || !/^https:\/\//i.test(value) || + /[\p{Cc}\p{Cf}\s\\]/u.test(value)) return false; + try { + const url = new URL(value); + return url.protocol === "https:" && !url.username && !url.password && !url.port && + (rule.allowed_hosts ?? []).includes(url.hostname); + } catch { return false; } +} + function scalarValid(value: unknown, rule: ArtifactScalarRule): boolean { if (rule.type === "integer" && (typeof value !== "number" || !Number.isSafeInteger(value))) return false; if (rule.type === "number" && (typeof value !== "number" || !Number.isFinite(value))) return false; if (rule.type === "string" && (typeof value !== "string" || (!rule.allow_empty && !value.length))) return false; if (rule.type === "date" && (typeof value !== "string" || !validDate(value))) return false; + if (rule.type === "https_url" && !validSourceUrl(value, rule)) return false; if (rule.type === "boolean" && typeof value !== "boolean") return false; if (rule.type === "array" && !Array.isArray(value)) return false; if (rule.type === "object" && (!value || typeof value !== "object" || Array.isArray(value))) return false; diff --git a/src/domain/artifact-contract.ts b/src/domain/artifact-contract.ts index 1ffe996..17f15e4 100644 --- a/src/domain/artifact-contract.ts +++ b/src/domain/artifact-contract.ts @@ -7,9 +7,14 @@ const safePath = z.string().min(1).max(1024).refine(path => !/^[A-Za-z]:/.test(path) && path.split("/").every(part => part && part !== "." && part !== ".."), "artifact paths must be safe and relative"); const hash = z.string().regex(/^[0-9a-f]{64}$/); +const allowedHost = z.string().min(3).max(253).regex( + /^(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z][a-z0-9-]{0,61}[a-z0-9]$/, + "source hosts must be exact lowercase DNS names", +); const scalar = z.object({ name: z.string().min(1).max(128), - type: z.enum(["string", "integer", "number", "boolean", "date", "array", "object"]), + type: z.enum(["string", "integer", "number", "boolean", "date", "https_url", "array", "object"]), + allowed_hosts: z.array(allowedHost).min(1).max(16).optional(), required: z.boolean().default(true), equals: z.union([z.string(), z.number().finite(), z.boolean()]).optional(), min: z.number().finite().optional(), max: z.number().finite().optional(), @@ -51,10 +56,12 @@ export const artifactContractSchema = z.object({ schema: z.literal("skillsync.ar const rules = file.format === "csv" ? file.columns : file.format === "json" ? file.fields : []; if (new Set(rules.map(rule => rule.name)).size !== rules.length) fail(); for (const rule of rules) { + if (rule.type === "https_url" ? !rule.allowed_hosts || + new Set(rule.allowed_hosts).size !== rule.allowed_hosts.length : rule.allowed_hosts !== undefined) fail(); if ((rule.min !== undefined || rule.max !== undefined) && !["integer", "number"].includes(rule.type)) fail(); if (rule.equals !== undefined && ((rule.type === "integer" && !Number.isSafeInteger(rule.equals)) || (rule.type === "number" && typeof rule.equals !== "number") || - (["string", "date"].includes(rule.type) && typeof rule.equals !== "string") || + (["string", "date", "https_url"].includes(rule.type) && typeof rule.equals !== "string") || (rule.type === "boolean" && typeof rule.equals !== "boolean") || ["array", "object"].includes(rule.type))) fail(); } if (file.format === "csv" && (file.min_rows > file.max_rows || diff --git a/tests/domain/artifact-source-url.test.ts b/tests/domain/artifact-source-url.test.ts new file mode 100644 index 0000000..401ee74 --- /dev/null +++ b/tests/domain/artifact-source-url.test.ts @@ -0,0 +1,81 @@ +import { describe, expect, it } from "vitest"; +import { parseArtifactContract } from "../../src/domain/artifact-contract"; +import { inspectArtifactContent } from "../../src/domain/artifact-content"; + +function contract(field: Record = {}) { + return { schema: "skillsync.artifacts/v1", + limits: { max_files: 1, max_file_bytes: 4096, max_total_bytes: 4096 }, + files: [{ path: "source.json", format: "json", fields: [{ + name: "source_url", type: "https_url", allowed_hosts: ["docs.example.test"], ...field, + }] }] }; +} + +function inspect(value: string) { + const rule = parseArtifactContract(contract()).files[0]; + return inspectArtifactContent(rule, Buffer.from(JSON.stringify({ source_url: value }))); +} + +describe("declared knowledge source URLs", () => { + it.each([ + "https://docs.example.test/reference", + "HTTPS://DOCS.EXAMPLE.TEST/reference?language=zh#section", + "https://docs.example.test:443/reference", + "https://docs.example.test/a%20reference", + ])("accepts HTTPS on the exact declared host: %s", value => { + expect(inspect(value).findings).toEqual([]); + }); + + it.each([ + "http://docs.example.test/reference", + "javascript:alert(1)", + "file:///synthetic/reference.md", + "/reference", + "https://docs.example.test.attacker.test/reference", + "https://extra.docs.example.test/reference", + "https://demo-reader@docs.example.test/reference", + "https://docs.example.test@attacker.test/reference", + "https://docs.example.test:8443/reference", + "https://docs.example.test./reference", + "https://127.0.0.1/reference", + "https://localhost/reference", + " https://docs.example.test/reference", + "https://docs.example.test/ref erence", + "https://docs.example.test/ref\nerence", + "https://docs.example.test/ref\u200Berence", + "https:\\docs.example.test\\reference", + "https://docs.example.test/" + "x".repeat(2048), + ])("rejects unsafe or undeclared addresses without returning their values: %s", value => { + const report = inspect(value); + expect(report.findings).toContainEqual({ code: "artifact.json-value", path: "source.json", field: "source_url" }); + expect(JSON.stringify(report.findings)).not.toContain(value); + }); + + it("uses the same URL rule for actual CSV fields", () => { + const rule = parseArtifactContract({ ...contract(), files: [{ path: "sources.csv", format: "csv", + columns: [{ name: "source_url", type: "https_url", allowed_hosts: ["docs.example.test"] }], max_rows: 5 }] }).files[0]; + expect(inspectArtifactContent(rule, Buffer.from('source_url\n"https://docs.example.test/reference?a=1,b=2"\n')).findings).toEqual([]); + expect(inspectArtifactContent(rule, Buffer.from("source_url\nhttps://docs.example.test.attacker.test/reference\n")).findings) + .toContainEqual({ code: "artifact.csv-value", path: "sources.csv", field: "source_url", row: 1 }); + }); + + it.each([ + { allowed_hosts: undefined }, { allowed_hosts: [] }, + { allowed_hosts: ["*.example.test"] }, { allowed_hosts: ["DOCS.EXAMPLE.TEST"] }, + { allowed_hosts: ["127.0.0.1"] }, { allowed_hosts: ["localhost"] }, + { allowed_hosts: ["docs.example.test."] }, { allowed_hosts: ["-docs.example.test"] }, + { allowed_hosts: ["docs.example.test", "docs.example.test"] }, + { allowed_hosts: Array.from({ length: 17 }, (_, index) => `host${index}.example.test`) }, + { type: "string", allowed_hosts: ["docs.example.test"] }, + { type: "https_url", equals: 1 }, + ])("requires a bounded exact host declaration: %j", field => { + expect(() => parseArtifactContract(contract(field))).toThrow("artifact contract is invalid"); + }); + + it("does not retain URL values in facts or findings", () => { + const value = "https://docs.example.test/private-synthetic-title"; + const report = inspect(value); + expect(report.findings).toEqual([]); + expect(report.facts.integers.size).toBe(0); + expect(JSON.stringify(report)).not.toContain(value); + }); +}); diff --git a/tests/integration/source-index.test.ts b/tests/integration/source-index.test.ts new file mode 100644 index 0000000..33e102c --- /dev/null +++ b/tests/integration/source-index.test.ts @@ -0,0 +1,65 @@ +import { cp, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { describe, expect, it } from "vitest"; +import { runArtifacts, renderArtifacts } from "../../src/cli/commands/artifacts"; +import { createCli } from "../../src/cli/index"; + +const source = resolve("fixtures/product/source-index"); +async function fixture(run: (contract: string, data: string) => Promise) { + const root = await mkdtemp(join(tmpdir(), "skillsync-source-index-")); + try { + await cp(join(source, "artifacts"), join(root, "artifacts"), { recursive: true }); + await cp(join(source, "contract.json"), join(root, "contract.json")); + await run(join(root, "contract.json"), join(root, "artifacts")); + } finally { await rm(root, { recursive: true, force: true }); } +} + +describe("physical knowledge source inventory", () => { + it("accepts two declared sources with a matching summary without executing a provider", async () => fixture(async (contract, data) => { + const report = await runArtifacts({ contract, path: data }); + expect(report.status).toBe("passed"); + expect(report.evidence).toBe("physical-files"); + expect(report.execution).toBe("not-run"); + expect(report.provenance).toBe("not-authenticated"); + expect(report.files.find(file => file.path === "sources.csv")?.rows).toBe(2); + expect(report.files).toHaveLength(2); + })); + + it("rejects a lookalike host through the CLI without exposing URL or local paths", async () => fixture(async (contract, data) => { + const file = join(data, "sources.csv"); + const value = "https://docs.example.test.attacker.test/PRIVATE_SYNTHETIC_TITLE"; + await writeFile(file, (await readFile(file, "utf8")).replace("https://docs.example.test/reference-one", value)); + const report = await runArtifacts({ contract, path: data }); + expect(report.findings).toContainEqual({ code: "artifact.csv-value", path: "sources.csv", field: "source_url", row: 1 }); + for (const format of ["json", "text"]) { + const output = renderArtifacts(report, format); + expect(output).not.toContain(value); + expect(output).not.toContain("PRIVATE_SYNTHETIC_TITLE"); + expect(output).not.toContain(data); + } + let output = "", exitCode = 0; + const program = createCli({ writeOut: text => { output += text; }, writeErr: () => undefined, setExitCode: code => { exitCode = code; } }); + await program.parseAsync(["node", "skillsync", "artifacts", "--contract", contract, "--path", data, "--format", "json"]); + expect(exitCode).toBe(1); + expect(JSON.parse(output).status).toBe("failed"); + expect(output).not.toContain(value); + })); + + it("independently rejects invented reference totals", async () => fixture(async (contract, data) => { + await writeFile(join(data, "summary.json"), JSON.stringify({ schema_version: 1, row_count: 2, total_references: 99 })); + const report = await runArtifacts({ contract, path: data }); + expect(report.status).toBe("failed"); + expect(report.findings).toContainEqual({ code: "artifact.summary-sum", path: "summary.json", field: "total_references" }); + })); + + it("retains duplicate-document and capacity checks", async () => fixture(async (contract, data) => { + const file = join(data, "sources.csv"); + await writeFile(file, (await readFile(file, "utf8")).replace("SYN-NOTE-002", "SYN-NOTE-001")); + expect((await runArtifacts({ contract, path: data })).findings.some(finding => finding.code === "artifact.csv-duplicate")).toBe(true); + const declaration = JSON.parse(await readFile(contract, "utf8")); + declaration.limits.max_file_bytes = 1; + await writeFile(contract, JSON.stringify(declaration)); + expect((await runArtifacts({ contract, path: data })).findings[0].code).toBe("artifact.capacity"); + })); +});