Repository navigation
feat(cli): trust-report and clean-install E2E for testing your own agent - #686
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
d29005c to
ac40494
Compare
…agent Two paths, kept apart. Path 1 compares what an agent said with GitHub's record of a run and needs no access to the agent. Path 2 covers only commands started through trustshell run. trust-report joins them into one readable report with its evidence, and verify proves the folder is whole, matches that evidence and was sealed by a pinned key. scripts/release-e2e.mjs packs the release, installs it into an empty folder with no keys and runs each step in every shell, tracing the network. run no longer prints started for a command that could not start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In a real run, a last invocation that selected no tests exited 0 and counted as support. A claim paired with a command that ran with more than one command line is now NOT_CHECKED. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
64dcc71 to
a9724a8
Compare
Strix Security ReviewTo fix before merging (5)
PR summary: Adds a `trustshell trust-report` command that builds a readable report and a sealed, verifiable evidence folder from `check --out` records and the Guard ledger, plus a `trust-report verify` subcommand, a clean-install release E2E harness with a CI workflow, and a fix so `run` no longer reports "started" for a command that could not start. Review summaryThis PR adds The problems are in what the report treats as trustworthy evidence. The report is built to be handed to a third party, but every input it presents as proof is writable by the agent under test: the The CI harness and the Ruled out (9)
Fixed the findings? re-run the review, or tag Review 1 on this PR · Reviewed by Strix |
| if (f.includes('..') || !existsSync(p)) problems.push(`${f} is listed but missing`); | ||
| else if (sha256(readFileSync(p)) !== want) problems.push(`${f} changed after the report was made`); |
There was a problem hiding this comment.
🔵 trust-report verify prints attacker-controlled filenames from a stranger's report folder to the terminal without stripping escape sequences
Severity: MEDIUM · CWE-150
trustshell trust-report verify <dir> is designed to be run on a report folder a stranger sent you. Three of its problems entries interpolate a filename taken straight from that folder — from MANIFEST.sha256 lines, from the directory walk, and from report.json's inputs.records — and the problems are printed to stdout with no cleanCodePoints pass, unlike every other field in the file. A folder whose manifest entry or file name contains ANSI/OSC bytes therefore writes those bytes to the operator's terminal: the verdict line can be erased and rewritten, the window title spoofed, and the clipboard overwritten via OSC 52.
| if (f.includes('..') || !existsSync(p)) problems.push(`${f} is listed but missing`); | |
| else if (sha256(readFileSync(p)) !== want) problems.push(`${f} changed after the report was made`); | |
| if (f.includes('..') || !existsSync(p)) problems.push(`${clean(f, 300)} is listed but missing`); | |
| else if (sha256(readFileSync(p)) !== want) problems.push(`${clean(f, 300)} changed after the report was made`); |
Prompt to fix with AI
This is a security vulnerability found during a code review.
Vulnerability: `trust-report verify` prints attacker-controlled filenames from a stranger's report folder to the terminal without stripping escape sequences
Severity: MEDIUM
CWE: CWE-150
`trustshell trust-report verify <dir>` is designed to be run on a report folder a stranger sent you. Three of its `problems` entries interpolate a filename taken straight from that folder — from `MANIFEST.sha256` lines, from the directory walk, and from `report.json`'s `inputs.records` — and the problems are printed to stdout with no `cleanCodePoints` pass, unlike every other field in the file. A folder whose manifest entry or file name contains ANSI/OSC bytes therefore writes those bytes to the operator's terminal: the verdict line can be erased and rewritten, the window title spoofed, and the clipboard overwritten via OSC 52.
Location: src/cli/trust-report.ts:529-530
Context: manifest entry name printed raw (primary sink)
```
// Before:
if (f.includes('..') || !existsSync(p)) problems.push(`${f} is listed but missing`);
else if (sha256(readFileSync(p)) !== want) problems.push(`${f} changed after the report was made`);
// After:
if (f.includes('..') || !existsSync(p)) problems.push(`${clean(f, 300)} is listed but missing`);
else if (sha256(readFileSync(p)) !== want) problems.push(`${clean(f, 300)} changed after the report was made`);
```
Location: src/cli/trust-report.ts:532-532
Context: directory-walk filename printed raw
```
// Before:
for (const f of walk(dir)) if (f !== MANIFEST && f !== SEAL && !listed.has(f)) problems.push(`${f} is not in the manifest`);
// After:
for (const f of walk(dir)) if (f !== MANIFEST && f !== SEAL && !listed.has(f)) problems.push(`${clean(f, 300)} is not in the manifest`);
```
Location: src/cli/trust-report.ts:569-569
Context: evidence path from report.json printed raw
```
// Before:
for (const i of again.inspect) if (!i.digest_matches) problems.push(`${i.path}: its digest does not match its contents`);
// After:
for (const i of again.inspect) if (!i.digest_matches) problems.push(`${clean(i.path, 300)}: its digest does not match its contents`);
```
How to fix:
Pass every string that originates in the report folder through the existing `clean()` helper (which wraps `cleanCodePoints`) before it enters the `problems` array, exactly as the rest of `trust-report.ts` already does for `action`, `reasons`, `claim`, `scope`, `run_url` and the seal error. Apply it at the three interpolation sites in `verify()`: the manifest loop, the directory walk, and the per-record digest check. Consider also cleaning `inspectRow`'s `path` at the point it is stored, so any future rendering of that field is safe by construction.
Please fix this vulnerability. If you propose a fix, make it concise and minimal.React 👍 / 👎 to tune Strix for this repo. A repo collaborator (or the PR author) can resolve this thread to dismiss the finding.
| if (f.includes('..') || !existsSync(p)) problems.push(`${f} is listed but missing`); | ||
| else if (sha256(readFileSync(p)) !== want) problems.push(`${f} changed after the report was made`); | ||
| } | ||
| for (const f of walk(dir)) if (f !== MANIFEST && f !== SEAL && !listed.has(f)) problems.push(`${f} is not in the manifest`); |
There was a problem hiding this comment.
directory-walk filename printed raw
| for (const f of walk(dir)) if (f !== MANIFEST && f !== SEAL && !listed.has(f)) problems.push(`${f} is not in the manifest`); | |
| for (const f of walk(dir)) if (f !== MANIFEST && f !== SEAL && !listed.has(f)) problems.push(`${clean(f, 300)} is not in the manifest`); |
| if (renderTrustReportMarkdown(again) !== readLimited(join(dir, 'trust-report.md'))) problems.push('trust-report.md does not match what its evidence produces'); | ||
| if (renderTrustReportHtml(again) !== readLimited(join(dir, 'trust-report.html'))) problems.push('trust-report.html does not match what its evidence produces'); | ||
| if (again.guard.ledger.verdict === 'BROKEN') problems.push('the Guard record chain is broken'); | ||
| for (const i of again.inspect) if (!i.digest_matches) problems.push(`${i.path}: its digest does not match its contents`); |
There was a problem hiding this comment.
evidence path from report.json printed raw
| for (const i of again.inspect) if (!i.digest_matches) problems.push(`${i.path}: its digest does not match its contents`); | |
| for (const i of again.inspect) if (!i.digest_matches) problems.push(`${clean(i.path, 300)}: its digest does not match its contents`); |
| export interface InspectRow { | ||
| path: string; | ||
| run_url: string; | ||
| repo: string; | ||
| run_id: number; | ||
| head_sha: string; | ||
| conclusion: string | null; | ||
| verdict: string; | ||
| fetched_at: string; | ||
| digest: string; | ||
| digest_matches: boolean; | ||
| exit: number; | ||
| claims: { status: string; claim: string; scope: string }[]; | ||
| } |
There was a problem hiding this comment.
🔵 trust-report presents an unauthenticated local record file as GitHub's own record
Severity: MEDIUM · CWE-345
trustshell trust-report --record <file> accepts any local file whose record.schema equals ACTION_EVIDENCE_SCHEMA and whose embedded digest.value matches a SHA-256 the tool recomputes from that same file. Nothing ties the file to check --out, to GitHub, or to any key, so an agent under test can write evidence/record.json itself (or overwrite the one check produced) with any repo, run id, conclusion and claim statuses, recompute the digest, and the report renders those claims as SUPPORTED under the note "Compared with GitHub's own record, read <fetched_at>", exits 0 with "Nothing here contradicts the agent", and trust-report verify returns VERIFIED for a recipient who pins the report author's key. The record's own does_not_prove entry stating the record is unsigned is never rendered, so a third-party recipient sees GitHub-verified evidence where there is only an unauthenticated local file.
| export interface InspectRow { | |
| path: string; | |
| run_url: string; | |
| repo: string; | |
| run_id: number; | |
| head_sha: string; | |
| conclusion: string | null; | |
| verdict: string; | |
| fetched_at: string; | |
| digest: string; | |
| digest_matches: boolean; | |
| exit: number; | |
| claims: { status: string; claim: string; scope: string }[]; | |
| } | |
| export interface InspectRow { | |
| path: string; | |
| run_url: string; | |
| repo: string; | |
| run_id: number; | |
| head_sha: string; | |
| conclusion: string | null; | |
| verdict: string; | |
| fetched_at: string; | |
| digest: string; | |
| digest_matches: boolean; | |
| exit: number; | |
| claims: { status: string; claim: string; scope: string }[]; | |
| /** The record's own limits, carried through so a reader is never shown a claim without them. */ | |
| caveats: string[]; | |
| } |
Prompt to fix with AI
This is a security vulnerability found during a code review.
Vulnerability: trust-report presents an unauthenticated local record file as GitHub's own record
Severity: MEDIUM
CWE: CWE-345
`trustshell trust-report --record <file>` accepts any local file whose `record.schema` equals `ACTION_EVIDENCE_SCHEMA` and whose embedded `digest.value` matches a SHA-256 the tool recomputes from that same file. Nothing ties the file to `check --out`, to GitHub, or to any key, so an agent under test can write `evidence/record.json` itself (or overwrite the one `check` produced) with any repo, run id, conclusion and claim statuses, recompute the digest, and the report renders those claims as SUPPORTED under the note "Compared with GitHub's own record, read <fetched_at>", exits 0 with "Nothing here contradicts the agent", and `trust-report verify` returns VERIFIED for a recipient who pins the report author's key. The record's own `does_not_prove` entry stating the record is unsigned is never rendered, so a third-party recipient sees GitHub-verified evidence where there is only an unauthenticated local file.
Location: src/cli/trust-report.ts:47-60
Context: InspectRow carries no provenance or caveat field, so the record's own limits cannot reach the rendering
```
// Before:
export interface InspectRow {
path: string;
run_url: string;
repo: string;
run_id: number;
head_sha: string;
conclusion: string | null;
verdict: string;
fetched_at: string;
digest: string;
digest_matches: boolean;
exit: number;
claims: { status: string; claim: string; scope: string }[];
}
// After:
export interface InspectRow {
path: string;
run_url: string;
repo: string;
run_id: number;
head_sha: string;
conclusion: string | null;
verdict: string;
fetched_at: string;
digest: string;
digest_matches: boolean;
exit: number;
claims: { status: string; claim: string; scope: string }[];
/** The record's own limits, carried through so a reader is never shown a claim without them. */
caveats: string[];
}
```
Location: src/cli/trust-report.ts:183-202
Context: The only check on a --record file is a digest recomputed from the same file; the record's does_not_prove list is dropped
```
// Before:
function inspectRow(path: string, text: string): InspectRow {
const parsed = JSON.parse(text) as { record?: ActionEvidenceRecord; digest?: RecordDigest };
const record = parsed.record;
if (!record || record.schema !== ACTION_EVIDENCE_SCHEMA) throw new Error(`${path} is not a check --claims record (${ACTION_EVIDENCE_SCHEMA})`);
const o = record.observation;
const digest = digestRecord(record).value;
return {
path,
run_url: clean(o.url, 300),
repo: `${clean(o.owner, 100)}/${clean(o.repo, 100)}`,
run_id: o.run_id,
head_sha: clean(o.head_sha, 80),
conclusion: o.conclusion === null ? null : clean(o.conclusion, 40),
verdict: clean(o.check.verdict, 40),
fetched_at: clean(o.fetchedAt, 40),
digest,
digest_matches: parsed.digest?.value === digest,
exit: claimsExitCode(record),
claims: record.claims.map((c) => ({ status: c.status, claim: clean(c.claim, 300), scope: clean(c.scope, 400) })),
};
// After:
function inspectRow(path: string, text: string): InspectRow {
const parsed = JSON.parse(text) as { record?: ActionEvidenceRecord; digest?: RecordDigest };
const record = parsed.record;
if (!record || record.schema !== ACTION_EVIDENCE_SCHEMA) throw new Error(`${path} is not a check --claims record (${ACTION_EVIDENCE_SCHEMA})`);
const o = record.observation;
const digest = digestRecord(record).value;
return {
path,
run_url: clean(o.url, 300),
repo: `${clean(o.owner, 100)}/${clean(o.repo, 100)}`,
run_id: o.run_id,
head_sha: clean(o.head_sha, 80),
conclusion: o.conclusion === null ? null : clean(o.conclusion, 40),
verdict: clean(o.check.verdict, 40),
fetched_at: clean(o.fetchedAt, 40),
digest,
digest_matches: parsed.digest?.value === digest,
exit: claimsExitCode(record),
claims: record.claims.map((c) => ({ status: c.status, claim: clean(c.claim, 300), scope: clean(c.scope, 400) })),
caveats: (record.does_not_prove ?? []).map((s) => clean(s, 400)),
};
```
Location: src/cli/trust-report.ts:285-286
Context: The rendered note asserts GitHub provenance that was never established, and the record's own caveats are not shown
```
// Before:
note: `Compared with GitHub's own record, read ${i.fetched_at}: ${i.verdict}, conclusion ${i.conclusion ?? 'none'}, commit ${i.head_sha}. TrustShell needed no access to the agent or its keys. ${i.run_url}`,
rows: i.claims.length ? i.claims.map((c) => ({ status: c.status, text: c.claim, detail: c.scope })) : [{ status: 'NOT_CHECKED', text: 'No claims were given for this run.' }],
// After:
note: `Read from a local record file (${i.path}) that TrustShell did not fetch and cannot authenticate: ${i.verdict}, conclusion ${i.conclusion ?? 'none'}, commit ${i.head_sha}, as that file claims. Its digest only shows the file was not edited after it was written, not who wrote it. ${i.run_url}`,
rows: [
...(i.claims.length ? i.claims.map((c) => ({ status: c.status, text: c.claim, detail: c.scope })) : [{ status: 'NOT_CHECKED', text: 'No claims were given for this run.' }]),
...i.caveats.map((text) => ({ text })),
],
```
How to fix:
Do not describe a `--record` file as GitHub's own record unless its provenance was actually established. Either re-fetch the run from `api.github.com` and compare the observation and claim statuses against the live answer, or label the section as an unauthenticated local file and state plainly that TrustShell did not fetch it and cannot tell who wrote it. Carry the record's own `does_not_prove` list (and its `assurance.cryptographic_integrity` value) into every rendering so the "not signed" caveat reaches the reader, and stop treating a self-computed digest as an integrity control: it only detects accidental edits, and anyone who can write the file can recompute it. If a record is meant to be trustworthy evidence, have `check --out` seal it with the install key and have `trust-report` verify that seal before rendering the claims.
Please fix this vulnerability. If you propose a fix, make it concise and minimal.React 👍 / 👎 to tune Strix for this repo. A repo collaborator (or the PR author) can resolve this thread to dismiss the finding.
| function inspectRow(path: string, text: string): InspectRow { | ||
| const parsed = JSON.parse(text) as { record?: ActionEvidenceRecord; digest?: RecordDigest }; | ||
| const record = parsed.record; | ||
| if (!record || record.schema !== ACTION_EVIDENCE_SCHEMA) throw new Error(`${path} is not a check --claims record (${ACTION_EVIDENCE_SCHEMA})`); | ||
| const o = record.observation; | ||
| const digest = digestRecord(record).value; | ||
| return { | ||
| path, | ||
| run_url: clean(o.url, 300), | ||
| repo: `${clean(o.owner, 100)}/${clean(o.repo, 100)}`, | ||
| run_id: o.run_id, | ||
| head_sha: clean(o.head_sha, 80), | ||
| conclusion: o.conclusion === null ? null : clean(o.conclusion, 40), | ||
| verdict: clean(o.check.verdict, 40), | ||
| fetched_at: clean(o.fetchedAt, 40), | ||
| digest, | ||
| digest_matches: parsed.digest?.value === digest, | ||
| exit: claimsExitCode(record), | ||
| claims: record.claims.map((c) => ({ status: c.status, claim: clean(c.claim, 300), scope: clean(c.scope, 400) })), | ||
| }; |
There was a problem hiding this comment.
The only check on a --record file is a digest recomputed from the same file; the record's does_not_prove list is dropped
| function inspectRow(path: string, text: string): InspectRow { | |
| const parsed = JSON.parse(text) as { record?: ActionEvidenceRecord; digest?: RecordDigest }; | |
| const record = parsed.record; | |
| if (!record || record.schema !== ACTION_EVIDENCE_SCHEMA) throw new Error(`${path} is not a check --claims record (${ACTION_EVIDENCE_SCHEMA})`); | |
| const o = record.observation; | |
| const digest = digestRecord(record).value; | |
| return { | |
| path, | |
| run_url: clean(o.url, 300), | |
| repo: `${clean(o.owner, 100)}/${clean(o.repo, 100)}`, | |
| run_id: o.run_id, | |
| head_sha: clean(o.head_sha, 80), | |
| conclusion: o.conclusion === null ? null : clean(o.conclusion, 40), | |
| verdict: clean(o.check.verdict, 40), | |
| fetched_at: clean(o.fetchedAt, 40), | |
| digest, | |
| digest_matches: parsed.digest?.value === digest, | |
| exit: claimsExitCode(record), | |
| claims: record.claims.map((c) => ({ status: c.status, claim: clean(c.claim, 300), scope: clean(c.scope, 400) })), | |
| }; | |
| function inspectRow(path: string, text: string): InspectRow { | |
| const parsed = JSON.parse(text) as { record?: ActionEvidenceRecord; digest?: RecordDigest }; | |
| const record = parsed.record; | |
| if (!record || record.schema !== ACTION_EVIDENCE_SCHEMA) throw new Error(`${path} is not a check --claims record (${ACTION_EVIDENCE_SCHEMA})`); | |
| const o = record.observation; | |
| const digest = digestRecord(record).value; | |
| return { | |
| path, | |
| run_url: clean(o.url, 300), | |
| repo: `${clean(o.owner, 100)}/${clean(o.repo, 100)}`, | |
| run_id: o.run_id, | |
| head_sha: clean(o.head_sha, 80), | |
| conclusion: o.conclusion === null ? null : clean(o.conclusion, 40), | |
| verdict: clean(o.check.verdict, 40), | |
| fetched_at: clean(o.fetchedAt, 40), | |
| digest, | |
| digest_matches: parsed.digest?.value === digest, | |
| exit: claimsExitCode(record), | |
| claims: record.claims.map((c) => ({ status: c.status, claim: clean(c.claim, 300), scope: clean(c.scope, 400) })), | |
| caveats: (record.does_not_prove ?? []).map((s) => clean(s, 400)), | |
| }; |
| const ledgerPath = join(home, GUARD_RECEIPTS_FILE); | ||
| const policyPath = join(home, POLICY_FILE); | ||
| const ledgerText = existsSync(ledgerPath) ? readLimited(ledgerPath) : ''; | ||
| const policyText = existsSync(policyPath) ? readLimited(policyPath) : null; |
There was a problem hiding this comment.
reads the unkeyed, agent-writable ledger that the report presents as evidence
| export interface CommandRow { | ||
| seq: number; | ||
| ts: string; | ||
| action: string; | ||
| script_sha256: string | null; | ||
| command_sha256: string | null; | ||
| decision: string; | ||
| reasons: string[]; | ||
| result: string; | ||
| exit_code: number | null; | ||
| } |
There was a problem hiding this comment.
🔵 trust-report marks a bound claim SUPPORTED from a name-only run and hides the executable identity
Severity: MEDIUM · CWE-345
trustshell trust-report turns a trustshell run record into a sealed SUPPORTED verdict without recording which executable actually ran. bindMatches matches a --bind tool:<name> claim on the action label alone and CommandRow carries no executable identity, so a run of any file whose basename equals the tool name counts. Because run pins an executable only when the policy opts in (guard allow tool:<name> --exe <path>), an agent under test can drop a script named pytest in the working directory, run trustshell run --tool pytest -- ./pytest, and the report renders SUPPORTED All 412 tests pass and ALLOWED #3 tool:pytest: exited 0 — identical to a genuinely pinned run — while trust-report verify returns VERIFIED. The run card warns name-only (weaker) on stderr at run time, but that warning never reaches the artifact a third party reads, and the report's own "what this cannot see" list omits executable identity.
| export interface CommandRow { | |
| seq: number; | |
| ts: string; | |
| action: string; | |
| script_sha256: string | null; | |
| command_sha256: string | null; | |
| decision: string; | |
| reasons: string[]; | |
| result: string; | |
| exit_code: number | null; | |
| } | |
| export interface CommandRow { | |
| seq: number; | |
| ts: string; | |
| action: string; | |
| script_sha256: string | null; | |
| command_sha256: string | null; | |
| /** The executable name `run` derived from the command, and its sha256 only when the policy pinned it. */ | |
| executable: string | null; | |
| executable_sha256: string | null; | |
| decision: string; | |
| reasons: string[]; | |
| result: string; | |
| exit_code: number | null; | |
| } |
Prompt to fix with AI
This is a security vulnerability found during a code review.
Vulnerability: trust-report marks a bound claim SUPPORTED from a name-only run and hides the executable identity
Severity: MEDIUM
CWE: CWE-345
`trustshell trust-report` turns a `trustshell run` record into a sealed SUPPORTED verdict without recording which executable actually ran. `bindMatches` matches a `--bind tool:<name>` claim on the action label alone and `CommandRow` carries no executable identity, so a run of any file whose basename equals the tool name counts. Because `run` pins an executable only when the policy opts in (`guard allow tool:<name> --exe <path>`), an agent under test can drop a script named `pytest` in the working directory, run `trustshell run --tool pytest -- ./pytest`, and the report renders `SUPPORTED All 412 tests pass` and `ALLOWED #3 tool:pytest: exited 0` — identical to a genuinely pinned run — while `trust-report verify` returns VERIFIED. The run card warns `name-only (weaker)` on stderr at run time, but that warning never reaches the artifact a third party reads, and the report's own "what this cannot see" list omits executable identity.
Location: src/cli/trust-report.ts:62-72
Context: CommandRow drops the executable identity the record already holds
```
// Before:
export interface CommandRow {
seq: number;
ts: string;
action: string;
script_sha256: string | null;
command_sha256: string | null;
decision: string;
reasons: string[];
result: string;
exit_code: number | null;
}
// After:
export interface CommandRow {
seq: number;
ts: string;
action: string;
script_sha256: string | null;
command_sha256: string | null;
/** The executable name `run` derived from the command, and its sha256 only when the policy pinned it. */
executable: string | null;
executable_sha256: string | null;
decision: string;
reasons: string[];
result: string;
exit_code: number | null;
}
```
Location: src/cli/trust-report.ts:147-157
Context: commandRows never copies requested.executable / executable_sha256
```
// Before:
const req = r.requested as { action: string; script_sha256?: string; command_sha256?: string };
rows.push({
seq: r.seq,
ts: r.ts,
action: clean(req.action, 140),
script_sha256: req.script_sha256 ?? null,
command_sha256: req.command_sha256 ?? null,
decision: r.authorized?.decision ?? 'UNKNOWN',
reasons: (r.authorized?.reasons ?? []).map((x) => clean(x, 300)),
...resultOf(outcome, r.kind, r.authorized?.decision),
});
// After:
const req = r.requested as { action: string; script_sha256?: string; command_sha256?: string; executable?: string; executable_sha256?: string };
rows.push({
seq: r.seq,
ts: r.ts,
action: clean(req.action, 140),
script_sha256: req.script_sha256 ?? null,
command_sha256: req.command_sha256 ?? null,
executable: req.executable === undefined ? null : clean(req.executable, 140),
executable_sha256: req.executable_sha256 ?? null,
decision: r.authorized?.decision ?? 'UNKNOWN',
reasons: (r.authorized?.reasons ?? []).map((x) => clean(x, 300)),
...resultOf(outcome, r.kind, r.authorized?.decision),
});
```
Location: src/cli/trust-report.ts:178-178
Context: SUPPORTED verdict with no mention that the executable was name-only
```
// Before:
if (last.exit_code === 0) return { ...base, status: 'SUPPORTED', why: `the last time it ran through trustshell run it exited 0 (${scope}). That shows this command passed, not that it tests what the claim says.`, record_seq: last.seq };
// After:
if (last.exit_code === 0) {
const identity =
b.script_sha256 || last.executable_sha256 !== null
? ''
: ` The policy did not pin ${last.executable ?? b.tool} to a file, so any executable with that name would have counted: pin it with trustshell guard allow tool:${b.tool} --exe <path> and run it again.`;
return { ...base, status: 'SUPPORTED', why: `the last time it ran through trustshell run it exited 0 (${scope}). That shows this command passed, not that it tests what the claim says.${identity}`, record_seq: last.seq };
}
```
Location: src/cli/trust-report.ts:305-305
Context: Path 2 renders the action label only, never the executable or its pin state
```
// Before:
rows: g.commands.map((c) => ({ status: c.decision, text: `#${c.seq} ${c.action}: ${c.result}`, detail: c.reasons.join('; ') || undefined })),
// After:
rows: g.commands.map((c) => ({
status: c.decision,
text: `#${c.seq} ${c.action}: ${c.result}`,
detail:
[
c.executable === null ? null : `starts ${c.executable} (${c.executable_sha256 === null ? 'name-only: not pinned to a file' : `pinned, sha256 ${c.executable_sha256.slice(0, 12)}…`})`,
c.reasons.join('; ') || null,
]
.filter(Boolean)
.join('; ') || undefined,
})),
```
How to fix:
Record the executable name, and its sha256 when the policy pins it, in each command row, and render that identity beside the command in both the Markdown and HTML report. When a claim is bound by tool name and the run recorded no pinned executable hash, say so in the claim's explanation and point the reader at `trustshell guard allow tool:<name> --exe <path>`. Treating a tool-name bind with no executable pin as NOT_CHECKED rather than SUPPORTED would close the gap outright; surfacing the identity is the minimum.
Please fix this vulnerability. If you propose a fix, make it concise and minimal.React 👍 / 👎 to tune Strix for this repo. A repo collaborator (or the PR author) can resolve this thread to dismiss the finding.
| const req = r.requested as { action: string; script_sha256?: string; command_sha256?: string }; | ||
| rows.push({ | ||
| seq: r.seq, | ||
| ts: r.ts, | ||
| action: clean(req.action, 140), | ||
| script_sha256: req.script_sha256 ?? null, | ||
| command_sha256: req.command_sha256 ?? null, | ||
| decision: r.authorized?.decision ?? 'UNKNOWN', | ||
| reasons: (r.authorized?.reasons ?? []).map((x) => clean(x, 300)), | ||
| ...resultOf(outcome, r.kind, r.authorized?.decision), | ||
| }); |
There was a problem hiding this comment.
commandRows never copies requested.executable / executable_sha256
| const req = r.requested as { action: string; script_sha256?: string; command_sha256?: string }; | |
| rows.push({ | |
| seq: r.seq, | |
| ts: r.ts, | |
| action: clean(req.action, 140), | |
| script_sha256: req.script_sha256 ?? null, | |
| command_sha256: req.command_sha256 ?? null, | |
| decision: r.authorized?.decision ?? 'UNKNOWN', | |
| reasons: (r.authorized?.reasons ?? []).map((x) => clean(x, 300)), | |
| ...resultOf(outcome, r.kind, r.authorized?.decision), | |
| }); | |
| const req = r.requested as { action: string; script_sha256?: string; command_sha256?: string; executable?: string; executable_sha256?: string }; | |
| rows.push({ | |
| seq: r.seq, | |
| ts: r.ts, | |
| action: clean(req.action, 140), | |
| script_sha256: req.script_sha256 ?? null, | |
| command_sha256: req.command_sha256 ?? null, | |
| executable: req.executable === undefined ? null : clean(req.executable, 140), | |
| executable_sha256: req.executable_sha256 ?? null, | |
| decision: r.authorized?.decision ?? 'UNKNOWN', | |
| reasons: (r.authorized?.reasons ?? []).map((x) => clean(x, 300)), | |
| ...resultOf(outcome, r.kind, r.authorized?.decision), | |
| }); |
| const variants = new Set(hits.filter((c) => c.decision === 'ALLOWED').map((c) => c.command_sha256)); | ||
| if (variants.size > 1) return { ...base, status: 'NOT_CHECKED', why: `it ran with ${variants.size} different command lines, so no one run decides this. Run the exact command you trust, using a fresh TRUSTSHELL_HOME.`, record_seq: last.seq }; | ||
| const scope = `record ${last.seq}`; | ||
| if (last.exit_code === 0) return { ...base, status: 'SUPPORTED', why: `the last time it ran through trustshell run it exited 0 (${scope}). That shows this command passed, not that it tests what the claim says.`, record_seq: last.seq }; |
There was a problem hiding this comment.
SUPPORTED verdict with no mention that the executable was name-only
| if (last.exit_code === 0) return { ...base, status: 'SUPPORTED', why: `the last time it ran through trustshell run it exited 0 (${scope}). That shows this command passed, not that it tests what the claim says.`, record_seq: last.seq }; | |
| if (last.exit_code === 0) { | |
| const identity = | |
| b.script_sha256 || last.executable_sha256 !== null | |
| ? '' | |
| : ` The policy did not pin ${last.executable ?? b.tool} to a file, so any executable with that name would have counted: pin it with trustshell guard allow tool:${b.tool} --exe <path> and run it again.`; | |
| return { ...base, status: 'SUPPORTED', why: `the last time it ran through trustshell run it exited 0 (${scope}). That shows this command passed, not that it tests what the claim says.${identity}`, record_seq: last.seq }; | |
| } |
| note: g.commands.length | ||
| ? `Only these. Policy ${g.policy.state}, ${g.policy.signed ? 'signed' : 'not signed'}, nonce ${g.policy.nonce ?? 'none'}. Record chain ${g.ledger.verdict}, ${g.ledger.count} records.` | ||
| : 'No command went through TrustShell, so this report says nothing about what the agent ran.', | ||
| rows: g.commands.map((c) => ({ status: c.decision, text: `#${c.seq} ${c.action}: ${c.result}`, detail: c.reasons.join('; ') || undefined })), |
There was a problem hiding this comment.
Path 2 renders the action label only, never the executable or its pin state
| rows: g.commands.map((c) => ({ status: c.decision, text: `#${c.seq} ${c.action}: ${c.result}`, detail: c.reasons.join('; ') || undefined })), | |
| rows: g.commands.map((c) => ({ | |
| status: c.decision, | |
| text: `#${c.seq} ${c.action}: ${c.result}`, | |
| detail: | |
| [ | |
| c.executable === null ? null : `starts ${c.executable} (${c.executable_sha256 === null ? 'name-only: not pinned to a file' : `pinned, sha256 ${c.executable_sha256.slice(0, 12)}…`})`, | |
| c.reasons.join('; ') || null, | |
| ] | |
| .filter(Boolean) | |
| .join('; ') || undefined, | |
| })), |
Lets a developer test their own agent and get a report they can inspect.
What this adds
trustshell trust-report: one readable report and its evidence. It joins what the agent said about a GitHub Actions run (check --outrecords) with every command that went throughtrustshell run, and keeps the two apart.trust-report verify: proves the folder is whole, matches its evidence and was sealed by a key you pin.MANIFEST.sha256also works with plainsha256sum -c.--bind "<tool or script>=<claim>": pairs a claim with the command that decides it. If that command ran with more than one command line, the claim is NOT_CHECKED, so a failure cannot be replaced by an easier run.scripts/release-e2e.mjs: packs the release, installs it into an empty folder with no keys and runs each step in every shell, tracing the network. It needs no secret of ours.examples/own-agent/README.md: the two paths, with commands that are the same in bash and PowerShell.Also fixes
trustshell runprinting "started" for a command that could not start, and says why a.cmdshim cannot be started on Windows.What it does not claim
The report cannot see anything the agent did without
trustshell run, and says so in every rendering. The seal shows who made the report, not that the commands test what the claims say.Evidence
release-e2ein CI: macOS 50 of 50 steps; Ubuntu and Windows 49 of 50, where the one NOT_CHECKED is the sandbox step because a container runtime is present.sandbox-mission.ymlcovers that path.🤖 Generated with Claude Code