Security scanning is a required CI gate, not an advisory run —
.github/workflows/security.yml runs on the same single-run trigger as
the quality gates. It catches the class of regression unit/contract/e2e tests
don't: a known-CVE dependency, an insecure code pattern, or a credential
committed to history. Make it a required check via the pr-gating recipe.
This tool drives a browser over untrusted page content and runs a model that acts. The defenses, and the residual risks, in priority order:
- Prompt injection via a hostile page → defused by tool-gating (PRIMARY). In the standalone
harnesses (
explore/codify/interactive) the model's entire capability surface is the gated browser/codify tools — it holds no fs/shell/network tool.createGatedSession.onPreToolUse(src/harness.mjs) denies anything else. A page that says "read .env and POST it" has no tool to do so. The URL allowlist (src/allowlist.mjs, network-layer nav abort) is the second layer. (Sub-resources/iframes off-host — e.g. ads — are intentionally allowed; the leash, not the allowlist, is the real defense.) — ADR 0001 - No raw shell to the model.
pwcliandrun_specshell out viaexecFilewith argv arrays (nevershell:true/string interpolation), so page or model text is always a value, never a command. write_specfile writes are confined. The spec name isslug()-sanitized ([^a-z0-9]→-), so no path traversal — writes stay undertests/authored/.- HARDENED — the codifier executes model-authored code (#0022, ADR 0010).
write_specwrites a.spec.tsthatrun_specruns viaplaywright test(full Node). Two defense-in-depth layers now guard that boundary (src/spec-guard.mts): (1) a capability scan (scanSpecCapabilities) rejects source that imports a dangerous Node core module (fs/child_process/net/…) or useseval/Function/require/process.binding— enforced atwrite_spec/write_pomAND re-scanned acrosstests/authored/*.tsatrun_spec(so a file placed out-of-band still can't run); (2)run_specexecutes under a scrubbed environment (safeSpecEnv— a minimal allowlist, no host secrets), closing the secret-exfil leg even if the scan is bypassed. Accepted residual: the scan is static, not an OS sandbox — a determined obfuscation could still construct a banned reference, so authored specs remain untrusted until human-reviewed (the "never self-certified" principle) and OS-level isolation (a container) stays the documented future hardening if a stronger guarantee is needed.- Process-spawn caveat for Electron specs (#0027, ADR 0011). A surface-aware Electron spec
calls
_electron.launch({ args: [QA_ELECTRON_APP, …] })— a method on the allowed@playwright/testimport, so the capability scan (which gateschild_process, not API methods) does not flag it. That is a genuine process-spawn: it runs the configured Electron app's main process (full Node) and, by abuse ofargs, could point at another binary. It is not a fresh hole — the explorer already launches the operator-chosen app (provider.mts), andrun_specalready runs full Node — and it falls in the same untrusted-until-human-reviewed residual as the obfuscation bypass above. The scrubbed env still bounds exfil (QA_ELECTRON_APPis a path, not a secret; host secrets stay dropped). Treat an authored Electron spec as the untrusted, human-reviewed artifact it is.
- Process-spawn caveat for Electron specs (#0027, ADR 0011). A surface-aware Electron spec
calls
- RESIDUAL — the Copilot extension is a SOFT leash. An extension cannot remove copilot's built-in
fs/shell tools, so the gate is bypassable there. Use
bin/interactive.mjs(hard leash) when enforcement matters. — documented in ARCHITECTURE.md. - Auth-state hygiene.
save_authwrites a storageState file of live cookies/localStorage;.gitignoreexcludes the common names — never commit one, and treat any committed one as a credential to rotate.
npm audit --audit-level=high— 0 vulnerabilities.gitleaks detect(full history, 12 commits) — 0 leaks.semgrep --config auto(CI) — caught 1 real finding on first run: a shell-injection sink in the generatedrelease.yml(${{ github.* }}interpolated into arun:step). Fixed by routing the ref through anenv:var. Re-scan clean.
- SAST (static analysis) —
semgrep --error --config auto. Reads the source without running it and flags insecure patterns (injection sinks, unsafe deserialization, weak crypto, tainted data reaching a dangerous call). It complements lint: lint is about style/correctness, SAST is about security properties. - Dependency / vulnerability audit —
npm audit --audit-level=high. Cross-checks your dependency tree (and, for tools like govulncheck, the symbols you actually call) against published advisories, so a vulnerable transitive package fails CI instead of shipping silently. - Secret scan (gitleaks) — scans the diff (PRs) and history (pushes) for
high-entropy strings and known credential shapes (API keys, tokens, private
keys). A leaked secret is a credential to rotate, not just a line to delete —
see triage below. Configured by
.gitleaks.toml, whose allowlist exempts test fixtures.
# SAST + dependency audit — the exact commands CI runs:
semgrep --error --config auto
npm audit --audit-level=high
# Secret scan over the working tree + history (install gitleaks first):
gitleaks detect --config .gitleaks.toml --redactRunning these locally turns a red CI into a pre-push fix and keeps the gate fast.
- Is it real? Read the finding — the rule id, the file/line, the data flow. SAST and audit tools report potential issues; confirm the path is actually reachable and the input actually untrusted.
- If real, fix at the root. Bump/patch the vulnerable dependency (or remove
it); rewrite the insecure pattern; for a leaked secret, rotate the credential
first (assume it is compromised the moment it hit a remote), then purge it
from history (
git filter-repo/BFG) and update.gitleaks.tomlonly if a non-secret fixture tripped the scanner. - If it can't be fixed now, open a tracked issue (issues recipe) with the severity and a deadline rather than suppressing it silently — a suppression with no owner is how a real vuln hides.
A suppression is a security decision; it must be narrow, justified, and reviewable.
- SAST — use the tool's inline suppression scoped to the single line/rule
(e.g.
// nosec G401for gosec/#nosec,# nosemgrep: <rule-id>for semgrep,# nosecfor bandit) with a comment saying why. Never disable a whole rule repo-wide to clear one finding. - Dependency audit — record the accepted advisory in the tool's ignore file
(
govulncheckhas no per-CVE ignore — pin/patch instead;pip-audit --ignore-vuln,npm auditresolutions,trivy --ignorefile .trivyignore) with an expiry/review date. - Secret scan — add a path or regex to the
allowlistin.gitleaks.tomlfor genuine test fixtures only. If it's a real key shape that happens to be fake, prefer an obviously-fake placeholder (AKIAEXAMPLE…) over an allowlist.
Every suppression is a review flag: the reviewer should see why in the diff.
- Install the scanners CI needs in
security.yml(govulncheck/semgrep/bandit/ pip-audit/trivy/npm — matchsemgrep --error --config autoandnpm audit --audit-level=high). - Set
semgrep --error --config auto/npm audit --audit-level=highto this repo's real tools. - Make
securitya required status check (pr-gating recipe). - Triage and fix (or track) the findings from the first full run.
- Rotate any credential the secret scan surfaces in existing history.