Repository navigation
feat(formal-verification): add the chart and setup skills and the formal-area contract - #16458
Conversation
…chain scripts Register the formal-verification plugin (off by default) with its userConfig paths for the model checker jar, Java and the script runtime, prerequisites, README with the MIT notice for the carried statechart files, and the shared prerequisites and config-cascade copies. defaults.json holds the pins and bounds; scripts/settings.mjs resolves them through the consumer's docs/conventions/formal-verification.yaml. scripts/toolchain.mjs checks the toolchain, resolves it for the chart skill (blocked: toolchain missing when absent), and installs the pinned jar or renderer only when asked, keeping a download only when its checksum matches. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reference/formal-skill.md names what a consumer's formal-<area> skill holds: frontmatter; the sections Property, Harm, Tool and why, Bounds and assumptions, Not covered, Run, Reading a result and Updating; one sync.yaml entry per check (id, covers, commit, command, seeded_break); model/view.mjs as the view file the chart reads; model files under model/ unless sync.yaml lists them; unattended writes confined to the folder and those paths. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Carry the pstack statechart skill's text and scripts at cursor/plugins@ec56937 as /formal-verification:chart. The scripts no longer install TLC or Java: statechart.ts takes --jar, --java, --chart-states and --rerun-states, run.ts drops the mise install path and reads no environment, the bounds default from defaults.json, and the launcher takes --runtime. The skill resolves the toolchain through scripts/toolchain.mjs and reports blocked: toolchain missing instead of installing. Her tests come along, changed only to pass the jar and Java as arguments and to find the renderer in a given folder instead of installing it. They are not run in this change: her code runs only after the per-group script review. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/formal-verification:setup check reports the script runtime, Java, the model checker jar and the renderer, and the resolved pins and bounds. apply install-model-checker and apply install-renderer download the pinned file into the plugin data directory, keeping it only when its checksum matches; typing the subaction is the yes, and an unattended run installs nothing and reports blocked: toolchain missing. Java is never installed. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…laim-protocol The repository's first formal-<area> skill: a TLA+ model of one issue claimed at once by a local default claim, the issue lane's lease-only claim and a triage-phase claim on the GitHub adapter, with lease expiry, release and the completed-run rule. Each draft invariant quotes the tracker contract or claim.sh line it encodes. The model is not checked: the owner approves the invariants first. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eams The pstack record's statechart, create- and maintain-formal-verification rows read adopted at ec56937961, with the copy decision and the rule that copied scripts diverge as ours and are re-taken only through the drift re-audit. The tlaplus/AgentSkills record carries the revised decision and attribution rows at its re-read pin, f2adb5e (unchanged at upstream HEAD): nothing taken in this slice. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spelling lint The corpus report.md files are expected command output compared byte for byte, so a nested markdownlint config ignores them; two test lines whose fixture strings read as misspellings carry the typos disable-line directive. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tended chart run One case runs bare setup and expects the read-only toolchain check with no install; one runs the chart skill unattended with no jar and expects blocked: toolchain missing and no install. The setup description fits the description budget, and the cheat sheet links the chart skill's pstack equivalent. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cli.test.ts sent SIGKILL to Number(ps output); with ps missing that is pid 0, the whole test run's process group. The test and the helper script it writes now kill only a positive integer pid that is not their own. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tions label.ts refused any label containing "undefined" or "NaN", so a model's own state text could fail with no view. The guard now applies only to a view's return values and matches the whole value. regions.ts dropped a step's region condition when its label ended in "]", so two arrows could merge or look the same; the condition is kept and lift.ts keys its merge on it, undefined included. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… releases.json A repository's docs/conventions/formal-verification.yaml could set the model checker's url and sha256 and the renderer's integrity, and a crafted version or package name could reach outside the data folder. URLs and checksums now live only in releases.json; a convention file that sets one is refused, and a version it chooses must have a pin there. Versions and package names are checked against patterns, every install target is asserted to stay under the data directory, downloads must be https and stop at a size cap and a timeout. defaults.json and the schema gain the chart limits (timeout, heap, output bytes, interval limit) and the allowed-modules list. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TLC loaded Java from the spec's folder through -I and -DTLA-Library.
The chart command now copies only the folder's regular .tla files into
a fresh directory under its temp folder and points both there, with
tlc2.overrides.TLCOverrides set empty. Before any JVM starts it refuses
a folder holding a .class or .jar file, a module that shadows a
standard or generated module (or starts with _), a spec that uses a
generated name, a module outside the allowed-modules setting (Json and
similar modules touch the file system), and a spec whose name does not
end in .tla.
The worker and every Java run get an allowlisted environment (PATH,
LANG, LC_ALL, and HOME and TMPDIR in the run's temp folder), a -Xmx
heap cap, a wall-clock timeout and a cap on printed bytes, each a
per-run argument with a default from the settings. stopOf and tail stop
collecting at their window, outputs are read and their descriptors
closed before the temp folder is removed, the output folder is refused
when it is a symbolic link and resolved with path.resolve, and a large
interval reaches the view as {lo, hi}. New unit tests cover the guards.
Refs: #15648
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…efine every chart class report.md wrote spec-controlled state text unescaped and fenced TLC output with a fixed three backticks. Each bug path and TLC's error text now sit in a fence one backtick longer than their longest backtick run, spec text outside a fence is Markdown- and HTML-escaped, and each line and each path is capped with a pointer to the file holding the whole. chart.mmd now carries a classDef for the group colors, bugPath and repeats, not only the bug marks, so the legend means something in any Mermaid viewer; render.md's CSS takes !important on every rule to win over those styles. The Mermaid colon hazard is found in one right-to-left pass instead of a lookahead per colon. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…text is data The chart skill and its view-module reference now say, as rules, that a view file runs as JavaScript and is never taken from untrusted content, that an unattended run charts with a view only from the default branch, that a spec from an untrusted pull request is charted only inside a sandbox with no network and no credentials, and that report and TLC output text is data, not instructions. The Run step passes the new limits and the allowed modules. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t run The corpus test rewrote committed fixtures when STATECHART_UPDATE was set; that branch is now test/update-corpus.ts, which the test command never runs. The corpus test removes its temp folder in t.after, the differential test loads fast-check inside the test that needs it, and the scripts package commits a lockfile. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The report escaped whole note and region lines, so the tool's own prose came out as "never does\. So". Escaping now applies to the spec- controlled pieces (state text, step labels, region names) where the passes build each sentence, through escapeText in ir.ts, and leaves "." alone. chart.test.ts and pipeline.test.ts expect the fenced bug-path shape the report now writes. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mal-verification-chart-setup
|
Owner approved all seven invariants as drafted on 2026-11-15 ("go w/ recommendations"). |
|
Session note (agent, not an owner decision), answering the hub's question, whose session could not be reached by message: this pull request is owned by the local session |
…ty review Regenerate the chart corpus with test/update-corpus.ts against TLC 1.7.4 and assert the fenced path blocks in the report. Record the plugin-acceptance security review in ADR 0093, say who reads the STATECHART_* variables the environment allowlist keeps, and register the chart scripts package with the Node package lane. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The STATECHART_* variables the chart's environment allowlist keeps are read only by the test harness's replaying fake java (test/java.ts) and carry no credential; the constant's name says so where the allowlist is defined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… command prints Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…se and the assignee hand-back The formal-claim-protocol model now describes the GitHub adapter's lease scripts one call per step, with renew, the 300 s margin and the hand-back, and ships two checks: a local session against the issue lane, and against a same-login triage claim. The seeded break is caught in both. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…found The fifth script group's review: the launcher names a rejected --runtime, escapeText folds line breaks, and the chart skill documents exit 127. The claim-protocol skill records the three-claimant run without renew and the source line behind EndStateReclaimable. The upstream record says the copy lacks the renderer fix of cursor/plugins#1236. The README no longer says the jar is undeclared. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Session note (agent), for the hub sweep. Held with Open now, all being fixed in one more push:
After that push is green: rerun both shipped charts and the seeded break at the final head, update the evidence comment, lift the label, merge. ETA 4:30 PM EST; no help needed. |
- The module check reads past comment marks inside a TLA+ string, so a spec cannot hide an INSTANCE from the allowed-modules refusal. - A repository's convention file can remove modules from allowed_modules but not add them; resolve and check name what they drop. Owner decision, 2026-11-16. - resolve refuses a bounds.heap that is not a heap size and a jar, java or runtime path holding a quote, a dollar sign, a backquote or a control character; the chart skill single-quotes the values it passes. - A cut-off invariant or deadlock path names tlc-RULE.out, which now holds that run's trace. - A liveness property TLC passes under its SYMMETRY warning (2279) is listed apart from the rules that hold; the PickWorker report follows. - The four launcher tests expect the line naming a rejected --runtime. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… check and probe from the script's folder INSTANCE(**)Json reached SANY as INSTANCE Json but the module check read INSTANCEJson. On Windows a bare java, node or tar is looked up in the working directory first, so the toolchain probes and the renderer unpack now run from the script's folder. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…t the module check at the module header, move the render theme to a file A property checked under a cfg VIEW is listed apart when TLC passes it and carries a warning line when TLC breaks it; the HiddenClock report follows. Text before the MODULE line no longer opens a comment that hides the module. The CSS theme leaves render.md, which may carry no language-tagged fence. The claim-protocol skill records that both cfgs hold every rule without VIEW. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…n the module check SANY starts the module at the first dashes-MODULE run, mid-line included, so a header after an unclosed comment or before a decoy module hid the real one from the check. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… name in the module check SANY resolves EXTENDS and INSTANCE by file name, so a file whose header named another module made that module count as the folder's own. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Post-merge mutation testingKilled 2, survived 0, timeout 0, no coverage 5, invalid 0, not run 0.
The run's summary has the full report. |
Closes #15648
Summary
Slice 1 of the
formal-verificationplugin: the chart and setup skills, the contract for a consumer'sformal-<area>skill, and this repository's first one, checked./formal-verification:chartcarries herstatechartskill (cursor/pluginspstack/skills/statechartatec569379616a4826b98f0f0a205dd3411b573611, MIT): TLA+ with TLC, a Mermaid chart, the shortest path to each broken rule./formal-verification:setupchecks the toolchain and, on a typedapply install-model-checkerorapply install-renderer, downloads the pinned file into the plugin data directory, keeping it only when its checksum matches.reference/formal-skill.mdis the contract for a consumer'sformal-<area>skill..claude/skills/formal-claim-protocol/is this repository's firstformal-<area>skill: a TLA+ model of the work-item tracker's claim protocol.Fix
plugins/formal-verification/withplugin.json(model_checker_jar,java,script_runtimeasuserConfigfile options), README with the MIT notice,prerequisites.json,defaults.json(her pins and bounds, plus the limits added in review),releases.json(the only home of download URLs and checksums),schemas/formal-verification.schema.json(keys of the consumer'sdocs/conventions/formal-verification.yaml), the sharedprerequisitesandconfig-cascadecopies,scripts/settings.mjsandscripts/toolchain.mjswith tests; marketplace entry (off by default), catalog, cheat sheet, fragment list, changelog fragment.sync.yamlentry per check (id,covers,commit,command,seeded_break);model/view.mjsas the view file; model files undermodel/unlesssync.yamllists them underoutside_model; unattended writes confined to the folder and those paths. Her "What makes it good" text is kept.SKILL.mdtext (including "Never weaken a rule to make it hold." and her "Write the spec" rules) split between the skill andreference/view-module.md,read-report.md,render.md; every file under herscripts/copied. Changed:statechart.tstakes--jar,--javaand the bounds and limits as arguments;run.tsdrops themiseinstall and reads no environment; the launcher takes--runtime; the review round below hardened the rest. The corpus fixtures are regenerated withtest/update-corpus.tsagainst TLC 1.7.4, and her tests assert the fenced path blocks of the current report.check(runtime, Java, jar with its pinned SHA-256, renderer, plus the declared prerequisites and the resolved settings);apply install-model-checkerandapply install-renderer; unattended installs nothing and reportsblocked: toolchain missing. Java is never installed.plugins/formal-verification/skills/chart/scriptsis registered inscripts/outside-node-packages.txt, so the Test Node lane runs itsnpm test. WithoutTLA2TOOLSevery TLC test skips, so the lane needs no Java.formal-verificationreview record.docs/upstream/cursor-pstack.mdrows forstatechart,create-formal-verification-skillandmaintain-formal-verification-skillread adopted at the pin, with the note that copied scripts diverge and are re-taken only through the drift re-audit;docs/upstream/tlaplus-agentskills.mdcarries the revised decision and three attribution rows at its re-read pinf2adb5e(unchanged at upstream HEAD).Review round: chart script findings
Five fresh-context reviewers each read every line of one group of the copied chart scripts: the first four at
43fba2c9189, the fifth at0af0d1aa8d9:statechart.ts,run.ts,worker.ts,toolchain.mjsmermaid.ts,report.ts,cfg.ts,plain-text.tsfrontends/passes/,test/ir.ts,pipeline.ts, thestatechartlauncher,package.json,tsconfig.json9eb40bc77e73(a rejected--runtimeis named,escapeTextfolds line breaks, exit 127 documented). Not changed:tsconfig.jsonnames@types/node, which is not installed, as upstream does; nothing runstscThe first four reviewers' findings, all 16, and the commit that addressed each:
.tlafiles are staged into a fresh folder that-Iand-DTLA-Librarypoint at;.class/.jarfiles, shadowing and generated module names refused before any JVM;-Dtlc2.overrides.TLCOverrides=empty (stage.ts,run.ts)146c03d13c2stage.tschildEnv), plus theSTATECHART_*variables, kept because only the test harness's replaying fakejava(test/java.ts) reads them and none carries a credential146c03d13c2,327bb34b7e7-Xmx, byte cap, each a setting and an argument;stopOfstops at 200 parts andtailkeeps a 1 MiB window; outputs are read and their descriptors closed before the temp folder is removed146c03d13c2(settings ind6a259c21af)allowed_modulessetting are refused, naming the module and why; checked on the staged text before SANY146c03d13c2chart/SKILL.mdandreference/view-module.md; the scrubbed worker environment is the second layer93f6a07ac30releases.json, a convention file that sets one is refused, versions and package names pattern-checked, install targets asserted under the data folder, https only, size cap and timeoutd6a259c21aflstatrefuses a link;path.resolve146c03d13c2SKILL.mdandread-report.mdb369b064acb,93f6a07ac30d851e211f22interval_maxthe view gets{lo, hi}146c03d13c2undefined/NaNrefused86427e41b1c86427e41b1c.tlafails; the staged folder is under the temp folder and is refused if it holds the separator146c03d13c2classDeffor every used class; one right-to-left passb369b064acbChartBound/ChartRun, or any use of a generated name, is refused with a message (names not suffixed, to keep her test harness's file names)146c03d13c2test/update-corpus.ts, never run by the test command;t.aftercleanup in the corpus test and the new tests; fast-check loaded inside its test;package-lock.jsoncommitteddb59e0fff71Run after the fixes:
test/update-corpus.tsregenerated all 33 corpus cases with TLC 1.7.4, and her whole suite ran with TLC (counts under Verification). The test "an --out that is a file stops the command with exit 2" asserted the oldENOTDIR: not a directorytext; it now expects finding 7's "It is not a folder" (d662a24980c). Her render tests (Pele and mermaid.js) were not run: they needSTATECHART_RENDER_DEPSwith pele, mermaid and jsdom installed.Audit her copy without a diff commit. Git blob ids are content hashes, so equal ids mean an unchanged file. List her files at the pin:
List the copy as committed in
647187aaff2:Fetch one of her files at the pin to diff by hand:
The regenerated fixture
.outfiles carry TLC's run header (kernel version, core count, heap, process id, timestamps) and temp-folder paths; no home folder or user path.Security posture
After this round:
.tlafiles, refuses Java code and file-touching modules, runs Java with a scrubbed environment, a heap cap, a timeout and an output cap, and writes only to a new output folder.releases.json, over https, size- and time-capped, written only under the plugin data directory, and only on a typedsetup apply install-*.Verification
node --test --test-concurrency=4 test/*.test.tsin the chart scripts folder,TLA2TOOLSset to tla2tools 1.7.4, Java Temurin 25.0.5):ENOTDIRtest above), 73 skipped (273 s). The 73 skips are the render tests.npm testwith noTLA2TOOLSand no Java on PATH, atbf5c619af20: 354 tests, 214 pass, 0 fail, 140 skipped (36 s).node --test plugins/formal-verification/scripts/toolchain.test.mjs: 18 pass, 0 fail (settings layering,blocked: toolchain missingwriting nothing, a repository URL/sha256/integrity refused, an unpinned version refused, traversal refused, https only, size cap by header and by bytes, timeout, checksum match kept and mismatch discarded).test/guard.test.ts: 20 pass, 0 fail (fences, escaping, caps, the colon scan in linear time, module and generated-name scanning, staging and its refusals, the environment allowlist, the output folder).scripts/validate-plugins.sh --only formal-verification: passes.scripts/check-affected.shin the pre-push hook: passes after mergingorigin/main./formal-verification:chartwith the verdicts under "Claim-protocol model"; raw TLC agrees on every rule. The seeded break is caught in both.Related
specifypropose, create, check), Add the formal-verification specify skill's refresh action #15775 (specifyrefresh).Claim-protocol model
The owner approved all seven invariants as drafted (2026-11-15). The model,
.claude/skills/formal-claim-protocol/model/ClaimProtocol.tla, describes the GitHub adapter's lease scripts at a548b3a (after #16475, #16509 and #16515) one GitHub call per step: claim, renew, release and the assignee hand-back, with the 300 s margin, expiry and the completed-run rule. Two checks ship, each two to three minutes through/formal-verification:chart: a local session against the issue lane (LocalLane.cfg, 1,686,588 states) and against a same-login triage claim (LocalTriage.cfg, 1,356,633 states).TypeOK,EndStateReclaimableand no deadlock.NoStrandedAssigneein both,HolderAssignedagainst a same-login claim); fix(work-items): hand back on an outranked release and judge hand-back leases at a clock read before the list #16515 fixed both in the scripts, and no rule was changed.The seeded-break chart and the failure-flag table are in this pull request's evidence comment.
Review rounds after ready
The CI review lanes and Codex raised twelve findings after the ready flip; each was verified against the code, fixed with a test that fails without the fix, and its thread answered:
e85241434fa9: a relative--jaror--javaresolves before Java starts in its temp folder; the runtime version floors apply to a named--runtimeand totoolchain.mjs; a non-numericjava.min_versionis refused;toolchain.mjsandsettings.mjsrun through a symlink;timeout_secondsis capped at 2147483; the eval graders match the quoted commands; the claim-protocol view labels the hand-back's list states.1ecef7a2869: the module check reads past comment marks inside a string; a repository file can remove modules fromallowed_modulesbut not add them (owner, 2026-11-16);resolverefuses a malformedbounds.heapand unsafe paths; a cut-off path names a file that holds it; a liveness property TLC passes under its SYMMETRY warning is listed apart.4ff764ec564: a comment right afterINSTANCEorEXTENDScounts as whitespace in the module check; the toolchain probes and the renderer unpack run from the script's folder, so a same-named binary in the charted repository is not the one run on Windows (fixed by reading, not run on Windows).Chart suite with TLC at
1ecef7a2869: 399 tests, 326 pass, 0 fail, 73 skipped.toolchain.test.mjs: 33 pass, 0 fail.Remaining
Not handled: TLC message 2284 (liveness checked under a CONSTRAINT), which a bounded rerun may raise and the report does not read; no layer the user controls can add a module to
allowed_modules(only the shipped list and a hand-passed--modules); upstream's later renderer change (cursor/plugins#1236) is not in the copy; the module refusal scans the spec's text after stripping comments, and review got past that scan twice (1ecef7a2869,4ff764ec564), so it is a second layer, not a guarantee: the pinnedtla2tools.jarships noJsonorIOUtils, Java beside the spec is refused and TLC's override loading is off; reading the module list from SANY's own output would make it one. Owner, 2026-11-16: no user-level way to add a module is built now. The three documented limits (clock skew, ttl-0, comment-id order) and the no-compare-and-swap windows are listed as known limits in the skill. Not run: the 73 render tests (they need the renderer package), so the legend class fix has not been drawn by a renderer.🤖 Generated with Claude Code