feat(bin): let the terminal record say a task failed - #1978
Open
sbracewell64 wants to merge 5 commits into
Open
Conversation
sbracewell64
force-pushed
the
fm/cfvc-12-task-outcome-failure
branch
from
August 9, 2026 00:01
937c956 to
30b8637
Compare
The fleet's terminal outcome was a constant. Teardown set outcome=landed and only --force changed it, so nothing anywhere produced failed: a record reading "40 terminal (landed 40)" was not a success rate, because the numerator could not move. Definition first. bin/fm-wake-ledger.sh now owns what a terminal outcome means and what evidence stands behind it. The enum stays three members pinned to the v1 line schema; every terminal record gains outcome_source naming where its outcome came from - declared, discarded, unreleased, or assumed. A record written before the field existed reads as assumed, which is exactly what those records were, so the append-only file needs no rewrite. Field second. Teardown derives the outcome from the task's own last declaration instead of a constant, and a --force discard still outranks it. A second producer covers the case teardown never sees: a task that fails and is never released was silent in the ledger, and silence there is indistinguishable from a task that never failed. `sweep` records those once, receipt-guarded, and a locked session start runs it. Diagnostic only, deliberately. The report breaks outcomes down by evidence and refuses to print a rate: this ledger counts released tasks while the no-mistakes pipeline counts validation runs, and until that divergence is reconciled any ratio would describe neither. The report names that gap on every run. The attempt counter and the unified terminal vocabulary are separate increments and are not absorbed here.
…ion-start test header
sbracewell64
force-pushed
the
fm/cfvc-12-task-outcome-failure
branch
from
August 9, 2026 01:01
30b8637 to
fefaee4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
CFVC-12: a task-outcome definition that can express failure.
GOAL: the fleet's terminal record can say a task FAILED. Before this change, bin/fm-teardown.sh set LEDGER_OUTCOME=landed, overridden to abandoned only under --force, and NOTHING produced failed - so "40 terminal (landed 40)" was not a success rate and every per-accepted-change metric was uncomputable. Two authoritative records disagreed about the same changes: the fleet said 40 landed while the no-mistakes pipeline said 1 completed of 61 runs.
OWNER: bin/fm-teardown.sh + bin/fm-wake-ledger.sh. Definition first, field second - deliberately in that order.
BINDING REQUIREMENTS, all accepted and all implemented:
DESIGN DECISIONS MADE, deliberate and not accidental:
derivesubcommand rather than being restated in teardown, because fm-wake-ledger.sh is the single owner of the outcome vocabulary (one-owner rule). Teardown calls it. Its output is space-separated rather than tab-separated on purpose: both tokens come from closed enums with no spaces, and invisible tab literals in shell source are a readability hazard.sweep, exists because teardown is not enough: a task that fails and is never released was silent in the ledger, and silence is indistinguishable from a task that never failed. It is guarded by a state/.terminal-recorded receipt so a rerun never double-records, and the receipt is written only AFTER the append is durable so a failed append still retries.ALREADY APPLIED IN AN EARLIER ROUND OF THIS SAME WORK (commit af1938d, accepted and correct - do not re-flag):
REBASE DECISION ALREADY MADE: the rebase step is skipped on purpose. This lane's fork trunk is the deliberate base and the commits ahead of upstream are the fork landing queue, never replayed or bundled into contribution history.
VERIFICATION ALREADY DONE ON THE PRE-REVIEW COMMIT: bin/fm-lint.sh clean; bin/fm-doc-audience-check.sh clean; tests/fm-wake-ledger.test.sh, tests/fm-teardown.test.sh and tests/fm-bootstrap.test.sh all green. A negative control confirmed the new failed-teardown test genuinely fails against the pre-change teardown. The full --changed suite reported 5 failures; ALL FIVE reproduce identically on the unmodified base commit (tests/fm-calm-pi-extension, tests/fm-pi-watch-extension and tests/fm-turnend-guard fail deterministically on a node/Pi environment problem, tests/fm-session-start fails its MISSING-node assertion on base too, and tests/fm-watcher-lock is flaky at the same 2-in-3 rate on base as with the change). None of those five are caused by this work.
ALSO ALREADY APPLIED AND ACCEPTED (commit a996b8c, do not re-flag): the remote-secondmate teardown path now removes state/.terminal-recorded alongside the other per-task state files, matching the main cleanup list, so a reused task id can never inherit a stale receipt that would suppress the sweep.
DELIBERATELY NOT CHANGED, already reviewed and accepted as a no-op tradeoff: ledger_derive_outcome runs inside command substitution, so the lazy fm-classify-lib.sh load does not persist across calls and status_line_verb forks a subshell per status line. This is bounded, non-blocking, and entirely off the documented wake path that the ledger's never-block contract protects. Avoiding it would change the call convention of status_line_verb, a shared parser owned by bin/fm-classify-lib.sh, which is out of scope here.
What Changed
landed:bin/fm-wake-ledger.shgains aderivesubcommand that reads the task's lastdone:/failed:status declaration (reusingfm-classify-lib.sh'sstatus_line_verb, loaded lazily off the wake path), and every terminal record gains one new field,outcome_source(declared,discarded,unreleased,assumed), on the existing v1 schema token — pre-existing lines with no field read asassumed, so the append-only ledger needs no migration.bin/fm-teardown.shcallsderive, with--forcediscard still outranking the task's own last word.fm-wake-ledger.sh sweeprecords declared-but-unreleased failures asfailed/unreleasedterminal records, guarded by a per-task receipt written only after the append is durable (bootstrap counts recorded sweeps from the durable-append summary, not discovery lines). Session-start bootstrap runs the sweep in its locked mutating section, falling back to a reporting dry run when the lock is refused, and remote-secondmate teardown clears the receipt with the rest of per-task state so a reused task id can't inherit it. New coverage lands in the wake-ledger, teardown, and bootstrap suites, each failure case with its own negative control.fm-wake-ledger.sh reportprints terminal outcome counts broken down byoutcome_source, refuses to print a success rate (the outcome is diagnostic-only until the fleet/pipeline count divergence is reconciled, which the report names as a known gap on every run), and a test asserts the absence of any ratio. Note: this branch is deliberately based on the fork trunk, so the raw diff against the base commit also carries the previously-landed fork queue (launcher menu, admission control, model registry, wake-outcome ledger, remote secondmates, LoopSpec); that work landed in earlier PRs and only the commits above are new here.Risk Assessment
✅ Low: The change is well-bounded to its two owner scripts with validated closed vocabularies, an untouched wake path, receipt-guarded idempotence covered on both teardown paths, and thorough colocated tests with negative controls for every new behavior; every binding intent requirement is satisfied and no forbidden behavior (success rate, widened enum, attempt counter) was introduced.
Testing
Ran the three targeted suites (wake-ledger, teardown, bootstrap) via bin/fm-test-run.sh — all green including every new CFVC-12 test — then manually exercised the real CLI end-to-end in a sandboxed fleet home, capturing a transcript that shows a declared failure becoming a
failedterminal record via both producers (teardown derivation and the unreleased-failure sweep), receipt-guarded idempotence, the closed evidence vocabulary refusing an unknown source, and the report refusing a success rate while naming the fleet/pipeline divergence; no UI surface exists for this change, so the CLI transcript is the reviewer-visible artifact.Evidence: End-to-end CLI transcript: failure expressed, sweep recorded, report refuses a rate
$ fm-wake-ledger.sh sweep unreleased failure: beta recorded 1 unreleased failure(s) as terminal records -- the ledger now carries a terminal record distinguishable from landed: | v1 task 1786230764 task=beta harness=pi model=sol effort=high mode=no-mistakes kind=ship project=unknown backend=tmux outcome=failed outcome_source=unreleased route=unknown escalated=unknown findings=unknown $ fm-wake-ledger.sh report tasks: 3 terminal (landed 2 failed 1) by evidence: declared 1 unreleased 1 assumed 1 counts are DIAGNOSTIC ONLY - not a success rate: assumed outcomes carry no evidence, and this ledger counts released tasks while the no-mistakes pipeline counts validation runs. that fleet/pipeline divergence is a known, named, unreconciled gap.Evidence: Targeted test results (wake-ledger 19 ok, teardown 71 ok, bootstrap sweep wiring ok)
Evidence: Reproducible demo script that generated the transcript
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
⏭️ **Rebase** - skipped
.agents/skills/afk/SKILL.md- branch carries 46 commit(s) that exist on your local main branch but were never pushed to origin/main; rebasing would bundle this unrelated work (212 file(s)) into the PR:Push main to origin, or rebase your branch onto origin/main, before gating.
bin/fm-bootstrap.sh:1017- In the locked bootstrap branch, a sweep that discovers a declared failure but cannot append it is fully silent at session start: sweep stderr is suppressed (2>/dev/null), the discovery lines are discarded, and no summary line is emitted, so 'nothing found' and 'found but unrecordable' are indistinguishable in bootstrap output. This is a consequence of the accepted af1938d design (count only durable appends) and self-heals because the missing receipt makes every subsequent locked session start retry the append; the detect-only branch does report discovered-but-unrecorded failures.bin/fm-wake-ledger.sh:735- The receipt write after a durable append is best-effort (printf 'failed\n' > "$receipt" 2>/dev/null || true), so if state/ is unwritable while data/ is writable, each locked session start appends one duplicate identical terminal record for the same task. Readers resolve the last record per task id, so semantics stay correct and growth is bounded to one line per session start; the receipt-after-durable-append ordering is deliberate per the design (a failed append must retry), and this is the complementary tradeoff of that ordering.✅ **Test** - passed
✅ No issues found.
bin/fm-test-run.sh tests/fm-wake-ledger.test.sh tests/fm-teardown.test.sh tests/fm-bootstrap.test.sh(total=3 failed=0)bash tests/fm-wake-ledger.test.shfull output — 19 ok including the four new terminal-outcome tests (derivation, closed outcome_source vocabulary, unreleased-failure sweep with negative controls, report refuses a rate)bash tests/fm-teardown.test.shfull output — 71 ok including the three new tests (declared failure records failed not landed, uncorroborated landed records assumed, teardown supersedes a sweep record and clears its receipt)Manual end-to-end CLI demo in a sandboxed fleet home:fm-wake-ledger.sh deriveon an evolving status log, refusal of an unknown--source,sweep --dry-runvssweepwith receipt idempotence and the undeclared task left unrecorded, legacy no-outcome_source line read as assumed, andreportprinting evidence counts with the diagnostic-only refusal and named gapgit status --porcelainclean after testing (demo sandbox self-removed)🔧 **Document** - 1 issue found → auto-fixed ✅
AGENTS.md:155- AGENTS.md step 2 still hardcodes a sweep count ('the seven bootstrap MUTATING sweeps') plus a full enumeration duplicating the list bin/fm-bootstrap.sh's header owns; the count has drifted repeatedly (the session-start test comment said five, and bootstrap's model entitlement probe describes itself as a mutating sweep yet appears in neither enumeration). A follow-up could reduce it to count-free pointer phrasing like bin/fm-session-start.sh's accepted fix; left as-is here because the line is currently accurate and was deliberately updated by this change.🔧 Fix: point AGENTS.md mutating-sweep list at bootstrap header owner
✅ Re-checked - no issues remain.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.