diff --git a/CHANGELOG.md b/CHANGELOG.md index 8167a145..11b1be41 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,11 @@ breaking changes may land in a minor release. ### Added +- **`repo_root` in run `state.json`** (#716). A run records the git root its code work happens in, + so an out-of-process reader — `bmad-loop resolve`'s re-arm — uses the tree the run measured + instead of re-deriving one. A `state.json` written before the field existed degrades to the + project directory, which is the pre-upgrade behavior. + - **Atomic writers gain an opt-in `require_writable_target` refusal** (#597). Callers over operator-curated files can ask for the `PermissionError` a plain `Path.write_text` used to raise on a read-only target. Off by default — what the other callers do today is a @@ -40,6 +45,88 @@ breaking changes may land in a minor release. ### Changed +- **`bmad-loop diagnose` routes the re-arm records by field name** (#640, #716). `spec_file` and + `overwritten` are aliased, and `repo` is dropped — an absolute host path that correlates nothing. + Routing is by field name across every entry rather than by kind, so no existing run's dump changes + shape and `SCHEMA_VERSION` is unaffected. + +- **`bmad-loop diagnose`'s default report shows the split code root and the task generation** + (#705, #716). Both fields reached `--json` but not the markdown renderer, which samples its fields + by hand — so the two conditions they exist to name were legible only to whoever thought to ask for + JSON. They render as a `code root differs from project` yes/no line and a `gen` column beside + `att`; the code root's path itself still never renders. + +- **`resolve` and the TUI's re-arm aim the code root before they re-arm, not after** (#716). Both + surfaces re-armed and THEN resumed, so resume's `repo_root` re-stamp landed after the re-arm had + already read the stale mirror out of process: a `repo_root:` edit made while a run was paused + advanced the attempt baseline in the tree the run had left, while the engine that resumed measured + in the new one, with no error anywhere. Both now re-stamp through one shared writer, after the + confirm — so a cancelled resolve still leaves the divergence for `resume` to report — and each + warns that the run has changed repositories. A config this process cannot read degrades to the + root the run recorded, and says so. The #414 isolation refusal is hoisted alongside it, ahead of + both writes: under `isolation = "worktree"` beside a `repo_root` override, both surfaces used to + re-stamp, advance the attempt baseline and report "re-armed" before resume refused the + configuration — spending an escalation `resolve` could no longer re-run, since the story was no + longer escalated. + +- **Re-arm refuses a story spec it cannot re-open, instead of re-driving onto a status the session + cannot route** (#640). A spec carrying no top-level `status:` failed the flip silently: the + operator was told "re-armed ", the run resumed in the same gesture, and the re-driven + session halted on `unrecognized status in existing story file`, spending the escalation. The skip + record now aborts the re-arm, leaving the escalation armed for a corrected spec, the run state + untouched and the spec byte-identical. Narrowed to the spec the re-drive will actually READ — an + unreachable path only warns, since the re-drive mounts a fresh worktree and reads the committed + spec regardless. + +- **Re-arm writes the spec the run actually used, and reports every write it could not make** + (#640). `StoryTask` persists `spec_file` relative to the worktree and re-arm resolved it against + the process cwd, where the main checkout carries the same layout — so the status flip and the + baseline re-stamp landed on the WRONG file while the worktree's real spec kept the escalated + attempt's sha. The recorded path is now re-anchored on the worktree before either write. + Separately, both frontmatter writers answer a spec they cannot move with `False` rather than an + exception and those returns were discarded; they now journal `rearm-baseline-restamp-skipped` and + `rearm-spec-flip-skipped`, on genuine failure only — re-arm reads the status back, so an ordinary + second re-arm is no longer told its spec "could not be re-opened". The skip record no longer nests + behind a successful git advance. + +- **Re-arm warns when its spec writes cannot reach the re-drive** (#640). Under worktree isolation + the status flip and baseline re-stamp land in a worktree the re-drive discards before reading it, + and the re-driven session reads the COMMITTED spec — so a correction left only in the working tree + is silently lost. Re-arm journals `rearm-spec-write-unreachable` and both surfaces tell the + operator to commit the corrected spec. It is raised only when the committed spec does not already + carry the status the re-drive needs; gating on isolation alone fired on every isolated re-arm, + where the advice is a no-op. That proof is read at the run's pinned target branch — the ref the + replacement worktree is cut from — not at the code root's current `HEAD`, which parts company with + it the moment an operator checks out another branch while the run is paused; the record carries + the branch so the remedy names the tree the re-drive will actually read. This record alone also + HOLDS the resume both surfaces fold in behind + the re-arm, since its advice is unactionable once the run has resumed, and `--resume` does not + override the hold — the re-arm stands, and `bmad-loop resume ` picks the story up once the + fix is committed. + +- **The re-arm baseline records reach the TUI operator too** (#640). Each surface carried its own + copy of the journal-kind → message routing and they had drifted: the TUI printed only + `re-armed ` and handled three kinds to `resolve`'s six, silently dropping the whole + `stale-restore-*` family — including the commits warning that is the only notice telling a human + to inspect the tree. Both now route through one table (`runs.rearm_event_notice`), raise the same + warnings before resuming, and agree on the ABORT path. Both read the journal through one shared + guard, so a corrupt journal costs the echo and never the gesture. The TUI omits the trailing + "before resuming" imperative, since it resumes in the same gesture. + +- **`bmad-loop resolve` re-stamps the spec's `baseline_revision` on both re-drive legs** (#640), + not only on a patch-restore. A from-scratch re-drive carried the escalated attempt's sha until + step-03 re-stamped it, so every gate reading a claimed baseline before then read a stale one. The + trade is recorded rather than hidden: a claim that genuinely diverged is journalled + (`rearm-baseline-restamped`) instead of being silently normalized, and a re-stamp is refused + outright when the baseline advance failed, so spec and task can never agree on a stale sha. The + record compares against the baseline the run RECORDED, so it fires only on a claim the run never + made. That refusal now also puts the spec back, as does the abort a failing `## Auto Run Result` + strip raises: those are the two that fire after the status flip has landed — the rest are + sequenced ahead of every write — so a spec carrying a movable `status:` beside an unmovable + `baseline_revision:`, or one whose strip could not be written, used to come back flipped while + the run still called the story escalated. A restore that itself fails raises rather than + degrading. + - **A published run archive now lands at mode `0600`** instead of a umask-derived mode (#591). It is staged through a file the orchestrator creates itself rather than one `tarfile` opens by name, so it inherits the private mode the rest of the `.bmad-loop` write path uses. @@ -74,6 +161,70 @@ breaking changes may land in a minor release. ### Fixed +- **`resume` re-stamps the run's recorded code root** (#716). Resume arms the engine against the + `repo_root` it re-reads from `_bmad/bmm/config.yaml` but left the `state.json` copy at its launch + value, so after an edit the engine worked in one tree while the out-of-process re-arm advanced the + attempt baseline in the other, with no error on either side. The mirror now follows the paths + resume adopts, and a move is announced rather than silent — the baselines, preserve refs and + branches already recorded name objects in the previous tree. A `state.json` from before the field + existed migrates without being reported as a move. + +- **The unreachable-spec-write warning no longer fires on a shared artifact directory** (#640). An + artifact directory configured outside the project is left where it is by `ProjectPaths.rebased`, + shared across checkouts instead of rebased onto each worktree — so the flip lands on the one file + every re-drive reads and there was never anything to commit, yet that layout took the warning on + every re-arm with a remedy naming a file outside the repository. Containment is decided on the + canonical paths, so a spec spelled out of but resolving back into the worktree still warns, as + does one the host cannot canonicalize. + +- **`bmad-loop resolve` still reports abandoned-restore residue when the re-arm aborts** (#640). + The residue is journalled before the re-stamp that can raise, so an abort discarded records + already written — including the commits warning. The echo now runs on both paths. + +- **A YAML boolean in a spec's baseline key no longer refuses the attempt** (#716). `no`, `off`, + `yes` and `on` parse as booleans, and the shared reader stringified them into `"False"`/`"True"` — + non-empty, so they were judged as a claimed sha and outranked a `baseline_commit` naming the + correct commit. Booleans are now treated as absent, exactly as a YAML null already was. + +- **A dev RETRY now notifies the operator, with the reason** (#640). RETRY was the only dev outcome + that REJECTS an attempt without raising a notice, and it is the one that discards a completed + implementation. The reason lived only in the `dev-decision` journal line, so a run could spend its + whole attempt budget throwing finished work away with nothing but the eventual exhaustion notice + reaching a human. The notice carries the reason's first line, capped and marked with `[…]` when + trimmed; the untruncated text stays in the journal. It can repeat for one attempt if the host dies + between the notice and the rollback. + +- **A stale `baseline_commit` no longer outranks the fresh `baseline_revision` a spec claims** + (#716). Both consumers of a claimed dev baseline ranked the legacy key first — and `bmad-loop +resolve` manufactures exactly that dual-key spec, inserting `baseline_revision` while never + removing a pre-existing `baseline_commit`, so the gate judged the leftover and failed an attempt + that had done everything right. One shared reader (`frontmatter.auto_dev_baseline_of`) now backs + both: `baseline_revision` wins whenever it is non-empty, `baseline_commit` remains the + backward-compatible fallback, and an empty or YAML-null value on either key reads as absent. + +- **The dev proof-of-work gate measures the tree the baseline was written in** (#716). Under an + explicit `repo_root:` with `isolation = "none"` the baseline is stamped in the git root, but every + gate probe — the commit-identity lookup, both ancestry checks and `has_changes_since` — asked the + BMAD project directory about it, so a marker only the project tree held satisfied proof of work + and a correct attempt was refused forever. The four probes now share the git root, and so do the + three exclude sources that feed them. `artifact_relpaths` is deliberately untouched: it has no + production caller, and rollback protection builds its own list against the workspace root. No + effect where the two roots coincide, which is every other configuration. + +- **`bmad-loop resolve` advances the re-arm baseline in the code tree, and says so when it cannot** + (#640). The advance read HEAD of the BMAD project directory rather than the git root, and + swallowed every failure with a bare `except Exception`, so a re-drive that silently rebuilt + against the pre-resolution tree looked exactly like one that adopted the human's fix. It is still + non-fatal outside a repository, but now narrowed to the typed git errors and journalled + (`rearm-baseline-advance-failed`); anything that is not a git answer propagates. + +- **A re-armed story can no longer replay the abandoned attempt's verdict** (#705). Re-arm resets + `attempt` to 0 and deliberately keeps `task.sessions`, so the next dispatch re-minted a session id + byte-equal to a record the abandoned attempt had already written — and a host death before the new + record landed resumed straight into that stale result. A per-task generation counter now + discriminates the id, emitted only above zero, so every id already on disk stays byte-identical + and a run resumed across the upgrade still finds its `tasks/` directories. + - **The three review gates run `[verify] commands` in the git root, not the BMAD project root** (#695). Under an explicit `repo_root:` with `isolation = "none"` they shelled out in the wrong tree — an operator's build/test verbs ran in the BMAD project dir rather than in @@ -279,6 +430,21 @@ decisions` and the TUI decision modal now also catch the state-root failure that ### Security +- **`bmad-loop diagnose` no longer ships a merge record's target branch verbatim** (#640). + The journal's `target` field carries a branch on `unit-merge-started`, `unit-merged` and + `resume-unit-merge` but a sprint status on the `board-advance-*` family, and per-field + routing is by field NAME — so the field was left unrouted and an identifier-shaped branch + name (`main`, `release`) passed through `scrub_json` into a bundle meant to be posted + publicly. The egress backstop only masked it: it repairs a value already in the legend, so + a run that journalled the same branch earlier was rescued while disclosing a + `backstop_repairs` gap, and a journal truncated past that event was not rescued at all. + Routing gains a narrow kind-scoped table for this one overloaded field; the sprint status + keeps rendering verbatim, since aliasing it would destroy what those records are read for. + The producers are deliberately NOT renamed — + `engine._replay_unlatched_ledger_carries` correlates the merge kinds on a tuple including + that field and reads journals written by earlier processes, so a rename would break the + carry replay across a version boundary. + - **The run-archive staging temp is minted by `mkstemp` beside the destination — exclusive, `0600`, freshly named per attempt** (#591). The fixed temp name was created and unlinked by name, so a symlink planted there was followed and the cleanup could remove a concurrent diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 50b1ad06..f46c7f33 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -48,7 +48,7 @@ See [README.md](../README.md) for the narrative overview and [setup-guide.md](se ### Verification (trust-nothing gate) - After each session, checks on-disk artifacts before proceeding: spec frontmatter status, independent baseline validity, non-empty diff (skipped only on the two legs that can legitimately produce none: a park — see parking under Failure handling below — and a stories plan halt) (#676), and sprint-status sync. The exact recorded commit remains valid. A different claim must uniquely resolve from its immutable object ID to a direct commit that descends from the recorded baseline and is reachable from the checkout's `HEAD`; symbolic or movable refs, ambiguous prefixes, non-commit objects, older claims (except the deferred-work bundle case below), diverged commits, and off-HEAD descendants are refused. Proof for an accepted descendant is re-anchored after that commit and counts only tracked, staged, or committed changes because no snapshot can date untracked files relative to the later claim. A deferred-work bundle may still use an older ancestor when it adopts a pre-existing story spec. In the default shared checkout, the gate proves later tracked work exists but cannot attribute it to a particular session; `[scm] isolation = "worktree"` is the provenance-preserving mode. -- Runs _your_ commands (`[verify].commands`, e.g. `pytest -q`, `ruff check .`) in the git root the code lives in (`repo_root`): your project dir by default, the mounted per-unit worktree under `[scm] isolation = "worktree"`, and an explicit `repo_root:` when you set one under `isolation = "none"`. The gate's own artifact reads — the spec, the sprint board, the deferred-work ledger — stay project-rooted either way; only the command `cwd` follows the code (#695). A broken build never reaches review or commit. +- Runs _your_ commands (`[verify].commands`, e.g. `pytest -q`, `ruff check .`) in the git root the code lives in (`repo_root`): your project dir by default, the mounted per-unit worktree under `[scm] isolation = "worktree"`, and an explicit `repo_root:` when you set one under `isolation = "none"`. The gate's own artifact READS — the spec's frontmatter, the sprint board, the deferred-work ledger — stay project-rooted either way (#695). What follows the code, besides the command `cwd`, is every git question the gate asks about it: the recorded baseline is written in the git root, so the commit-identity lookup, both ancestry checks and the non-empty-diff probe are all asked there, and the pathspecs they exclude are spelled relative to that same root (#716). Anchoring those on the project dir meant a correct attempt could be refused forever under an explicit `repo_root:`. A broken build never reaches review or commit. ### Adversarial review (review stage) @@ -69,9 +69,52 @@ See [README.md](../README.md) for the narrative overview and [setup-guide.md](se - A confirmation is resumable. Every write is checked — the spec is read back from disk, so a story is never declared done over a write that did not land — and the park record is dropped last, so a failure part-way leaves the story findable. Interrupted between the spec writes and the board write, what survives is a signed-off spec at `done` with the entry still pointing at it; re-running `confirm` **finishes** that rather than refusing it as stale, with no second prompt and no second audit section (the section on disk _is_ the acknowledgment, and the check is fence-aware). It resumes equally from a board a human fixed by hand, which is what the failure message asks for — advancing an already-`done` board is idempotent. `--list`, `--json` (`resumable`, `confirmation_recorded`) and `validate` (`operator.confirm-interrupted`) name that state rather than calling it stale. - Dispatched sessions are told the sprint board is orchestrator-owned (#437) — the sibling of the park contract above, injected into the prompt the same way. The board advances as soon as dev verifies, but the story's single commit lands only after the review loop, so a session dispatched in between opens on an uncommitted, unattributed change to `sprint-status.yaml` with nothing in the repo naming its author (one read it as a spec violation, reverted it, and tripped the sign-off-regression gate on a story both sessions agreed was finished). Story dev prompts and the review prompts of sprint and sweep runs carry the same prohibition: never write the board, never revert it, and a row at `done` or `awaiting-operator` is the orchestrator's own bookkeeping — not a defect to fix, and not proof that the work is verified, deliberately, since the row is written _before_ the deterministic dev verification runs and a repair session opens on a red tree under a `done` row. Only the **review** prompt adds where to go instead: a story that cannot be finished without a human decision is finalized to `status: blocked` with a reason — the one hand-back that both withholds the commit and reaches a human, where any other non-terminal status just burns the review budget onto a defer that rolls the work back. A dev prompt gets no such invitation, because `blocked` halts the whole run — the exact failure park exists to avoid — and a dev session that cannot finish already has park. A deferred-work bundle's dev prompt carries nothing (a bundle has no board row) while a bundle's _review_ prompt does, since a sweep runs inside a project whose board exists and is just as revertible; every injected plugin-workflow session carries the prohibition too — `post_dev_phase`, `post_review_result` and `pre_commit_gate` all fire inside that same window — as its own `## Sprint board` section appended _after_ the session-gate hooks, so a plugin prompt rewrite cannot strip it, and without the `blocked` redirect for the same reason a dev prompt has none; stories mode carries none of it, having no board at all. - Typed escalations: `CRITICAL` pauses the run + notifies (desktop + `ATTENTION` file); `PREFERENCE` is journaled and continues. +- A rejected dev attempt notifies too, with its reason (#640). RETRY was the only dev outcome that rejected an attempt silently, and it is the one that discards a completed implementation — the non-fixable leg resets the tree to baseline. The notice fires once per rejected attempt in an uninterrupted run (so ordinarily at most `max_dev_attempts` per story) and has no suppression knob of its own; it follows `[notify]` like every other notice. One attempt can raise it twice: the notice precedes the rollback, so a host that dies in between replays that verdict on resume and announces it again — treat the count as a floor on attempts rejected, not an exact tally. The reason is reduced to its first line and capped, with a `[…]` marker when it was trimmed, because a `Decision.reason` routinely carries a verify-output tail that would otherwise spill into `ATTENTION` and a desktop bubble; the untruncated reason stays in the `dev-decision` journal entry. It fires above the fixable/non-fixable split, so on a leg that goes on to pause for manual recovery the operator sees both notices. - Environment faults pause without burning budget (#194): a session whose coding CLI never reached the API — a verify command whose _environment_ is broken (`sh` reports rc `126`/`127`; on Windows a missing tool is caught by its `is not recognized` message or by resolving the command's leading token, and a command naming a file `cmd` cannot execute — a `.sh`, or any extension outside `PATHEXT`, which cmd hands to the file association and which exits `0` without running anything — is a fault rather than a silent rc `0` pass, #302) **or** a session whose log matches the profile's `env_fault_patterns` (an `API Error … Connection refused`-class transport failure, or a provider quota/usage-limit refusal, that idled out the session clock) — pauses the run with the matched evidence instead of charging the attempt and deferring the story as if its code were broken. Re-arm restores the budget. Patterns are per-profile: `claude` seeds three, reproducing only complete error sentences its CLI was captured printing (connection loss, and the two captured provider 5xx refusals — statuses enumerated, never ranged, so an uncaptured `503` stays prose), so a story that merely writes _about_ a provider error cannot trip them (#507); `opencode` seeds a provider quota/rate-limit and connection pair (#323), matched against the `opencode serve` process's own stdout, which the model cannot write to; the other four profiles ship none. Each adapter matches them against the log named by its `ENV_FAULT_LOG_SUFFIX` — the tmux pane capture `logs/.log`, or `.server.out` (the `opencode serve` process's own stdout) for `opencode-http`, never that adapter's model-written transcript. A pattern is only sound against a log the model cannot write to; where that does not hold — the pane capture — the pattern has to reproduce a whole captured sentence, because an error token plus a cause on the same line is precisely the shape a story writing about the error emits, and that framing is what the guard now refuses (#507). A usage-limit / quota cause stays unseeded on the pane-capture profiles for the same evidentiary reason: no captured line exists for them (#323). Extend or disable them in a project profile overlay. - A session the multiplexer lost says so (#489). Sessions complete on a hook `Stop` or on window death, and a window is gone whether the CLI exited or something destroyed the whole mux session out from under the run — an external reaper, a concurrent prune or `bmad-loop stop`, an operator `kill-session`, a server crash, the host sleeping. Both are `crashed`, so the retry/defer reason an operator reads said only `dev session crashed` — pointing at the agent when the host was at fault. The crash verdict now asks whether the _session_ still exists and, when it does not, says so in the reason (`… session crashed: the multiplexer no longer reports the session, so the window's disappearance is not evidence the CLI exited`), as `session_vanished` on `dev-decision` and `fix-decision` either way, beside the routing each fed, on every role's `session-end` journal entry when it is true (the convention `env_fault` already uses there), and as a `session-vanished` breadcrumb in `session-lifecycle.jsonl`. The repair path carries it the same way: when fix attempts are exhausted the defer names the lost session instead of blaming the tree for repairs that never ran. The wording states what the evidence _withdraws_, not what it proves: `has_session` maps every nonzero backend result to False, so a negative lookup is "the backend did not confirm it" rather than proof the session is gone — enough to stop an operator reading window death as a CLI exit, not enough to name a destroyer. It composes with an environment-fault pause instead of being swallowed by it. A session reaped _after_ flushing its result still scores `completed` and is not diagnosed — it produced something. Diagnosis only — the routing is unchanged, and a retry re-creates the session. -- CRITICAL resolution: `bmad-loop resolve ` opens an interactive resolve agent seeded with the escalation + frozen spec; you disambiguate, it re-arms the story (`escalated → pending`, spec reset to `ready-for-dev`) and resumes. `--no-interactive` skips to re-arm if you fixed the spec yourself. +- CRITICAL resolution: `bmad-loop resolve ` opens an interactive resolve agent seeded with the escalation + frozen spec; you disambiguate, it re-arms the story (`escalated → pending`, spec reset to `ready-for-dev`) and resumes. `--no-interactive` skips to re-arm if you fixed the spec yourself. The re-arm advances the story's + baseline in the **code tree** and is honest when it cannot: a failed advance is narrowed to typed git + errors, journalled, echoed to stderr, and explicitly NOT followed by a re-stamp it did not earn, so + spec and task never silently agree on a stale sha (#640). A re-stamp that does overwrite a differing + claim records what it replaced, and warns on either leg: the record fires only when the spec claimed + a baseline the run never recorded, which is the only remaining trace of a divergence the gate can no + longer report. `spec_file` is persisted relative to a worktree for an isolated task, so re-arm re-anchors it on + that worktree before writing — resolved against the process cwd it named the main checkout's copy + of the same story spec, and both writes landed on a file the run never used. A spec re-arm still + cannot read has its baseline re-stamp skipped rather than silently no-oped + (`rearm-baseline-restamp-skipped`), and a status flip that quietly changed nothing is reported + too (`rearm-spec-flip-skipped`) — though not when the spec was simply already at the target status, + which is an ordinary re-arm rather than a failure; neither record depends on the git advance having + succeeded. On a spec the re-drive will actually READ, that skip also refuses the re-arm outright: the + routing status is what the re-drive runs on, and a spec without one halts the re-driven session on + an unrecognized status, so the escalation stays armed for a corrected spec instead of being spent + on a session that cannot route. The spec is left byte-identical, down to the stale `## Auto Run +Result` section. Every other spec keeps warn-and-continue, and the record says which happened + (`refused`) so neither surface prints a refusal's remedy for a re-arm that completed. One a re-arm + cannot read at all is warn-and-continue because the failed flip says nothing about what the + re-drive will read: the path is worktree-relative and the re-drive mounts a fresh worktree and + reads the COMMITTED spec either way. A worktree-local one is warn-and-continue for the sharper + version of the same reason — the readable file is the copy that fresh mount destroys, so refusing + over it would demand a repair to a file nothing opens, even once the corrected spec is committed + and routable. Under worktree isolation those writes land in a worktree the re-drive discards, and the + re-driven session reads the COMMITTED spec, so re-arm says so (`rearm-spec-write-unreachable`) and + asks you to commit the corrected spec — but only when the spec really is worktree-local and the + committed spec does not already carry the status the re-drive needs, so an isolated re-arm whose + correction is already committed says nothing, and neither does one whose spec sits in an artifact + directory configured outside the project: those are shared across checkouts rather than rebased + onto each worktree, so the flip lands on the one file every re-drive reads. That record also HOLDS + the resume: both surfaces re-arm and resume in one gesture, which made its own "commit the + corrected spec before resuming" advice unactionable the moment it printed and then burned the + escalation on a session that halts blocked. They now stop after the re-arm — the story stays armed, + `bmad-loop resume ` picks it up once the fix is committed, and `--resume` does not override + it, since the record is written on proof rather than suspicion. The advisory warnings do not hold. + All of these warnings reach the TUI's re-arm as well as `resolve`'s — both route every + kind through one shared table, so neither surface can silently learn a kind the other drops, + though each still owns where it calls the echo from and the TUI drops the trailing "before + resuming" advice, since it otherwise resumes in the same gesture. Each re-arm also bumps a per-task + **generation**, so the re-minted session id cannot collide with the abandoned attempt's record — ids + already on disk keep their exact spelling, since the suffix appears only above generation zero + (#705). - Attempt-owned sprint-spec recovery (#123, #630): a bound plain attempt whose only residue is its own lifecycle flip is normalized back to its pre-attempt lifecycle status, proven Git-clean, and retried. Every bound retry chain snapshots its first spec input byte-for-byte and retains it across both dev-verification and review-verification repair sessions; a resolved re-drive therefore retains the operator-corrected `ready-for-dev` input rather than a failed child's later body. Repair entry points validate retained authority before constructing a prompt that can reset the spec. A non-fixable retry parks the failed child first, restores that snapshot, and re-establishes the promised route after resetting sibling residue. The same snapshot restores pre-launch operator edits when a plain child puts a tracked spec back at Git baseline. Git-ignored and pre-existing-untracked bound specs use the byte snapshot as their dirtiness oracle and are force-included only in the private recovery ref before restoration; index-only force-adds and cached removals also trigger cleanup and restore baseline index ownership. That real repair reports `rollback-owned-spec-restored`, never `rollback-skipped-clean`. Missing, unreadable, deleted, retargeted, changed external, or unsafe legacy authority pauses once with spec-specific adoption instructions and clears the unusable pair so manual recovery can converge; recovery also refuses a reset whose baseline would replace the canonical path or a parent directory with a symlink, tree, file, or other unsafe shape. An initial Sprint binding fault may safely degrade to an unbound bare-key launch; an existing Stories folder+id target instead aborts unless it can be snapshotted. Once an explicit binding is durable, a later snapshot fault aborts before child launch while retaining that authority for recovery. Fresh sprint tasks with no recorded path remain bare-key dispatches; other substantive changes or sibling residue follow rollback policy; Stories remains folder+id; Sweep remains intent-bundle routing; snapshots are retired after commit; and recovery never auto-commits the human correction. - Intent-gap patch-restore (BMAD-METHOD#2564): when review halts on an `intent gap`, the dev primitive saves the attempted change as a patch file (referenced from the halt output) before reverting the tree. If that reading turns out to be correct, the resolve agent adds `"restore_patch": ""` to its `resolution.json`; the orchestrator re-arms the spec to `in-review` (not `ready-for-dev`) and re-applies the patch after every reset, so the re-driven session resumes _review_ on the restored diff instead of re-implementing. `bmad-loop resolve --no-interactive --restore-patch ` does the same by hand. A patch that fails to apply escalates rather than dispatching onto a half-restored tree. Sweep bundles get the same recovery. @@ -89,7 +132,7 @@ See [README.md](../README.md) for the narrative overview and [setup-guide.md](se - The sprint board is **worktree-canonical for the duration of a story** (#350). A board your project _tracks_ needs nothing special: the story's advance is an ordinary modification of a checked-out file and rides the unit commit through the merge. A **gitignored** board is neither checked out nor delivered by one, so it is seeded into the worktree alongside the deferred-work ledger — the orchestrator writes the board through the worktree and `verify_dev` then reads the file it just wrote, which without the seed is a missing file the run dies on rather than a lost write. Its advance is re-applied to the main checkout's board **after** the merge, journaled `board-advance-carried` — or `board-advance-carry-uncommitted` where `git add` refused the ignored path, which is the ordinary outcome for such a board and not a fault (the status on disk is the value; the commit is best-effort). The carry replays from its record if a crash lands between the merge and its latch, and `sprintstatus.advance` never regresses, so a double application is a no-op. On that replay leg the merge — and with it the pre-flight above — has already happened, so the carry proves its own ownership before committing: it recomputes HEAD's content through `advance` and asks **git** whether the board holds that and nothing else, which a crashed pass's half-written advance does and an operator's edit does not. Sameness is git's own — both sides are hashed through the path's clean filter — so a board spelled CRLF by one host and LF by another still matches, where a byte compare would have to guess which spelling is on disk and would refuse a pristine board on every host that chose the other. **Both** places git holds the path are proved, because the carry's `git add` overwrites both: the working tree it copies into the commit, and the index it stages over — an edit staged and then restored in the working tree exists nowhere afterwards, so proving the working tree alone would authorize destroying it. That proof guards the **commit**, which is one write too late for the story's own row: `advance` would already have replaced an operator's status with the target, leaving exactly the bytes the proof accepts — and skipping the commit saves nothing when the value scheduling reads is the one on disk. So that one row is checked **before** the advance as well, and a status that is neither HEAD's nor this pass's own is refused there with nothing written. Both refusals journal `board-advance-carry-foreign-dirt`, and they refuse at different points with different stakes: the pre-advance row check refuses with **nothing written**, so your status is still on the board; the post-advance proof withholds only the **commit** — the advance is on disk by then — and the dirt it declined to take is still escalated by the next run's merge pre-flight. A board that is **gone** by carry time is refused before either question is asked and journaled `board-advance-carry-failed`, the same record a vanished row gets. Every probe fails closed, and git's own dirt answer decides whether either comparison is asked at all — a board nobody wrote is not their question (#618). Both comparisons apply to a board git **tracks**. A gitignored one has no baseline anywhere in git to be compared against, so a replayed carry can still overwrite a row you edited on it while the host was down; the commit half cannot arise there at all, since `git add` refuses an ignored path outright. Scheduling is what depends on it: `_pick_next` reads the **main** board, so an advance that never came back hands finished work to the next run's dev session. - Only the orchestrator writes the board — the session prompts forbid it, and `sprintstatus.advance` is the sole write path. The consequence under isolation, stated rather than left as a surprise: in a _gitignored_ board a session's edit to any OTHER story's row lives only in the worktree copy and evaporates when the worktree is removed. Only the story's own advance is carried back, because only that one is the orchestrator's own write. - `bmad-loop confirm` leaves a gitignored board **out** of its commit (#577). `git add` refuses an explicitly named ignored path and refuses the whole operand list with it, which would take the spec's flip to `done` and the park record's deletion down too — leaving the tree dirty for the next run's preflight to refuse. The board is still advanced on disk; a board git will not track has no commit to ride. -- Merge-back is serialized; `max_parallel` is a validated knob clamped to `1` until parallel fan-out is built. The `repo_root` key in `_bmad/bmm/config.yaml` (defaults to the project dir) decouples where git/code work happens from where run state lives (monorepos), and your `[verify] commands` run there too while the orchestrator's own artifact reads stay project-rooted (#695) — but it is **mutually exclusive with `isolation = "worktree"`**, which seeds from `repo_root` while the preflight probes `project`; `validate` reports the pair and the run/sweep/resume preflight refuses it (#414; #443 lifts the restriction). +- Merge-back is serialized; `max_parallel` is a validated knob clamped to `1` until parallel fan-out is built. The `repo_root` key in `_bmad/bmm/config.yaml` (defaults to the project dir) decouples where git/code work happens from where run state lives (monorepos), and your `[verify] commands` run there too, as does every git question the dev gate asks about the code — baseline identity, ancestry, and the non-empty-diff proof, with their exclude pathspecs spelled relative to the same root (#695, #716) — while the orchestrator's own artifact reads stay project-rooted — but it is **mutually exclusive with `isolation = "worktree"`**, which seeds from `repo_root` while the preflight probes `project`; `validate` reports the pair and the run/sweep/resume preflight refuses it (#414; #443 lifts the restriction). - `commit_message_template` (`{story_key}` / `{run_id}` / `{story_title}` substituted) customizes story/bundle commit messages. ### Plugins (extensibility) @@ -111,7 +154,7 @@ See [README.md](../README.md) for the narrative overview and [setup-guide.md](se - Every run is a resumable on-disk state machine: `bmad-loop resume ` continues from a gate, escalation, or interruption. - A graceful stop (`stop --graceful` / TUI `S`) is resumable too: unlike a hard stop killed mid-item, it lets the in-flight item finish through commit and finalizes cleanly, ending as a `stopped` run that `resume` picks up at the next item. -- All run state in `.bmad-loop/runs//` (gitignored): `state.json`; `journal.jsonl` (every decision, including the `session-synthesized-from-frontmatter` catch and its `spec-marker-repaired` repair, #276, plus one `verify-command-result` per verifier command — emitted on the dev and repair legs alike — whose stream pointers name the `verify/` directory below); `tasks//` (per-session prompt + `result.json` + breadcrumbs — `session-lifecycle.jsonl` records timeout fires, budget-guard trips (`budget-tripped` / `over-budget-fired`), transport-failure classification (`env-fault-classified`, #194), a mux session lost under the run (`session-vanished`, #489) and the #276 forensics (`spec-status-transition-observed`, `frontmatter-unmodified-refused`, `contract-nudge-sent`); `heartbeat.json` is the wait loop's proof-of-life; `resultless-stops.jsonl` records give-up Stops with a verdict — `no-artifact`, `ambiguous-frontmatter`, `unmodified-since-launch`, `terminal-frontmatter-pending`); `logs/`; `verify/` (verifier command stdout/stderr, one file per stream, pointed at by the journal's `verify-command-result` records — its own store, because every name in `logs/` is a session task id the TUI resolves as a pane log; each stream is retained tail-first up to `[verify] stream_capture_kb` (256 KiB, `0` = keep nothing) and the record carries the full byte count, a truncation flag, and a `capture_error` when the write itself failed); `deferred/`; `resolve/`; `ATTENTION`; `ctl-window` (the control-session window id the last TUI launch minted, so attach/stop follow the live window, #482). +- All run state in `.bmad-loop/runs//` (gitignored): `state.json` (which records `repo_root`, the git root code work happens in, so an out-of-process re-arm reads back the tree the run measured, #716 — resume re-stamps it from the config.yaml it just re-read, since that is the tree it arms the engine against, and warns when the root moved: the baselines, preserve refs and branches already recorded name objects in the previous one. `resolve` and the TUI's re-arm re-stamp it themselves, before they re-arm: both re-arm and then resume in one gesture, so resume's own re-stamp lands after the re-arm has already advanced the baseline in whichever tree the mirror still named); `journal.jsonl` (every decision, including the `session-synthesized-from-frontmatter` catch and its `spec-marker-repaired` repair, #276, plus one `verify-command-result` per verifier command — emitted on the dev and repair legs alike — whose stream pointers name the `verify/` directory below); `tasks//` (per-session prompt + `result.json` + breadcrumbs — `session-lifecycle.jsonl` records timeout fires, budget-guard trips (`budget-tripped` / `over-budget-fired`), transport-failure classification (`env-fault-classified`, #194), a mux session lost under the run (`session-vanished`, #489) and the #276 forensics (`spec-status-transition-observed`, `frontmatter-unmodified-refused`, `contract-nudge-sent`); `heartbeat.json` is the wait loop's proof-of-life; `resultless-stops.jsonl` records give-up Stops with a verdict — `no-artifact`, `ambiguous-frontmatter`, `unmodified-since-launch`, `terminal-frontmatter-pending`); `logs/`; `verify/` (verifier command stdout/stderr, one file per stream, pointed at by the journal's `verify-command-result` records — its own store, because every name in `logs/` is a session task id the TUI resolves as a pane log; each stream is retained tail-first up to `[verify] stream_capture_kb` (256 KiB, `0` = keep nothing) and the record carries the full byte count, a truncation flag, and a `capture_error` when the write itself failed); `deferred/`; `resolve/`; `ATTENTION`; `ctl-window` (the control-session window id the last TUI launch minted, so attach/stop follow the live window, #482). - One piece deliberately lives **outside** that directory: the hook-event channel (#494) is at `///events/` under the user-scoped state root (`BMAD_LOOP_STATE_DIR`, see the [transport section](#hook-based-transport-no-pane-scraping) below and the README's env-var table), not `/events/`. The orchestrator still polls the legacy in-tree location, so a project whose installed relay predates the move keeps completing its sessions. `delete`, `archive` and `clean` remove the out-of-tree counterpart along with the run dir, and `clean` sweeps counterparts whose run dir is already gone; an **archived** run's tarball therefore no longer contains `events/` — those files are transient completion signals, consumed while the run was live, and everything an archive is read for later is in the run dir. - `journal.jsonl` records `session-end` for every session unconditionally — even a teardown that throws still lands one (status `aborted` when the outcome is unknowable). A timed-out session's entry carries `fired_at` (wall time the deadline was declared), `teardown_s` (wall seconds from that fire to this entry — the teardown gap), and `expired_clock` (`monotonic` / `wall` / `both` — `wall` alone fingerprints a host suspend that froze the monotonic clock). Every entry whose usage was read carries `tokens` (raw) and `tokens_weighted` (cache reads at `limits.cache_read_weight`), keeping per-session spend reconstructible; both are `null` when the usage read failed, and both are absent on an `aborted` end. `tokens_weighted` is the end-of-session total — distinct from a tripped session's `budget_weighted`, the guard's mid-session sample at trip time. diff --git a/src/bmad_loop/cli.py b/src/bmad_loop/cli.py index f537be18..96a33a8a 100644 --- a/src/bmad_loop/cli.py +++ b/src/bmad_loop/cli.py @@ -12,7 +12,7 @@ import time from enum import IntEnum from pathlib import Path -from typing import TYPE_CHECKING +from typing import TYPE_CHECKING, Any from . import ( __version__, @@ -2503,10 +2503,30 @@ def _resume_paused_run(project: Path, run_dir: Path) -> int: if pinned is None: pinned = state.trusted_config_digest security_config_changed = bool(pinned) and new_digest != pinned + # The recorded code root against the one THIS process just loaded. Resume + # re-reads config.yaml, so a `repo_root:` key added, changed or removed while the + # run was paused re-points the engine at a different git tree — `compose_resume` + # below builds the Workspace off `paths`, not off state.json. The persisted + # mirror is what `runs.rearm_escalation` reads back OUT OF PROCESS + # (`RunState.code_root`), so leaving it at its launch value makes the two readers + # disagree exactly when the config moved: `resolve` would advance the attempt + # baseline in the old tree while the resumed engine resets and measures in the + # new one. Re-stamped below, with the snapshot and the digest. + # + # An exact string compare, deliberately, with no canonicalization: both sides are + # `str(paths.repo_root)` off `bmadconfig.load_paths`, which resolves every member + # or raises, so they are spelled the same way whenever they name the same tree. + # The `bool(state.repo_root)` guard is what keeps a legacy state.json — written + # before the field existed, and read back as "" — out of the comparison: it is a + # missing value, not a divergent one, and the re-stamp migrates it silently. + code_root_changed = bool(state.repo_root) and state.repo_root != str(paths.repo_root) fields: dict[str, object] = { # Scalars only, per the note above: a bool records THAT the pinned surface # moved without journaling a command, a binary path or a plugin name. "security_config_changed": security_config_changed, + # Same treatment, same reason: a bool, never either path. `diagnose` renders + # the split as a presence flag for exactly this reason (`repo_root_diverges`). + "code_root_changed": code_root_changed, "was_paused": state.paused_reason, "cache_read_weight": pol.limits.cache_read_weight, # Compare JSON-normalized, the way save_state persists it: to_dict() @@ -2534,6 +2554,21 @@ def _resume_paused_run(project: Path, run_dir: Path) -> int: " make that edit.", file=sys.stderr, ) + if code_root_changed: + # Loud, because the re-stamp below is not a repair: every sha this run already + # recorded — each task's `baseline_commit`, its preserve refs, its unit + # branches — names an object in the PREVIOUS tree, and nothing here can move + # them. The re-stamp only stops the two readers from disagreeing about which + # tree the run is in from here on; whether the new tree can honor those shas + # is the operator's call, and this is the moment they can still make it. + print( + f"warning: run {run_dir.name}: the code root in _bmad/bmm/config.yaml has" + " changed since this run started — the resumed engine works in the tree" + " configured now, while the baselines, preserve refs and branches this run" + " already recorded name objects in the previous one. Restore the previous" + " `repo_root:` value if you did not intend the move.", + file=sys.stderr, + ) # Re-stamp: the snapshot must describe the policy THIS process enforces, for # its whole lifetime (Policy is loaded once here and frozen — the engine never # re-reads policy.toml). Enforcement already reads the reloaded `pol` (the @@ -2569,6 +2604,11 @@ def _resume_paused_run(project: Path, run_dir: Path) -> int: # BMAD_LOOP_STATE_DIR. Tampering that removes the out-of-tree file is a # different problem with no fix at equal privilege — #571. state.trusted_config_digest = new_digest + # ...and the code root, for the same reason and at the same moment: this process + # arms an engine against `paths.repo_root` (compose_resume -> Workspace), so that + # is the tree `runs.rearm_escalation` must read back. Unconditional, so it also + # migrates a pre-field state.json onto the root it was already using. + state.repo_root = str(paths.repo_root) state.clear_pause() runs.write_pid(run_dir) # Persist before the engine starts: status, the TUI and diagnose only ever @@ -2740,34 +2780,56 @@ def _resolve_restore_patch( return str(patch), None -def _echo_stale_restore(run_dir: Path, seen_entries: int) -> None: - """Surface the `stale-restore-*` events a just-completed re-arm journaled about - the restore attempt it abandoned (runs._stale_restore_residue). The commits - variant is the one the human must act on — nothing else will.""" - for entry in Journal(run_dir).entries()[seen_entries:]: - kind = entry.get("kind", "") - if kind == "stale-restore-excluded": - files = ", ".join(entry.get("files", [])) - print( - f"note: excluded the abandoned restore's new files from the " - f"re-drive baseline: {files}", - file=sys.stderr, - ) - elif kind == "stale-restore-unparseable": - print( - f"warning: could not read the abandoned restore patch " - f"({entry.get('patch', '?')}) — its new files may be swept into the " - "next commit; check `git status` before resuming", - file=sys.stderr, - ) - elif kind == "stale-restore-commits": - n = len(entry.get("commits", [])) - print( - f"warning: {n} commit(s) sit below the re-drive's new baseline " - f"({entry.get('old_baseline', '?')[:12]}..) — if any came from the " - "abandoned attempt rather than your resolve, revert them now", - file=sys.stderr, - ) +def _echo_rearm_events(run_dir: Path, before: list[dict[str, Any]] | None) -> bool: + """Surface the events a just-completed re-arm journaled: the `stale-restore-*` + residue of the restore attempt it abandoned (runs._stale_restore_residue), and the + `rearm-*` records the status flip, the advance and the re-stamp write. The commits + variant is the one the human must act on — nothing else will. + + Named for the re-arm, not for the stale restore: it began as a `stale-restore-*` + echo and now carries the baseline family too, so a name from the narrower era + would send the next re-arm record somewhere else. + + Routing lives in `runs.rearm_event_notice`, not here, because the TUI re-arms + through the same journal and used to carry its own divergent copy of this chain — + it surfaced three of the kinds and silently dropped the rest. One table, two + renderings: this one appends the `next_step` imperative, the TUI omits it because + it resumes in the same gesture. + + The baseline records are echoed because a failed advance means the re-drive + rebuilds against the tree as it stood BEFORE the resolve, and the re-stamp then + deliberately refuses to write a sha it did not earn. All of it is warn-only by + contract (a project that is not a repo must not fail re-arm), so without an echo + the whole degrade is journal-only — the invisibility #640(b) exists to end, not to + relocate. + + Returns True when one of those records HOLDS the resume + (`runs.rearm_holds_the_resume`): the caller re-arms and resumes in a single gesture, + and a record proving the re-drive cannot route has to break that gesture, or its own + "before resuming" imperative is already unactionable the moment it prints. The + question is asked here because this is the one walk over the entries the re-arm + added, and the answer has to survive the `finally` it is computed in.""" + after = runs.journal_entries_or_none(run_dir) + if before is None or after is None: + # Either end of the diff is unreadable, so there is no trustworthy "new since + # the re-arm" window. Skip rather than guess: this runs from a `finally`, and a + # raise here would replace the `RearmError` the operator needs, while treating a + # failed read as "no entries seen" would replay the whole journal as new. The + # hold degrades with the echo, for the same reason: an unproven hold is a guess, + # and this is what the gesture did before either existed. + return False + holds = False + for entry in after[len(before) :]: + # asked of every entry, BEFORE the routing table can drop it — a `None` notice + # means "nothing to print here", never "nothing to decide here" + holds = runs.rearm_holds_the_resume(entry) or holds + notice = runs.rearm_event_notice(entry) + if notice is None: + continue + severity, message, next_step = notice + tail = f"; {next_step}" if next_step else "" + print(f"{severity}: {message}{tail}", file=sys.stderr) + return holds def cmd_resolve(args: argparse.Namespace) -> int: @@ -2870,13 +2932,75 @@ def cmd_resolve(args: argparse.Namespace) -> int: if args.resume is None and not _confirm(f"re-arm {story_key} and resume run {args.run_id}?"): print("cancelled — run is still paused at the escalation") return 0 - seen_entries = len(Journal(run_dir).entries()) + # The code root `runs.rearm_escalation` reads back OUT OF PROCESS + # (`RunState.code_root`). `_resume_paused_run` re-stamps it because the engine it + # arms works in `paths.repo_root` — but on THIS path the re-arm runs FIRST, so that + # re-stamp lands too late to aim it: a `repo_root:` key added, changed or removed + # while the run was paused split the two readers exactly the way the re-stamp exists + # to prevent — the attempt baseline advanced (and `baseline_revision` re-stamped) in + # the tree the run has LEFT, while the engine resumed at the bottom of this function + # reset and measured in the new one, with no error anywhere. + # + # AFTER the confirm, deliberately: a cancelled resolve writes nothing, so the + # divergence is still there for `resume` to report on its own terms. + try: + paths = bmadconfig.load_paths(project) + except (bmadconfig.BmadConfigError, OSError) as e: + # An observation, so it degrades: without the config this process cannot NAME + # the tree, and re-pointing the mirror at a guess is the one outcome worse than + # leaving it alone. The re-arm then reads the root the run recorded — precisely + # what it did before this seam existed — and the default flow's + # `_resume_paused_run` raises on the same config moments later. Reported, never + # silent: the write this could not aim is the whole subject of the block above. + print( + f"warning: run {args.run_id}: cannot read the project config to confirm the " + f"code root ({e}) — re-arming against the root this run recorded", + file=sys.stderr, + ) + else: + # The SAME refusal `_resume_paused_run` makes, hoisted ahead of both writes + # below — because aiming the mirror at the tree config.yaml names is only + # correct for a configuration the orchestrator will actually run, and this is + # not one. `worktree_isolation_conflict` fires exactly when `repo_root` is an + # override beside `isolation = "worktree"`, so on that config the re-stamp + # persisted the unsupported root and `rearm_escalation` then advanced the + # attempt baseline (and re-stamped the spec's `baseline_revision`) against it + # — all of it before `_resume_paused_run` at the bottom of this function + # reached the refusal and returned 1. + # + # Everything about that was spent: the operator was told "re-armed " + # and then refused, and the story was no longer ESCALATED, so `resolve` — which + # requires an escalation — could not re-run to correct it. The escalation was + # burned on a gesture the orchestrator had already decided it would not honor. + # + # It also falsified the premise the baseline advance is built on. `runs` + # reasons that `repo_root == project` "in every reachable configuration" + # BECAUSE this refusal exists, and reads the code tree's HEAD on that basis; + # a path that mutates first and refuses second made the unreachable + # configuration reachable, in the one function that had ruled it out. + # + # Ordered after the confirm with the re-stamp, not before it: a cancelled + # resolve still writes nothing, and an operator who declines is not owed a + # config lecture about a gesture they did not make. + if (rc := _reject_isolation_conflict(paths, pol)) is not None: + return rc + if (moved := runs.restamp_code_root(run_dir, paths.repo_root)) is not None: + print(f"warning: {moved}", file=sys.stderr) + before_entries = runs.journal_entries_or_none(run_dir) + hold_resume = False try: runs.rearm_escalation(run_dir, story_key, restore_patch=restore_patch) except runs.RearmError as e: print(f"error: {e}", file=sys.stderr) return 1 - _echo_stale_restore(run_dir, seen_entries) + finally: + # In the `finally`, not after the `try`: `_stale_restore_residue` journals + # BEFORE the re-stamp block that raises `RearmError`, so on that path the + # records were already written and returning early threw them away — including + # `stale-restore-commits`, the one record whose whole point is that nothing + # else will tell the human. An abort is when that residue matters most: the + # re-arm half-ran and the operator has to decide what to do with the tree. + hold_resume = _echo_rearm_events(run_dir, before_entries) print( f"re-armed {story_key}" + (" (restoring the attempted change for review)" if restore_patch else "") @@ -2884,6 +3008,18 @@ def cmd_resolve(args: argparse.Namespace) -> int: if args.resume is False: print(f"resume when ready: bmad-loop resume {args.run_id}") return 0 + if hold_resume: + # The re-arm SUCCEEDED — the task is armed and persisted — so this is a 0, and it + # stops the GESTURE, not the run. `--resume` does not override it: that flag + # skips the confirmation prompt, while the hold is not a question but a proof + # that resuming now spends the escalation on a session that cannot route + # (`runs.rearm_holds_the_resume`). The escape hatch is the command this prints, + # which the operator reaches the moment their correction is committed. + print( + "NOT resuming in this gesture — the correction has to reach the re-drive " + f"first (see the warning above). Then: bmad-loop resume {args.run_id}" + ) + return 0 from .tui import launch # import-safe: launch.py has no textual imports if launch.in_ctl_session(): diff --git a/src/bmad_loop/devcontract.py b/src/bmad_loop/devcontract.py index 0d163f35..0c5ff0dc 100644 --- a/src/bmad_loop/devcontract.py +++ b/src/bmad_loop/devcontract.py @@ -29,7 +29,7 @@ from . import deferredwork from .fences import fenced as _fenced -from .frontmatter import _edit_frontmatter_block, status_of +from .frontmatter import _edit_frontmatter_block, auto_dev_baseline_of, status_of from .platform_util import atomic_write_bytes, atomic_write_bytes_confined from .verify import DEV_WORKFLOW, operator_actions_of, read_frontmatter @@ -386,8 +386,9 @@ def synthesize_result( status = fm_status or arr.status consistent = (not arr.present) or (not arr.status) or (arr.status == status) - # The skill names the baseline `baseline_revision`; verify reads `baseline_commit`. - baseline = str(fm.get("baseline_commit", fm.get("baseline_revision", ""))).strip() + # One reader, shared with verify's baseline-match gate, so the two halves of + # the contract cannot disagree about which key wins (#716). + baseline = auto_dev_baseline_of(fm) escalations: list[dict[str, Any]] = [] if status == BLOCKED or arr.status == BLOCKED: diff --git a/src/bmad_loop/diagnostics.py b/src/bmad_loop/diagnostics.py index 634cea0c..0ed34a58 100644 --- a/src/bmad_loop/diagnostics.py +++ b/src/bmad_loop/diagnostics.py @@ -111,6 +111,11 @@ "target_branch": "branch", "commit": "commit", "baseline": "commit", + # The baseline a re-arm's re-stamp replaced (`rearm-baseline-restamped`). Its + # OWN entry rather than the `baseline` one beside it, because both shas ride a + # single record: alias one and leave the other and a dump pseudonymizes half a + # comparison, which is worse than either doing both or doing neither. + "overwritten": "commit", # A spec name IS the customer's feature name — `Pseudonymizer`'s own docstring # has always listed "spec filenames" among what it exists to alias, so the # omission here was a routing gap, not a policy. A producer that journals a @@ -124,6 +129,59 @@ # whose epic could not be resolved. See `_JOURNAL_BASENAME_NAMESPACES` for why # the value is normalized first: the producers do NOT agree on a bare basename. "spec": "spec", + # The same value also arrives under a second field NAME. `runs.rearm_escalation` + # is the only journal producer of `spec_file`, across FOUR kinds — + # `rearm-spec-write-unreachable`, `rearm-spec-flip-skipped`, + # `rearm-baseline-restamp-skipped` and `rearm-baseline-restamped`. Routing is by + # field NAME, not by kind, so the list is documentation rather than a gate — but an + # enumeration that undercounts is how the next reader concludes a kind is unrouted. + # `engine._park_awaiting_operator` passes + # `spec_file=` to `operatoractions.record_park`, which is a record file, not the + # journal. So the divergence is BETWEEN FIELDS, not between two producers of this + # one — but BOTH fields are mixed-shape, and neither is the reliable one: + # `spec_file` is now always ABSOLUTE, because all four kinds journal + # `str(_task_spec_path(...))`, whose anchors (`task.worktree_path`, `state.project`) + # are absolute in every production path; while `spec` is NOT uniformly absolute — + # engine's reconcile and marker-repair kinds journal an absolute `str(spec_path)`, + # but `stories_engine`'s `checkpoint-pause` journals the raw persisted + # `task.spec_file`, which is worktree-relative for a task that ran under isolation. + # Same value, same hazard, same namespace. Do NOT read this as "one field is + # already normalized, so the basename step is dead": `_JOURNAL_BASENAME_NAMESPACES` + # keys on the NAMESPACE rather than the field precisely so both spellings reduce to + # one alias whichever shape either happens to carry. + "spec_file": "spec", +} +# Kind-scoped routing, consulted BEFORE the by-name table above and losing to +# `_JOURNAL_DROP_FIELDS`, which is stricter than any alias. +# +# It exists for ONE field, and the by-name rule genuinely cannot express it: `target` +# carries the target BRANCH on the three merge kinds below and a sprint STATUS on the +# `board-advance-*` family (`board-advance-carried`, `-carry-failed`, +# `-carry-foreign-dirt`, `-carry-uncommitted`). Aliasing it by NAME would pseudonymize +# statuses as branches, turning a legible `"target": "done"` into `branch-3f2a` and +# destroying the field a maintainer reads those kinds for; leaving it unrouted — what +# happened until now — ships an identifier-shaped branch name VERBATIM. A normal run +# journals the same string as `branch` first (`ensure_target_branch`), so the egress +# backstop repairs it and discloses a `backstop_repairs` gap, but that is a defense in +# depth that only works because the value is already in the legend: a journal truncated +# past that event, or a `unit-merged` read out of a bundle on its own, has nothing to +# repair from. +# +# Deliberately NOT fixed by renaming the producers to `target_branch`, which is the +# obvious move and is unsafe: `engine._replay_unlatched_ledger_carries` correlates +# `unit-merge-started` against `unit-merged` on a tuple that INCLUDES this field, and it +# reads a journal written by an earlier process — and possibly an earlier version. A +# rename would make a run started before the change and resumed after it fail to +# correlate, silently skipping the carry replay so a resumed sweep re-triages work that +# already landed. The scrub is what is wrong, so the scrub is where the fix belongs. +# +# A closed set, not a growing one: any NEW producer should pick a name the by-name table +# already routes (`branch`, or `target_branch` — see `runs.rearm_escalation`) rather than +# add a row here. +_JOURNAL_KIND_ALIAS_FIELDS: dict[str, dict[str, str]] = { + "unit-merge-started": {"target": "branch"}, + "unit-merged": {"target": "branch"}, + "resume-unit-merge": {"target": "branch"}, } # Namespaces whose journalled value arrives in more than one shape and must be # reduced to its basename before it is aliased. `spec` is one: engine.py's @@ -176,6 +234,19 @@ "capture_error", "stdout_path", "stderr_path", + # An absolute host path naming the run's code tree + # (`rearm-baseline-advance-failed`). Dropped rather than aliased: unlike a + # spec filename it correlates nothing across events — one run has one code + # root — while carrying the customer's directory layout. Dropped rather + # than left to the `scrub_json` fallback: that fallback happens to redact an + # absolute path (`_IDENTIFIER_RE` forbids `/`, `\` and `:`, so any real root + # collapses to ``), but it fails closed only by accident of + # shape — a root that did parse as a bare identifier would pass verbatim, and + # nothing here should depend on a path never looking like one. The `error=` + # field on the very same record is dropped too, but under this set's + # free-text rule above (it is a `GitError` string quoting git's own stderr), + # not this identifier-shape argument — same set, different rationale. + "repo", } ) # Journal fields whose value is a LIST of story keys (sprint unknown-keys). @@ -247,6 +318,10 @@ class TaskDiag: deferred_with_reason: bool spec_present: bool worktree_isolated: bool + # The discriminator a #705-class replay turns on. Without it a collided re-drive + # dumps as `rearmed=True, attempt=1, n_sessions=2` — byte-identical to a HEALTHY + # post-re-arm task. A counter, so it carries no customer content. + generation: int dw_count: int n_sessions: int sessions: SessionTally @@ -278,6 +353,13 @@ class RunDiag: paused: bool paused_stage: str | None paused_reason_present: bool + # Whether this run's code tree is a DIFFERENT directory from its project root. + # A presence flag in the `paused_reason_present` / `worktree_isolated` style, never + # the path — `_JOURNAL_DROP_FIELDS` drops `repo` as an absolute host path, and that + # drop otherwise removes the last trace of the split from a dump. The split layout + # is exactly the one in which the re-anchored baseline probes behave differently, + # so a bug report that cannot show it cannot be triaged. + repo_root_diverges: bool current_epic: int | None sweep_cycle: int sweeps_triggered: list[str] @@ -520,6 +602,7 @@ def _task_diag(task: StoryTask, pseudo: sanitize.Pseudonymizer, weight: float) - deferred_with_reason=bool(task.defer_reason), spec_present=bool(task.spec_file), worktree_isolated=bool(task.worktree_path), + generation=task.generation, dw_count=len(task.dw_ids), n_sessions=len(task.sessions), sessions=_session_tally([task]), @@ -608,16 +691,21 @@ def _scrub_entry( out["ts_offset"] = round(ts - first_ts, 3) kind = str(entry.get("kind", "?")) out["kind"] = kind if sanitize.looks_like_identifier(kind) else "" + # Read off the RAW kind, not the redacted spelling above: a kind that failed + # `looks_like_identifier` is not one of the three below anyway, and keying on the + # placeholder would silently unroute every entry in a dump that had one. + by_kind = _JOURNAL_KIND_ALIAS_FIELDS.get(kind, {}) for k, v in entry.items(): if k in ("ts", "kind"): continue + kind_ns = by_kind.get(k) if k in _JOURNAL_DROP_FIELDS: out[f"{k}_present"] = v is not None and v != "" elif k in _JOURNAL_KEYLIST_FIELDS and isinstance(v, list): ns = "story" if k == "keys" else "dw" out[k] = [pseudo.alias(x, ns=ns, epic=epic_by_key.get(str(x))) for x in v] - elif k in _JOURNAL_ALIAS_FIELDS: - ns = _JOURNAL_ALIAS_FIELDS[k] + elif kind_ns is not None or k in _JOURNAL_ALIAS_FIELDS: + ns = kind_ns or _JOURNAL_ALIAS_FIELDS[k] v = _alias_input(v, ns) epic = epic_by_key.get(str(v)) if ns == "story" else None out[k] = pseudo.alias(v, ns=ns, epic=epic) @@ -731,6 +819,7 @@ def collect_run(run_dir: Path, *, pseudo: sanitize.Pseudonymizer, cap: int) -> R paused=state.paused, paused_stage=state.paused_stage, paused_reason_present=state.paused_reason is not None, + repo_root_diverges=bool(state.repo_root) and Path(state.repo_root) != Path(state.project), current_epic=state.current_epic, sweep_cycle=state.sweep_cycle, sweeps_triggered=[ @@ -794,6 +883,7 @@ def _unreadable_run(run_dir: Path, err: Exception) -> RunDiag: paused=False, paused_stage=None, paused_reason_present=False, + repo_root_diverges=False, current_epic=None, sweep_cycle=0, sweeps_triggered=[], @@ -884,6 +974,13 @@ def render_markdown( for r in d.runs: out.append(f"## Run `{r.run_id}` ({r.run_type})") out.append(_fmt_kv("project", f"`{r.project_alias}`")) + # Rendered here and not only in `--json`: this is the report an operator + # produces by default and hands a maintainer, and the split layout is the + # one the re-anchored baseline probes behave differently in. A dump that + # cannot show it cannot be triaged — the whole warrant for the field. + out.append( + _fmt_kv("code root differs from project", "yes" if r.repo_root_diverges else "no") + ) out.append(_fmt_kv("started", r.started_date or "—")) out.append( _fmt_kv( @@ -918,16 +1015,21 @@ def render_markdown( out.append("### Tasks") if r.tasks: + # `gen` rides beside `att` because the pair is the discriminator: a + # #705-class replay and a healthy post-re-arm task agree on every other + # column here, so dropping it from the human report leaves the one field + # that separates them visible only under `--json`. out.append( - "| alias | epic | phase | att | rev | committed | spec | dw | sessions " + "| alias | epic | phase | att | gen | rev | committed | spec | dw | sessions " "| weighted | raw |" ) - out.append("|---|---|---|---|---|---|---|---|---|---|---|") + out.append("|---|---|---|---|---|---|---|---|---|---|---|---|") for t in r.tasks: out.append( - f"| `{t.alias}` | {t.epic} | {t.phase} | {t.attempt} | {t.review_cycle} " - f"| {t.committed} | {t.spec_present} | {t.dw_count} | {t.n_sessions} " - f"| {t.tokens.get('weighted', 0)} | {t.tokens.get('total', 0)} |" + f"| `{t.alias}` | {t.epic} | {t.phase} | {t.attempt} | {t.generation} " + f"| {t.review_cycle} | {t.committed} | {t.spec_present} | {t.dw_count} " + f"| {t.n_sessions} | {t.tokens.get('weighted', 0)} " + f"| {t.tokens.get('total', 0)} |" ) else: out.append("_no tasks._") diff --git a/src/bmad_loop/engine.py b/src/bmad_loop/engine.py index 76c3e626..0dee3579 100644 --- a/src/bmad_loop/engine.py +++ b/src/bmad_loop/engine.py @@ -360,13 +360,79 @@ def render(self) -> str: stalled and its work may be discarded.""" -def _session_task_id(story_key: str, part: str, seq: int) -> str: +def _session_task_id(story_key: str, part: str, seq: int, generation: int) -> str: """Single composition point for session task ids. Sanitize the whole composition, not the parts: two individually capped parts can still compose past a Windows filename segment limit, and ``safe_segment``'s digest suffix differs between the two orders. ``_resumable_session``'s resume match must - be byte-identical to what ``_run_session`` stored, so both MUST call this.""" - return safe_segment(f"{story_key}-{part}-{seq}") + be byte-identical to what ``_run_session`` stored, so both MUST call this. + + ``generation`` is ``StoryTask.generation``, bumped once per human re-arm + (``runs.rearm_escalation``). Re-arm resets ``attempt`` to 0 and the next + dispatch bumps it back to 1, so without this the re-minted id was BYTE-EQUAL + to a record the abandoned attempt already appended to ``task.sessions`` — and + ``_resumable_session``, which scans that append-only list, replayed the + abandoned attempt's verdict for the fresh one (#705). + + REQUIRED, with no default, for the reason ``verify_dev_exclude_relpaths``' ``root`` + is: an implicit ``generation=0`` is correct in every run that never re-armed — which + is nearly every test — and wrong only on the re-armed one, so a fourth mint site that + omitted it would look right everywhere it was exercised and silently re-open #705. + Requiring it turns OMISSION into a type error; it does not police a wrong value. + + The suffix is composed INSIDE the f-string, before ``safe_segment``, because + the whole-composition cap and digest contract above is what makes the id a + legal single segment; appending after sanitization could push it back over + ``MAX_SEGMENT``. It is emitted ONLY when ``generation > 0``, so every id an + existing run already wrote to disk stays byte-identical and a run resumed + across this upgrade still finds its ``tasks/`` directories.""" + gen = f"-g{generation}" if generation > 0 else "" + return safe_segment(f"{story_key}-{part}-{seq}{gen}") + + +# Longest single-line `reason` a notification channel carries — the returned string +# runs to AT MOST NOTICE_REASON_MAX + len(" […]"), the bound +# `test_notice_reason_caps_a_long_single_line_and_marks_the_trim` pins. At most, not +# exactly, in two ways: the slice is `.rstrip()`ed, so a cut landing on whitespace +# returns less; and `trimmed` is set for ANY multi-line reason regardless of length, so +# a short first line followed by evidence is marked far below the cap. Not a display +# preference: `gates.notify` normally writes one `[stamp] title: message` line into +# ATTENTION and hands the same string to a desktop toast, while a `Decision.reason` +# is routinely MULTI-line — `verify.verify_command_results_outcome` appends the +# captured output tail below the command line on purpose, because a repair session +# reads that tail as its feedback. Pasted through verbatim, one failing verify +# command spills a whole build log into ATTENTION as many un-prefixed lines (the +# file's own `[stamp] title:` grammar breaks with it) and into a notification bubble. +# +# "Normally" is exact, not hedging: `_notify_park` deliberately writes a newline-joined +# numbered action list through the same call, so one-line-per-notice is a property of +# the reason-carrying notices, NOT of the ATTENTION file. Any test asserting the shape +# over the whole file is really asserting that no park fired in that run. +NOTICE_REASON_MAX = 200 + + +def _notice_reason(reason: str) -> str: + """``reason`` as ONE bounded line, for a notification channel. + + Keeps the first non-empty line and caps it. Every producer front-loads the + classification there — ``verify command failed (rc=1): pytest -q``, ``spec + baseline … does not match orchestrator-recorded baseline …`` — and puts the + evidence underneath, so the first line is exactly the part a human deciding + whether to intervene needs. Nothing is lost: the untruncated reason is already + in the ``dev-decision`` journal entry every caller writes before notifying, + which is where a maintainer reads it. + + A trim is MARKED (``[…]``) rather than silent, so a reader can tell a reason + that ended there from one that was cut — a bare truncation reads as the whole + story and is how a "no changes since baseline" gets mistaken for the complete + diagnosis. + """ + first = next((line.strip() for line in reason.splitlines() if line.strip()), "") + trimmed = first != reason.strip() + if len(first) > NOTICE_REASON_MAX: + first = first[:NOTICE_REASON_MAX].rstrip() + trimmed = True + return f"{first} […]" if trimmed else first def _at_or_past(landed: str | None, target: str) -> bool: @@ -1585,7 +1651,7 @@ def _resumable_session(self, task: StoryTask) -> tuple[str, SessionResult] | Non role, seq = "review", task.review_cycle else: return None - task_id = _session_task_id(task.story_key, role, seq) + task_id = _session_task_id(task.story_key, role, seq, task.generation) for record in reversed(task.sessions): if record.task_id != task_id: continue @@ -1601,7 +1667,7 @@ def _resumable_session(self, task: StoryTask) -> tuple[str, SessionResult] | Non def _current_dev_session_index(self, task: StoryTask) -> int | None: """Index of the newest primary dev record for the current attempt.""" - task_id = _session_task_id(task.story_key, "dev", task.attempt) + task_id = _session_task_id(task.story_key, "dev", task.attempt, task.generation) for index in range(len(task.sessions) - 1, -1, -1): if task.sessions[index].task_id == task_id: return index @@ -2297,6 +2363,29 @@ def _dev_phase(self, task: StoryTask, resume_result: SessionResult | None = None return False return True if decision.action == Action.RETRY: + # Tell the operator WHY the attempt is being redone (#640d). RETRY + # was the only dev outcome that REJECTS an attempt without raising a + # notice (PROCEED raises none either, but it accepts the work rather + # than discarding it), and it is the arm that DISCARDS a completed + # implementation — the non-fixable leg below rolls the tree back to + # baseline. Without + # this the only record was the `dev-decision` journal line, so a + # run could burn its whole attempt budget throwing away finished + # work with nothing on the operator's phone but the eventual + # exhaustion notice. + # + # Placed at the TOP of the branch, ahead of the fixable/non-fixable + # split: it is the only point where both `decision.reason` and + # `task.attempt` are known-good for this decision, and it is before + # `_rollback_or_pause` can raise `RunPaused` and skip the notice for + # exactly the attempt whose loss most needs announcing. + gates.notify( + self.policy, + self.run_dir, + f"dev retry: {task.story_key} (attempt {task.attempt})", + _notice_reason(decision.reason) + or "dev attempt rejected with no reason recorded", + ) if outcome is not None and outcome.fixable: # work exists and the failure is concrete: keep the tree, # hand the failing output to a repair session @@ -4905,22 +4994,30 @@ def _harvest_gate_exclude(self, task: StoryTask) -> tuple[str, ...]: a session-authored ledger-only change as valid work even when the same attempt also records a frontmatter deferral. Standing down exposes more of the tree to the gate, which is the conservative direction. + + The relpath is derived against ``paths.repo_root``, the tree the gate + invokes git in, NOT ``paths.project`` (#716). The two are the same object + in every configuration but the `repo_root` override, and under that + override the ledger sits outside the code tree — where it cannot satisfy + proof-of-work, so ``()`` is the right answer rather than a pathspec git + would silently match nothing against. """ if not task.harvest_wrote_ledger or task.ledger_changed_before_harvest: return () paths = self.workspace.paths + root = paths.repo_root try: - rel = paths.deferred_work.resolve().relative_to(paths.project.resolve()) + rel = paths.deferred_work.resolve().relative_to(root.resolve()) except ValueError: - # The proof-of-work gate only sees the project tree, so an external - # ledger cannot satisfy it and needs no exclusion. + # The proof-of-work gate only sees the code tree, so a ledger outside it + # cannot satisfy the gate and needs no exclusion. return () except (OSError, RuntimeError): # ProjectPaths are normalized when loaded. If filesystem resolution - # nevertheless faults, keep a lexically in-project ledger excluded: + # nevertheless faults, keep a lexically in-tree ledger excluded: # uncertainty must not turn the engine's append into session proof. try: - rel = paths.deferred_work.relative_to(paths.project) + rel = paths.deferred_work.relative_to(root) except ValueError: return () return (rel.as_posix(),) @@ -5178,7 +5275,7 @@ def _run_session( ) -> SessionResult: # ``label`` names a non-standard session (a plugin-provided workflow) so # its task_id stays distinct from the role's own dev/review attempts. - task_id = _session_task_id(task.story_key, label if label else role, seq) + task_id = _session_task_id(task.story_key, label if label else role, seq, task.generation) adapter = self.adapters[role] cfg = self.policy.adapter.resolved(role) env = { diff --git a/src/bmad_loop/frontmatter.py b/src/bmad_loop/frontmatter.py index dde11107..cb4e3efd 100644 --- a/src/bmad_loop/frontmatter.py +++ b/src/bmad_loop/frontmatter.py @@ -63,18 +63,16 @@ def _split_frontmatter(text: str) -> tuple[str, str, str] | None: return None -def read_frontmatter(path: Path) -> dict[str, Any]: - if not path.is_file(): - return {} - try: - text = path.read_text(encoding="utf-8") - except UnicodeDecodeError: - # A non-UTF-8 file carries no readable frontmatter — degrade exactly like - # unparseable YAML below. Every status gate then reads status "" and - # returns a clean retry/repair outcome instead of crashing mid-verify - # (UnicodeDecodeError is a ValueError, so it slipped past callers' - # except-OSError guards). - return {} +def parse_frontmatter(text: str) -> dict[str, Any]: + """The frontmatter mapping ``text`` carries, or ``{}`` when it carries none. + + Split out of `read_frontmatter` so a spec that never touches the filesystem — a + blob read back out of git with `verify.file_bytes_at_revision`, say — parses + through the SAME reader as a live file rather than through a second copy free to + drift from it. Degrades rather than raising on every shape: no frontmatter block, + unparseable YAML, or a document that is not a mapping. Callers tell "absent" from + "present but empty" by asking the `*_of` readers, never by inspecting this. + """ split = _split_frontmatter(text) if split is None: return {} @@ -85,6 +83,21 @@ def read_frontmatter(path: Path) -> dict[str, Any]: return doc if isinstance(doc, dict) else {} +def read_frontmatter(path: Path) -> dict[str, Any]: + if not path.is_file(): + return {} + try: + text = path.read_text(encoding="utf-8") + except UnicodeDecodeError: + # A non-UTF-8 file carries no readable frontmatter — degrade exactly like the + # unparseable-YAML arm in `parse_frontmatter` above. Every status gate then + # reads status "" and returns a clean retry/repair outcome instead of crashing + # mid-verify (UnicodeDecodeError is a ValueError, so it slipped past callers' + # except-OSError guards). + return {} + return parse_frontmatter(text) + + def status_of(fm: dict[str, Any]) -> str: """Normalized spec status from a frontmatter dict: stripped + lowercased. @@ -138,6 +151,62 @@ def operator_actions_of(fm: dict[str, Any]) -> tuple[str, ...]: return tuple(dict.fromkeys(a for a in items if a)) +# The two frontmatter keys a bmad-build-auto spec can carry a dev baseline under, +# in precedence order. `baseline_revision` is what the skill's step-03 actually +# stamps; `baseline_commit` is the legacy spelling (the name the orchestrator's +# synthesized result.json uses) kept readable for specs written before the rename. +_BASELINE_KEYS = ("baseline_revision", "baseline_commit") + + +def auto_dev_baseline_of(fm: dict[str, Any]) -> str: + """The dev baseline a bmad-build-auto spec CLAIMS: the first non-empty value + among ``baseline_revision`` then ``baseline_commit``, stripped; ``""`` when the + spec claims neither. + + Deliberately not the bare ``baseline_of`` the siblings' naming would suggest. + ``status_of`` and ``operator_actions_of`` are field-generic — they read one key + and normalize it — whereas this precedence belongs to the bmad-build-auto + contract specifically: the skill stamps ``baseline_revision``, the orchestrator's + own result.json says ``baseline_commit``, and a spec re-armed by + ``runs.rearm_escalation`` can carry both. A bare ``baseline_of`` would read as + universal when it is not (#716). + + ``baseline_revision`` WINS whenever it is non-empty, even against a + ``baseline_commit`` that would have matched. Both consumers — the dev + devcontract's synthesized result and verify's baseline-match gate — read + through here so the two cannot drift, and the two byte-identical + ``fm.get("baseline_commit", fm.get("baseline_revision", ""))`` expressions + this replaces had the precedence the other way round. That flip is deliberate + and tightening: the legacy key is a leftover the re-arm never removes, so + ranking it first let a stale sha silently outrank the fresh value the skill + had just written, killing the gate on an attempt that did everything right. + + An EMPTY legacy key is skipped rather than returned. ``dict.get``'s default + only fires on a MISSING key, so ``baseline_commit: ''`` used to be selected and + yield ``""`` — which every consumer reads as "no claim" and which therefore + disabled the baseline-match gate outright. + + A YAML-null value (a bare ``baseline_commit:`` line, or ``: null``) is treated + as absent for the same reason ``status_of`` guards it: ``str(None)`` is the + token ``"None"``, which is not a sha but IS non-empty, so it would flow into + the gate as a claim and fail an attempt that never made one (#358). A YAML + BOOLEAN is the same trap class and is skipped with it: PyYAML resolves ``no``, + ``off`` and ``false`` to ``False`` (``yes``/``on``/``true`` to ``True``), and + ``str(False)`` is the token ``"False"`` — again not a sha, again non-empty. + Worse than null: because the truthiness test is on the STRINGIFIED value, a + bool on ``baseline_revision`` outranks and SHADOWS a correct ``baseline_commit`` + sitting right beside it, refusing an attempt whose legacy claim was good. + """ + for key in _BASELINE_KEYS: + raw = fm.get(key) + if raw is None or isinstance(raw, bool): + continue + value = str(raw).strip() + if value: + return value + return "" + + class FrontmatterWriteError(Exception): """A frontmatter block carries a key the reader can see but no minimal line edit can safely rewrite. diff --git a/src/bmad_loop/model.py b/src/bmad_loop/model.py index 6fde30b3..71d2308c 100644 --- a/src/bmad_loop/model.py +++ b/src/bmad_loop/model.py @@ -188,6 +188,17 @@ class StoryTask: # rather than burning another cycle. Reset to 0 by runs.rearm_escalation so a # human-resolved re-drive gets a fresh damping budget. Survives the round-trip. followup_reviews_spent: int = 0 + # How many times a human re-arm (`runs.rearm_escalation`) has re-opened this + # task. Re-arm resets `attempt` to 0 and the next dispatch bumps it back to 1, + # so without a discriminator the re-minted session task_id is byte-equal to a + # record the ABANDONED attempt already appended to the append-only `sessions` + # list — and `Engine._resumable_session`, which matches on that id, replays the + # abandoned attempt's verdict for the fresh one (#705). Feeds + # `engine._session_task_id`, which emits the suffix only above zero, so every + # id already on disk stays byte-identical across the upgrade. `task.sessions` + # is deliberately NOT cleared at re-arm: the run-dir audit trail it indexes is + # read by a second resolve cycle. + generation: int = 0 # set from the bmad-build-auto session's `followup_review_recommended` # frontmatter (PR #2505): when True and review.trigger = "recommended", the # orchestrator runs a follow-up review pass (bmad-build-auto re-invoked on the @@ -390,6 +401,7 @@ def to_dict(self) -> dict[str, Any]: "attempt": self.attempt, "review_cycle": self.review_cycle, "followup_reviews_spent": self.followup_reviews_spent, + "generation": self.generation, "followup_review_recommended": self.followup_review_recommended, "baseline_commit": self.baseline_commit, "baseline_untracked": self.baseline_untracked, @@ -470,6 +482,7 @@ def from_dict(cls, d: dict[str, Any]) -> "StoryTask": attempt=int(d.get("attempt", 0)), review_cycle=int(d.get("review_cycle", 0)), followup_reviews_spent=int(d.get("followup_reviews_spent", 0)), + generation=int(d.get("generation", 0)), followup_review_recommended=bool(d.get("followup_review_recommended", False)), baseline_commit=d.get("baseline_commit"), baseline_untracked=( @@ -540,6 +553,15 @@ class RunState: run_id: str project: str started_at: str + # The git root this run's code work happens in — `paths.repo_root`, which is + # `paths.project` unless `_bmad/bmm/config.yaml` sets a `repo_root:` override. + # Persisted because `runs.rearm_escalation` runs OUT OF PROCESS from the engine + # and had only `project` to reach for, so it advanced the attempt baseline by + # reading HEAD of a repo the proof-of-work gate never measures. Empty means a + # state.json written before this field existed; `code_root` then falls back to + # `project`, which is exactly the pre-upgrade behavior and the correct answer + # for every run without the override. + repo_root: str = "" policy_snapshot: dict[str, Any] = field(default_factory=dict) # SECONDARY copy of the host-exec baseline (#498) — runsetup.config_digest over # the agent-writable config that reaches HOST code execution: verify commands, @@ -625,6 +647,17 @@ class RunState: def paused(self) -> bool: return self.paused_reason is not None + @property + def code_root(self) -> Path: + """The tree git runs against for this run — ``repo_root`` when the run + recorded one, else ``project``. + + The single reader of the pair, so an out-of-process consumer + (``runs.rearm_escalation``) cannot pick the wrong one, and a pre-upgrade + state.json (empty ``repo_root``) degrades to precisely what it did before + rather than to a path that does not exist.""" + return Path(self.repo_root or self.project) + def handled_keys(self) -> set[str]: """Story keys this run already drove to a terminal phase.""" return {k for k, t in self.tasks.items() if t.terminal} @@ -660,6 +693,7 @@ def to_dict(self) -> dict[str, Any]: return { "run_id": self.run_id, "project": self.project, + "repo_root": self.repo_root, "started_at": self.started_at, "policy_snapshot": self.policy_snapshot, "trusted_config_digest": self.trusted_config_digest, @@ -690,6 +724,7 @@ def from_dict(cls, d: dict[str, Any]) -> "RunState": return cls( run_id=d["run_id"], project=d["project"], + repo_root=str(d.get("repo_root", "")), started_at=d["started_at"], policy_snapshot=d.get("policy_snapshot", {}), trusted_config_digest=str(d.get("trusted_config_digest", "")), diff --git a/src/bmad_loop/runs.py b/src/bmad_loop/runs.py index a6d2fa5f..c4b7f1b8 100644 --- a/src/bmad_loop/runs.py +++ b/src/bmad_loop/runs.py @@ -17,9 +17,11 @@ import time from dataclasses import dataclass from pathlib import Path +from typing import Any, Literal from . import devcontract, envvars, verify from .adapters.multiplexer import MultiplexerError, get_multiplexer, mux_usable +from .frontmatter import auto_dev_baseline_of, parse_frontmatter, status_of from .journal import STATE_FILE, VERIFY_DIR, Journal, load_state, save_state from .model import PAUSE_ESCALATION, Phase, RunState, StoryTask from .platform_util import ( @@ -27,6 +29,7 @@ UnconfinedWriteError, _mkstemp_beside, atomic_replace, + atomic_write_bytes_confined, atomic_write_text_confined, create_exclusive_confined, has_parent_ref, @@ -2172,6 +2175,285 @@ def validate_restore_latch( return None +def _task_spec_path(task: StoryTask, state: RunState) -> Path: + """The recorded spec path, re-anchored on the tree it was persisted relative to. + + `StoryTask._serialized_worktree_path` (`model.py`) persists a worktree-local spec + RELATIVE to the mounted worktree root, and `from_dict` reads it back raw. Resolving + that against the process cwd is not merely unreachable — it is actively wrong: + `bmad-loop resolve` runs from the project root, where the MAIN CHECKOUT carries the + same `_bmad-output/specs/...` layout, so a bare `Path(task.spec_file)` names the main + checkout's copy of the story spec. `is_file()` then answers True, `confine_root` + accepts it (it genuinely is under `project`), and the status flip and the baseline + re-stamp both land on a file the run never used while the worktree's real spec is + left on the escalated attempt's sha. + + Absolute paths pass through: a spec outside the worktree is persisted verbatim. + """ + raw = Path(task.spec_file or "") + if raw.is_absolute(): + return raw + return _task_spec_root(task, state) / raw + + +def _task_spec_root(task: StoryTask, state: RunState) -> Path: + """The tree a relative `task.spec_file` is anchored on — and confined to. + + One definition backs both halves because they must not disagree: the root + `_task_spec_path` resolves against and the `confine_root` the writers validate the + result against are the same claim about which tree owns this spec. Passing + `state.project` while resolving against the worktree does not REFUSE the mismatch — + `set_frontmatter_status`, `devcontract.strip_auto_run_result` and + `verify.set_frontmatter_field` all answer an out-of-root path by silently dropping + to the plain no-follow write, losing the confined arm's O_NOFOLLOW walk of the + parent components (#593) with no signal at all. + + Worktrees normally resolve under `/.bmad-loop/runs/...`, so the confined + arm is taken by construction rather than by luck — no policy or env var can + relocate them. The one escape is that `workspace.open_unit_workspace` stores a + `.resolve()`d path: a symlinked `.bmad-loop`, `runs` or `worktrees` lands the spec + outside `project`, and before this anchor moved that silently degraded all three + writes. + """ + return Path(task.worktree_path or state.project) + + +def _spec_is_shared_with_the_redrive(state: RunState, task: StoryTask) -> bool: + """True when an isolated unit's recorded spec lives outside BOTH checkouts, so the + re-arm's status flip survives the worktree's disposal and the re-drive reads it. + + The case: artifact dirs configured outside the project tree. `ProjectPaths.rebased` + leaves those exactly where they are ("configured outside the project tree; doesn't + move") — they are SHARED across checkouts, not per-worktree — so the spec the dev + session reported resolves to one file that every worktree sees. The re-drive reads it + back through `verify.resolve_spec_path`, whose absolute branch passes the value + through untouched, and `engine._dispatched_spec_for_attempt` then accepts it because + the rebased `implementation_artifacts` is still that same external directory. + + Both roots are load-bearing, and neither implies the other: + + - INSIDE the worktree — the file the fresh mount destroys. Unreachable. + - inside the PROJECT but outside the worktree — the main checkout's copy. The write + lands, but the re-drive cannot use it: under isolation `workspace.paths` is rebased + onto the fresh worktree, so `verify.spec_within_roots` measures the main + checkout's path against worktree-local roots and rejects it. Unreachable, and this + is the one shape the worktree test alone would wrongly exempt. + - outside both — the shared artifact dir above. Reachable. + + (The two are not nested: worktrees normally sit under `/.bmad-loop/runs/`, + but `workspace.open_unit_workspace` stores a `.resolve()`d path, so a symlinked + `.bmad-loop` puts the mount outside the project.) + + The recorded spelling opens the question but does not answer it. + `StoryTask._serialized_worktree_path` persists a spec RELATIVE whenever it sits + under the mounted worktree and verbatim (absolute) otherwise, so an absolute value + on a task that HAS a worktree is the only shape that can be shared — but that + relativize is a LEXICAL `relative_to` against the same `worktree_path` read here, so + all an absolute value proves is that the two spellings did not share a prefix. A + spec reported through a symlink or a `..` segment sits inside the worktree and is + persisted absolute all the same, and answering "shared" for it would suppress the + warning on a spec that really is destroyed with the worktree. + + So containment is decided on the CANONICAL paths, and a host that cannot canonicalize + one of them answers "not shared". That degrade is the safe direction and the reason + this does not use `resolve_or_lexical`: its fallback is `absolute()`, which does not + fold `..`, so a spec spelled through either checkout would come back looking external + and go silent — trading a wrong warning for no warning at all.""" + raw = Path(task.spec_file or "") + if not task.worktree_path or not raw.is_absolute(): + return False + try: + # the house pair — `resolve()` raises RuntimeError, not OSError, for a symlink + # loop on the 3.11/3.12 floor + real = raw.resolve() + return not real.is_relative_to(Path(task.worktree_path).resolve()) and not ( + real.is_relative_to(Path(state.project).resolve()) + ) + except (OSError, RuntimeError): + return False + + +def _redrive_base_ref(state: RunState, task: StoryTask) -> str: + """The ref whose committed tree the re-drive will actually read this unit's spec + from: the run's PINNED `target_branch` for an isolated unit, ``HEAD`` otherwise. + + Not `HEAD` in both cases, because the isolated re-drive never reads the main + checkout's working ref. `engine._finish_inflight` discards the escalated worktree + and its branch and `_run_story` mounts a replacement, and + `workspace.open_unit_workspace` cuts that fresh branch from the `base` it is handed + — `worktree_flow.run_isolated` passes `state.target_branch`, pinned once at run + start so resume keeps targeting the same branch. An operator who checks out another + branch in the main checkout while the escalation is paused therefore moves `HEAD` + off the tree the re-drive reads, in either direction: a correction committed on the + now-current branch is invisible to the re-drive, and one committed on the target + branch is invisible to `HEAD`. + + That mattered once `rearm-spec-write-unreachable` began holding the resume + (`rearm_holds_the_resume`): reading the wrong ref does not merely mis-word a + warning, it either resumes a re-drive that re-wedges on the target branch's + terminal status, or holds a resume whose work is already committed where the + re-drive will find it. + + The two guards are the same proxy the caller already uses. `task.worktree_path` is + how this file recognizes an isolated unit at all (`_task_spec_root`, + `_spec_is_shared_with_the_redrive`) — every isolated escalation carries a mounted + one. An empty `target_branch` beside it is a MISSING value, not a divergent one: + `ensure_target_branch` pins the field before any worktree mounts, so only a + state.json predating it can reach here, and that shape degrades to exactly the ref + it read before — the same migration `restamp_code_root` gives an unrecorded root. + Answering ``""`` instead would hold the resume on a per-configuration constant, the + failure the record's narrowing exists to avoid. + """ + if task.worktree_path and state.target_branch: + return state.target_branch + return "HEAD" + + +def _restore_rearmed_spec( + spec_path: Path, original: bytes | None, task: StoryTask, state: RunState +) -> None: + """Put back the bytes a re-arm FOUND on the spec, for the aborts that can fire after + a write has already landed. + + `rearm_escalation` holds an invariant its own refusals depend on: an aborted re-arm + leaves the spec byte-identical, so the escalation stays armed and the human can fix + the file and re-run resolve. TWO of its four refusals earn that by SEQUENCING alone — + the flip's read-back check and the `FrontmatterWriteError` arm both raise before + `devcontract.strip_auto_run_result` runs, which is why that strip is deliberately + ordered after them, and `set_frontmatter_status` decides it cannot move a `status:` + before it writes anything. The other two cannot be sequenced out of the hazard, and + both call this: + + * The baseline re-stamp needs `task.baseline_commit` from the advance, and the + advance must itself run after the spec block (a just-cleared stories sentinel would + otherwise be captured into `baseline_untracked` as phantom pre-existing residue). + * The `(OSError, UnicodeDecodeError)` arm spans BOTH spec helpers, and the strip is + the later one — a fault raised inside it is raised after the flip published. + + By the time either can fail, the status flip has landed and `save_state` has not — so + the abort would otherwise leave the run's task ESCALATED against a spec already + flipped to the re-drive's status and (for the re-stamp) stripped of the terminal + `## Auto Run Result` the next resolve session reads as its context. That is exactly + the "one edit nothing else records" the sequencing exists to prevent. + + Writes only what it can prove it changed. `original` is `None` when the spec was + unreadable before the first write (there is then nothing to restore, and nothing + could have been written either), and a spec that is gone or unreadable NOW is not a + state this undo can improve — recreating a file another process removed would fight + a concurrent actor rather than restore this function's own edit. Bytes equal to + `original` mean nothing landed, so nothing is rewritten and the mtime is left alone. + + Byte-verbatim and CONFINED, matching the writes it undoes: `atomic_write_text_confined` + would re-encode and translate newlines, so a CRLF spec would come back subtly + different from the file this re-arm found, and an unconfined write would drop the + `O_NOFOLLOW` walk of the parent components (#593) that every other write to this path + takes. A restore that itself fails RAISES rather than degrading — the spec is then + half-written and only the operator can settle it, which is the loudest thing this can + be. `UnconfinedWriteError` is an `OSError`, so the one arm covers both. + """ + if original is None: + return + try: + if spec_path.read_bytes() == original: + return + except OSError: + return + try: + atomic_write_bytes_confined(spec_path, original, confine_root=_task_spec_root(task, state)) + except OSError as e: + raise RearmError( + f"cannot restore {spec_path} after a failed re-arm " + f"({e.__class__.__name__}: {e}) — the spec carries this re-arm's status flip " + "and has lost its `## Auto Run Result` section, while the story is still " + "escalated; restore the spec from git, then re-run resolve" + ) from e + + +def _committed_spec_status(state: RunState, task: StoryTask) -> str: + """The spec's status as COMMITTED in the tree the re-drive reads, or ``""`` when + unprovable. + + Under isolation the re-drive reads the committed spec and never a working-tree + write (see `rearm_escalation`'s note on `rearm-spec-write-unreachable`), so this is + the value that decides whether the operator still has anything to do. Anchored on + `state.code_root` — the same tree the baseline advance reads — at the ref + `_redrive_base_ref` names, which is the run's pinned `target_branch` for an isolated + unit rather than that tree's current `HEAD`. + + Degrades to ``""`` on every uncertainty: a spec recorded absolute (nothing names + its position in the tree), an absent or non-blob path at that ref, a non-UTF-8 blob, + or any `GitError` — which includes the project simply not being a repository, and a + `target_branch` the code root no longer carries. ``""`` never equals a target + status, so the caller's record still fires. Suppression therefore requires PROOF + that the work is already done, and the non-repo case stays non-fatal, as the story's + Boundaries require. + + The absolute arm is narrower than it looks: the caller has already answered the one + absolute shape whose write the re-drive DOES read — the shared external spec — with + `_spec_is_shared_with_the_redrive`. What still reaches here is an absolute spelling + of a path inside one of the two checkouts, which is genuinely unreachable and + genuinely unprovable, so degrading it to a warning is the right answer, not a gap. + """ + raw = Path(task.spec_file or "") + if not task.spec_file or raw.is_absolute(): + return "" + try: + blob = verify.file_bytes_at_revision( + state.code_root, _redrive_base_ref(state, task), raw.as_posix() + ) + except verify.GitError: + return "" + if blob is None: + return "" + try: + text = blob.decode("utf-8") + except UnicodeDecodeError: + return "" + return status_of(parse_frontmatter(text)) + + +def restamp_code_root(run_dir: Path, repo_root: Path) -> str | None: + """Re-point a paused run's persisted code-root mirror at `repo_root` — the tree + the caller is about to act in — and return the warning an operator must see when + that MOVED a root the run had recorded (`None` when it already agreed, or when the + run predates the field). + + Exists because `rearm_escalation` reads that mirror OUT OF PROCESS + (`RunState.code_root`) and has no `ProjectPaths` to consult, while `repo_root:` is + re-read from config.yaml by every process that arms an engine. `cli._resume_paused_run` + folds the same re-stamp into the one `save_state` that also carries the policy + snapshot and the config digest — this is the seam for the surfaces that re-arm + BEFORE they resume (`cli.cmd_resolve`, `TuiApp._do_rearm`), where that write lands + too late to aim the re-arm. + + The compare is exact and uncanonicalized, matching resume's: both sides are + `str(paths.repo_root)` off `bmadconfig.load_paths`, which resolves every member or + raises, so they are spelled the same way whenever they name the same tree. An empty + recorded root is a MISSING value, not a divergent one — a state.json written before + the field existed — so it is migrated silently and reported as no move. + + The message names neither tree, like resume's: what an operator needs is that the + run has changed repositories, and the paths are the half that would put an + attacker-controlled string on their terminal. + """ + state = load_state(run_dir) + new = str(repo_root) + if state.repo_root == new: + return None + moved = bool(state.repo_root) + state.repo_root = new + save_state(run_dir, state) + if not moved: + return None + return ( + f"run {run_dir.name}: the code root in _bmad/bmm/config.yaml has changed since " + "this run started — the re-drive works in the tree configured now, while the " + "baselines, preserve refs and branches this run already recorded name objects " + "in the previous one. Restore the previous `repo_root:` value if you did not " + "intend the move." + ) + + def rearm_escalation( run_dir: Path, story_key: str | None = None, *, restore_patch: str | None = None ) -> str: @@ -2180,14 +2462,31 @@ def rearm_escalation( Flips the escalated task out of its terminal ESCALATED phase back to PENDING — which makes `_finish_inflight` reset the tree to the story's baseline and re-run it (clean rebuild) against the now-corrected frozen - spec. The baseline itself is advanced to the project's current HEAD (and - the untracked snapshot refreshed) so commits and files the resolve session + spec. The baseline itself is advanced to the CODE TREE's current HEAD + (`state.code_root`, which is `paths.repo_root` — the tree the dev writer + stamps from and the proof-of-work gate measures, and the same directory as + `state.project` in every configuration without a `repo_root:` override) and + the untracked snapshot refreshed, so commits and files the resolve session produced count as the rebuild's starting point, not as attempt debris to roll back. Strips the escalated attempt's stale `## Auto Run Result` section so the re-drive cannot read as terminal from its first save, and sets the spec's frontmatter status so step-01 routes to the right stage. Does NOT clear the pause; the caller resumes the run separately. + Two consequences of the reset are load-bearing and easy to undo by accident: + + - `task.generation` is bumped, because `attempt` returning to 0 would + otherwise let the re-drive re-mint a session id byte-equal to one the + abandoned attempt already recorded (#705). `task.sessions` is deliberately + NOT cleared — a second resolve cycle reads that run-dir audit trail — so + the id is what has to change. + - The spec's `baseline_revision` is re-stamped on BOTH legs, and only when the + advance above actually RAN — `advanced` records that both git reads succeeded, + not that HEAD changed, so a resolve session that committed nothing still + re-stamps (with the same sha, harmlessly). What it will not do is re-stamp + after a FAILED advance (see the block that does it for why each half of that + is the way it is). + Two re-drive modes, selected by `restore_patch`: - **from-scratch** (default, ``restore_patch=None``): status → ``ready-for-dev`` @@ -2250,6 +2549,14 @@ def rearm_escalation( # engine._finish_inflight): a clean re-attempt against the corrected spec. task.phase = Phase.PENDING task.attempt = 0 + # A new generation of this task. `attempt` going back to 0 (and the next + # dispatch bumping it to 1) would otherwise re-mint a session task_id + # byte-equal to one the abandoned attempt already recorded, and + # `Engine._resumable_session` — matching that id over the append-only + # `task.sessions`, which this function deliberately does NOT clear — would + # replay the abandoned verdict for the fresh attempt (#705). Bumped BEFORE any + # dispatch, so the id is unique from the re-drive's first session onward. + task.generation += 1 task.review_cycle = 0 task.followup_reviews_spent = 0 # human-resolved re-drive gets a fresh damping budget task.defer_reason = None @@ -2259,8 +2566,12 @@ def rearm_escalation( # a prior restore attempt the human then chose to redo from scratch. task.restore_patch = restore_patch + # The bytes this re-arm found on the spec, for `_restore_rearmed_spec`. Declared out + # here because the baseline re-stamp that consumes it sits in a SECOND + # `if task.spec_file:` block, past the advance it depends on. + spec_before: bytes | None = None if task.spec_file: - spec_path = Path(task.spec_file) + spec_path = _task_spec_path(task, state) # Stories mode only: a fixed-slug pre-planning-halt sentinel # (`-unresolved.md` / `-ambiguous.md`) is cleared by deletion, not a # status flip. Clear it ONLY when the run recorded this task AS a sentinel at @@ -2279,20 +2590,217 @@ def rearm_escalation( task.spec_file = None task.sentinel_kind = "" # verdict discharged; the re-dispatch is clean else: + # A WORKTREE-LOCAL spec's writes below land in the unit's worktree + # (`_task_spec_path`) — which the re-drive destroys before reading anything. + # A re-armed task (phase PENDING, `defer_reason` cleared, and no resumable + # session because `generation` was just bumped) falls to + # `engine._finish_inflight`'s final arm, which calls `discard_worktree` and + # lets `_run_story` mount a fresh one. The re-driven session then resolves + # its spec through `verify.resolve_spec_path(task.spec_file, + # workspace.paths)` (`engine._dispatched_spec_for_attempt`), and under + # isolation `workspace.paths` is rebased onto that FRESH worktree, which + # checks out TRACKED files only. So the re-drive reads the COMMITTED spec. + # + # No working-tree write reaches it — not this one, and not a write to the + # main checkout either: the fresh worktree comes from git rather than from a + # copy of that tree, and `seed_adapter_defaults` seeds adapter config files, + # not the output folder. The channel that DOES work is the human committing + # the corrected spec from the resolve session, which runs with `cwd=project`. + # The writes below are kept (they are correct for the in-place case, and + # harmless here), but the operator is told — a flip that cannot land is + # exactly the silent re-wedge #640(b) exists to end. + # + # "Worktree-local" is the load-bearing qualifier, and isolation does not + # imply it: an artifact dir configured OUTSIDE the project tree is shared + # across checkouts by `ProjectPaths.rebased`, so a spec that landed there is + # one file the fresh worktree reads through the very absolute path this + # writes to. `_spec_is_shared_with_the_redrive` carves out that case, and only + # that one: the main checkout's copy is outside the worktree too, and stays + # unreachable because the re-drive measures it against worktree-local roots. + # Route /bmad-build-auto via the spec's frontmatter status (decision + # table): patch-restore -> in-review -> step-04 (resume review on + # the restored diff); from-scratch -> ready-for-dev -> step-03 + # (re-implement). Independent of the resolve agent having set it. + target_status = "in-review" if restore_patch else "ready-for-dev" + # Whether the writes below are the copy the re-driven session actually + # reads. Hoisted out of the record's condition because TWO decisions turn on + # it, and only one of them used to: the warning below, and the flip's + # REFUSAL one screen down, which was gated on `spec_path.is_file()` alone. + # Under isolation that readable file is the doomed worktree copy, so the + # refusal demanded a repair to the one file the re-drive destroys before + # reading anything — and demanded it even when `_committed_spec_status` had + # already proven the committed spec carries the status the re-drive routes + # on. See `_spec_is_shared_with_the_redrive` for why an isolated unit's spec + # is nevertheless reachable when it sits in an artifact dir configured + # outside the project tree. + write_reaches_the_redrive = not task.worktree_path or _spec_is_shared_with_the_redrive( + state, task + ) + # Narrowed to the case an operator can ACT on. Every isolated escalation + # carries a mounted `worktree_path` — `worktree_flow.escalate_unit` never + # clears it, and `keep_branch_and_escalate` deliberately leaves the worktree + # up — so gating on that alone fired this warning on 100% of re-arms under + # `isolation = "worktree"`: a per-configuration constant, not an event, and + # the same "trains the operator to scroll past the meaningful one" failure + # that the `flipped` read-back below and the `overwritten != old_baseline` + # guard were each narrowed to avoid. The remedy it prints ("commit the + # corrected spec") is already a no-op once the committed spec carries the + # target status, which is precisely when the re-drive reads what it needs. + # Suppression requires PROOF: an unreadable blob, a non-repo project, or any + # git fault leaves `""` and the record fires. The proof is read at + # `_redrive_base_ref`, NOT at the code root's current `HEAD` — the two part + # company as soon as the operator checks out another branch while the + # escalation is paused, and this record now holds the resume. + # + # The branch rides along because the remedy needs it: on exactly the shape + # the ref fix rescues, "commit the corrected spec" without a branch sends + # the operator to commit again on the branch the re-drive does not read, and + # the next re-arm prints the same sentence. Empty for the migrated shape + # `_redrive_base_ref` degrades to `HEAD` for, and the notice drops the + # clause rather than naming a ref it cannot source. + # + # Spelled `target_branch` and NOT `base`, because `diagnostics` routes the + # scrub by field NAME: `target_branch` is already in `_JOURNAL_ALIAS_FIELDS` + # under the `branch` namespace (with no journal producer until now), while + # any new spelling falls through to `scrub_json`, which waves an + # identifier-shaped branch name through verbatim. In a normal run + # `ensure_target_branch` has already journalled the same string as `branch`, + # so the egress backstop would repair it and disclose a `backstop_repairs` + # routing gap; in a truncated journal missing that event nothing would catch + # it and the branch would ship in a shareable bundle. `target` — the + # spelling the merge kinds use — is NOT available: `board-advance-*` puts a + # sprint STATUS in that same field, and routing is by name, so aliasing it + # to `branch` would pseudonymize statuses as branches. + if ( + not write_reaches_the_redrive + and _committed_spec_status(state, task) != target_status + ): + journal.append( + "rearm-spec-write-unreachable", + story_key=key, + spec_file=str(spec_path), + status=target_status, + target_branch=state.target_branch, + ) + # Captured immediately before the FIRST write, so an abort further down can + # put the spec back exactly as found. Unreadable degrades to `None`: the + # writes below answer such a path with `False` rather than an exception, so + # there would be nothing to undo either. + try: + spec_before = spec_path.read_bytes() + except OSError: + spec_before = None try: - # Route /bmad-build-auto via the spec's frontmatter status (decision - # table): patch-restore -> in-review -> step-04 (resume review on - # the restored diff); from-scratch -> ready-for-dev -> step-03 - # (re-implement). Independent of the resolve agent having set it. - target_status = "in-review" if restore_patch else "ready-for-dev" - verify.set_frontmatter_status( - spec_path, target_status, confine_root=Path(state.project) + flipped = verify.set_frontmatter_status( + spec_path, target_status, confine_root=_task_spec_root(task, state) ) + # `set_frontmatter_status` answers "nothing to change" with `False` + # for FOUR causes, not three — its own docstring lists them: no file, + # no frontmatter block, no top-level `status:`, and ALREADY AT THE + # TARGET (`_edit_frontmatter_block` returns None on + # `original[key] == value`). Only the first three are failures. The + # fourth is an ordinary, fully-successful re-arm: a second resolve + # cycle on an already-flipped spec, or the documented + # `resolve --no-interactive` flow where a human fixed the spec + # themselves — the case the comment above calls "Independent of the + # resolve agent having set it". Journalling it fired the operator + # warning ("could not be re-opened … may re-wedge on it") on a spec + # that was byte-identical and CORRECT, which is the "trains the + # operator to scroll past the meaningful one" failure the re-stamp's + # `overwritten != old_baseline` guard exists to prevent one screen + # below. Read the status back to tell the two apart: `read_frontmatter` + # degrades a missing/unreadable/unparseable spec to `{}` and `status_of` + # then answers `""`, so all three real failures still record. + if not flipped and verify.status_of(verify.read_frontmatter(spec_path)) != ( + target_status + ): + # Discarding that return is how the flip + # became a SILENT no-op: the re-drive is dispatched anyway, step-01 + # reads the unchanged terminal status, routes the session to "ingest + # as context, do not resume", and the story re-wedges with nothing on + # the record. The `FrontmatterWriteError` arm below covers only the + # shapes that RAISE; this covers the ones that lie quietly. + # `refused` is written ON the record because ONE kind now covers + # two outcomes and the operator surfaces must tell them apart — + # they read the journal OUT OF PROCESS, with neither the task nor + # the tree to re-derive it from. Printing the refusal's remedy + # ("add a top-level `status:`") for a re-arm that COMPLETED sends + # the human to repair a file nothing will read. + refused = spec_path.is_file() and write_reaches_the_redrive + journal.append( + "rearm-spec-flip-skipped", + story_key=key, + spec_file=str(spec_path), + status=target_status, + refused=refused, + ) + # ...and then ABORT — but only for a spec that IS a readable file + # here AND is the copy the re-drive reads. The first half is the same + # `is_file` split the baseline re-stamp below already draws, and for + # the same reason. On THAT shape the failure is + # a REPAIR that did not land on the very file the re-drive reads, so it + # aborts for the same reason the `FrontmatterWriteError` arm does: + # journalling alone left the two default surfaces telling the operator + # "re-armed " and resuming in the same gesture, so the record's + # own imperative was already unactionable when it rendered — while + # step-01's contract for what reaches here is not a maybe. A spec with + # no `status:` HALTs blocked on `unrecognized status in existing story + # file`; one still carrying the escalated attempt's terminal status + # routes to "ingest as context, do not resume". Either way the re-drive + # re-wedges and the escalation is burned. Refusing keeps it armed: nothing + # is persisted yet (`save_state` runs below), the spec is byte-identical + # (the `## Auto Run Result` strip is deliberately sequenced AFTER this + # check so an abort leaves nothing half-done), and the human fixes the + # frontmatter and re-runs resolve. + # + # A spec that is NOT a file from here keeps warn-and-continue, because + # there the flip's failure says nothing about what the re-drive will + # read: `spec_file` is persisted RELATIVE to a worktree, an isolated + # task's worktree may already be gone, and the re-drive mounts a fresh + # one and reads the COMMITTED spec regardless. Aborting on it would + # refuse the re-arms that the `rearm-baseline-restamp-skipped` and + # `rearm-spec-write-unreachable` records exist to report rather than + # prevent — an unreadable path is an observation, and observations + # degrade. + # + # A worktree-local spec that IS readable takes that same lane, for a + # sharper version of the same reason: `_task_spec_root` anchors this + # write on the mounted worktree, so the readable file is the copy the + # re-drive DISCARDS. The refusal's own remedy could not fix anything + # there — an operator who added a `status:` to that file and re-ran + # resolve would flip a spec that is deleted before it is read, while + # the committed spec, the one thing that decides routing, went + # untouched. Worse, the refusal fired even when the correction was + # already committed: `_committed_spec_status` had just PROVEN the + # re-drive routes correctly, and the re-arm was refused anyway over an + # obsolete copy. The real remedy on that shape is + # `rearm-spec-write-unreachable`'s ("commit the corrected spec"), + # which fires from the block above on exactly the legs that need it + # and now holds the resume rather than merely printing. + # + # The record is written on BOTH sides of that split: the abort message + # reaches stderr only, and the journal is the run's audit trail — + # `_echo_rearm_events` surfaces it from a `finally` on this path. + if refused: + raise RearmError( + f"cannot re-open story spec {spec_path} to `{target_status}` " + "for the re-drive: it has no frontmatter `status:` this re-arm " + "can set, so the re-driven session would wedge on the status " + "it reads — add a top-level `status:` to the spec's " + "frontmatter block, then re-run resolve" + ) # drop the stale `## Auto Run Result` section along with the status flip # (mirrors engine._reset_spec_for_repair): find_result_artifact keys on # that heading, so leaving it would let the re-driven session's first # save of the spec parse as the prior attempt's terminal outcome. - devcontract.strip_auto_run_result(spec_path, confine_root=Path(state.project)) + # + # Sequenced AFTER the read-back check above, not with the flip it mirrors: + # that check now raises, and an aborted re-arm must leave the spec exactly + # as it found it — a stripped result section on a spec the re-arm then + # refused would be the one edit nothing else records. + devcontract.strip_auto_run_result( + spec_path, confine_root=_task_spec_root(task, state) + ) except verify.FrontmatterWriteError as e: # The spec reads fine but carries `status:` in a shape no line # edit can move (a block scalar, a flow mapping, a value continued @@ -2313,6 +2821,24 @@ def rearm_escalation( # flip the re-drive would just re-wedge — abort BEFORE any state # is persisted (save_state runs below) with an actionable error # instead of a traceback; the escalation stays armed for a retry. + # + # ...and this arm is the SECOND refusal that can fire after a write has + # landed, which the sequencing argument above does not cover. It guards + # BOTH helpers, and `strip_auto_run_result` is the later one: by the + # time its own read/decode or its atomic write faults (an + # `atomic_write_bytes_confined` that cannot land — ENOSPC, EIO, a + # component swapped for a link under the `O_NOFOLLOW` walk — or a spec + # replaced under us between the two writes), the flip has already been + # published and `save_state` has not. Ordering the strip after the + # read-back check bought that check its byte-identical abort; it buys + # this one nothing, because the fault is IN the strip. So the same undo + # the re-stamp carries applies here, on the same terms. + # + # On the arm's other shape — the flip itself faulting on an + # unreadable/undecodable spec — nothing was written, `spec_before` still + # equals the bytes on disk, and `_restore_rearmed_spec` proves that and + # returns without touching the file or its mtime. + _restore_rearmed_spec(spec_path, spec_before, task, state) raise RearmError( f"cannot re-open story spec {spec_path} for the re-drive " f"({e.__class__.__name__}: {e}) — fix or replace the file " @@ -2330,12 +2856,39 @@ def rearm_escalation( # sentinel must not be snapshotted), and before it because it feeds it. # Nothing is deleted here: the re-drive's reset (verify.safe_rollback) removes # whatever the refreshed snapshot no longer blesses, at the right moment. - stale_residue = _stale_restore_residue( - Path(state.project), journal, key, old_latch, old_baseline - ) + # The CODE tree, not `state.project`: every git read below (and every baseline + # the proof-of-work gate later measures against) must name the repository the + # dev writer stamps. + # + # That is `paths.repo_root` for every run this function can be reached from, but + # NOT because `paths.repo_root == workspace.root` universally — it does not. + # `Workspace.default` sets `root=paths.repo_root`, while the isolation constructor + # mounts `root=/worktrees/` and rebases a fresh `ProjectPaths` onto + # it, so under `isolation = "worktree"` the run-level `repo_root` is the main + # checkout and the baseline is stamped in the worktree. + # + # `bmadconfig.worktree_isolation_conflict` refuses worktree isolation beside a + # `repo_root:` OVERRIDE — a narrower fact than it looks. It forces + # `repo_root == project`; it says nothing about `repo_root` vs `workspace.root`. + # Under plain isolation with NO override those two still diverge and isolation is + # ON, so "wherever the roots could diverge, isolation is off" is false, and a rule + # built on it licenses treating `state.code_root` as the tree the dev writer + # stamped — which under isolation it is not. + # + # What is true, and the only claim to carry forward: `repo_root == project` in + # every reachable configuration, so reading HEAD here is right for the in-place + # case; and under isolation this value is deliberately SUPERSEDED rather than + # relied on — `engine._finish_inflight` discards the worktree and `_dev_phase` + # re-stamps `task.baseline_commit` from the fresh worktree's HEAD before any gate + # reads it. Do not carry an identity into new code; carry this argument. + # + # A pre-upgrade state.json with no recorded root degrades to `project` exactly as + # before. + repo = state.code_root + stale_residue = _stale_restore_residue(repo, journal, key, old_latch, old_baseline) - # Advance the attempt baseline to the project's current HEAD and refresh the - # untracked snapshot: whatever the human-driven resolve session left on the + # Advance the attempt baseline to the CODE TREE's current HEAD (`repo`, above) + # and refresh the untracked snapshot: whatever the human-driven resolve session left on the # branch (a committed fixture, a corrected ledger, ...) is authorized input # for the re-drive, not failed-attempt debris. Without this, the re-drive's # reset-to-baseline in engine._rollback_or_pause parks the resolution @@ -2349,41 +2902,160 @@ def rearm_escalation( # pre-existing untracked file. The two locals are computed before either task # field is assigned, so a failure on either git call can't advance # baseline_commit while baseline_untracked stays stale, or vice versa. + advanced = False try: - repo = Path(state.project) head = verify.rev_parse_head(repo) untracked = sorted(verify.untracked_files(repo) - stale_residue) + except verify.GitError as e: + # `verify.GitError` is a TOTAL replacement for the `except Exception` that + # stood here, not a narrowing that leaks: both calls go through `_run_git`, + # which translates spawn (`GitSpawnError`), timeout (`GitTimeoutError`) and + # decode faults into this one taxonomy, and a non-zero rc into a plain + # `GitError`. Still swallowed rather than raised — a project that is not a + # git repo must not fail re-arm — but no longer SILENT: the degrade is the + # difference between "the re-drive starts from the resolution" and "it + # rebuilds against the tree the human just corrected away", and the + # re-stamp below now refuses to paper over it. + journal.append( + "rearm-baseline-advance-failed", + story_key=key, + repo=str(repo), + baseline=old_baseline or "", + error=f"{e.__class__.__name__}: {e}", + ) + else: task.baseline_commit = head task.baseline_untracked = untracked - except Exception: # nosec B110 - best-effort git read, must not fail re-arm - pass - - # Patch-restore only: re-stamp the spec's own baseline to the advanced one. - # The in-review route skips step-03 — the only step that stamps - # `baseline_revision` — so without this the re-driven step-04 would build its - # review diff (and, on an intent-gap/bad-spec re-triage, revert) "since" the - # ORIGINAL pre-attempt sha, clawing back the very resolve-session commits the - # advance above just blessed as the re-drive's starting point. Loud on - # failure: a silently stale spec baseline is exactly the hazard being closed - # (the spec block above already proved the file readable, so this is remote). - if restore_patch and task.spec_file and task.baseline_commit: - try: - verify.set_frontmatter_field( - Path(task.spec_file), - "baseline_revision", - task.baseline_commit, - confine_root=Path(state.project), + advanced = True + + # Re-stamp the spec's own baseline to the advanced one, on BOTH re-drive legs. + # + # The patch-restore leg needs it because the in-review route skips step-03 — + # the only step that stamps `baseline_revision` — so without it the re-driven + # step-04 would build its review diff (and, on an intent-gap/bad-spec + # re-triage, revert) "since" the ORIGINAL pre-attempt sha, clawing back the + # very resolve-session commits the advance above just blessed as the re-drive's + # starting point. + # + # The from-scratch leg gets it too (#640a). Its step-03 re-stamps the key + # itself, so the write is redundant on the happy path — but only ON that path: + # until step-03 runs, the spec carries the escalated attempt's sha, and every + # gate that reads a claimed baseline before then reads a stale one. The cost is + # recorded rather than hidden: re-stamping removes the gate's INDEPENDENT + # signal on this leg (it then compares a value the orchestrator itself wrote), + # so a claim that genuinely diverged is journalled on the way out instead of + # being silently normalized. + # + # Gated on `advanced`, not on truthiness of `task.baseline_commit`: a failed + # advance leaves the OLD sha in that field, which passes a truthiness test + # identically to a freshly advanced one. Writing it would make spec and task + # agree on a stale value — the one state in which nothing downstream can tell + # that the advance never happened, and the re-drive rebuilds from the wrong + # point with no error anywhere. Skipping keeps the failure legible (the degrade + # is journalled above) and keeps re-arm non-fatal outside a repo. + # + # Loud on WRITE failure: a silently stale spec baseline is exactly the hazard + # being closed. + # + # Guarded on `is_file` FIRST, because a spec this process cannot reach is not a + # write failure here — it is a SILENT one. Both frontmatter writers answer such a + # path with `False` rather than an exception (`verify.set_frontmatter_status`, + # `verify.set_frontmatter_field`), so without a check the re-stamp no-ops with + # nothing on the record and the spec keeps the escalated attempt's sha. + # + # `_task_spec_path` re-anchors the recorded path before we get here, which is what + # makes `is_file` mean what it says. Resolved raw it meant something else and worse: + # `spec_file` is persisted RELATIVE to the worktree for an isolated task, and the + # main checkout carries the same layout, so the check passed on the wrong file and + # the write landed there. The restore leg cannot reach any of this (its precondition + # rejects a truthy `task.worktree_path`); the from-scratch leg has no such guard, + # which is exactly why that precondition has to exist. + # + # `is_file` is necessary but not sufficient: a spec that EXISTS with no frontmatter + # block also returns `False` from both writers. That shape is caught by the flip's + # `flipped` check above and, here, by `overwritten` staying empty. + if task.spec_file: + spec_path = _task_spec_path(task, state) + if not spec_path.is_file(): + # OUTSIDE the `advanced` gate on purpose. Nesting this record inside it + # made the two #640 legs shadow each other: on a project that is not a + # repo the advance fails, `advanced` is False, and an unreadable spec + # then produced NO record at all — the journal blamed git while the + # status flip above had silently no-opped for an entirely different + # reason. The two degrades compose; they do not substitute. + journal.append( + "rearm-baseline-restamp-skipped", + story_key=key, + spec_file=str(spec_path), + baseline=task.baseline_commit or "", ) - except (OSError, UnicodeDecodeError, verify.FrontmatterWriteError) as e: - # FrontmatterWriteError joins the tuple rather than getting its own - # arm: the remedy is the same sentence ("fix the file"), and the - # exception already says which shape it could not move. What matters - # is that it aborts here — the stale-baseline hazard this block exists - # to close is exactly what a swallowed write would leave behind. - raise RearmError( - f"cannot re-stamp baseline_revision on {task.spec_file} " - f"({e.__class__.__name__}: {e}) — fix the file, then re-run resolve" - ) from e + elif advanced and task.baseline_commit: + try: + # Read through the same reader both consumers of a claimed baseline use, + # so what gets journalled as "overwritten" is the value the gate would + # have judged — not whichever key happened to be inspected here (#716). + # + # INSIDE the try, with the write it describes. `read_frontmatter` opens + # the file itself, so an OSError here would otherwise escape as a + # traceback from the one block whose whole contract is to turn a spec + # this re-arm cannot move into an actionable `RearmError`. What it does + # NOT rescue: `read_frontmatter` DEGRADES an unparseable YAML block to + # `{}` rather than raising, so on such a spec `overwritten` is `""`, the + # guard below is falsy, and no divergence record is written even though + # the insert lands. That is the reader's deliberate observe-degrade + # contract, not something to defeat here — the value is unknowable, and + # inventing one would be worse than the silence. + overwritten = auto_dev_baseline_of(verify.read_frontmatter(spec_path)) + verify.set_frontmatter_field( + spec_path, + "baseline_revision", + task.baseline_commit, + confine_root=_task_spec_root(task, state), + ) + except (OSError, UnicodeDecodeError, verify.FrontmatterWriteError) as e: + # FrontmatterWriteError joins the tuple rather than getting its own + # arm: the remedy is the same sentence ("fix the file"), and the + # exception already says which shape it could not move. What matters + # is that it aborts here — the stale-baseline hazard this block exists + # to close is exactly what a swallowed write would leave behind. + # + # ...and that the abort leaves the spec as this re-arm FOUND it. This is + # the LAST of the two refusals that can fire after a write has landed — + # the flip and the result strip are both behind us, `save_state` is not — + # so it carries the undo the sequenced refusals get for free (the other + # is the spec block's `(OSError, UnicodeDecodeError)` arm, which the + # strip raises through after the flip has published). Without + # it a spec with a movable `status:` beside an unmovable + # `baseline_revision:` came back flipped to the re-drive's status and + # stripped of the terminal result, while the run still called the story + # escalated. + _restore_rearmed_spec(spec_path, spec_before, task, state) + raise RearmError( + f"cannot re-stamp baseline_revision on {spec_path} " + f"({e.__class__.__name__}: {e}) — fix the file, then re-run resolve" + ) from e + if overwritten and overwritten != old_baseline: + # Compared against `old_baseline` — what the RUN recorded for the + # escalated attempt — NOT against `task.baseline_commit`, which the + # advance above has already moved to the new HEAD. Measuring against the + # advanced value made this fire on every ordinary from-scratch re-arm + # whose resolve session committed anything: the spec and the run agreed + # exactly, and the operator was still told they diverged. A record that + # fires on the routine case is the "trains the operator to scroll past + # the meaningful one" failure the `restore` split exists to prevent. + # + # What survives is the real signal, on BOTH legs: the spec claimed a + # baseline the run never recorded. That is the only trace left of a + # divergence the gate can no longer report, because the re-stamp is + # about to normalize it away. + journal.append( + "rearm-baseline-restamped", + story_key=key, + spec_file=str(spec_path), + overwritten=overwritten, + baseline=task.baseline_commit, + restore=bool(restore_patch), + ) save_state(run_dir, state) journal.append( @@ -2395,6 +3067,222 @@ def rearm_escalation( return key +def journal_entries_or_none(run_dir: Path) -> list[dict[str, Any]] | None: + """This run's journal entries, or ``None`` when the journal cannot be read. + + The re-arm surfaces read the journal TWICE to diff what a re-arm appended, and + before that echo existed they read it not at all — so `Journal.entries()`' strict + UTF-8 decode would turn a corrupt journal into a re-arm the operator can no longer + perform, which is strictly worse than the missing echo and a regression against the + gesture's own history. Shared by `cli.cmd_resolve` and `TuiApp._do_rearm` rather + than living on one of them: the CLI's copy was left unguarded when the TUI's was + hardened, and the CLI's echo now runs from a `finally`, where a raise would replace + the `RearmError` the operator actually needs to see. + + ``None`` rather than ``[]`` because the two callers DIFF two reads. Degrading a + failed FIRST read to ``[]`` sets the watermark to zero, and a second read that + succeeds then replays every historical `rearm-*`/`stale-restore-*` entry as if this + re-arm had just produced it. A caller that cannot establish both ends of the diff + must skip the echo, not guess at it. + """ + try: + # Non-mapping lines are dropped HERE so the annotation is true for every + # caller: `Journal.entries()` appends `json.loads(line)` with no shape filter, + # so a bare `3` or `null` on its own line survives as a non-dict entry and its + # `list[dict[str, Any]]` return type is a claim about first-party producers, + # not a guarantee — pyright sees `Any` and is satisfied. Both reads apply the + # same filter, so the `len(before)` watermark stays exact. + return [e for e in Journal(run_dir).entries() if isinstance(e, dict)] + except (OSError, UnicodeDecodeError): + return None + + +def _journal_sequence(value: Any) -> tuple[Any, ...]: + """A journal list field read back as a sequence, whatever the line actually held. + + Every read in `rearm_event_notice` runs inside both operator surfaces' `finally`, + where a `TypeError` replaces the outcome the operator needs — on the TUI, whose + `_do_rearm` runs on Textual's message loop with no `_handle_exception` override, + it ends the app. `", ".join` and `len` are the two reads that raise on a shape the + journal admits (`"files": 3`, `"files": null`, `[1, 2]`); every sibling read is + already `str()`-wrapped or f-string-interpolated and cannot. + + A bare string is deliberately NOT iterated: `", ".join("abc")` renders `"a, b, c"`, + which is worse than useless. It is wrapped as a single element instead, and `None` + — which `.get(key, default)` returns whenever the key EXISTS holding null, so the + default never applies — reads as empty. + """ + if isinstance(value, (list, tuple)): + return tuple(value) + return () if value is None else (value,) + + +def rearm_event_notice( + entry: dict[str, Any], +) -> tuple[Literal["note", "warning"], str, str] | None: + """`(severity, message, next_step)` for a re-arm record an operator must see. + + ONE table, two surfaces. `cli._echo_rearm_events` prints `message` followed by + `next_step`; `TuiApp._do_rearm` shows `message` alone. That split is the whole + reason this returns three fields instead of a formatted line: the TUI re-arms and + RESUMES in a single gesture, so an instruction to check something "before + resuming" is already unactionable by the time it renders — but the finding it + reports is not, and dropping the record to avoid the dead imperative is what left + the TUI silent on three kinds `resolve` echoed. + + Returns None for journal kinds no operator has to act on, so a caller can walk + every new entry and let the table decide. + + Severity is `"note"` or `"warning"`; each surface maps those onto its own channel. + """ + if not isinstance(entry, dict): + return None + kind = entry.get("kind", "") + if kind == "stale-restore-excluded": + files = ", ".join(str(f) for f in _journal_sequence(entry.get("files"))) + return ( + "note", + f"excluded the abandoned restore's new files from the re-drive " f"baseline: {files}", + "", + ) + if kind == "stale-restore-unparseable": + return ( + "warning", + f"could not read the abandoned restore patch ({entry.get('patch', '?')}) " + "— its new files may be swept into the next commit", + "check `git status` before resuming", + ) + if kind == "stale-restore-commits": + n = len(_journal_sequence(entry.get("commits"))) + return ( + "warning", + f"{n} commit(s) sit below the re-drive's new baseline " + f"({str(entry.get('old_baseline', '?'))[:12]}..) — if any came from the " + "abandoned attempt rather than your resolve, revert them now", + "", + ) + if kind == "rearm-baseline-advance-failed": + return ( + "warning", + f"could not advance the re-drive baseline ({entry.get('error', '?')}) — it " + f"still names {str(entry.get('baseline', '') or '(none)')[:12]}, so the " + "re-drive rebuilds against the tree as it stood before your resolve; the " + "spec was deliberately NOT re-stamped", + "Check the baseline before resuming", + ) + if kind == "rearm-spec-write-unreachable": + # The branch is the half an operator cannot infer: the re-drive cuts its fresh + # worktree from the run's PINNED target branch, so a correction committed on + # whatever the main checkout happens to have checked out is not the one it + # reads. Named only when the record carries it — a run predating the field + # leaves it empty, and a remedy that names no ref beats one that names a guess. + base = str(entry.get("target_branch", "") or "") + where = f" on `{base}`" if base else "" + return ( + "warning", + f"this story ran under worktree isolation, so the re-arm's spec writes " + f"({entry.get('spec_file', '?')}) land in a worktree the re-drive discards " + "— the re-driven session reads the COMMITTED spec, so commit the corrected " + f"spec{where} or the story re-wedges on the escalated attempt's status", + f"Commit the corrected spec{where} before resuming", + ) + if kind == "rearm-spec-flip-skipped": + # ONE kind, TWO outcomes, told apart by the flag the producer writes rather + # than by anything readable from here: `rearm_escalation` raises `RearmError` + # right after journalling this only when the flip failed on the very copy the + # re-drive reads. It also journals it — and completes — when that copy is + # unreadable from this process, or is a worktree-local file the re-drive + # discards. This row used to claim the abort unconditionally, which told an + # operator whose re-arm had SUCCEEDED that it "was REFUSED" and sent them to + # add a `status:` to a file the re-drive never opens. + spec = entry.get("spec_file", "?") + status = entry.get("status", "?") + if entry.get("refused"): + # The message names the refusal rather than predicting a re-wedge, because + # there is no re-drive left to wedge — and the next_step is the repair, not + # an inspection, for the same reason. + return ( + "warning", + f"the recorded spec for this story ({spec}) could not be re-opened to " + f"`{status}` — it carries no frontmatter `status:` to set, so the " + "re-arm was REFUSED rather than re-driving a session that would wedge " + "on the status it reads", + "Add a top-level `status:` to the spec, then re-run resolve", + ) + # No next_step, and deliberately: on this leg there is nothing to do to THIS + # file. Whether anything is left to do at all is decided by the committed spec, + # and `rearm-spec-write-unreachable` — journalled from the same block, on + # exactly the legs where the committed spec is not already at the target — + # carries that imperative, and holds the resume behind it. + return ( + "warning", + f"the recorded spec for this story ({spec}) could not be re-opened to " + f"`{status}` — the re-arm was NOT refused, because that copy is not what " + "the re-driven session reads: it mounts a fresh worktree and reads the " + "COMMITTED spec", + "", + ) + if kind == "rearm-baseline-restamp-skipped": + return ( + "warning", + f"the recorded spec for this story ({entry.get('spec_file', '?')}) is not a " + "readable file from here, so the baseline re-stamp was skipped — the spec " + "still names the escalated attempt's baseline", + "Check the recorded spec path before resuming", + ) + if kind == "rearm-baseline-restamped": + head = ( + f"re-stamped the spec baseline " + f"{str(entry.get('overwritten', '?'))[:12]}.. -> " + f"{str(entry.get('baseline', '?'))[:12]}.." + ) + # NOT differentiated on the `restore` flag any more. That split predated the + # record's condition moving to `overwritten != old_baseline` (compared against + # what the RUN recorded, not against the just-advanced value): the record now + # fires ONLY when the spec claimed a baseline the run never recorded, which is + # equally exceptional on both legs. Keeping the split meant the patch-restore + # leg's real divergence was the one downgraded to a note. The flag stays ON the + # record because it says which leg produced it — not how routine it is. + return ( + "warning", + f"{head} — the spec claimed a DIFFERENT baseline than the run recorded, " + "and this re-stamp is the only trace of it; the gate can no longer report " + "that divergence", + "", + ) + return None + + +def rearm_holds_the_resume(entry: dict[str, Any]) -> bool: + """True for a re-arm record whose remedy has to land BEFORE the re-drive reads the + tree — so a surface that re-arms and resumes in ONE gesture must stop after the + re-arm and leave `bmad-loop resume` to the operator. + + Exactly one kind qualifies, and the discriminator is PROOF, not urgency. + `rearm-spec-write-unreachable` is written only once `_committed_spec_status` has + established that the committed spec does NOT carry the status the re-drive routes + on, and only for a spec the working-tree flip cannot reach. Resuming on it is not + risky, it is futile: the re-drive discards the worktree, mounts a fresh one from + git, and step-01 reads a status it cannot route — `unrecognized status in existing + story file` halts it blocked, and the escalation is spent. The record's own + next_step already said "commit the corrected spec before resuming"; both default + surfaces then resumed in the same breath, which made the imperative unactionable at + the moment it rendered. The interactive resolve agent cannot close that gap either + — its skill forbids it from committing. + + The other warnings stay advisory and do NOT hold. `stale-restore-commits`, + `stale-restore-unparseable` and `rearm-baseline-advance-failed` each report + something an operator may need to act on, but none of them PROVES the re-drive + cannot route, and holding on a maybe would turn the ordinary degrade path into a + two-command gesture for an outcome nothing decided. + + Not folded into `rearm_event_notice`'s tuple, because they are different questions + asked of the same entry: that table answers "what do I tell the operator", this + answers "may this gesture still resume". Both surfaces ask both, in one walk. + """ + return isinstance(entry, dict) and entry.get("kind") == "rearm-spec-write-unreachable" + + def _stale_restore_residue( repo: Path, journal: Journal, diff --git a/src/bmad_loop/runsetup.py b/src/bmad_loop/runsetup.py index c685de41..071c8ada 100644 --- a/src/bmad_loop/runsetup.py +++ b/src/bmad_loop/runsetup.py @@ -773,6 +773,7 @@ def build_run_state( *, run_id: str, project: Path, + repo_root: Path, policy: Policy, epic_filter: int | None, story_filter: str | None, @@ -791,10 +792,16 @@ def build_run_state( tree by :func:`compose_run` (#498). The out-of-tree file is the one resume trusts; this copy is the secondary that travels with the run directory — see ``RunState.trusted_config_digest`` for why a run that outlives its state key - needs one.""" + needs one. + + ``repo_root`` records the git root code work happens in (``paths.repo_root``), + which equals ``project`` unless the BMAD config sets a `repo_root:` override. + ``runs.rearm_escalation`` runs out of process and reads it back to advance the + attempt baseline in the tree the proof-of-work gate actually measures.""" return RunState( run_id=run_id, project=str(project), + repo_root=str(repo_root), started_at=time.strftime("%Y-%m-%dT%H:%M:%S"), policy_snapshot=policy.to_dict(), epic_filter=epic_filter, @@ -1009,6 +1016,7 @@ def compose_run( state = build_run_state( run_id=run_id, project=project, + repo_root=paths.repo_root, policy=policy, epic_filter=epic_filter, story_filter=story_filter, @@ -1144,6 +1152,7 @@ def compose_sweep( state = RunState( run_id=run_id, project=str(project), + repo_root=str(paths.repo_root), started_at=time.strftime("%Y-%m-%dT%H:%M:%S"), policy_snapshot=policy.to_dict(), run_type="sweep", diff --git a/src/bmad_loop/signals.py b/src/bmad_loop/signals.py index 2a547630..0c6c33b5 100644 --- a/src/bmad_loop/signals.py +++ b/src/bmad_loop/signals.py @@ -134,11 +134,18 @@ def wait_for( none may be lost. Events older than `since_ns` (wall-clock ns, the session's launch time) - are dropped: a resumed/re-armed run reuses task_ids, and a fresh watcher - re-sees the events directory from scratch, so a prior cycle's Stop event - would otherwise replay instantly and the old result.json be read as a - bogus completion. Sessions run sequentially, so since_ns only advances; - anything below the current floor is genuinely stale and safe to discard. + are dropped: a resumed run reuses task_ids, and a fresh watcher re-sees the + events directory from scratch, so a prior cycle's Stop event would + otherwise replay instantly and the old result.json be read as a bogus + completion. Sessions run sequentially, so since_ns only advances; anything + below the current floor is genuinely stale and safe to discard. + + A re-ARMED run no longer reuses them — `runs.rearm_escalation` bumps + `StoryTask.generation`, which `engine._session_task_id` folds into the id + (#705). That removes one source of collision; it does not make this floor + redundant, because a plain resume re-mints the SAME id by design (that is + how crash replay finds its record) and is the case this guard was written + for. """ deadline = clock() + timeout_s while True: diff --git a/src/bmad_loop/sweep.py b/src/bmad_loop/sweep.py index ca835227..22dff5f5 100644 --- a/src/bmad_loop/sweep.py +++ b/src/bmad_loop/sweep.py @@ -946,6 +946,12 @@ def _ensure_migration(self, text: str) -> None: raise RunPaused(reason, PAUSE_STORY_GATE, MIGRATE_KEY) if not task.baseline_commit: + # `self.workspace.root`, which is `paths.repo_root` — the same anchor the + # dev writer (`Engine._dev_phase`), the re-arm writer + # (`runs.rearm_escalation`) and every proof-of-work probe in + # `verify._verify_shared_gates` use. Under the `repo_root` override it is + # NOT `paths.project`, and a baseline stamped in one tree and measured in + # the other names a commit the measuring repo has never heard of (#716). task.baseline_commit = verify.rev_parse_head(self.workspace.root) task.baseline_untracked = sorted(verify.untracked_files(self.workspace.root)) diff --git a/src/bmad_loop/tui/app.py b/src/bmad_loop/tui/app.py index 8ac9812f..4dd933cc 100644 --- a/src/bmad_loop/tui/app.py +++ b/src/bmad_loop/tui/app.py @@ -15,7 +15,7 @@ import time from collections.abc import Callable from pathlib import Path -from typing import TypeVar +from typing import Any, TypeVar from rich.text import Text from textual import work @@ -821,6 +821,37 @@ def _do_replan(self, run_id: str, spec_path: Path) -> None: self.notify("plan reset to draft — the next dispatch re-plans") self._do_resume(run_id) + def _echo_rearm_events(self, run_dir: Path, before: list[dict[str, Any]] | None) -> bool: + """Toast the re-arm records `cli._echo_rearm_events` prints, same table. + + Reads through `runs.journal_entries_or_none`, shared with the CLI so the two + surfaces cannot drift on robustness the way they drifted on routing. Both ends + of the diff must be readable: a failed FIRST read degraded to `[]` would set the + watermark to zero and replay every historical record as a fresh toast, so an + unreadable journal costs the echo and keeps the gesture. + + The table's `next_step` is deliberately dropped: it reads "... before + resuming", and this path resumes in the same gesture. + + Returns True when a record HOLDS that gesture (`runs.rearm_holds_the_resume`), + which is the one case where the dropped imperative was load-bearing rather than + moot — `_do_rearm` stops instead of resuming, and says so in its own words. + """ + after = runs.journal_entries_or_none(run_dir) + if before is None or after is None: + return False + holds = False + for entry in after[len(before) :]: + # before the routing table can drop it: a `None` notice means "nothing to + # toast", never "nothing to decide" + holds = runs.rearm_holds_the_resume(entry) or holds + notice = runs.rearm_event_notice(entry) + if notice is None: + continue + severity, message, _next_step = notice + self.notify(message, severity="warning" if severity == "warning" else "information") + return holds + def _do_rearm( self, run_id: str, run_dir: Path, story_key: str, *, restore_recorded: bool = False ) -> None: @@ -828,11 +859,62 @@ def _do_rearm( path (rearm_escalation handles sentinel auto-delete-with-preservation).""" if self._resolve_blocked_by_liveness(run_id, run_dir): return + # Same seam as `cli.cmd_resolve`, for the same reason and at the same moment: + # `runs.rearm_escalation` reads the persisted code root back out of the run + # state, and only a process that has just read config.yaml can tell whether a + # `repo_root:` edit made while the run was paused has moved it. Resume re-stamps + # it, but this gesture re-arms BEFORE it resumes, so the mirror has to be aimed + # here or the re-arm advances the baseline in the tree the run has left. + try: + paths = bmadconfig.load_paths(self.project) + except (bmadconfig.BmadConfigError, OSError) as e: + self.notify( + f"cannot read the project config to confirm the code root ({e}) — " + "re-arming against the root this run recorded", + severity="warning", + ) + else: + # Same hoist as `cli.cmd_resolve`, for the same reason: this gesture + # re-arms and THEN resumes, so the isolation refusal the detached CLI makes + # in `_resume_paused_run` landed after the re-stamp had persisted the + # unsupported root and `rearm_escalation` had advanced the attempt baseline + # against it. The operator saw "re-armed " and then a pane that + # refused, with the story no longer escalated for `resolve` to correct. + # + # An unreadable policy falls THROUGH to the re-arm rather than blocking, + # matching this surface's launch guard above: the check cannot tell "no + # conflict" from "could not look", and the detached CLI reads the same file + # and fails loudly on it. `paths` is already in hand, so only the policy + # read is guarded here. + try: + conflict = bmadconfig.worktree_isolation_conflict( + paths, policy.load(self.project / POLICY_FILE).scm.isolation + ) + except (policy.PolicyError, OSError): + conflict = None + if conflict is not None: + self.notify(conflict, severity="error") + return + if (moved := runs.restamp_code_root(run_dir, paths.repo_root)) is not None: + self.notify(moved, severity="warning") + before_entries = runs.journal_entries_or_none(run_dir) + hold_resume = False try: runs.rearm_escalation(run_dir, story_key) except RearmError as e: self.notify(f"re-arm failed: {e}", severity="error") return + finally: + # In the `finally`, matching `cli.cmd_resolve`. `_stale_restore_residue` + # journals BEFORE the re-stamp block that raises `RearmError`, so on that + # path the records were already written and returning early threw them + # away — including `stale-restore-commits`, the one record whose whole + # point is that nothing else will tell the human. This surface used to + # `return` there while the CLI echoed, so the two DID drift on the abort + # path even after they were unified on routing — and an abort is when the + # residue matters most: the re-arm half-ran and the operator has to decide + # what to do with the tree. + hold_resume = self._echo_rearm_events(run_dir, before_entries) if restore_recorded: self.notify( "recorded restore patch NOT honored — this re-arm re-drives from " @@ -840,6 +922,18 @@ def _do_rearm( severity="warning", ) self.notify(f"re-armed {story_key}") + if hold_resume: + # The half of the gesture that still worked is kept: the story IS re-armed + # and persisted. What stops is the resume this surface folds in behind it, + # because the warning above proved the re-drive would read a spec it cannot + # route on and burn the escalation. Worded for a surface that drops + # `next_step`, and worded as an instruction the operator can finish here — + # the run stays paused and resumable from this same screen. + self.notify( + "not resuming: commit the corrected spec, then resume this run", + severity="warning", + ) + return self._do_resume(run_id) def _resolve_blocked_by_liveness(self, run_id: str, run_dir: Path) -> bool: diff --git a/src/bmad_loop/tui/data.py b/src/bmad_loop/tui/data.py index 26408367..14f2ccf5 100644 --- a/src/bmad_loop/tui/data.py +++ b/src/bmad_loop/tui/data.py @@ -605,8 +605,14 @@ class ActiveAgent: def _story_key_from_task_id(task_id: str, role: str) -> str: """Recover the story key from a session task_id when the journal entry predates story-key stamping (#153 phase 1). The id is - ``safe_segment(f"{story_key}-{part}-{seq}")`` where ``part`` is the role, or - a workflow label for labeled plugin sessions — so peel the trailing + ``safe_segment(f"{story_key}-{part}-{seq}{gen}")`` where ``part`` is the role, or + a workflow label for labeled plugin sessions, and ``gen`` is a ``-g`` re-arm + generation suffix emitted only above zero (#705). This parser handles the + unsuffixed shape ONLY: a ``-g1`` tail fails the ``seq.isdigit()`` test below and + returns the whole id. That is unreachable rather than latent-correct — every + session-start has carried ``story_key`` since #153 phase 1, so the entries this + fallback sees are exactly the ones that predate generations — but widen the + fallback and this is the assumption that breaks. So peel the trailing ``-{part}-{seq}``: drop the numeric seq, then the recorded role when it matches (the common case), else one more ``-`` group (best-effort, since a label is not recoverable from the entry).""" diff --git a/src/bmad_loop/verify.py b/src/bmad_loop/verify.py index a9a311a6..702e38ef 100644 --- a/src/bmad_loop/verify.py +++ b/src/bmad_loop/verify.py @@ -29,6 +29,7 @@ from .frontmatter import ( _edit_frontmatter_block, _split_frontmatter, + auto_dev_baseline_of, operator_actions_of, read_frontmatter, status_of, @@ -724,9 +725,10 @@ def has_changes_since( `exclude` is repo-relative posix dir prefixes whose changes don't count — used by the dev/bundle proof-of-work gate to ignore the orchestrator-owned - BMAD artifact folders (see `artifact_relpaths`), so a session that only - rewrites its own spec (e.g. the frontmatter-status reconcile) under those - folders doesn't register as real implementation work. Mirrors + BMAD artifacts (composed by `verify_dev_exclude_relpaths`, relative to the same + root this is invoked against), so a session that only rewrites its own spec + (e.g. the frontmatter-status reconcile) under them doesn't register as real + implementation work. Mirrors `attempt_dirty`'s exclusion. Default `()` keeps the unscoped behavior. `baseline_untracked` is the untracked-file snapshot taken when the baseline @@ -3112,9 +3114,15 @@ def artifact_relpaths(paths: ProjectPaths) -> tuple[str, ...]: """Repo-relative posix prefixes of the orchestrator-owned BMAD artifact folders (the output root and the implementation/planning artifact dirs), relative to ``paths.project``. Folders configured outside the project tree - are skipped — nothing to exclude there. The same set as - ``Engine._protected_relpaths``; the dev/bundle proof-of-work gate passes - these to ``has_changes_since`` so spec-only edits never count as real work.""" + are skipped — nothing to exclude there. + + NO PRODUCTION CALLER. Both consumers it was written for have moved: the + dev/bundle proof-of-work gate now composes its excludes file-granularly through + ``verify_dev_exclude_relpaths``, rooted on ``paths.repo_root`` where git runs + (#716), and rollback protection builds its own list against the workspace root in + ``RecoveryFlow.protected_relpaths``. Its ``paths.project`` anchor is therefore + inert rather than correct — do not cite it as evidence that project-rooting is + right for anything, and re-derive the root if a caller is ever added.""" out: list[str] = [] for folder in ( paths.output_folder, @@ -3133,12 +3141,19 @@ def artifact_relpaths(paths: ProjectPaths) -> tuple[str, ...]: def verify_dev_exclude_relpaths( - paths: ProjectPaths, spec_path: Path, restore_patch: str | None = None + paths: ProjectPaths, + spec_path: Path, + restore_patch: str | None = None, + *, + root: Path, ) -> tuple[str, ...]: """Repo-relative posix paths the dev/bundle proof-of-work gate excludes from `has_changes_since` — file-granularity, unlike `artifact_relpaths`' whole-folder - exclusion (still used as-is by `Engine._protected_relpaths` for rollback - protection, a different job). Deliberately does NOT exclude `output_folder`: + exclusion. `artifact_relpaths` has NO production caller left: rollback + protection builds its own list in `recovery_flow.protected_relpaths` against + `workspace.root`, and `Engine._protected_relpaths` merely delegates there. Do + not adopt it as a shortcut — it is still anchored on `paths.project`, which is + #716's root cause. Deliberately does NOT exclude `output_folder`: in the standard layout it is the parent directory of `implementation_artifacts`/ `planning_artifacts`, so excluding it as a directory prefix would swallow those two folders' content right back out of view via the same git-pathspec prefix @@ -3165,15 +3180,32 @@ def verify_dev_exclude_relpaths( an un-normalized `..`/`.` segment would still resolve to the real on-disk file (the OS resolves it), but as a raw string it wouldn't match git's own normalized path output, silently defeating this exclude and letting a bare - status flip on the session's own spec count as real work.""" + status flip on the session's own spec count as real work. + + ``root`` is the tree the resulting pathspecs are relative to, and MUST be the + same root the caller invokes git against — `paths.repo_root` for the + proof-of-work gate, which is where `has_changes_since` runs. REQUIRED, with no + default: an implicit `paths.project` anchor is #716's own root cause, and the + two roots collapse in every configuration but the `repo_root` override, so a + defaulted caller would look correct everywhere it was tested and be wrong only + on the one config that matters. Requiring it turns OMITTING the root into a + type error; it does not police a WRONG one — ``root=paths.project`` type-checks + cleanly and silently excludes nothing, which is the failure the next paragraph + describes. The requirement buys a caller who must think about the root, not a + checker that knows the right answer. + + A relpath computed against the wrong root does not raise: it simply + matches nothing on git's side, so the exclusion silently disappears and a bare + status flip starts counting as real work. The latched `restore_patch` is + anchored on the SAME root for the same reason (a relative latch names a path + in the tree it will be applied to).""" candidates: list[Path] = [paths.sprint_status, spec_path] if restore_patch: - candidates.append(resolve_restore_path(restore_patch, paths.project)) + candidates.append(resolve_restore_path(restore_patch, root)) out: list[str] = [] - project = paths.project for path in candidates: try: - rel = path.resolve().relative_to(project).as_posix() + rel = path.resolve().relative_to(root).as_posix() except (OSError, RuntimeError, ValueError): continue # outside or uncertain; nothing safe to exclude here if rel and rel != ".": @@ -3315,14 +3347,34 @@ def _verify_shared_gates( # `baseline_commit` — that name exists only in the result.json devcontract # synthesizes, which this gate does not consult (it re-reads frontmatter). # An absent key skips the check below, so reading `baseline_commit` alone - # made this gate dead code for every generic-skill session. Read both, the - # same idiom as `devcontract.synthesize_result`. - claimed_baseline = str(fm.get("baseline_commit", fm.get("baseline_revision", ""))).strip() + # made this gate dead code for every generic-skill session. Both keys are read + # through the one shared reader `devcontract.synthesize_result` also calls, so + # the value this gate judges and the value the result.json reports are the same + # value by construction rather than by two expressions agreeing (#716). + claimed_baseline = auto_dev_baseline_of(fm) proof_baseline: str = task.baseline_commit or "" include_untracked_proof = True + # Every probe below runs against `paths.repo_root`, the CODE tree, never + # `paths.project`. Both baseline writers stamp `workspace.root` + # (`Engine._dev_phase`, `SweepEngine`'s migration task) and re-arm now does the + # same, and `Workspace.default` sets `root = paths.repo_root` while + # `ProjectPaths.rebased` sets both roots to the worktree — so `repo_root` is + # the one root that names the same repository as the recorded baseline in every + # configuration. Under the `repo_root` override (`isolation = "none"` plus a + # `repo_root:` config key, the only shape where the two differ — + # `bmadconfig.worktree_isolation_conflict` refuses the other) the session's cwd + # IS the code tree, so a `project`-anchored probe judged a tree the session never + # touched. WHICH probe burned the attempt depends on the layout, and the burn is + # not `has_changes_since` in both: it fails OPEN (`rc != 0` -> True), so wherever + # `project` is not a checkout the failing git call PASSES that gate. Nested + # (`project` a subdirectory of the code tree) the call succeeds but is scoped to + # that subdirectory, and the "no changes" forever-burn is real. Disjoint + # (`project` beside the checkout) git fails and the burn moves to the probes that + # fail CLOSED: `_canonical_commit_oid` returns None -> "does not match", and + # `is_ancestor` / `commit_reachable_above_baseline` read the failure as False. if task.baseline_commit and claimed_baseline not in ("", "NO_VCS"): try: - canonical_claimed = _canonical_commit_oid(paths.project, claimed_baseline) + canonical_claimed = _canonical_commit_oid(paths.repo_root, claimed_baseline) except GitError as e: return VerifyOutcome.escalate(str(e)) if canonical_claimed is None: @@ -3340,7 +3392,7 @@ def _verify_shared_gates( # history (a superset of the unit's changes), which is sound; a # diverged or unknown baseline still fails. older_ok = allow_ancestor_baseline and is_ancestor( - paths.project, canonical_claimed, task.baseline_commit + paths.repo_root, canonical_claimed, task.baseline_commit ) # The other direction needs no opt-in flag: an intervening commit # before step-03 stamps `baseline_revision` makes the claim newer @@ -3348,7 +3400,7 @@ def _verify_shared_gates( # HEAD reaches that canonical descendant; stale, diverged, unknown, # and off-HEAD commits still fail. newer_ok = commit_reachable_above_baseline( - paths.project, canonical_claimed, task.baseline_commit + paths.repo_root, canonical_claimed, task.baseline_commit ) # Accepting a newer claim moves the proof-of-work reference onto it: # under `isolation = "none"` the claimed commit may have arrived in @@ -3366,10 +3418,18 @@ def _verify_shared_gates( ) if extra_exclude is not None and task.baseline_commit: - exclude = verify_dev_exclude_relpaths(paths, spec_path, task.restore_patch) + extra_exclude + # The exclude pathspecs are rooted where git is invoked: `repo_root` here + # and `repo_root` in every producer that composes into `extra_exclude` + # (`Engine._harvest_gate_exclude`, `_stories_relpaths`). A pathspec relative + # to a different root is not merely wrong, it is SILENTLY wrong — git + # matches nothing and the exclusion evaporates. + exclude = ( + verify_dev_exclude_relpaths(paths, spec_path, task.restore_patch, root=paths.repo_root) + + extra_exclude + ) try: if not has_changes_since( - paths.project, + paths.repo_root, proof_baseline, exclude=exclude, baseline_untracked=task.baseline_untracked, @@ -3477,12 +3537,15 @@ def verify_dev( covers EVERY park, including one that wrote nothing and listed plausible actions, because the actions gate tests list non-emptiness and never content. - ``engine_written`` names project-relative paths the orchestrator itself - wrote above this gate during the attempt. They compose with the mode's normal - proof-of-work exclusions so engine bookkeeping cannot masquerade as session - work; see :meth:`Engine._harvest_gate_exclude`. On the parked leg they are not - passed at all — proof-of-work is skipped there, so there is no exclusion set - left for them to compose with. + ``engine_written`` names paths the orchestrator itself wrote above this gate + during the attempt, relative to ``paths.repo_root`` — the tree the gate invokes + git in, and therefore the root every pathspec composed into this exclusion set + must share (#716). They compose with the mode's normal proof-of-work exclusions + so engine bookkeeping cannot masquerade as session work; see + :meth:`Engine._harvest_gate_exclude`, which is their producer and states what a + ledger outside the code tree resolves to. On the parked leg they are not passed + at all — proof-of-work is skipped there, so there is no exclusion set left for + them to compose with. """ rj = result_json or {} spec_file = rj.get("spec_file") @@ -3681,7 +3744,12 @@ def verify_dev_stories( paths, expected_status=expected, extra_exclude=( - None if plan_halt else _stories_relpaths(paths.project, spec_folder) + engine_written + None + if plan_halt + # Rooted where the proof-of-work gate invokes git (`paths.repo_root`), + # not on `paths.project`: a pathspec relative to the other root matches + # nothing and the exclusion evaporates without an error (#716). + else _stories_relpaths(paths.repo_root, spec_folder) + engine_written ), ) if gate is not None: @@ -3691,14 +3759,19 @@ def verify_dev_stories( return VerifyOutcome.passed() -def _stories_relpaths(project: Path, spec_folder: Path) -> tuple[str, ...]: +def _stories_relpaths(root: Path, spec_folder: Path) -> tuple[str, ...]: """Proof-of-work exclude prefixes for the story record + manifest: the spec - folder's ``stories/`` subdir and its ``stories.yaml``, project-relative. Empty - when the spec folder is outside the project tree (nothing to exclude there).""" + folder's ``stories/`` subdir and its ``stories.yaml``, relative to ``root``. + Empty when the spec folder is outside that tree (nothing to exclude there). + + ``root`` is the tree git is invoked against — `paths.repo_root` at the one + production call site, which under the `repo_root` override is NOT + `paths.project` (the spec folder then sits outside the code tree and this + correctly returns ``()``).""" from .stories import STORIES_FILENAME, STORIES_SUBDIR try: - rel = spec_folder.resolve().relative_to(project.resolve()).as_posix() + rel = spec_folder.resolve().relative_to(root.resolve()).as_posix() except (OSError, RuntimeError, ValueError): return () base = "" if rel == "." else f"{rel}/" @@ -4288,8 +4361,10 @@ def resolve_restore_path(raw: str, root: Path) -> Path: `model.StoryTask.restore_patch` documents the field as repo-relative-or-absolute, and every consumer must resolve it against the base it actually reads the tree from — the engine's live workspace root (the unit worktree under isolation), - `paths.project` for the proof-of-work exclude, the CLI's `--project`. Hence the - caller-supplied `root` rather than one baked-in base. + `paths.repo_root` for the proof-of-work exclude (which is where + `has_changes_since` runs, so the latch has to name a path in that tree; #716), + the CLI's `--project`. Hence the caller-supplied `root` rather than one + baked-in base. In practice `cli._resolve_restore_patch` always latches an already-`.resolve()`d absolute path, so the relative branch is exercised only by a hand-written state diff --git a/tests/conftest.py b/tests/conftest.py index 2a96fe60..ad968004 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -718,15 +718,34 @@ def render_deferred(items) -> str: return "\n".join(lines) + "\n" +# `write_spec`'s "this key is not present at all" marker. A plain `None` cannot +# serve: for `legacy_baseline` it means the YAML-null shape (a bare +# `baseline_commit:` line), which is a distinct case the reader must treat as +# absent WITHOUT turning it into the token "None" (#358). +class _Omit: + """Type of :data:`OMIT`, so a parameter accepting the sentinel can still be + annotated. A bare ``object()`` forced ``baseline: object``, which silently + disabled checking for every ordinary caller passing a sha.""" + + __slots__ = () + + def __repr__(self) -> str: # pragma: no cover - debugging aid + return "OMIT" + + +OMIT = _Omit() + + def write_spec( path: Path, status: str, - baseline: str, + baseline: str | _Omit, *, prose_status: str | None = None, closes_deferred: object = None, operator_actions: object = None, deferred=None, + legacy_baseline: str | None | _Omit = OMIT, ) -> None: """Write a spec the way the real bmad-dev-auto skill does. The skill's step-03 stamps `baseline_revision` and NEVER `baseline_commit` (that name exists only @@ -734,6 +753,20 @@ def write_spec( same key — a reader that only knows `baseline_commit` must fail a test here, not sail through production (issue #89). + ``legacy_baseline`` adds the OTHER key, and exists because until #716 this + fixture *could not express the bug*: a spec that carries both is exactly what + `runs.rearm_escalation` manufactures (it inserts `baseline_revision` and never + removes a pre-existing `baseline_commit`), and no fixture could produce one, so + the precedence between them was untestable. ``OMIT`` writes no key at all + (the default, and what every pre-existing caller gets — `tests/test_verify.py` + asserts on that absence). ``None`` writes a bare ``baseline_commit:`` line, the + YAML-null shape. Any other value is written as a quoted scalar, including ``""`` + for the empty-value shape that a ``dict.get(k, default)`` chain selects but a + non-empty test cannot see. + + ``baseline`` accepts ``OMIT`` too, for the legacy-only spec: `baseline_revision` + is then absent and `baseline_commit` is the only claim on the file. + ``closes_deferred`` writes the story-declared ledger-closure field (#234): a list renders as a YAML flow sequence, and a bare string renders as a scalar — the wrong-container mistake whose handling must not depend on which file it @@ -763,9 +796,16 @@ def write_spec( declare += f"operator_actions: {operator_actions}\n" if deferred is not None: declare += render_deferred(deferred) + claims = "" if baseline is OMIT else f"baseline_revision: '{baseline}'\n" + if legacy_baseline is not OMIT: + claims += ( + "baseline_commit:\n" + if legacy_baseline is None + else f"baseline_commit: '{legacy_baseline}'\n" + ) body = ( f"---\ntitle: 'test'\ntype: 'feature'\nstatus: '{status}'\n" - f"baseline_revision: '{baseline}'\n{declare}---\n\n## Intent\n\ntest spec\n" + f"{claims}{declare}---\n\n## Intent\n\ntest spec\n" ) if prose_status is not None: # mirror bmad-dev-auto's terminal finalize: it appends a `## Auto Run @@ -1270,9 +1310,16 @@ def escalated_run( ``with_session`` appends the completed review SessionRecord the resolve-context builder reads. ``git_project`` makes ``state.project`` a REAL repo (spec files already written are committed, run state is gitignored) so `rearm_escalation`'s - baseline snapshot refresh actually runs and `baseline_commit` defaults to HEAD — - in a bare tmp_path its best-effort `except` swallows every git call and the - refresh silently no-ops. + baseline snapshot refresh actually runs and `baseline_commit` defaults to HEAD. + That refresh reads `state.code_root`, not `state.project`; the two name the same + directory for this fixture only because the RunState below records no + `repo_root`, and `code_root` is defined as `repo_root or project` — a caller that + ever adds an override must git-init THAT tree, not this one. In a bare tmp_path + the refresh's git calls raise `verify.GitError`, which re-arm swallows (a + non-repo project must not fail re-arm) but journals as + `rearm-baseline-advance-failed` while the old baseline stands — degraded and + visible, not silent, so a test asserting the advance must pass ``git_project`` + rather than read a no-op as success. """ project = Path(project) if git_project: diff --git a/tests/test_cli.py b/tests/test_cli.py index 036adb86..bcc03ec1 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -2560,6 +2560,169 @@ def test_resolve_no_interactive_rearms_and_resumes(tmp_path, monkeypatch, capsys assert "ready-for-dev" in spec.read_text() +# ------------------------------------ resolve aims the code root before it re-arms + +# `_resume_paused_run` re-stamps the persisted code root because the engine it arms +# works in `paths.repo_root` — but `resolve` re-arms FIRST, so on this path that +# re-stamp lands too late to aim `runs.rearm_escalation`, which reads the mirror out of +# process. Three rows: the move is adopted and announced before the re-arm sees it, a +# cancelled confirm writes nothing, and a config this process cannot read degrades +# instead of guessing a tree. + + +def _resolve_run_with_a_moved_code_root(project, monkeypatch): + """An escalated run whose recorded code root is NOT the one config.yaml now names. + Returns (run_dir, the tree config names, the tree the run recorded).""" + from bmad_loop.journal import load_state, save_state + + install_bmad_config(project) + moved = project.project / "moved-code" + moved.mkdir() + _configure_repo_root(project, moved) + run_dir = _escalated_run(project.project, "r1") + recorded = project.project / "old-code" + state = load_state(run_dir) + state.repo_root = str(recorded) + save_state(run_dir, state) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + return run_dir, moved, recorded + + +def test_resolve_restamps_the_code_root_before_it_rearms(project, monkeypatch, capsys): + """The ordering IS the fix. `rearm_escalation` advances the attempt baseline and + re-stamps `baseline_revision` against `RunState.code_root`; the engine resumed at the + bottom of the same command works in `paths.repo_root`. With the re-stamp left to + `_resume_paused_run`, a `repo_root:` edit made while the run was paused split those + two readers with no error anywhere — the re-arm armed one repository and the run + continued in another. + + Asserted at the moment of the re-arm, not afterwards: a re-stamp that lands after + `rearm_escalation` returns is exactly the bug, and reading state.json at the end + cannot tell the two apart. + + Ablation: move the `runs.restamp_code_root(...)` call below the `try:` that re-arms + and this reddens on the stale root while the warning assertion still passes. + """ + from bmad_loop import runs + from bmad_loop.journal import load_state + + run_dir, moved, _ = _resolve_run_with_a_moved_code_root(project, monkeypatch) + seen: list = [] + + def fake_rearm(rd, key, *, restore_patch=None): + seen.append(load_state(rd).code_root) + return key + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + + argv = ["resolve", "--project", str(project.project), "r1", "--no-interactive", "--resume"] + assert cli.main(argv) == 0 + + assert seen == [moved.resolve()] + err = capsys.readouterr().err + assert "the code root in _bmad/bmm/config.yaml has changed" in err + assert str(moved) not in err # the warning names neither tree, matching resume's + + +def test_resolve_declined_at_the_confirm_leaves_the_code_root_for_resume( + project, monkeypatch, capsys +): + """The re-stamp sits AFTER the confirm on purpose. A cancelled resolve must write + nothing: adopting the new root there would silence the loud `code_root_changed` + warning `_resume_paused_run` raises on its own terms, and the operator would never + hear about the move they did not go through with. + + Ablation: hoist the re-stamp above the `args.resume is None` confirm and this + reddens on both the persisted root and the silence. + """ + from bmad_loop import runs + from bmad_loop.journal import load_state + + run_dir, _moved, recorded = _resolve_run_with_a_moved_code_root(project, monkeypatch) + monkeypatch.setattr(cli, "_confirm", lambda _prompt: False) + monkeypatch.setattr( + runs, "rearm_escalation", lambda *a, **k: pytest.fail("re-armed after a decline") + ) + + assert cli.main(["resolve", "--project", str(project.project), "r1", "--no-interactive"]) == 0 + + assert load_state(run_dir).repo_root == str(recorded) + assert "code root" not in capsys.readouterr().err + + +def test_resolve_refuses_worktree_isolation_before_it_mutates_anything( + project, monkeypatch, capsys +): + """`resolve` re-arms and THEN resumes, so the isolation refusal `_resume_paused_run` + makes used to land after the whole re-arm had already been persisted. + + On `isolation = "worktree"` beside a `repo_root` override, the re-stamp wrote the + unsupported root onto the mirror and `rearm_escalation` then advanced the attempt + baseline — and re-stamped the spec's `baseline_revision` — against it, all before + the refusal at the bottom of the command returned 1. The escalation was spent + either way: the story came back PENDING, and `resolve` needs an ESCALATED story, so + the operator could not re-run it to undo the damage after fixing the config. + + It also made a configuration reachable that `runs.rearm_escalation` documents as + unreachable — it reads the code tree's HEAD on the stated premise that + `repo_root == project` "in every reachable configuration", which is true only + BECAUSE this refusal exists. + + Both post-conditions are asserted, because a refusal that merely returns 1 is not + the fix — writing nothing is. `_resolve_run_with_a_moved_code_root` stubs + `_resume_paused_run` to 0, so the rc can only come from the hoisted check. + + Ablation: move the `_reject_isolation_conflict(...)` call below the + `runs.restamp_code_root(...)` line and the persisted-root assertion reddens; delete + it outright and the rc assertion reddens too. + """ + from bmad_loop import runs + from bmad_loop.journal import load_state + from bmad_loop.model import Phase + + run_dir, _moved, recorded = _resolve_run_with_a_moved_code_root(project, monkeypatch) + _write_policy(project.project, ISOLATION_WORKTREE_POLICY) + monkeypatch.setattr( + runs, + "rearm_escalation", + lambda *a, **k: pytest.fail("re-armed under a configuration the run refuses"), + ) + + argv = ["resolve", "--project", str(project.project), "r1", "--no-interactive", "--resume"] + assert cli.main(argv) == 1 + + assert REFUSAL in capsys.readouterr().err + state = load_state(run_dir) + assert state.repo_root == str(recorded) # the mirror was never re-pointed + assert state.tasks["s1"].phase == Phase.ESCALATED # still armed for a corrected config + + +def test_resolve_degrades_when_the_config_cannot_name_the_code_root(tmp_path, monkeypatch, capsys): + """Reading config.yaml to learn the tree is an OBSERVATION, so it degrades: without + it this process cannot name the code root, and re-pointing the mirror at a guess is + the one outcome worse than leaving it alone. The re-arm proceeds against the root the + run recorded — what it did before this seam existed — and says so. + + Ablation: turn the `except (bmadconfig.BmadConfigError, OSError)` arm into a + `return 1` and this reddens on the exit code; delete the warning it prints and it + reddens on the silence while the re-arm assertion still passes. + """ + from bmad_loop import runs + from bmad_loop.journal import load_state + + run_dir = _escalated_run(tmp_path, "r1") # no _bmad/bmm/config.yaml anywhere + rearmed: list = [] + monkeypatch.setattr(runs, "rearm_escalation", lambda rd, key, **k: rearmed.append(key) or key) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + + argv = ["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"] + assert cli.main(argv) == 0 + + assert rearmed == ["s1"] + assert load_state(run_dir).repo_root == "" # nothing guessed onto the mirror + assert "cannot read the project config to confirm the code root" in capsys.readouterr().err + + def test_resolve_echoes_this_rearms_stale_restore_events(tmp_path, monkeypatch, capsys): """#90's journal entries reach the operator. The commits variant is warn-only — stderr is the only place it ever surfaces. Entries from *earlier* re-arms are @@ -2590,6 +2753,361 @@ def fake_rearm(rd, key, *, restore_patch=None): assert "FROM-LAST-TIME.txt" not in err +def test_resolve_echoes_the_rearm_baseline_records(tmp_path, monkeypatch, capsys): + """The two `rearm-baseline-*` records reach the operator on the same seam. + + Both are warn-only by contract — a project that is not a git repo must not fail + re-arm — so stderr is the only place either ever surfaces. A failed advance is the + most actionable outcome of the whole re-arm: the re-drive rebuilds against the + tree as it stood BEFORE the resolve, and the re-stamp then refuses to write a sha + it did not earn, so spec and task stay honestly out of step rather than silently + agreeing on a stale value. Journal-only, that is invisible to the human running + `bmad-loop resolve` — the invisibility #640(b) exists to end. + + Ablation: drop either `rearm-baseline-*` arm from `runs.rearm_event_notice` + (the shared table both surfaces route through) and the matching assertion + reddens. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + _escalated_run(tmp_path, "r1") + + def fake_rearm(rd, key, *, restore_patch=None): + journal = Journal(rd) + journal.append( + "rearm-baseline-advance-failed", + story_key=key, + repo=str(tmp_path), + baseline="a" * 40, + error="GitError: not a git repository", + ) + journal.append( + "rearm-baseline-restamped", + story_key=key, + spec_file="spec.md", + overwritten="b" * 40, + baseline="c" * 40, + restore=False, + ) + return key + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 0 + ) + + err = capsys.readouterr().err + assert "could not advance the re-drive baseline (GitError: not a git repository)" in err + assert "aaaaaaaaaaaa" in err + assert "deliberately NOT re-stamped" in err + assert "re-stamped the spec baseline bbbbbbbbbbbb.. -> cccccccccccc.." in err + + +def test_resolve_restamp_echo_warns_on_both_legs(tmp_path, monkeypatch, capsys): + """The record fires only on a REAL divergence, so both legs warn. + + The `restore` split predated the record's condition moving to + `overwritten != old_baseline` — compared against what the RUN recorded, not against + the value the advance just wrote. Under that condition the record is written only + when the spec claimed a baseline the run never recorded, which is equally + exceptional whichever leg produced it; keeping the old "routine on a restore + re-drive" note meant the patch-restore leg's genuine divergence was the one + downgraded. The flag stays ON the record to say which leg it was. + + Ablation: restore the `if entry.get("restore")` branch in `runs.rearm_event_notice` + and the `restore=True` leg reddens — it goes back to printing "routine" for a + divergence. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + def rearm_with(restore: bool): + def fake_rearm(rd, key, *, restore_patch=None): + Journal(rd).append( + "rearm-baseline-restamped", + story_key=key, + spec_file="spec.md", + overwritten="b" * 40, + baseline="c" * 40, + restore=restore, + ) + return key + + return fake_rearm + + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + + _escalated_run(tmp_path, "r1") + monkeypatch.setattr(runs, "rearm_escalation", rearm_with(True)) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 0 + ) + err = capsys.readouterr().err + assert "DIFFERENT baseline" in err # the restore leg is NOT exempt + assert "routine on a restore re-drive" not in err + + _escalated_run(tmp_path, "r2") + monkeypatch.setattr(runs, "rearm_escalation", rearm_with(False)) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r2", "--no-interactive", "--resume"]) == 0 + ) + err = capsys.readouterr().err + assert "DIFFERENT baseline" in err + assert "routine on a restore re-drive" not in err + # both legs reached the same sentence; only the flag on the record differs + + +@pytest.mark.parametrize("outcome", ["ok", "rearm-error"]) +def test_resolve_survives_a_corrupt_journal(tmp_path, monkeypatch, capsys, outcome): + """An undecodable byte in journal.jsonl costs the echo, never the gesture — and + never the exit code. + + The counterpart to `test_escalation_rearm_survives_a_corrupt_journal` in the TUI, + which had no CLI twin: the TUI's reads were guarded while `cmd_resolve`'s two were + left calling `Journal(run_dir).entries()` straight, and the echo then MOVED into a + `finally`. On the `RearmError` path that `finally` runs before `return 1`, so a + raise from the echo would replace an actionable error with a traceback and swallow + the original — the one path where the residue matters most. + + Ablation: call `Journal(run_dir).entries()` directly in `_echo_rearm_events` and + both legs redden — `ok` with a UnicodeDecodeError escaping `cmd_resolve`, and + `rearm-error` with the same, losing the `RearmError` message entirely. + """ + from bmad_loop import runs + from bmad_loop.journal import JOURNAL_FILE + + def fake_rearm(rd, key, *, restore_patch=None): + if outcome == "rearm-error": + raise runs.RearmError("cannot re-open story spec /x/spec.md") + return key + + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + run_dir = _escalated_run(tmp_path, "r1") + (run_dir / JOURNAL_FILE).write_bytes( + b'{"ts": 1.0, "kind": "session-start", "task_id": "t1"}\n\xff\xfe not utf-8\n' + ) + + rc = cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) + err = capsys.readouterr().err + + if outcome == "rearm-error": + # the operator gets the real error, not a decode traceback from the `finally` + assert rc == 1 + assert "cannot re-open story spec" in err + else: + assert rc == 0 + + +def test_resolve_echoes_a_skipped_restamp(tmp_path, monkeypatch, capsys): + """The unreadable-spec skip is warn-only like its two siblings, so stderr is the + only place it ever surfaces. Without the echo the operator resumes a re-drive + whose spec still names the escalated attempt's baseline. + + Ablation: drop the `rearm-baseline-restamp-skipped` arm from + `runs.rearm_event_notice` and this reddens. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + _escalated_run(tmp_path, "r1") + + def fake_rearm(rd, key, *, restore_patch=None): + Journal(rd).append( + "rearm-baseline-restamp-skipped", + story_key=key, + spec_file="wt/specs/s1.md", + baseline="c" * 40, + ) + return key + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 0 + ) + + err = capsys.readouterr().err + assert "wt/specs/s1.md" in err + assert "not a readable file from here" in err + + +def test_resolve_echoes_the_residue_even_when_the_rearm_aborts(tmp_path, monkeypatch, capsys): + """An abort is when the residue matters MOST, so the echo lives in a `finally`. + + `runs._stale_restore_residue` journals BEFORE the re-stamp block that raises + `RearmError`, so on that path the records are already on disk when the abort + happens — and an echo placed after an early `return 1` threw away records the + re-arm had genuinely written. The one it threw away is the one that cannot be + recovered from anywhere else: `stale-restore-commits` names commits now sitting + below a baseline the operator is looking at in a half-run re-arm, and nothing + but this line will tell them. The failure and the residue are both true, and the + operator needs both to decide what to do with the tree. + + Ablation (residue echo): move `_echo_rearm_events` out of the `finally` back under + the `try` and the commits assertion reddens while the `error:` line still prints. + + Ablation (success output): deleting the gate outright does NOT grade the last + assertion. Drop `return 1` from `cmd_resolve`'s `except runs.RearmError` arm and the + success line does leak to stdout, but `main` then answers 0 and the exit-code + assertion above reddens first, so this line is never reached — a bare rc is a + worthless oracle for an absent-output claim. The mutation that grades it keeps + `return 1` and adds `print(f"re-armed {story_key}")` to that arm: + `AssertionError: assert 're-armed' not in 're-armed s1\n'`. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + _escalated_run(tmp_path, "r1") + + def fake_rearm(rd, key, *, restore_patch=None): + # journalled first, exactly as the real residue pass is ordered + Journal(rd).append( + "stale-restore-commits", story_key=key, old_baseline="f" * 40, commits=["c1", "c2"] + ) + raise runs.RearmError("could not re-stamp the spec baseline") + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 1 + ) + + out, err = capsys.readouterr() + assert "error: could not re-stamp the spec baseline" in err # the abort still reports + assert "2 commit(s) sit below the re-drive's new baseline (ffffffffffff..)" in err + assert "re-armed" not in out # ...and the failure is not dressed up as a success + + +def test_resolve_holds_the_resume_when_the_correction_cannot_reach_the_redrive( + tmp_path, monkeypatch, capsys +): + """The one record that PROVES a wedge breaks the re-arm+resume gesture. + + `rearm-spec-write-unreachable` fires only once the re-arm has established that the + committed spec does not carry the status the re-drive routes on, and its own + next_step reads "Commit the corrected spec before resuming". This command printed + that and then resumed two lines later, so the imperative was already unactionable + when it rendered — and the interactive resolve agent cannot close the gap either, + since its skill forbids it from committing. The fresh worktree then checked out the + still-terminal committed spec, step-01 halted blocked on an unrecognized status, and + the escalation was spent. + + `--resume` does not override it: that flag skips the confirmation prompt, while the + hold is a proof rather than a question. The re-arm itself SUCCEEDED, so this stays a + 0, with the run armed and the resume command named. + + The advance-failed leg is the control, and it is what makes this a narrowing rather + than "warnings stop resumes": it is the most actionable degrade the re-arm has, it + prints its own "before resuming" imperative, and it still resumes — because nothing + about it proves the re-drive cannot route. + + Ablation: drop the `if hold_resume:` arm from `cmd_resolve` and the first leg + reddens on `assert [] == ['r1']`; make `runs.rearm_holds_the_resume` answer True for + every entry and the control leg reddens instead. Discard `_echo_rearm_events`' return + (keep `hold_resume = False`) and the first leg reddens alone. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + def rearm_journalling(kind, **fields): + def fake_rearm(rd, key, *, restore_patch=None): + Journal(rd).append(kind, story_key=key, **fields) + return key + + return fake_rearm + + resumed: list[str] = [] + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: resumed.append(rd.name) or 0) + + _escalated_run(tmp_path, "r1") + monkeypatch.setattr( + runs, + "rearm_escalation", + rearm_journalling( + "rearm-spec-write-unreachable", spec_file="wt/specs/s1.md", status="ready-for-dev" + ), + ) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 0 + ) + out, err = capsys.readouterr() + assert "Commit the corrected spec before resuming" in err + assert "NOT resuming in this gesture" in out + assert "bmad-loop resume r1" in out # the escape hatch, reachable once it is committed + assert resumed == [] # the gesture stopped; the story stays armed and resumable + + _escalated_run(tmp_path, "r2") + monkeypatch.setattr( + runs, + "rearm_escalation", + rearm_journalling( + "rearm-baseline-advance-failed", + repo=str(tmp_path), + baseline="a" * 40, + error="GitError: not a git repository", + ), + ) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r2", "--no-interactive", "--resume"]) == 0 + ) + out, err = capsys.readouterr() + assert "Check the baseline before resuming" in err # equally imperative... + assert "NOT resuming in this gesture" not in out + assert resumed == ["r2"] # ...and it still resumes: an advisory is not a proof + + +def test_resolve_appends_the_next_step_imperative(tmp_path, monkeypatch, capsys): + """This surface renders `severity: message; next_step`; the TUI renders `message`. + + That split is the entire reason `runs.rearm_event_notice` returns three fields + instead of one formatted line — the TUI re-arms and RESUMES in a single gesture, + so "before resuming" is already moot there, while `resolve` leaves the run parked + and the imperative is the actionable half. Dropping it here would leave the field + with no observable purpose on either surface and nothing red to say so. + + Graded both ways in one run, because the empty string is a real table value: a + row that carries a next_step prints it after `; `, and a row that carries "" must + print no dangling separator. + + Ablation: drop `tail` from `_echo_rearm_events`' f-string and the first assertion + reddens; hard-code it to `f"; {next_step}"` and the second does. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + _escalated_run(tmp_path, "r1") + + def fake_rearm(rd, key, *, restore_patch=None): + journal = Journal(rd) + journal.append( # table row with a next_step + "rearm-baseline-advance-failed", + story_key=key, + repo=str(tmp_path), + baseline="a" * 40, + error="GitError: not a git repository", + ) + journal.append( # table row whose next_step is "" + "stale-restore-commits", story_key=key, old_baseline="f" * 40, commits=["c1"] + ) + return key + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 0 + ) + + lines = capsys.readouterr().err.splitlines() + advance = next(ln for ln in lines if "could not advance the re-drive baseline" in ln) + assert advance.startswith("warning: ") + assert advance.endswith("; Check the baseline before resuming") + commits = next(ln for ln in lines if "commit(s) sit below" in ln) + assert commits.endswith("revert them now") + + def test_resolve_interactive_runs_session_then_rearms(tmp_path, monkeypatch): from bmad_loop import resolve from bmad_loop.journal import load_state @@ -3665,6 +4183,106 @@ def test_resume_restamps_policy_snapshot_for_sweep_runs(project, monkeypatch): assert load_state(run_dir).cache_read_weight() == 0.5 +# ------------------------------------------- resume re-stamps the code root + +# `repo_root:` is the one ProjectPaths member that decides which git TREE the run +# works in, and resume re-reads config.yaml, so it is also the one the persisted +# mirror (`RunState.repo_root`, which `runs.rearm_escalation` reads back out of +# process) can fall behind. The three rows below are the whole contract: a move is +# adopted and announced, a run that did not move reports nothing, and a state.json +# from before the field existed migrates without being called a move. + + +def _configure_repo_root(project, root: Path) -> None: + """Add a `repo_root:` override to the config.yaml `install_bmad_config` wrote. + Appended rather than rewritten so the artifact keys keep their real values — + `load_paths` requires them, and a stub would move what the test is not about.""" + cfg = project.project / "_bmad" / "bmm" / "config.yaml" + cfg.write_text( + cfg.read_text(encoding="utf-8") + f"repo_root: '{root.as_posix()}'\n", encoding="utf-8" + ) + + +def test_resume_restamps_the_code_root_when_the_config_moved(project, monkeypatch, capsys): + """Resume adopts the code root now on disk, so the mirror must follow it. + + `compose_resume` builds the Workspace off the freshly loaded `paths`, never off + state.json, so after a `repo_root:` edit the engine works in the new tree while + `runs.rearm_escalation` — a separate process, reading `RunState.code_root` — would + still advance the attempt baseline and re-stamp `baseline_revision` in the old one. + Two readers, two trees, no error: `resolve` would report a re-arm that armed the + wrong repository. + + Durable BEFORE the engine starts, like the snapshot and the digest beside it: + `rearm_escalation` reads the file, and `Engine._save()` may not fire for minutes. + + Ablation: delete `state.repo_root = str(paths.repo_root)` from `_resume_paused_run` + and this reddens on the stale root (the legacy-migration row below reddens with + it); neutralize `code_root_changed` to a literal `False` and it reddens on the + journal field and the warning instead. + """ + seen: list = [] + run_dir = _paused_run_for_resume( + project, monkeypatch, repo_root=str(project.project / "old-code") + ) + moved = project.project / "moved-code" + moved.mkdir() + _configure_repo_root(project, moved) + monkeypatch.setattr(cli, "Engine", _state_reading_engine(seen)) + + assert cli._resume_paused_run(project.project, run_dir) == 0 + + (at_start,) = seen + assert at_start.code_root == moved.resolve() + assert _resume_entry(run_dir)["code_root_changed"] is True + err = capsys.readouterr().err + assert "the code root in _bmad/bmm/config.yaml has changed" in err + # the warning names neither tree: a journalled scalar, an operator-facing sentence + assert str(moved) not in err + + +def test_resume_reports_no_code_root_change_when_the_config_did_not_move( + project, monkeypatch, capsys +): + """The ordinary resume, and the reason the compare is exact rather than a + canonicalizing one: both sides are `str(paths.repo_root)` off `load_paths`, which + resolves every member or raises, so an unmoved config must compare equal on the + nose. A row that reported a change here would fire the warning on every resume — + the per-configuration constant the `rearm-spec-write-unreachable` narrowing exists + to avoid, one file over. + """ + run_dir = _paused_run_for_resume( + project, monkeypatch, repo_root=str(Path(project.project).resolve()) + ) + monkeypatch.setattr(cli, "Engine", _StubEngine) + + assert cli._resume_paused_run(project.project, run_dir) == 0 + + assert _resume_entry(run_dir)["code_root_changed"] is False + assert "code root" not in capsys.readouterr().err + + +def test_resume_migrates_a_legacy_state_without_calling_it_a_move(project, monkeypatch, capsys): + """A state.json written before `repo_root` existed reads back "" — a MISSING value, + not a divergent one. The `bool(state.repo_root)` guard is what keeps it out of the + compare, and the unconditional re-stamp is what migrates it onto the root the run + was already using. + + Ablation: drop that guard and this reddens on both the journal field and the + warning, on every legacy run's first resume — the exact false alarm the guard buys. + """ + from bmad_loop.journal import load_state + + run_dir = _paused_run_for_resume(project, monkeypatch) # no repo_root: pre-field + monkeypatch.setattr(cli, "Engine", _StubEngine) + + assert cli._resume_paused_run(project.project, run_dir) == 0 + + assert load_state(run_dir).repo_root == str(Path(project.project).resolve()) + assert _resume_entry(run_dir)["code_root_changed"] is False + assert "code root" not in capsys.readouterr().err + + def test_resume_tolerates_a_corrupt_sweep_json(project, monkeypatch): """A torn/corrupt sweep.json (a crash mid-write on an older run) must not abort resume — the recovery path. compose_resume guards the read and falls back to the diff --git a/tests/test_devcontract.py b/tests/test_devcontract.py index ee12b35e..aa46c375 100644 --- a/tests/test_devcontract.py +++ b/tests/test_devcontract.py @@ -7,6 +7,15 @@ import pytest +# The shared "this key is absent entirely" marker, for `legacy_baseline`. Imported +# rather than redefined: `conftest.write_spec` states the same rule for the same +# reason, and two sentinels for one contract is one place for them to disagree. A +# plain `None` cannot serve: there it means the YAML-null shape (a bare +# `baseline_commit:` line), a distinct case the reader must treat as absent WITHOUT +# turning it into the token "None" (#358). +from conftest import OMIT as _OMIT +from conftest import _Omit + from bmad_loop import devcontract, platform_util, verify @@ -16,14 +25,27 @@ def _spec( status: str = "done", baseline_field: str = "baseline_revision", baseline: str = "abc123def456abc123def456abc123def456abcd", + legacy_baseline: str | None | _Omit = _OMIT, auto_run: str | None = "done", body_extra: str = "", followup: bool | None = None, actions: str | None = None, ) -> Path: + """``legacy_baseline`` writes the OTHER baseline key alongside `baseline_field`. + + It exists because this fixture could not previously express the spec shape + `runs.rearm_escalation` actually manufactures: that function inserts + `baseline_revision` and never removes a pre-existing `baseline_commit`, so a + re-armed spec carries BOTH — and until #716 the precedence between them was + therefore untestable here. `_OMIT` writes no second key (the default, and what + every pre-existing caller gets); `None` writes a bare `baseline_commit:` line; + any other value is written as a quoted scalar, `""` included.""" fm = f"---\ntitle: 'x'\ntype: 'feature'\nstatus: '{status}'\n" if baseline: fm += f"{baseline_field}: '{baseline}'\n" + if legacy_baseline is not _OMIT: + other = "baseline_commit" if baseline_field == "baseline_revision" else "baseline_revision" + fm += f"{other}:\n" if legacy_baseline is None else f"{other}: '{legacy_baseline}'\n" if followup is not None: fm += f"followup_review_recommended: {str(followup).lower()}\n" if actions is not None: @@ -163,6 +185,69 @@ def test_synth_success_maps_baseline_revision(tmp_path): assert "dw_ids" not in rj +_FRESH_SHA = "b" * 40 +_STALE_SHA = "a" * 40 + + +def test_synth_dual_key_spec_reports_the_fresh_revision(tmp_path): + """#716's other half. `synthesize_result` and `verify._verify_shared_gates` each + carried a copy of `fm.get("baseline_commit", fm.get("baseline_revision", ""))`, + and the whole point of routing both through `frontmatter.auto_dev_baseline_of` + is that they can no longer drift apart — so BOTH halves need a pin, or restoring + the inline expression on this side goes unnoticed. + + The spec shape is the one `runs.rearm_escalation` manufactures: it inserts + `baseline_revision` and never removes a pre-existing `baseline_commit`. The + synthesized result's key is still called `baseline_commit` (that name exists + only in the orchestrator's own result.json), but its VALUE must be the fresh + revision the skill just stamped. + + Ablation: restore the inline `fm.get("baseline_commit", fm.get(...))` and this + reddens with the stale sha. + """ + sp = _spec(tmp_path / "s.md", baseline=_FRESH_SHA, legacy_baseline=_STALE_SHA) + body = sp.read_text(encoding="utf-8") + assert "baseline_revision:" in body and "baseline_commit:" in body # the dual-key shape + assert ( + devcontract.synthesize_result(sp, story_key="1-1-a").result_json["baseline_commit"] + == _FRESH_SHA + ) + + +@pytest.mark.parametrize("legacy", ["", None]) +def test_synth_skips_an_unusable_legacy_key(tmp_path, legacy): + """An EMPTY (`baseline_commit: ''`) or YAML-null (bare `baseline_commit:`) legacy + key must not shadow the fresh claim. `dict.get`'s default fires only on a MISSING + key, so the empty value used to be SELECTED and synthesized as `""` — which every + consumer reads as "no baseline claimed", the state that skips the gate entirely. + + Ablation: drop the `if value:` guard in `auto_dev_baseline_of` and the empty row + reports `""`; drop the `if raw is None` guard and the null row reports `"None"`. + """ + sp = _spec(tmp_path / "s.md", baseline=_FRESH_SHA, legacy_baseline=legacy) + rj = devcontract.synthesize_result(sp, story_key="1-1-a").result_json + assert rj["baseline_commit"] == _FRESH_SHA + + +def test_synth_reads_a_legacy_only_spec(tmp_path): + """Back-compat: a spec predating the rename claims only `baseline_commit`.""" + sp = _spec(tmp_path / "s.md", baseline_field="baseline_commit", baseline=_STALE_SHA) + rj = devcontract.synthesize_result(sp, story_key="1-1-a").result_json + assert rj["baseline_commit"] == _STALE_SHA + + +def test_synth_and_the_verify_gate_read_one_dual_key_spec_identically(tmp_path): + """The drift guard itself: the two consumers must agree by CONSTRUCTION, so the + contract is asserted as an equality between them rather than twice in parallel. + + Ablation: change the precedence on EITHER side alone and this reddens — which is + exactly what neither module's own tests could see before. + """ + sp = _spec(tmp_path / "s.md", baseline=_FRESH_SHA, legacy_baseline=_STALE_SHA) + rj = devcontract.synthesize_result(sp, story_key="1-1-a").result_json + assert rj["baseline_commit"] == verify.auto_dev_baseline_of(verify.read_frontmatter(sp)) + + def test_synth_blocked_frontmatter_becomes_critical(tmp_path): sp = _spec(tmp_path / "s.md", status="blocked", auto_run="blocked") out = devcontract.synthesize_result(sp, story_key="1-1-a") diff --git a/tests/test_diagnostics.py b/tests/test_diagnostics.py index aa75ecac..a0e207bd 100644 --- a/tests/test_diagnostics.py +++ b/tests/test_diagnostics.py @@ -20,7 +20,7 @@ import pytest from bmad_loop import diagnostics, sanitize -from bmad_loop.journal import Journal, save_state +from bmad_loop.journal import Journal, load_state, save_state from bmad_loop.model import Phase, RunState, SessionRecord, StoryTask, TokenUsage from bmad_loop.policy import Policy @@ -29,6 +29,13 @@ STORY_KEY = "1.2-AcmeQuantumBillingEngine" PROPRIETARY = "AcmeQuantumBillingEngine" BRANCH = "feature/AcmeSecret" +# A branch name with NO separator, for the one row that grades branch-field ROUTING. +# `BRANCH` cannot: `scrub_json`'s `_IDENTIFIER_RE` forbids `/`, so a slashed name is +# collapsed to `` by the fallback and a canary sweep over it stays green +# with the routing entry deleted — the same false green `repo` has, documented on +# `test_rearm_journal_fields_are_routed`. Bare `main`/`develop`-style names are the +# common case anyway, and they are exactly the ones the fallback waves through verbatim. +REARM_BRANCH = "AcmeSecretRelease" SECRET_GH = "ghp_CANARYxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx01" SECRET_OPENAI = "sk-CANARYxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx99" SECRET_AWS = "AKIACANARY0123456789" @@ -504,6 +511,210 @@ def test_verify_command_free_text_drops_to_presence_booleans(): assert canary not in rendered, f"LEAK: {canary!r}" +def test_rearm_records_leak_neither_the_code_root_nor_a_spec_name(): + """The two records `runs.rearm_escalation` added must be routed by FIELD NAME, + not left to the `scrub_json` fallback (#640, #716). + + `repo` is an absolute host path naming the run's code tree, and the fallback + fails closed only by accident of shape — `looks_like_identifier` forbids `/`, + so a POSIX root collapses, but a one-segment root would ship verbatim. It is + DROPPED rather than aliased: one run has one code root, so it correlates + nothing, and the `error` field on the same record is already dropped. + + `spec_file` is the customer's feature name (the very hazard + `_JOURNAL_ALIAS_FIELDS`' `spec` entry exists for) and is ALIASED, so a + maintainer can still follow one spec across events. + + `overwritten` and `baseline` are both shas on one record, so aliasing one and + leaving the other would pseudonymize half a comparison — the assertion below + is that BOTH come back aliased and DIFFERENT from each other. + + Ablation: drop `repo` from `_JOURNAL_DROP_FIELDS` and the PRESENCE assertion + below reddens — not the canary sweep, which stays green because `scrub_json` + already collapses an absolute path to `` (verified by running that + ablation: `repo` comes back as `''`, so the canary never appears). + That is the whole point of the drop: the fallback happens to redact THIS path + shape, so only an assertion on the field's absence can grade a routing decision + taken for a shape the fallback would not catch. Drop `spec_file` or `overwritten` + from `_JOURNAL_ALIAS_FIELDS` and the alias assertions redden (`spec_file` on the + canary sweep too). Drop `target_branch` and the branch row reddens on BOTH the alias + lookup and the canary sweep — that field is identifier-shaped by design, so unlike + `repo` the fallback does not accidentally rescue it. + """ + other_sha = "f" * 40 + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + advance_failed = diagnostics._scrub_entry( + { + "ts": 1.0, + "kind": "rearm-baseline-advance-failed", + "story_key": STORY_KEY, + "repo": HOME_PATH, + "baseline": SHA, + "error": f"GitError: git rev-parse HEAD failed in {HOME_PATH}", + }, + pseudo, + {}, + 1.0, + ) + restamped = diagnostics._scrub_entry( + { + "ts": 2.0, + "kind": "rearm-baseline-restamped", + "story_key": STORY_KEY, + "spec_file": SPEC_ABS, + "overwritten": other_sha, + "baseline": SHA, + "restore": False, + }, + pseudo, + {}, + 1.0, + ) + + assert "repo" not in advance_failed and advance_failed["repo_present"] is True + assert "error" not in advance_failed # the sibling that was already routed + # aliased, not dropped: the key stays and the VALUE is replaced, which is what + # keeps the record correlatable across events + alias = next(a for ns, orig, a in pseudo.entries() if ns == "spec" and orig == SPEC_NAME) + assert restamped["spec_file"] == alias + # the absolute spelling reduced to the basename first, so this spec has ONE + # alias and the home path never entered the legend + assert [orig for ns, orig, _a in pseudo.entries() if ns == "spec"] == [SPEC_NAME] + # both shas aliased, and distinguishable from each other + assert restamped["overwritten"] != other_sha and restamped["baseline"] != SHA + assert restamped["overwritten"] != restamped["baseline"] + assert restamped["restore"] is False # a plain flag still ships + + # The OTHER three kinds `runs.rearm_escalation` journals `spec_file` on. Routing is + # by field NAME, so these ride the same `_JOURNAL_ALIAS_FIELDS` entry as + # `rearm-baseline-restamped` and are correct today for free — which is exactly why + # they belong in the sweep: the canary is what catches a field added to one of + # these kinds later, and a sweep that covers two of four grades the routing of a + # record shape nobody re-checks. + siblings = [ + diagnostics._scrub_entry( + {"ts": 3.0, "kind": kind, "story_key": STORY_KEY, "spec_file": SPEC_ABS, **extra}, + pseudo, + {}, + 1.0, + ) + for kind, extra in ( + ("rearm-spec-write-unreachable", {"target_branch": REARM_BRANCH}), + ("rearm-spec-flip-skipped", {"status": "ready-for-dev"}), + ("rearm-baseline-restamp-skipped", {"baseline": SHA}), + ) + ] + # every one of them aliases to the SAME alias as the restamped record above: one + # spec, one alias, however many kinds carry it + assert [s["spec_file"] for s in siblings] == [alias, alias, alias] + assert [orig for ns, orig, _a in pseudo.entries() if ns == "spec"] == [SPEC_NAME] + + # `rearm-spec-write-unreachable` names the branch the re-drive cuts its replacement + # worktree from, so the operator is told WHERE to commit. It is journalled as + # `target_branch` rather than a fresh spelling for exactly this reason: routing is + # by field NAME, and that name is already in `_JOURNAL_ALIAS_FIELDS` under the + # `branch` namespace. Graded on a SEPARATOR-FREE name (see `REARM_BRANCH`) because + # a slashed one dies at `scrub_json` and would pass with the routing deleted. + # + # In a real run `ensure_target_branch` journals the same string as `branch` first, + # so an unrouted spelling would be caught by the egress backstop and disclosed as a + # `backstop_repairs` gap — but a truncated journal missing that event has nothing to + # repair from, and the branch would ship verbatim in a shareable bundle. + branch_alias = next( + a for ns, orig, a in pseudo.entries() if ns == "branch" and orig == REARM_BRANCH + ) + assert siblings[0]["target_branch"] == branch_alias != REARM_BRANCH + + rendered = json.dumps([advance_failed, restamped, *siblings]) + for canary in (SHA, other_sha, SPEC_NAME, PROPRIETARY, HOME_PATH, REARM_BRANCH, *CANARIES): + assert canary not in rendered, f"LEAK: {canary!r}" + + +def test_target_field_routes_by_kind_because_it_carries_two_kinds_of_value(): + """`target` is a BRANCH on the merge kinds and a sprint STATUS on `board-advance-*`. + + That overload is why the field is absent from `_JOURNAL_ALIAS_FIELDS`: routing there + is by field NAME, so a single entry would have to be wrong for one of the two + families. Leaving it unrouted was the wrong half to be wrong on — a bare + `main`/`release`-style branch name is identifier-shaped, so `scrub_json` ships it + VERBATIM into a bundle whose guiding assumption is that it will be posted publicly. + + The backstop is not the answer here. It repairs only values already in the legend, + so it rescues this exactly when `ensure_target_branch` journalled the same string as + `branch` earlier in the same file — and then discloses a `backstop_repairs` gap on a + routine run. A journal truncated past that event has nothing to repair from. + + Both directions are graded, because either alone is satisfied by a wrong fix: + + - the three merge kinds ALIAS, to the branch namespace, and to the SAME alias as the + `branch` field beside them when the value matches — a by-name entry would pass + this too. + - `board-advance-carried` keeps its `target` VERBATIM. A by-name entry reddens here, + rendering the sprint status a maintainer reads that kind for as `branch-`. + + Separator-free names throughout: `_IDENTIFIER_RE` forbids `/`, so a `feature/x` + spelling is collapsed by the FALLBACK and every assertion below would pass with the + routing deleted (the same false green documented for `repo` above). + + Ablation: empty `_JOURNAL_KIND_ALIAS_FIELDS` and the three merge rows redden on the + legend lookup (`StopIteration`); move `"target": "branch"` into + `_JOURNAL_ALIAS_FIELDS` instead and the board-advance row reddens on the status. + """ + target_branch = "AcmeSecretIntegration" + unit_branch = "AcmeSecretUnit" + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + merges = [ + diagnostics._scrub_entry( + { + "ts": 1.0, + "kind": kind, + "story_key": STORY_KEY, + "branch": unit_branch, + "target": target_branch, + }, + pseudo, + {}, + 1.0, + ) + for kind in ("unit-merge-started", "unit-merged", "resume-unit-merge") + ] + board = diagnostics._scrub_entry( + { + "ts": 2.0, + "kind": "board-advance-carried", + "story_key": STORY_KEY, + "target": "done", + "status": "done", + }, + pseudo, + {}, + 1.0, + ) + + target_alias = next( + a for ns, orig, a in pseudo.entries() if ns == "branch" and orig == target_branch + ) + unit_alias = next( + a for ns, orig, a in pseudo.entries() if ns == "branch" and orig == unit_branch + ) + # every merge kind aliases the same target to the same alias — one branch, one alias, + # however many kinds name it + assert [m["target"] for m in merges] == [target_alias] * 3 + # ...and the unit branch beside it stays DISTINGUISHABLE, so a maintainer can still + # read "this branch merged into that one" off the scrubbed record + assert [m["branch"] for m in merges] == [unit_alias] * 3 + assert target_alias != unit_alias + + # the other family keeps the same field verbatim: it is a sprint status, and + # aliasing it would destroy the only thing the record is read for + assert board["target"] == "done" and board["status"] == "done" + assert not [orig for ns, orig, _a in pseudo.entries() if ns == "branch" and orig == "done"] + + rendered = json.dumps([*merges, board]) + for canary in (target_branch, unit_branch, *CANARIES): + assert canary not in rendered, f"LEAK: {canary!r}" + + def test_structure_is_preserved(project): run_dir = _seed_run(project.project) diag, _pseudo, _combined = _render_all([run_dir]) @@ -1338,3 +1549,117 @@ def test_env_git_version_is_none_when_the_probe_fails(monkeypatch): env = diagnostics.collect_env(ANY_PROJECT) assert env.git_version is None + + +def test_diag_surfaces_the_split_code_root_and_the_task_generation(project): + """The two state fields this wave ADDED must be legible in a bug report. + + `RunState.repo_root` and `StoryTask.generation` are new here, and both name the + exact conditions under which the re-anchored gates and the re-minted session ids + behave differently — yet neither reached the projection. `repo` was simultaneously + dropped from journal records (correctly: an absolute host path), which removed the + last trace of a split root from a dump altogether. + + `generation` matters because without it a #705-class replay dumps as + `rearmed=True, attempt=1, n_sessions=2` — byte-identical to a HEALTHY post-re-arm + task, so the one field that separates "minted a fresh id" from "collided with the + abandoned record" is the one a triager cannot see. + + Both are privacy-safe by construction and are asserted to be: a boolean in the + `paused_reason_present` / `worktree_isolated` style, and a small counter. The path + itself must NOT appear — that is what `_JOURNAL_DROP_FIELDS` drops. + + Ablation: delete `repo_root_diverges=` from `collect_run` (or `generation=` from + `_task_diag`) and this reddens on the corresponding assertion; deleting the field + from the dataclass reddens as a TypeError at construction. + """ + run_dir = _seed_run(project.project) + state = load_state(run_dir) + state.repo_root = str(project.project / "code-tree") + state.tasks[STORY_KEY].generation = 2 + save_state(run_dir, state) + + diag, _pseudo, combined = _render_all([run_dir]) + (run,) = diag.runs + + assert run.repo_root_diverges is True + assert run.tasks[0].generation == 2 + # a presence flag, never the path — the same rule `repo` is dropped under + assert "code-tree" not in combined + + +def test_diag_repo_root_diverges_is_false_for_the_ordinary_layout(project): + """The flag distinguishes; it is not simply always on. + + Without this the assertion above passes for a hardcoded `True`, and the field + stops carrying the one bit it exists to carry. + """ + run_dir = _seed_run(project.project) + diag, _pseudo, _combined = _render_all([run_dir]) + (run,) = diag.runs + + assert run.repo_root_diverges is False + assert run.tasks[0].generation == 0 + + +def _md_task_row(md: str) -> list[str]: + """The one task row of the report's task table, split into its cells.""" + (row,) = [ln for ln in md.splitlines() if ln.startswith("| `")] + return [c.strip() for c in row.strip("|").split("|")] + + +def test_the_markdown_report_carries_the_split_root_and_the_generation(project): + """Both fields must reach the report `diagnose` emits BY DEFAULT, not only `--json`. + + The sibling tests above grade the collector, which is what `--json` dumps whole via + `asdict`. The markdown report is a different renderer that samples fields by hand, + and it is the artifact an operator actually produces and hands a maintainer — the + header says so ("Safe to share"). A field whose entire warrant is "a bug report that + cannot show this cannot be triaged" is not delivered until it renders here, so the + warrant is graded where it is spent. + + `generation` rides beside `attempt` because that is the column pair a #705-class + replay turns on: a collided re-drive and a healthy post-re-arm task agree on every + other cell in this row. + + Ablation: drop the `code root differs from project` line from `render_markdown` and + both this test and the sibling below redden on their first assertion. Drop + `{t.generation}` from the row f-string together with its header and separator cells + and this test reddens at `names[4]` (`"rev" != "gen"`) while the sibling reddens at + the row cell — as `"1" != "0"`, the review cycle shifted left rather than a missing + key, which is why the cell is read positionally and the three widths are compared. + """ + run_dir = _seed_run(project.project) + state = load_state(run_dir) + state.repo_root = str(project.project / "code-tree") + state.tasks[STORY_KEY].generation = 2 + save_state(run_dir, state) + + pseudo = sanitize.Pseudonymizer() + diag = diagnostics.collect([run_dir], pseudo=pseudo, project=ANY_PROJECT) + md = diagnostics.render_markdown(diag, pseudo=pseudo) + + assert "- **code root differs from project:** yes" in md + cells = _md_task_row(md) + (header,) = [ln for ln in md.splitlines() if ln.startswith("| alias |")] + (rule,) = [ln for ln in md.splitlines() if ln.startswith("|---|")] + names = [c.strip() for c in header.strip("|").split("|")] + assert names[4] == "gen" + # header, separator and row must agree on width or the table renders skewed + assert len(cells) == len(names) == len(rule.strip("|").split("|")) == 12 + assert cells[3] == "2" # attempt, seeded by `_seed_run` + assert cells[4] == "2" # generation — NOT the review cycle, which is 1 + assert cells[5] == "1" # review cycle, still in its own column + # still a flag and a counter: the path itself never renders + assert "code-tree" not in md + + +def test_the_markdown_report_says_no_for_the_ordinary_layout(project): + """The rendered line distinguishes; a hardcoded "yes" would pass the test above.""" + run_dir = _seed_run(project.project) + pseudo = sanitize.Pseudonymizer() + diag = diagnostics.collect([run_dir], pseudo=pseudo, project=ANY_PROJECT) + md = diagnostics.render_markdown(diag, pseudo=pseudo) + + assert "- **code root differs from project:** no" in md + assert _md_task_row(md)[4] == "0" diff --git a/tests/test_engine.py b/tests/test_engine.py index 1788d5b5..88743b9c 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -39,12 +39,15 @@ from bmad_loop.adapters.base import SessionResult from bmad_loop.adapters.mock import MockAdapter from bmad_loop.engine import ( + NOTICE_REASON_MAX, Engine, RunPaused, RunStopped, _digest_of, _LedgerAnchor, + _notice_reason, _run_depth, + _session_task_id, ) from bmad_loop.journal import LOGS_DIR, VERIFY_DIR, Journal, load_state from bmad_loop.model import ( @@ -2078,6 +2081,344 @@ def test_resumable_session_matches_sanitized_task_id(project, key): assert result.result_json == {"status": "done"} +def test_session_task_id_is_byte_identical_at_generation_zero(): + """The upgrade contract (#705): an in-flight run resumed across it must find its + `tasks/` directories, so generation 0 renders as no suffix AT ALL — not `-g0`. + + Ablation: emit the suffix unconditionally and this reddens, which is the shape + that would strand every existing run's session directory. + """ + # `generation` is REQUIRED, so there is no omitted spelling to assert: omitting it + # is a type error, which is the point (an implicit 0 is right in every run that + # never re-armed and silently re-opens #705 at a new mint site). + assert _session_task_id("1-1-a", "dev", 1, 0) == "1-1-a-dev-1" + assert _session_task_id("1-1-a", "dev", 1, 1) == "1-1-a-dev-1-g1" + # composed INSIDE the f-string: the sanitizer still sees one whole segment, so + # a key long enough to overflow the cap comes back capped WITH the suffix folded + # in rather than appended past it. + long_key = "k" * 130 + long_ids = [_session_task_id(long_key, "dev", 1, g) for g in (0, 1, 2)] + assert all(len(i) <= platform_util.MAX_SEGMENT for i in long_ids) + # ... and the generations stay DISTINCT through that fold. This is the one thing + # that could silently fail here: over the cap the suffix is not appended, it is + # truncated away and the id ends in `safe_segment`'s digest of the raw input + # instead — so distinctness rests on the digest seeing the suffix, not on the + # suffix surviving. A cap applied AFTER composition would collapse all three to + # one id and re-open #705 for exactly the long keys that already stress the path. + assert len(set(long_ids)) == 3, long_ids + + +def test_resumable_session_ignores_a_pre_rearm_record(project): + """#705. `rearm_escalation` resets `attempt` to 0 and deliberately does NOT + clear `task.sessions`, so the next dispatch's `attempt += 1` re-mints an id + byte-equal to a record the ABANDONED attempt already appended. A host death in + the window between that bump and `record_session` then resumes into + `_resumable_session`, which matches on the id and replays the abandoned + attempt's verdict for a session that never ran — the run wedges re-deciding a + story on a stale result. + + The collision is asserted, not assumed: the first two lines below show the + pre-re-arm record's id is exactly what generation 0 would mint now. + + Ablation: drop the generation argument from `_resumable_session`'s + `_session_task_id` call (or the suffix from the composition) and the stale + record matches again — this row reddens with a replayed result. + """ + write_sprint(project, {"1-1-a": "ready-for-dev"}) + engine, _ = make_engine( + project, [SessionResult(status="completed", result_json={"status": "abandoned"})] + ) + task = StoryTask(story_key="1-1-a", epic=1) + engine.state.tasks[task.story_key] = task + engine._save() + engine._run_session(task, role="dev", prompt="p", seq=1) + (stale,) = task.sessions + assert stale.task_id == _session_task_id(task.story_key, "dev", 1, 0) + assert stale.status == "completed" and stale.result_json is not None + + # the human re-arm, then the re-drive's first dispatch, then a host kill in the + # window before `record_session` (engine.py: `attempt += 1` … advance … _save) + task.generation += 1 + task.attempt = 1 + task.phase = Phase.DEV_RUNNING + + assert engine._resumable_session(task) is None + assert task.sessions == [stale] # the audit trail is kept, just not matched + + +def test_resumable_session_matches_within_the_same_generation(project): + """The counterpart the row above needs to be worth anything: a re-armed task + that DOES record a session under its new generation still resumes from it, so + the discriminator has not simply disabled crash replay.""" + write_sprint(project, {"1-1-a": "ready-for-dev"}) + engine, _ = make_engine( + project, [SessionResult(status="completed", result_json={"status": "done"})] + ) + task = StoryTask(story_key="1-1-a", epic=1, generation=1) + engine.state.tasks[task.story_key] = task + engine._save() + engine._run_session(task, role="dev", prompt="p", seq=1) + task.phase = Phase.DEV_RUNNING + task.attempt = 1 + + resumable = engine._resumable_session(task) + assert resumable is not None and resumable[1].result_json == {"status": "done"} + + +def test_current_dev_session_index_follows_the_generation(project): + """The third mint site (`_current_dev_session_index`, which drives + `accepted_dev_session_index`) must move with the other two: left at generation + 0 it would point the PROCEED receipt at the abandoned attempt's record.""" + write_sprint(project, {"1-1-a": "ready-for-dev"}) + engine, _ = make_engine( + project, + [ + SessionResult(status="completed", result_json={"status": "abandoned"}), + SessionResult(status="completed", result_json={"status": "done"}), + ], + ) + task = StoryTask(story_key="1-1-a", epic=1) + engine.state.tasks[task.story_key] = task + engine._save() + engine._run_session(task, role="dev", prompt="p", seq=1) # pre-re-arm record + + task.generation += 1 + task.attempt = 1 + assert engine._current_dev_session_index(task) is None # the stale record is not it + + engine._run_session(task, role="dev", prompt="p", seq=1) # the re-drive's record + assert engine._current_dev_session_index(task) == 1 + + +def test_notice_reason_caps_a_long_single_line_and_marks_the_trim(): + """The LENGTH half of `_notice_reason`'s contract, which no engine-level row + reaches. + + `test_dev_retry_notice_collapses_a_multiline_reason` grades only the first-line + collapse: its long run sits on the third line, so `first` never exceeds the cap + and its `len(retry) < 300` bound passes for any cap value — including none. A + single-line reason is the shape that actually crosses it: a `[verify] commands` + entry whose invocation carries many paths makes `verify command failed (rc=N): + ` one line well past the cap, and with the cap gone that whole line + lands verbatim in ATTENTION and in the toast payload. + + Pinned as a direct unit row rather than through the engine because the cap is a + property of the helper, and routing a 500-char command through a dev session to + observe it would grade the plumbing instead. + + Ablation: delete the `if len(first) > NOTICE_REASON_MAX:` block and this reddens + on the length assertion; the sibling engine row stays green. + """ + capped = _notice_reason("z" * 500) + assert capped == "z" * NOTICE_REASON_MAX + " […]" + assert len(capped) == NOTICE_REASON_MAX + len(" […]") + + # exactly at the cap is not a trim — the marker would otherwise claim a cut that + # did not happen, which is the ambiguity the marker exists to remove + assert _notice_reason("z" * NOTICE_REASON_MAX) == "z" * NOTICE_REASON_MAX + + # the reasonless RETRY: the helper returns "" so the call site's `or` fallback + # ("dev attempt rejected with no reason recorded") is what reaches the operator + assert _notice_reason("") == "" + assert _notice_reason(" \n\n ") == "" + + +def test_dev_retry_notice_collapses_a_multiline_reason(project, monkeypatch): + """The FIXABLE leg, which is where a `Decision.reason` is routinely multi-line: + `verify.verify_command_results_outcome` appends the captured output tail below + the command line on purpose, because the repair session reads that tail as its + feedback. + + `gates.notify` writes exactly one `[stamp] title: message` line and hands the + same string to a desktop toast, so a raw reason spills a whole build log into + ATTENTION as many un-prefixed lines — breaking the file's own grammar for every + later reader — and into a notification bubble. The sibling row above passes + without this only because a baseline mismatch happens to be single-line. + + Nothing is lost: the untruncated reason is in the `dev-decision` journal entry, + which this asserts explicitly so the trim can never be mistaken for a drop. + + Ablation: pass `decision.reason` raw and the one-line assertion reddens with the + tail's lines loose in the file. + """ + write_sprint(project, {"epic-1": "backlog", "1-1-a": "ready-for-dev"}) + engine, _ = make_engine( + project, + [ + dev_effect(project, "1-1-a", followup_review=False), + dev_effect(project, "1-1-a", followup_review=False), + ], + policy=Policy( + gates=GatesPolicy(mode="none"), + notify=QUIET, + review=ReviewPolicy(enabled=False), + limits=LimitsPolicy(max_dev_attempts=2), + verify=VerifyPolicy(commands=["pytest -q"]), + ), + ) + tail = "FAILED tests/test_a.py::test_one\nFAILED tests/test_b.py::test_two\n" + "x" * 400 + # fail exactly the first invocation, pass every later one: the run makes more + # verify calls than the retry alone (the commit gate re-runs them), and a + # fixed-length script exhausts into a StopIteration crash rather than the + # single retry this row is about. + failing = iter([[verify.CommandResult("pytest -q", 1, tail)]]) + monkeypatch.setattr( + verify, + "run_verify_commands", + lambda policy, cwd: next(failing, [verify.CommandResult("pytest -q", 0, "ok")]), + ) + + assert engine.run().done == 1 + + attention = (engine.run_dir / "ATTENTION").read_text(encoding="utf-8") + lines = [ln for ln in attention.splitlines() if ln.strip()] + # every line the notice wrote still carries the file's `[stamp] title: ...` shape + assert all(ln.startswith("[") for ln in lines), attention + (retry,) = [ln for ln in lines if "dev retry: 1-1-a" in ln] + assert "verify command failed (rc=1): pytest -q" in retry + assert retry.endswith("[…]") # the trim is marked, not silent + assert "FAILED tests/test_a.py" not in attention # the tail stayed out + assert len(retry) < 300 + + # ... and the whole reason is still on the record a maintainer reads + (decision,) = [ + e + for e in engine.journal.entries() + if e["kind"] == "dev-decision" and e["action"] == "retry" + ] + assert "FAILED tests/test_b.py::test_two" in decision["reason"] + + +def test_harvest_gate_exclude_is_rooted_on_the_code_tree(project, tmp_path): + """The engine's own ledger append is excluded from proof of work by PATH, and + that path must be relative to the tree the gate invokes git in. + + Under a `repo_root` override the ledger sits outside the code tree, where it + cannot satisfy proof-of-work at all — so `()` is the right answer. A + `project`-relative pathspec would instead be resolved by git against the CODE + tree and silently exclude whatever happens to live at that relative path there. + + Ablation: put the relpath back on `paths.project` and the second half reddens + with the project-relative ledger entry. + """ + from bmad_loop.workspace import Workspace + + engine, _ = make_engine(project, []) + task = StoryTask(story_key="1-1-a", epic=1) + task.harvest_wrote_ledger = True + + # default config: the two roots are the same object, ledger inside the tree + assert engine._harvest_gate_exclude(task) == ( + "_bmad-output/implementation-artifacts/deferred-work.md", + ) + + art = tmp_path / "artifacts-root" + (art / "_bmad-output" / "implementation-artifacts").mkdir(parents=True) + diverged = dataclasses.replace( + project, + project=art, + implementation_artifacts=art / "_bmad-output" / "implementation-artifacts", + planning_artifacts=art / "_bmad-output" / "planning-artifacts", + output_folder=art / "_bmad-output", + repo_root=project.project, + ) + engine.workspace = Workspace(root=project.project, paths=diverged) + assert engine._harvest_gate_exclude(task) == () + + +def test_harvest_gate_exclude_degrade_arm_is_rooted_on_the_code_tree( + project, tmp_path, monkeypatch +): + """The DEGRADE arm of the same exclude follows the same root. + + The row above grades the resolved path. A filesystem resolve fault takes the + LEXICAL fallback instead, and that spelling moved with it — but no row observed + the move: the existing fault-injection row runs on the `project` fixture, where + the two roots are the same object, and the row above injects no fault. + + Under the override the ledger sits outside the code tree, so `()` is the honest + answer. A `paths.project` spelling SUCCEEDS lexically there and emits a relpath + that git, running in the code tree, resolves against the wrong directory — + silently excluding whatever happens to sit at that relative path in the checkout, + which turns a real change into "no changes since baseline". Uncertainty must not + turn the engine's own append into session proof, and it must not turn a session's + work into nothing either. + + Ablation: put the lexical fallback back on `paths.project` and this reddens with + the project-relative ledger entry. + """ + from bmad_loop.workspace import Workspace + + engine, _ = make_engine(project, []) + task = StoryTask(story_key="1-1-a", epic=1) + task.harvest_wrote_ledger = True + + art = tmp_path / "artifacts-root" + (art / "_bmad-output" / "implementation-artifacts").mkdir(parents=True) + diverged = dataclasses.replace( + project, + project=art, + implementation_artifacts=art / "_bmad-output" / "implementation-artifacts", + planning_artifacts=art / "_bmad-output" / "planning-artifacts", + output_folder=art / "_bmad-output", + repo_root=project.project, + ) + engine.workspace = Workspace(root=project.project, paths=diverged) + + real_resolve = Path.resolve + + def resolve_fault(self, *args, **kwargs): + if self == diverged.deferred_work: + raise OSError("injected ledger resolve fault") + return real_resolve(self, *args, **kwargs) + + monkeypatch.setattr(Path, "resolve", resolve_fault) + assert engine._harvest_gate_exclude(task) == () + + +def test_dev_retry_notifies_the_operator_with_the_reason(project): + """#640(d): RETRY was the only dev outcome that notified nothing, and it is the + outcome that DISCARDS a completed implementation — the non-fixable leg rolls the + tree back to baseline. The reason lived only in the `dev-decision` journal line, + so a run could burn its whole attempt budget throwing finished work away with + nothing on the operator's phone. + + Ablation: delete the `gates.notify` call at the top of the RETRY branch and this + reddens alone. Ablate the CONTENT instead — pass a fixed string in place of + `decision.reason` — and it reddens on the reason assert, which is the point of + asserting the reason at all: "retry, attempt 1" tells a human nothing about + whether to intervene. + """ + write_sprint(project, {"epic-1": "backlog", "1-1-a": "ready-for-dev"}) + engine, _ = make_engine( + project, + [ + _baseline_liar_effect(project), # non-fixable: rejected AFTER the work + dev_effect(project, "1-1-a", followup_review=False), + ], + policy=_harvest_policy(attempts=2), + ) + + assert engine.run().done == 1 + + attention = (engine.run_dir / "ATTENTION").read_text(encoding="utf-8") + retries = [line for line in attention.splitlines() if "dev retry: 1-1-a" in line] + assert len(retries) == 1 # exactly the one rejected attempt, not the accepted one + assert "(attempt 1)" in retries[0] + assert "does not match orchestrator-recorded baseline" in retries[0] + # the decision the notice describes is the one the journal recorded + (decision,) = [ + e + for e in engine.journal.entries() + if e["kind"] == "dev-decision" and e["action"] == "retry" + ] + # the notice carries the reason's FIRST LINE, which is what `_notice_reason` + # promises — asserting the whole untrimmed reason passes only while that reason + # happens to stay single-line and under `NOTICE_REASON_MAX`, so it would go green + # for the wrong reason the moment a producer appended an evidence tail. + assert decision["reason"].splitlines()[0].strip() in retries[0] + + def test_token_budget_discounts_cache_reads(project): """Raw totals dominated by cache reads must not trip the budget; the weighted total (cache reads at 0.1x) is what's checked.""" @@ -15257,3 +15598,33 @@ def test_a_park_record_rollback_refused_as_unconfined_is_journaled(project): assert "UnconfinedWriteError" in journal # journaled by NAME, not a bare errno escaped = _json.loads((outside / record.name).read_text(encoding="utf-8")) assert escaped["actions"] == ACTIONS # this run's record, NOT the prior put back + + +def test_notice_reason_bound_is_an_upper_bound_not_an_equality(): + """`NOTICE_REASON_MAX + len(" […]")` is a ceiling the return need not attain. + + The sibling row pins it with `"z" * 500` — whitespace-free, so the slice never + rstrips and the equality holds. Two shapes make it strictly less, and the comment + on the constant used to state the bound as though neither existed: + + * a cut landing on whitespace, since the slice is `.rstrip()`ed; + * ANY multi-line reason, since `trimmed` is set by the line collapse regardless of + length — which is the common case, `verify.verify_command_results_outcome` + putting its output tail under a short classification line. + + Behaviour is correct in every case; what was wrong was the claim about it, and a + test that pins one whitespace-free instance cannot tell the claim from the truth. + + Ablation: this row grades the COMMENT, so the meaningful ablation is textual — + restore "runs to NOTICE_REASON_MAX + len(...)" without "AT MOST" and the assertions + below contradict it. For the code half, delete the `.rstrip()` and the + whitespace-boundary assertion reddens on `len(capped) == 204`. + """ + capped = _notice_reason("a" * (NOTICE_REASON_MAX - 1) + " " + "b" * 300) + assert capped.endswith(" […]") # a trim happened, and is marked + assert len(capped) < NOTICE_REASON_MAX + len(" […]") # yet lands BELOW the bound + assert not capped.startswith("a" * NOTICE_REASON_MAX) # because the slice rstripped + + short = _notice_reason("short first line\nthe evidence lives here") + assert short == "short first line […]" # marked well under the cap + assert len(short) < NOTICE_REASON_MAX diff --git a/tests/test_frontmatter.py b/tests/test_frontmatter.py index 3ac67136..8fd6708b 100644 --- a/tests/test_frontmatter.py +++ b/tests/test_frontmatter.py @@ -731,3 +731,62 @@ def test_set_frontmatter_status_refuses_a_readonly_spec(tmp_path): assert spec.read_bytes() == _PLAIN.encode("utf-8") assert list((root / "artifacts").glob("*.tmp")) == [] # a refusal stages nothing + + +_FRESH = "b" * 40 +_STALE = "a" * 40 + + +@pytest.mark.parametrize( + "fm,expected", + [ + # the key the skill actually stamps, alone + ({"baseline_revision": _FRESH}, _FRESH), + # legacy-only spec: the orchestrator's own name, kept readable + ({"baseline_commit": _STALE}, _STALE), + # THE #716 CASE. `rearm_escalation` inserts `baseline_revision` and never + # removes a pre-existing `baseline_commit`, so a re-armed spec carries both. + # The replaced expression was `fm.get("baseline_commit", fm.get(...))`, which + # ranked the leftover FIRST and failed an attempt that did everything right. + ({"baseline_revision": _FRESH, "baseline_commit": _STALE}, _FRESH), + # `dict.get`'s default fires only on a MISSING key, so an empty legacy value + # was SELECTED and yielded "" — which every consumer reads as "no claim", + # disabling the baseline-match gate outright. + ({"baseline_revision": _FRESH, "baseline_commit": ""}, _FRESH), + # the fallback is on the VALUE, not the key: an empty fresh key defers + ({"baseline_revision": "", "baseline_commit": _STALE}, _STALE), + # YAML-null on either key is absent, never the token "None" (#358) + ({"baseline_commit": None}, ""), + ({"baseline_revision": None, "baseline_commit": _STALE}, _STALE), + ({"baseline_revision": None, "baseline_commit": None}, ""), + # A YAML boolean is the same trap class: PyYAML resolves `no`/`off`/`false` + # to False, `str(False)` is the token "False", and the truthiness test runs on + # that STRING — so an unguarded bool reads back as a claimed sha. + ({"baseline_revision": False}, ""), + ({"baseline_revision": True}, ""), + ({"baseline_commit": False}, ""), + # the sharp case: a bool on the WINNING key must defer to a valid legacy sha + # rather than shadow it. Unguarded this returns "False", which the gate's + # non-empty filter admits and `_canonical_commit_oid` then rejects — refusing + # an attempt whose correct baseline was sitting on the very next line. + ({"baseline_revision": False, "baseline_commit": _STALE}, _STALE), + ({"baseline_revision": True, "baseline_commit": _STALE}, _STALE), + ({"baseline_revision": f" {_FRESH} "}, _FRESH), # stripped + ({}, ""), # claims nothing + ({"baseline_revision": 123}, "123"), # a non-string scalar still reads back + ], +) +def test_auto_dev_baseline_of_precedence(fm, expected): + """The one reader both consumers of a claimed baseline go through (#716). + + Lives here rather than in `test_verify.py` (which reached it through verify's + re-export) because AGENTS.md has the flat `tests/` mirror src modules by name and + the symbol is defined in `frontmatter.py`. + + Ablation for the negative rows: delete the ``raw is None`` guard and the + YAML-null rows read back the token ``"None"``; delete the ``isinstance(raw, bool)`` + guard and the boolean rows read back ``"True"``/``"False"``, including in place of + the legacy sha they must defer to; delete the ``if value:`` guard and the + empty-legacy-key row reads back ``""``. + """ + assert frontmatter.auto_dev_baseline_of(fm) == expected diff --git a/tests/test_model.py b/tests/test_model.py index d23e96a6..6bc1850d 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -2,6 +2,7 @@ import binascii import json +from pathlib import Path import pytest @@ -38,6 +39,27 @@ def test_run_state_stories_fields_default_and_round_trip(): assert back.spec_folder == "_bmad-output/epic-1" +def test_run_state_repo_root_round_trips_and_backs_code_root(): + """The git root a run's code work happens in, persisted because + `runs.rearm_escalation` runs OUT OF PROCESS from the engine and had only + `project` to reach for.""" + state = _state(repo_root="/code") + back = RunState.from_dict(state.to_dict()) + assert back.repo_root == "/code" + assert back.code_root == Path("/code") + + +def test_run_state_code_root_falls_back_to_project_for_legacy_state(): + """A state.json written before the field existed reads back empty, and + `code_root` then answers `project` — exactly the pre-upgrade behavior, and the + correct answer for every run without a `repo_root:` override.""" + d = _state().to_dict() + del d["repo_root"] # state.json from before the field existed + back = RunState.from_dict(d) + assert back.repo_root == "" + assert back.code_root == Path("/p") + + def test_run_state_stories_fields_default_when_absent_from_dict(): # a pre-stories state.json (no source/spec_folder keys) reads as sprint mode d = _state().to_dict() @@ -176,6 +198,20 @@ def test_followup_reviews_spent_defaults_zero_for_legacy_state(): assert StoryTask.from_dict(doc).followup_reviews_spent == 0 +def test_generation_round_trips(): + task = StoryTask(story_key="1-1-a", epic=1, generation=2) + assert StoryTask.from_dict(task.to_dict()).generation == 2 + + +def test_generation_defaults_zero_for_legacy_state(): + """A run in flight across the upgrade must resume at generation 0, which is the + value `engine._session_task_id` renders as no suffix at all — so every task id + already on disk still matches and its `tasks/` directory is still found (#705).""" + doc = StoryTask(story_key="1-1-a", epic=1).to_dict() + del doc["generation"] # state.json from before the field existed + assert StoryTask.from_dict(doc).generation == 0 + + def test_resolved_redrive_round_trips(): task = StoryTask(story_key="1-1-a", epic=1, resolved_redrive=True) assert StoryTask.from_dict(task.to_dict()).resolved_redrive is True diff --git a/tests/test_resolve.py b/tests/test_resolve.py index eef31889..78d86ab2 100644 --- a/tests/test_resolve.py +++ b/tests/test_resolve.py @@ -40,16 +40,34 @@ def _escalated_run( source="sprint-status", sentinel_kind="", worktree_path="", + baseline_commit="abc123", + restore_patch=None, + repo_root=None, + target_branch=None, ): """conftest's builder with this module's shape: a review-cycle-1 task carrying a - completed review session (what `build_context` reads), returning the full triple.""" + completed review session (what `build_context` reads), returning the full triple. + + ``baseline_commit`` / ``restore_patch`` are forwarded for the rows that need a + task ALREADY carrying a latch and a real sha before re-arm runs — the state + `_stale_restore_residue` reads (`old_latch`, `old_baseline`) and returns early + without touching git when the latch is absent. + + ``repo_root`` records the divergent CODE tree on the saved state, so the + divergent-root rows do not each re-open, mutate and re-save the file they just + built. A row that forgot that save would silently degrade into the same-root + case it exists to distinguish, which is the failure this kwarg removes. + ``target_branch`` is there for the same reason and with the same hazard: it pins + the branch the isolated re-drive cuts its fresh worktree from, and leaving it unset + degrades `_redrive_base_ref` to `HEAD` — the pre-fix anchor, which is exactly the + case the rows that pass it exist to separate themselves from.""" run = escalated_run( project, run_id, story_key="6-4-cli-list-command", epic=6, review_cycle=1, - baseline_commit="abc123", + baseline_commit=baseline_commit, started_at="2026-06-13T11:14:29", paused_reason="CRITICAL escalation from review session: names not unique", spec_file=spec_file, @@ -57,7 +75,14 @@ def _escalated_run( source=source, sentinel_kind=sentinel_kind, worktree_path=worktree_path, + restore_patch=restore_patch, ) + if repo_root is not None or target_branch is not None: + if repo_root is not None: + run.state.repo_root = str(repo_root) + if target_branch is not None: + run.state.target_branch = target_branch + save_state(run.run_dir, run.state) return run.run_dir, run.state, run.task @@ -600,6 +625,162 @@ def test_rearm_strips_stale_terminal_section(tmp_path): assert devcontract.find_result_artifact(tmp_path, since_ns=0) is None +def test_rearm_warns_when_an_isolated_tasks_spec_writes_cannot_reach_the_redrive(tmp_path): + """A worktree-isolated task's spec writes land in a directory the re-drive destroys. + + `_task_spec_path` re-anchors the recorded spec on `task.worktree_path`, but a + re-armed task falls to `engine._finish_inflight`'s final arm, which calls + `discard_worktree` and lets `_run_story` mount a fresh one — and the re-driven + session resolves its spec against THAT worktree + (`engine._dispatched_spec_for_attempt` -> `verify.resolve_spec_path(..., + workspace.paths)`, rebased onto the mount under isolation), which checks out tracked + files only. So the re-drive reads the COMMITTED spec and no working-tree write + reaches it — the main checkout's copy included. The writes stay (they are correct + in-place, harmless here); the operator is told, because a flip that cannot land is + the silent re-wedge #640(b) exists to end. + + The main-checkout copy is the row this shape pins, and it is the reason + `_spec_is_shared_with_the_redrive` tests the PROJECT as well as the worktree. That + write does land — the spec is outside `confine_root`, which + `set_frontmatter_status` answers with its plain no-follow arm — but the re-drive + still cannot use it: `engine._dispatched_spec_for_attempt` measures the absolute + path with `verify.spec_within_roots` against `workspace.paths`, rebased onto the + fresh worktree, and a main-checkout path is under none of those roots. "Outside the + worktree" alone would exempt it; only "outside both checkouts" is shared. + + Ablation: delete the `if task.worktree_path:` term and this reddens while the + no-worktree control still passes. Drop the `state.project` half of + `_spec_is_shared_with_the_redrive` and this reddens alone — the shared-artifact-dir + row is the one that must stay silent, and it does. + """ + _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + spec.write_text(SPEC, encoding="utf-8") + run_dir, _, _ = _escalated_run( + tmp_path, spec_file=str(spec), worktree_path=str(tmp_path / "wt" / "u1") + ) + + runs.rearm_escalation(run_dir) + + (rec,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] + assert rec["story_key"] == "6-4-cli-list-command" + assert rec["spec_file"] == str(spec) + # and it is routed to BOTH operator surfaces, not journal-only + severity, message, next_step = runs.rearm_event_notice( + {"kind": "rearm-spec-write-unreachable", **rec} + ) + assert severity == "warning" + assert "commit the corrected spec" in message + assert next_step + + +def test_rearm_does_not_warn_about_unreachable_writes_without_a_worktree(tmp_path): + """The control for the row above: the in-place case is where those writes DO land, + so a record there would fire on every ordinary re-arm.""" + _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + spec.write_text(SPEC, encoding="utf-8") + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) + + runs.rearm_escalation(run_dir) + + assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] == [] + + +@pytest.mark.parametrize("shape", ["ordinary", "no-frontmatter", "already-at-target"]) +def test_rearm_journals_a_status_flip_that_silently_did_nothing(tmp_path, shape): + """`verify.set_frontmatter_status` answers "nothing to change" with `False`, never + an exception. Its return was DISCARDED, so on such a spec the flip no-opped + invisibly: the re-drive was dispatched anyway, step-01 read the unchanged terminal + status, routed the session to "ingest as context, do not resume", and the story + re-wedged with nothing on the record explaining why. The `FrontmatterWriteError` + arm below it covers only the shapes that RAISE; this covers the ones that lie + quietly. + + `False` has FOUR causes, not three — no file, no frontmatter block, no top-level + `status:`, and ALREADY AT THE TARGET — and only the first three are failures. Three + legs, so the record is graded on the distinction rather than on the bool: + + - `no-frontmatter` — a spec that EXISTS and is readable but carries no `---` block: + the writer returns False and leaves the file byte-identical, so the record is the + ONLY thing that distinguishes this from a flip that landed. It also ABORTS, because + the spec IS a readable file here: the routing status is what the re-drive runs on, + and step-01's contract for a spec without one is not a maybe — it HALTs blocked on + `unrecognized status in existing story file`. So the re-arm refuses instead of + burning the escalation on a session that cannot route, and the run is left exactly + as it was found: the task still ESCALATED (nothing is persisted before this point), + and the spec byte-identical down to the `## Auto Run Result` section, whose strip is + sequenced after the check. The other half of that split — a spec that is NOT a file + from here, where the flip's failure says nothing about what the re-drive will read + — is graded by `test_rearm_journals_a_skip_when_the_recorded_spec_is_not_readable`. + - `ordinary` — the control. A spec that really moves must not produce the record, or + the row would pass for a guard that fires on everything. + - `already-at-target` — the REGRESSION leg. A second re-arm, or + `resolve --no-interactive` after a human fixed the spec, hits a spec already at + `ready-for-dev`. The writer returns False and the operator used to be warned the + spec "could not be re-opened" and might re-wedge, while the file was correct. It + must NOT abort either — this is the ordinary flow, and refusing it would wedge + every second resolve cycle. + + Ablation: delete the `if not flipped ...` arm and the `no-frontmatter` leg reddens + on `DID NOT RAISE` while both other legs still pass. Drop the + `status_of(read_frontmatter(...)) != target` conjunct and `already-at-target` + reddens alone, now on that same raise — which is the discrimination this row exists + for, and the leg that shows why the abort had to be narrowed to the same conjunct + the record is. Delete the `raise RearmError(...)` and keep the append and the + no-frontmatter leg reddens on `DID NOT RAISE` while its record assertions still + pass — the record is not the refusal. Move `strip_auto_run_result` back above the + check and the byte-identity assertion reddens alone. + """ + _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + # every leg carries the stale result section, so the strip's SEQUENCING is graded + # rather than assumed: on the two legs that proceed it must be gone, and on the leg + # that aborts it must still be there. + stale = "\n## Auto Run Result\n\nstatus: blocked\n" + body = { + "ordinary": SPEC, + "no-frontmatter": "# Spec\n\nno frontmatter block\n", + "already-at-target": SPEC.replace("status: in-review", "status: ready-for-dev"), + }[shape] + stale + assert shape != "already-at-target" or "status: ready-for-dev" in body # fixture is real + spec.write_text(body, encoding="utf-8") + before = spec.read_bytes() + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) + + if shape == "no-frontmatter": + with pytest.raises(runs.RearmError, match="no frontmatter `status:`"): + runs.rearm_escalation(run_dir) + else: + runs.rearm_escalation(run_dir) + + records = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-flip-skipped"] + if shape == "already-at-target": + # the ordinary re-arm of an already-correct spec: no record, no refusal, and the + # status the re-drive needs is exactly what it was + assert records == [] + assert verify.read_frontmatter(spec)["status"] == "ready-for-dev" + assert "## Auto Run Result" not in spec.read_text(encoding="utf-8") + elif shape == "ordinary": + assert records == [] + assert verify.read_frontmatter(spec)["status"] == "ready-for-dev" + assert "## Auto Run Result" not in spec.read_text(encoding="utf-8") + else: + (skipped,) = records + assert skipped["story_key"] == "6-4-cli-list-command" + assert skipped["spec_file"] == str(spec) + assert skipped["status"] == "ready-for-dev" + # the flag the operator surfaces render the refusal's remedy from; the sibling + # rows that DON'T abort carry it False + assert skipped["refused"] is True + # the refusal left NOTHING behind but the record: not the re-stamp (which + # `set_frontmatter_field` declines on the same missing block), not the result + # strip (sequenced after the check), and not the task reset — the escalation is + # still armed for the corrected spec, which is what makes the abort recoverable + assert spec.read_bytes() == before + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.ESCALATED + + def test_rearm_journals_event(tmp_path): run_dir, _, _ = _escalated_run(tmp_path) runs.rearm_escalation(run_dir) @@ -655,6 +836,390 @@ def test_rearm_keeps_stale_baseline_outside_a_repo(tmp_path): assert task.baseline_commit == "abc123" +def test_rearm_journals_a_failed_baseline_advance(tmp_path): + """#640(b): the advance was `except Exception: pass # nosec B110`, so a + re-drive that silently rebuilt against the pre-resolution tree looked exactly + like one that adopted the human's fix. Still non-fatal — a project that is not + a repo must not fail re-arm — but no longer silent. + + A non-repo project reaches the arm through `git rev-parse HEAD` returning 128, + which `_run_git` turns into a plain `GitError`; the spawn and timeout faults + arrive as its `GitSpawnError` / `GitTimeoutError` subclasses, so the narrowed + `except verify.GitError` is a total replacement for the bare `Exception`. + + Ablation: delete the `journal.append` and this row reddens alone. + """ + run_dir, _, _ = _escalated_run(tmp_path) # tmp_path is not a git repo + + runs.rearm_escalation(run_dir) + + (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-advance-failed"] + assert entry["story_key"] == "6-4-cli-list-command" + assert entry["baseline"] == "abc123" # the sha that consequently still stands + assert "GitError" in entry["error"] + assert load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit == "abc123" + + +def test_rearm_does_not_swallow_a_non_git_fault_from_the_advance(monkeypatch, tmp_path): + """The `except` arm is NARROWED, not merely observed. `_run_git` translates + every reachable git fault — non-zero rc, spawn failure, timeout, undecodable + output — into the `verify.GitError` taxonomy, so nothing legitimate is lost; + what the old bare `except Exception` also swallowed was every fault that is NOT + a git answer, and those are real bugs that must not be filed away as "the + project probably isn't a repo". + + Ablation: widen the arm back to `except Exception` and this reddens with + DID NOT RAISE, which is exactly how the class of fault it names used to end. + """ + _resolve_repo(tmp_path) + run_dir, _, _ = _escalated_run(tmp_path) + + def boom(repo): + raise MemoryError("not a git answer") + + monkeypatch.setattr(runs.verify, "untracked_files", boom) + with pytest.raises(MemoryError): + runs.rearm_escalation(run_dir) + + +@pytest.mark.parametrize("restore", [None, "artifacts/attempt.patch"]) +def test_rearm_does_not_restamp_a_baseline_the_advance_did_not_move(monkeypatch, tmp_path, restore): + """#640(a)+(b) couple in ONE expression. The re-stamp guard tested + `task.baseline_commit` for truthiness only, and a failed advance leaves the OLD + sha in that field — which passes a truthiness test identically to a freshly + advanced one. Writing it would make spec and task agree on a stale value, the + one state in which nothing downstream can tell the advance never happened. + + Both legs, because dropping the `restore_patch` guard is what makes the + from-scratch leg reach this write at all. + + Ablation: gate the re-stamp on `task.baseline_commit` instead of `advanced` and + both rows redden — the spec comes back carrying the stale sha. + """ + old_head = _resolve_repo(tmp_path) + run_dir, spec, new_head = _escalated_spec_run(tmp_path, old_head) + + def boom(repo): + raise verify.GitError("simulated failure") + + monkeypatch.setattr(runs.verify, "untracked_files", boom) + runs.rearm_escalation(run_dir, restore_patch=restore) + + fm = verify.read_frontmatter(spec) + assert fm["baseline_revision"] == old_head # NOT re-stamped with the stale sha + assert fm["baseline_revision"] != new_head + assert load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit == "abc123" + assert [e["kind"] for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] == [] + assert [e["kind"] for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-advance-failed"] + + +def test_rearm_bumps_the_task_generation(tmp_path): + """#705: re-arm resets `attempt` to 0 and never clears `task.sessions`, so the + next dispatch re-mints a session task_id byte-equal to a record the abandoned + attempt already appended. The generation counter is what makes the new id + unique; `engine._session_task_id` emits it only above zero so every id already + on disk stays byte-identical. + + Clearing `task.sessions` was the rejected alternative — it costs the run-dir + audit trail that a second resolve cycle reads — so the records below must still + be there after the re-arm. + """ + run_dir, _, _ = _escalated_run(tmp_path) + before = load_state(run_dir).tasks["6-4-cli-list-command"] + assert before.generation == 0 and len(before.sessions) == 1 + + runs.rearm_escalation(run_dir) + + task = load_state(run_dir).tasks["6-4-cli-list-command"] + assert task.generation == 1 + assert task.attempt == 0 + assert len(task.sessions) == 1 # the audit trail survives the re-arm + + save_state(run_dir, _rearmable(run_dir)) + runs.rearm_escalation(run_dir) + assert load_state(run_dir).tasks["6-4-cli-list-command"].generation == 2 + + +def test_rearm_advances_the_baseline_in_the_code_tree(tmp_path): + """Under a `repo_root` override the run's code + git live somewhere other than + `state.project`, and re-arm's advance used to read HEAD of `Path(state.project)` + — a directory the proof-of-work gate never measures. + + Ablation: put the advance back on `Path(state.project)` and this reddens on the + journalled degrade (`artifacts-root` is not a repo), which is the *visible* + version of what the swallowed `except Exception` used to do silently. + """ + code = tmp_path / "code" + code.mkdir() + head = _resolve_repo(code) + art = tmp_path / "artifacts-root" + art.mkdir() + run_dir, _, _ = _escalated_run(art, repo_root=code) + + (code / "fixture.txt").write_text("resolution fixture\n") + git(code, "add", "-A") + git(code, "commit", "-q", "-m", "resolution fixture") + (code / "leftover.txt").write_text("keep me\n") + + runs.rearm_escalation(run_dir) + + task = load_state(run_dir).tasks["6-4-cli-list-command"] + assert task.baseline_commit == git(code, "rev-parse", "HEAD") != head + assert "leftover.txt" in task.baseline_untracked + assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-advance-failed"] == [] + + +def test_rearm_reads_stale_restore_residue_from_the_code_tree(tmp_path): + """`_stale_restore_residue`'s repo argument moved to the code tree with the + advance, and the cost of getting it wrong here is not a lost notice — it is + CONTAMINATED baseline state. + + The helper does two things with that root: it anchors a RELATIVE `restore_patch` + latch on it, and it runs `commits_above` in it. Anchored on the wrong root the + patch is not found, the abandoned restore's new files are never subtracted, and + they enter `baseline_untracked` as "pre-existing" — after which every rollback + preserves them and `finalize_commit`'s `add -A` sweeps the abandoned attempt into + the corrected story's commit (#90). The committed-variant notice is lost in the + same breath, because `commits_above` faults in a directory that is not a repo and + the warn-only arm degrades to no record at all. + + Ablation: revert the argument to `Path(state.project)` and this reddens three + ways — `newfile.txt` re-enters `baseline_untracked`, `stale-restore-excluded` + becomes `stale-restore-unparseable`, and `stale-restore-commits` disappears. + """ + code = tmp_path / "code" + code.mkdir() + old_head = _resolve_repo(code) + art = tmp_path / "artifacts-root" + art.mkdir() + + # the abandoned restore attempt: a patch under the CODE tree, latched RELATIVE + # (which is what makes the anchoring root load-bearing), plus the untracked file + # it created + (code / "artifacts").mkdir() + (code / "artifacts" / "attempt.patch").write_text( + "diff --git a/newfile.txt b/newfile.txt\n" + "new file mode 100644\n" + "--- /dev/null\n" + "+++ b/newfile.txt\n" + "@@ -0,0 +1 @@\n" + "+from the abandoned attempt\n", + encoding="utf-8", + ) + (code / "newfile.txt").write_text("from the abandoned attempt\n") + (code / "leftover.txt").write_text("genuinely pre-existing\n") + + run_dir, _, _ = _escalated_run( + art, baseline_commit=old_head, restore_patch="artifacts/attempt.patch", repo_root=code + ) + + # the resolve session's own commit, above the old baseline + (code / "fixture.txt").write_text("resolution fixture\n") + git(code, "add", "fixture.txt") + git(code, "commit", "-q", "-m", "resolution fixture") + new_head = git(code, "rev-parse", "HEAD") + + runs.rearm_escalation(run_dir) # from scratch: the latch is dropped + + task = load_state(run_dir).tasks["6-4-cli-list-command"] + assert task.baseline_commit == new_head + # the patch's new file is SUBTRACTED; the genuine leftover is kept + assert "newfile.txt" not in task.baseline_untracked + assert "leftover.txt" in task.baseline_untracked + + kinds = _kinds(run_dir) + (excluded,) = [e for e in kinds if e["kind"] == "stale-restore-excluded"] + assert excluded["files"] == ["newfile.txt"] + assert [e for e in kinds if e["kind"] == "stale-restore-unparseable"] == [] + # the committed variant the human has to classify by hand + (commits,) = [e for e in kinds if e["kind"] == "stale-restore-commits"] + assert commits["old_baseline"] == old_head and commits["commits"] == [new_head] + + +def test_rearm_falls_back_to_project_when_no_code_root_was_recorded(tmp_path): + """A state.json written before `RunState.repo_root` existed carries no root, and + must degrade to exactly the pre-upgrade behavior rather than to a path that does + not exist. `d.get(key, default)` on the load side is what makes that true. + + What actually grades this is the BASELINE assertion, not the deletion: the + `_escalated_run` fixture never sets `repo_root`, so the serialized value is + already `""` and the `del` below only makes the legacy shape explicit — it + changes nothing observable and would pass with the fallback broken. The final + assertion is what discriminates, because `RunState.code_root` spells the degrade + `Path(self.repo_root or self.project)`: drop the `or self.project` and the empty + string becomes `Path("")` — the process CWD, a different repository under pytest — + and the recorded baseline no longer matches this project's HEAD. + + The positive direction (a recorded root actually being used) is pinned separately + by the divergent-root rows above. + """ + head = _resolve_repo(tmp_path) + run_dir, _, _ = _escalated_run(tmp_path) + raw = json.loads((run_dir / "state.json").read_text(encoding="utf-8")) + del raw["repo_root"] # state.json from before the field existed + (run_dir / "state.json").write_text(json.dumps(raw), encoding="utf-8") + + assert load_state(run_dir).repo_root == "" + runs.rearm_escalation(run_dir) + assert load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit == head + + +def test_rearm_writes_the_worktree_spec_not_the_main_checkouts_copy(monkeypatch, tmp_path): + """The recorded spec path is re-anchored on the tree it was persisted RELATIVE to. + + `StoryTask._serialized_worktree_path` (`model.py`) persists a worktree-local spec + relative to the mounted worktree root — no worktree prefix — and `from_dict` reads + it back raw, so a bare `Path(task.spec_file)` resolves against the process cwd. + That is not merely unreachable, it is actively WRONG: `bmad-loop resolve` runs from + the project root, and the main checkout carries the very same + `_bmad-output/specs/...` layout. `is_file()` answered True on the wrong file, + `confine_root` accepted it (it genuinely is under `project`), and both the status + flip AND the baseline re-stamp landed on a spec the run never used, while the + worktree's real spec kept the escalated attempt's sha and the re-drive re-wedged. + + `_task_spec_path` now anchors a relative path on `task.worktree_path` (falling back + to `state.project`) and passes an absolute one through. + + The cwd is set EXPLICITLY: pytest's own cwd is not this sandbox, so without the + `chdir` the old code would simply fail to resolve the relative path and the row + would pass for the wrong reason instead of reproducing the hazard. The two copies + carry distinguishable `baseline_revision` claims for the same reason — "the right + file was written" has to be checkable against "the other one was not". + + Ablation: revert `_task_spec_path`'s body to `return Path(task.spec_file or "")` + and this reddens on the worktree copy with + `AssertionError: assert 'blocked' == 'ready-for-dev'` — the flip went to the main + checkout — and the byte-identity assertion on the main copy reddens behind it. + """ + head = _resolve_repo(tmp_path) + rel = "_bmad-output/specs/6-4-cli-list-command.md" + wt = tmp_path / "wt" + for root, claim in ((wt, "sha-of-the-escalated-attempt"), (tmp_path, "sha-in-main-checkout")): + spec = root / rel + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text( + f"---\nstatus: blocked\nbaseline_revision: {claim}\n---\n\n## Intent\n\nx\n", + encoding="utf-8", + ) + main_spec = tmp_path / rel + untouched = main_spec.read_bytes() + + run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(wt)) + monkeypatch.chdir(tmp_path) # what `bmad-loop resolve` actually runs from + + runs.rearm_escalation(run_dir) + + fm = verify.read_frontmatter(wt / rel) + assert fm["status"] == "ready-for-dev" # the flip landed in the WORKTREE + assert fm["baseline_revision"] == head # and so did the re-stamp + assert main_spec.read_bytes() == untouched # the main checkout's copy is unread + + +def test_rearm_journals_a_skip_when_the_recorded_spec_is_not_readable(tmp_path): + """A spec path that does not resolve must not be a SILENT no-op. + + `StoryTask` persists `spec_file` relative to a worktree + (`model._serialized_worktree_path`) and `from_dict` reads it back raw, so a + re-arm of a task that ran under isolation can hold a path that resolves against + nothing from this process's cwd. Every writer in the spec block answers that + with `False` rather than an exception — `set_frontmatter_status`, + `strip_auto_run_result` and `set_frontmatter_field` all guard on `is_file` — and + all three return values are dropped, so without this guard the status flip AND + the baseline re-stamp both do nothing and the spec keeps the escalated attempt's + sha with no record anywhere. + + It is also the NEGATIVE control for the flip abort one screen above it. A spec that + exists and cannot take a status refuses the re-arm; an unreachable one must not, + because the flip's failure says nothing about what the re-drive will read — the path + is worktree-relative, the worktree may already be gone, and the re-drive mounts a + fresh one and reads the COMMITTED spec either way. Refusing here would turn the two + records this row exists to report into a wedge. + + Ablation: delete the `if not spec_path.is_file():` arm in `rearm_escalation` and + this reddens on the missing record — re-arm still "succeeds", which is the whole + problem. Delete the `spec_path.is_file()` conjunct guarding the flip's `RearmError` + and it reddens on the raise instead, before any assertion runs. + """ + _resolve_repo(tmp_path) + run_dir, _, _ = _escalated_run(tmp_path, spec_file="wt/_bmad-output/specs/gone.md") + + runs.rearm_escalation(run_dir) # must not raise: the flip's no-op is not a refusal + + kinds = _kinds(run_dir) + (skipped,) = [e for e in kinds if e["kind"] == "rearm-baseline-restamp-skipped"] + assert skipped["spec_file"].endswith("gone.md") + assert skipped["baseline"] == load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit + # and the re-stamp record is NOT written, since nothing was stamped + assert [e for e in kinds if e["kind"] == "rearm-baseline-restamped"] == [] + # the flip records its no-op on this shape too, and the task is re-armed anyway — + # and says so on the record, so neither surface prints the refusal's remedy for it + (flip,) = [e for e in kinds if e["kind"] == "rearm-spec-flip-skipped"] + assert flip["refused"] is False + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.PENDING + + +@pytest.mark.parametrize("repo", [True, False]) +def test_rearm_records_an_unreachable_spec_even_when_the_advance_failed(tmp_path, repo): + """The two #640 degrades COMPOSE; they do not substitute for each other. + + The restamp-skip record used to sit INSIDE the `advanced and ...` gate, so on a + project that is not a git repo — where the advance raises `verify.GitError` and + `advanced` stays False — an unreachable spec produced NO record at all. The journal + blamed git, while the status flip had silently no-opped for an entirely different + reason and the spec still carried the escalated attempt's sha. One warning was + standing in for two independent failures. + + The `repo=True` leg is the sibling row's case (advance succeeds, spec still + unreachable) carried here so the parametrization states the composition rather than + asserting only the half that used to be shadowed. + + Ablation: nest the `rearm-baseline-restamp-skipped` append back under + `elif advanced and task.baseline_commit:` and the `repo=False` row reddens with + `ValueError: not enough values to unpack (expected 1, got 0)` while `repo=True` + keeps passing — which is precisely the shadowing that hid it. + """ + if repo: + _resolve_repo(tmp_path) + run_dir, _, _ = _escalated_run(tmp_path, spec_file="wt/_bmad-output/specs/gone.md") + + runs.rearm_escalation(run_dir) + + kinds = _kinds(run_dir) + (skipped,) = [e for e in kinds if e["kind"] == "rearm-baseline-restamp-skipped"] + assert skipped["story_key"] == "6-4-cli-list-command" + assert skipped["spec_file"].endswith("gone.md") + # the advance's own degrade is reported alongside it, not instead of it + failed = [e for e in kinds if e["kind"] == "rearm-baseline-advance-failed"] + if repo: + assert failed == [] + else: + (advance_failed,) = failed + assert "GitError" in advance_failed["error"] + assert advance_failed["baseline"] == "abc123" # the stale sha that still stands + + +def test_rearm_restamps_normally_when_the_spec_resolves(tmp_path): + """The control for the row above: same code path, readable spec, no skip record. + + Without this the guard could refuse every spec and the negative row would still + pass — a skip record is present for the right reason only if the ordinary case + still stamps. + """ + _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + spec.write_text("---\nstatus: 'escalated'\nbaseline_revision: 'old'\n---\n\nbody\n") + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) + + runs.rearm_escalation(run_dir) + + kinds = _kinds(run_dir) + assert [e for e in kinds if e["kind"] == "rearm-baseline-restamp-skipped"] == [] + head = load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit + # unquoted: `_replace_value` drops the quotes it found, which is its own + # documented behavior and not what this row is about + assert f"baseline_revision: {head}" in spec.read_text() + + def test_rearm_clears_sentinel_preserving_a_copy(tmp_path): """Stories mode: a fixed-slug sentinel (`-unresolved.md`) is cleared by deletion, not a status flip — re-arm preserves a copy, journals the blocking @@ -901,25 +1466,269 @@ def test_rearm_restore_patch_restamps_spec_baseline(tmp_path): assert load_state(run_dir).tasks[key].baseline_commit == new_head -def test_rearm_from_scratch_leaves_spec_baseline_alone(tmp_path): - """A from-scratch re-arm routes ready-for-dev -> step-03, which re-stamps - `baseline_revision` itself — the re-arm must not touch it.""" - old_head = _resolve_repo(tmp_path) +def _escalated_spec_run(tmp_path, baseline: str, *, extra: str = "", recorded: str = "abc123"): + """A real repo + an escalated run whose spec claims `baseline`, plus one + resolution commit on top. Returns (run_dir, spec, new_head). + + ``recorded`` sets what the RUN recorded for the escalated attempt + (`task.baseline_commit`), which defaults to the fixture's `"abc123"` and therefore + differs from every spec claim a caller can write. The divergence-reference rows + need it equal to the claim: that is the only shape in which the two candidate + references (`old_baseline` vs the advanced `task.baseline_commit`) disagree. + """ spec = tmp_path / "spec.md" spec.write_text( - f"---\nstatus: blocked\nbaseline_revision: {old_head}\n---\n\n## Intent\n\nx\n", + f"---\nstatus: blocked\nbaseline_revision: {baseline}\n{extra}---\n\n## Intent\n\nx\n", encoding="utf-8", ) - run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec), baseline_commit=recorded) (tmp_path / "fixture.txt").write_text("resolution fixture\n") git(tmp_path, "add", "-A") git(tmp_path, "commit", "-q", "-m", "resolution fixture") + return run_dir, spec, git(tmp_path, "rev-parse", "HEAD") + + +def _kinds(run_dir): + return [ + json.loads(line) + for line in (run_dir / "journal.jsonl").read_text(encoding="utf-8").splitlines() + ] + + +def test_rearm_restamps_spec_baseline_on_the_from_scratch_leg_too(tmp_path): + """#640(a): the re-stamp used to be gated on `restore_patch`, so a from-scratch + re-drive left the escalated attempt's sha on the spec until step-03 ran — and + every gate reading a claimed baseline before then read a stale one. + + The design call is recorded rather than hidden: extending the re-stamp to this + leg also removes the gate's INDEPENDENT signal here (it then compares a value + the orchestrator itself wrote), which is why the overwrite is journalled — see + the row below. + """ + old_head = _resolve_repo(tmp_path) + run_dir, spec, new_head = _escalated_spec_run(tmp_path, old_head) runs.rearm_escalation(run_dir) # no restore fm = verify.read_frontmatter(spec) - assert fm["baseline_revision"] == old_head # untouched; step-03 owns the stamp + assert fm["baseline_revision"] == new_head assert fm["status"] == "ready-for-dev" + assert load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit == new_head + + +def test_rearm_restores_the_spec_when_the_baseline_restamp_aborts(tmp_path): + """An aborted re-arm leaves the spec byte-identical — INCLUDING the abort that fires + after two writes have already landed. + + Every other refusal in `rearm_escalation` earns that invariant by sequencing: the + flip's read-back check raises before `strip_auto_run_result` runs, which is the whole + reason that strip is ordered after it. The baseline re-stamp cannot be sequenced the + same way — it needs `task.baseline_commit` from an advance that must itself run after + the spec block, or a just-cleared stories sentinel is captured as phantom untracked + residue — so by the time it can fail, the status flip and the result strip are both + behind it and `save_state` is not. + + This spec is the shape that separates the two writes: a plain, perfectly movable + `status:` beside a `baseline_revision:` block scalar, which + `frontmatter._edit_frontmatter_block` refuses (no line edit re-parses to the intended + value, so no candidate verifies). Without the undo the operator was left with the + worst of both: the run still calling the story ESCALATED, and a spec already flipped + to `ready-for-dev` and stripped of the `## Auto Run Result` section the next resolve + session reads as its context — the one edit nothing else records. + + Ablation: drop the `_restore_rearmed_spec(...)` call from the re-stamp's except arm + and this reddens on the byte comparison (the status flip and the strip both stand), + while the `RearmError` and the ESCALATED phase keep passing — which is exactly why + those two alone do not grade this. + """ + old_head = _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + spec.write_text( + "---\n" + "status: blocked\n" + "baseline_revision: |\n" + f" {old_head}\n" + "---\n\n## Intent\n\nx\n\n## Auto Run Result\n\nterminal verdict\n", + encoding="utf-8", + ) + before = spec.read_bytes() + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) + # a resolve-session commit, so the advance really runs and the re-stamp is reached + (tmp_path / "fixture.txt").write_text("resolution fixture\n") + git(tmp_path, "add", "-A") + git(tmp_path, "commit", "-q", "-m", "resolution fixture") + + with pytest.raises(runs.RearmError, match="baseline_revision"): + runs.rearm_escalation(run_dir) + + assert spec.read_bytes() == before # flip AND strip both undone + # nothing was persisted either, so the escalation is still armed for a corrected spec + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.ESCALATED + + +def test_rearm_restores_the_spec_when_the_result_strip_faults(tmp_path, monkeypatch): + """The re-stamp is not the only abort that fires after a write has landed — the spec + block's own `(OSError, UnicodeDecodeError)` arm is the other, and it needs the same + undo. + + The sequencing argument that buys the read-back check its byte-identical abort does + not reach here. That arm guards BOTH spec helpers, and `strip_auto_run_result` is the + later one: ordering the strip after the check protects the CHECK, but a fault raised + inside the strip itself is raised with the status flip already published and + `save_state` still ahead. `strip_auto_run_result` documents that it lets a + present-but-unreadable spec and a failing write raise rather than swallowing them + (silently skipping the strip is the worse bug), so this is its contracted behavior, + not an accident — and its write is the confined atomic writer, which raises on ENOSPC, + on EIO, and on a parent component swapped for a link under the `O_NOFOLLOW` walk. + + Injected rather than provoked, because no in-process fault can order itself between + the two writes: both helpers decode the same file as UTF-8 and write through the same + `require_writable_target=True` path, so every natural fault that reddens the strip + reddens the flip first and leaves nothing to restore. The injection stands in for the + faults above, which are real and are exactly what the atomic writers exist for. + + Ablation: drop the `_restore_rearmed_spec(...)` call from that arm and this reddens on + the byte comparison alone — the `RearmError` and the ESCALATED phase both still pass, + since the flip landing is precisely what neither observes. + """ + _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + spec.write_text( + "---\nstatus: blocked\n---\n\n## Intent\n\nx\n\n## Auto Run Result\n\nterminal verdict\n", + encoding="utf-8", + ) + before = spec.read_bytes() + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) + + def boom(spec_path, *, confine_root): + raise OSError(28, "No space left on device") + + # patched on the module under test, so the flip runs for real and PUBLISHES + monkeypatch.setattr(runs.devcontract, "strip_auto_run_result", boom) + + with pytest.raises(runs.RearmError, match="No space left on device"): + runs.rearm_escalation(run_dir) + + assert spec.read_bytes() == before # the published flip is rolled back + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.ESCALATED + + +def test_rearm_journals_the_spec_baseline_it_overwrote(tmp_path): + """A claim the re-stamp normalizes away is the only trace of a divergence the + gate can no longer report, so it lands in the journal on the way out — read + back through the SAME reader the gate uses, so what is recorded is the value + the gate would have judged (#716). + + Ablation: drop the `rearm-baseline-restamped` append and the row reddens; drop + the `overwritten != old_baseline` guard and the second half reddens + (a re-arm that changed nothing would report an overwrite). + """ + old_head = _resolve_repo(tmp_path) + run_dir, _spec, new_head = _escalated_spec_run(tmp_path, old_head) + + runs.rearm_escalation(run_dir) + + (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] + assert entry["overwritten"] == old_head + assert entry["baseline"] == new_head + assert entry["restore"] is False + + # a second re-arm has nothing left to overwrite: no duplicate record + save_state(run_dir, _rearmable(run_dir)) + runs.rearm_escalation(run_dir) + assert len([e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"]) == 1 + + +def test_rearm_does_not_report_a_divergence_the_run_never_had(tmp_path): + """`rearm-baseline-restamped` measures the spec's claim against what the RUN + RECORDED — `old_baseline`, captured before the advance — and NOT against + `task.baseline_commit`, which the advance has already moved to the new HEAD. + + Against the advanced value the record fired on every ordinary from-scratch re-arm + whose resolve session committed anything: the spec and the run agreed exactly, HEAD + had simply moved on, and the operator was still told they diverged. A warning that + fires on the routine case is the "trains the operator to scroll past the meaningful + one" failure the `restore` split exists to prevent. + + This is the regression direction: it is the row that must redden if someone puts + `task.baseline_commit` back. The re-stamp is asserted to have LANDED first, so the + row cannot pass because the whole block was skipped. + + Ablation: restore `overwritten != task.baseline_commit` and this reddens with + `AssertionError: assert [{...'kind': 'rearm-baseline-restamped'...}] == []` — the + record fires on a re-arm with nothing to report. + """ + old_head = _resolve_repo(tmp_path) + run_dir, spec, new_head = _escalated_spec_run(tmp_path, old_head, recorded=old_head) + + runs.rearm_escalation(run_dir) + + # the re-stamp itself ran: this row is about what was REPORTED, not what was skipped + assert verify.read_frontmatter(spec)["baseline_revision"] == new_head + assert load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit == new_head + assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] == [] + + +def test_rearm_reports_a_claim_the_advanced_head_would_have_masked(tmp_path): + """The other direction of the same reference choice — and the one the old guard got + backwards. The spec claims the tree's CURRENT head while the run recorded a + different baseline: a real divergence (the spec names a sha this run never used), + yet compared against `task.baseline_commit` — already advanced to that same head — + the two came out equal and the record was dropped. + + Paired with the row above, the two bracket the guard: one reddens if the reference + moves forward to the advanced sha, the other if it does. Neither can be satisfied + by deleting the comparison, because `test_rearm_journals_the_spec_baseline_it_ + overwrote` pins the no-duplicate case. + + Ablation: restore `overwritten != task.baseline_commit` and this reddens with + `ValueError: not enough values to unpack (expected 1, got 0)` — no record at all. + """ + old_head = _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec), baseline_commit=old_head) + (tmp_path / "fixture.txt").write_text("resolution fixture\n") + git(tmp_path, "add", "-A") + git(tmp_path, "commit", "-q", "-m", "resolution fixture") + new_head = git(tmp_path, "rev-parse", "HEAD") + # written AFTER the commit: the claim has to name the sha the advance will adopt + spec.write_text( + f"---\nstatus: blocked\nbaseline_revision: {new_head}\n---\n\n## Intent\n\nx\n", + encoding="utf-8", + ) + + runs.rearm_escalation(run_dir) + + (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] + assert entry["overwritten"] == new_head # the claim, carried verbatim + assert entry["baseline"] == new_head # which the advance happens to agree with + assert entry["restore"] is False + + +def test_rearm_prefers_the_fresh_revision_when_the_spec_carries_both_keys(tmp_path): + """A re-armed spec carries BOTH keys — this function is what puts them there. + What it journals as overwritten must therefore be the value the gate reads, + which is `baseline_revision`, not the stale legacy leftover (#716).""" + old_head = _resolve_repo(tmp_path) + run_dir, spec, new_head = _escalated_spec_run( + tmp_path, old_head, extra=f"baseline_commit: {'a' * 40}\n" + ) + + runs.rearm_escalation(run_dir) + + (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] + assert entry["overwritten"] == old_head # NOT the stale baseline_commit + fm = verify.read_frontmatter(spec) + assert fm["baseline_revision"] == new_head + assert fm["baseline_commit"] == "a" * 40 # the legacy key is never removed + + +def _rearmable(run_dir): + """Re-escalate the task so a second `rearm_escalation` is legal.""" + state = load_state(run_dir) + task = state.tasks["6-4-cli-list-command"] + task.phase = Phase.ESCALATED + return state # -------------------------------------------------- non-UTF-8 robustness (bug class) @@ -1156,3 +1965,550 @@ def test_build_context_sprint_mode_has_no_stories_block(tmp_path): resolve.build_context(state, run_dir, "6-4-cli-list-command").read_text(encoding="utf-8") ) assert "stories" not in ctx + + +@pytest.mark.parametrize( + ("committed_status", "warns"), + [("ready-for-dev", False), ("blocked", True)], +) +def test_rearm_warns_about_an_unreachable_spec_write_only_when_it_is_actionable( + tmp_path, monkeypatch, committed_status, warns +): + """`rearm-spec-write-unreachable` must name an EVENT, not a configuration. + + Every escalated task under `isolation = "worktree"` carries a mounted + `worktree_path` — `worktree_flow.escalate_unit` never clears it — so gating the + record on that alone fired it on 100% of re-arms in that configuration. The advice + it prints ("commit the corrected spec") is already a no-op once the committed spec + carries the status the re-drive needs, which is exactly the state in which the + re-drive reads what it needs. A record that fires on the routine case is the + "trains the operator to scroll past the meaningful one" failure that the `flipped` + read-back and the `overwritten != old_baseline` guard were each narrowed to avoid. + + Both legs keep the WORKTREE spec at `blocked`, so the only thing separating them is + what git has committed — which is the whole claim. + + Ablation: restore the gate to a bare `if task.worktree_path:` and the + `committed_status="ready-for-dev"` row reddens on `assert True is False`, while the + `"blocked"` row keeps passing. That asymmetry IS the narrowing; a gate that fires + for both is indistinguishable from no gate at all. + """ + rel = "_bmad-output/specs/6-4-cli-list-command.md" + _resolve_repo(tmp_path) + main_spec = tmp_path / rel + main_spec.parent.mkdir(parents=True, exist_ok=True) + main_spec.write_text( + f"---\nstatus: {committed_status}\n---\n\n## Intent\n\nx\n", encoding="utf-8" + ) + git(tmp_path, "add", "-A") + git(tmp_path, "commit", "-q", "-m", "spec") + wt_spec = tmp_path / "wt" / rel + wt_spec.parent.mkdir(parents=True, exist_ok=True) + wt_spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + + run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + unreachable = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] + assert bool(unreachable) is warns + if warns: + assert unreachable[0]["status"] == "ready-for-dev" + + +@pytest.mark.parametrize( + ("corrected_on_target", "warns"), + [(True, False), (False, True)], +) +def test_rearm_reads_the_committed_spec_from_the_redrive_base_not_the_current_head( + tmp_path, monkeypatch, corrected_on_target, warns +): + """The proof is read at the run's PINNED target branch, not at the code root's HEAD. + + An isolated re-drive never reads the main checkout's working ref: + `engine._finish_inflight` discards the escalated worktree AND its branch, and + `workspace.open_unit_workspace` cuts the replacement from the `base` handed to it — + `worktree_flow.run_isolated` passes `state.target_branch`, pinned once at run start. + `ensure_target_branch` leaves the main checkout ON that branch, so the two agree + until an operator checks out something else while the escalation is paused, which is + a wholly ordinary thing to do with a run parked for a human. + + Both rows leave the main checkout on `side` and put the two candidate refs in + DISAGREEMENT, so neither can pass by reading the other: + + - corrected on `main` (the target) — the re-drive will read `ready-for-dev` and + route. Reading `HEAD` sees `side`'s terminal status and holds a resume whose work + is already committed exactly where the re-drive looks for it. + - corrected on `side` (the current branch) — the re-drive reads `main`'s terminal + status and re-wedges. Reading `HEAD` sees the correction and SUPPRESSES the + record, which since `rearm_holds_the_resume` is not a mis-worded warning but the + default resolve flow resuming into the wedge it was meant to clear. + + The worktree copy is held at `blocked` on both rows, so the only moving part is + which committed ref carries the correction. + + Ablation: restore the revision argument in `_committed_spec_status` to a literal + `"HEAD"` and BOTH rows redden (`assert False is True` / `assert True is False`) — + the pair is the discriminator; either row alone also passes for the anchor it is + meant to reject. + """ + rel = "_bmad-output/specs/6-4-cli-list-command.md" + _resolve_repo(tmp_path) + spec = tmp_path / rel + spec.parent.mkdir(parents=True, exist_ok=True) + + def _commit(status, message): + spec.write_text(f"---\nstatus: {status}\n---\n\n## Intent\n\nx\n", encoding="utf-8") + git(tmp_path, "add", "-A") + git(tmp_path, "commit", "-q", "-m", message) + + _commit("blocked", "escalated spec") # on `main`, the target branch + git(tmp_path, "checkout", "-q", "-b", "side") + if corrected_on_target: + git(tmp_path, "checkout", "-q", "main") + _commit("ready-for-dev", "corrected on the target branch") + git(tmp_path, "checkout", "-q", "side") # the operator wandered off again + else: + _commit("ready-for-dev", "corrected on the wrong branch") + + wt_spec = tmp_path / "wt" / rel + wt_spec.parent.mkdir(parents=True, exist_ok=True) + wt_spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + + run_dir, _, _ = _escalated_run( + tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt"), target_branch="main" + ) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + unreachable = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] + assert bool(unreachable) is warns + if warns: + # the branch rides on the record because the remedy is worthless without it: + # "commit the corrected spec" is what this operator just DID, on `side` + # spelled `target_branch` so `diagnostics._JOURNAL_ALIAS_FIELDS` routes it to + # the `branch` namespace; a new field name would fall to `scrub_json`, which + # ships an identifier-shaped branch verbatim + assert unreachable[0]["target_branch"] == "main" + assert "`main`" in runs.rearm_event_notice(unreachable[0])[2] + + +def test_rearm_base_ref_degrades_to_head_for_a_run_that_pinned_no_target(tmp_path, monkeypatch): + """An unrecorded `target_branch` is a MISSING value, not a divergent one. + + `ensure_target_branch` pins the field before any worktree can mount, so only a + state.json predating it reaches here with a `worktree_path` and no target — and it + must degrade to the ref it read before the fix rather than to `""`. Answering `""` + would make `_committed_spec_status` unprovable for every re-arm of such a run and + hold the resume on a per-configuration constant, the exact failure the record's + narrowing exists to avoid. + + The committed spec here already carries the target status, so a degrade to `""` + is separable from the read succeeding: only an anchor that actually resolves can + suppress. Ablation: make `_redrive_base_ref` return `""` on the migrated shape and + this reddens on the record appearing. + """ + rel = "_bmad-output/specs/6-4-cli-list-command.md" + _resolve_repo(tmp_path) + spec = tmp_path / rel + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text("---\nstatus: ready-for-dev\n---\n\n## Intent\n\nx\n", encoding="utf-8") + git(tmp_path, "add", "-A") + git(tmp_path, "commit", "-q", "-m", "corrected spec") + wt_spec = tmp_path / "wt" / rel + wt_spec.parent.mkdir(parents=True, exist_ok=True) + wt_spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + + # no `target_branch=`: the pre-upgrade shape, worktree_path set all the same + run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] == [] + + +@pytest.mark.parametrize("committed_status", ["ready-for-dev", "blocked"]) +def test_rearm_does_not_refuse_a_flip_the_redrive_never_reads( + tmp_path, monkeypatch, committed_status +): + """The flip's REFUSAL is gated on reachability, not merely on readability. + + A worktree-local spec is the one file the re-drive is guaranteed NOT to read: the + re-arm's final arm discards the worktree and `_run_story` mounts a fresh one, which + checks out TRACKED files only. Gating the abort on `spec_path.is_file()` alone + refused the re-arm over exactly that copy, and the refusal's remedy — "add a + top-level `status:` to the spec" — named a file deleted before anything opens it. An + operator who complied would flip a doomed copy, re-run resolve, and change nothing + about how the re-drive routes. + + Both legs keep the worktree spec unflippable (no `---` block at all) and differ only + in what git has committed, which is what makes this a reachability claim rather than + a readability one: + + - `ready-for-dev` — `_committed_spec_status` has already PROVEN the re-drive reads + the status it routes on. Refusing blocked an otherwise-complete re-arm over an + obsolete copy, with nothing for the operator to do at all. + - `blocked` — the correction really is outstanding, and the remedy is + `rearm-spec-write-unreachable`'s ("commit the corrected spec"), which fires from + the block above and holds the resume. The flip's own refusal would have named the + wrong file for the right problem. + + Ablation: restore the abort's gate to a bare `if spec_path.is_file():` and BOTH legs + redden on `RearmError` before any assertion runs. Hard-code `write_reaches_the_redrive` + False instead and `test_rearm_journals_a_status_flip_that_silently_did_nothing[no-frontmatter]` + reddens on `DID NOT RAISE` — the half this must not take with it. + """ + rel = "_bmad-output/specs/6-4-cli-list-command.md" + _resolve_repo(tmp_path) + main_spec = tmp_path / rel + main_spec.parent.mkdir(parents=True, exist_ok=True) + main_spec.write_text( + f"---\nstatus: {committed_status}\n---\n\n## Intent\n\nx\n", encoding="utf-8" + ) + git(tmp_path, "add", "-A") + git(tmp_path, "commit", "-q", "-m", "spec") + wt_spec = tmp_path / "wt" / rel + wt_spec.parent.mkdir(parents=True, exist_ok=True) + wt_spec.write_text("# Spec\n\nno frontmatter block\n", encoding="utf-8") + + run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) # must not raise: this flip cannot reach the re-drive + + kinds = _kinds(run_dir) + (skipped,) = [e for e in kinds if e["kind"] == "rearm-spec-flip-skipped"] + assert skipped["refused"] is False # what the surfaces render the remedy from + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.PENDING + # ...and the record that DOES carry an actionable remedy fires only when it has one + unreachable = [e for e in kinds if e["kind"] == "rearm-spec-write-unreachable"] + assert bool(unreachable) is (committed_status != "ready-for-dev") + + +def test_rearm_suppresses_the_unreachable_warning_only_on_proof(tmp_path, monkeypatch): + """A project that is not a repo must still WARN, not fall silent. + + `_committed_spec_status` degrades to `""` on every uncertainty — a `GitError` + (which covers "not a repository"), an absent blob, a non-UTF-8 blob. `""` never + equals a target status, so the record fires. This is the direction the narrowing + must fail in: suppressing a warning demands proof the work is done, and the + re-arm advance stays non-fatal outside a repo, as the story's Boundaries require. + + Ablation: make `_committed_spec_status` return `target_status` on `GitError` + instead of `""` and this reddens on the missing record — the re-arm still + "succeeds", which is exactly the silence #640(b) exists to end. + """ + rel = "_bmad-output/specs/6-4-cli-list-command.md" # no _resolve_repo: not a repo + wt_spec = tmp_path / "wt" / rel + wt_spec.parent.mkdir(parents=True, exist_ok=True) + wt_spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + + run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + kinds = _kinds(run_dir) + (unreachable,) = [e for e in kinds if e["kind"] == "rearm-spec-write-unreachable"] + assert unreachable["status"] == "ready-for-dev" + # and the advance's own degrade is reported alongside it, not instead of it + assert [e for e in kinds if e["kind"] == "rearm-baseline-advance-failed"] + + +def test_rearm_does_not_warn_when_the_spec_dir_is_shared_with_the_redrive(tmp_path, monkeypatch): + """A spec in an artifact dir configured OUTSIDE the project is reachable. + + `ProjectPaths.rebased` leaves an artifact dir outside the project tree exactly + where it is — shared across checkouts, not per-worktree — and + `verify.resolve_spec_path` passes an absolute value through untouched, so the fresh + worktree's re-drive opens the very file this flip writes. Under isolation that is + also the only shape `model._serialized_worktree_path` can persist ABSOLUTE: a spec + under the mounted worktree is stored relative to it. + + So the warning's whole premise — "the re-drive destroys the worktree before reading + anything" — is false here, and firing it told the operator to commit a file that is + not in the repository at all, on 100% of re-arms in that layout. The status + assertion is what makes this a reachability claim rather than a silence claim: the + write lands on the shared path (`set_frontmatter_status` keeps the plain no-follow + arm for a spec outside `confine_root`, which its docstring names as this exact + supported configuration), so there is nothing left for the operator to do. + + Ablation: drop the `not _spec_is_shared_with_the_redrive(task)` term from the gate + in `rearm_escalation` and this reddens on the record's presence + (`assert [] == []` fails with the entry). + """ + _resolve_repo(tmp_path) + shared = tmp_path.parent / "shared-artifacts" # outside the project tree + spec = shared / "specs" / "6-4-cli-list-command.md" + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + + run_dir, _, _ = _escalated_run( + tmp_path, spec_file=str(spec), worktree_path=str(tmp_path / "wt") + ) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] == [] + # and the flip really landed on the shared file the re-drive will read + assert "status: ready-for-dev" in spec.read_text(encoding="utf-8") + + +def test_rearm_still_warns_for_a_spec_spelled_out_of_but_resolving_into_the_worktree( + tmp_path, monkeypatch +): + """The exemption is containment, not spelling. + + `model._serialized_worktree_path` relativizes with a LEXICAL `relative_to`, so any + spelling that does not share a literal prefix with `worktree_path` is persisted + absolute — including one that walks back INTO the worktree through a sibling. That + spec is discarded with the worktree like any other, so the warning must survive; a + helper that read `is_absolute()` as "shared" would go silent on it. (The lexical + prefix test cannot separate the two rows: it is the same comparison, on the same + two operands, that decided the value was stored absolute in the first place.) + + The mount is deliberately outside the project, so the helper's project half cannot + carry this row — `test_rearm_warns_when_an_isolated_tasks_spec_writes_cannot_reach_the_redrive` + grades that one. Dropping the worktree half reddens here and nowhere else. + + Ablation: replace the canonical comparison in `_spec_is_shared_with_the_redrive` + with `return True` — accepting the recorded spelling as proof — and this reddens on + the missing record (`ValueError: not enough values to unpack`), while the + shared-artifact-dir row above keeps passing. That asymmetry is the containment test. + """ + _resolve_repo(tmp_path) + # the mount sits OUTSIDE the project, so the project half of the test cannot answer + # this row — `workspace.open_unit_workspace` stores a resolved path, and a symlinked + # `.bmad-loop` puts it here. Only the worktree half can keep the warning. + wt = tmp_path.parent / "u1-worktree" + (tmp_path.parent / "u1-side").mkdir(exist_ok=True) # real, so `u1-side/..` folds + real = wt / "_bmad-output" / "specs" / "6-4-cli-list-command.md" + real.parent.mkdir(parents=True, exist_ok=True) + real.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + # lexically outside `wt` (the components diverge at `u1-side`), canonically inside it + spelled = tmp_path.parent / "u1-side" / ".." / wt.name / "_bmad-output" / "specs" / real.name + + run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spelled), worktree_path=str(wt)) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + (unreachable,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] + assert unreachable["status"] == "ready-for-dev" + + +def test_rearm_warns_when_the_spec_cannot_be_placed_against_the_worktree(tmp_path, monkeypatch): + """A host that cannot canonicalize the spec keeps the warning. + + `_spec_is_shared_with_the_redrive` decides on `resolve()`, which raises on the hosts + #552 is about (a registered-but-not-serving WSL UNC provider) — and an uncertain + answer must not buy silence, the same direction `_committed_spec_status` degrades + in. This row is the shared-artifact-dir row above with resolution taken away, so it + is the fault, not the layout, that flips the outcome. + + Ablation: drop the `except (OSError, RuntimeError)` arm and this reddens with the + OSError escaping `rearm_escalation` — a re-arm that crashes on an observation is + strictly worse than one that warns. + """ + _resolve_repo(tmp_path) + shared = tmp_path.parent / "shared-artifacts-unresolvable" + spec = shared / "specs" / "6-4-cli-list-command.md" + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") + + real_resolve = platform_util.Path.resolve + + def _refuse(self, *a, **kw): + if "shared-artifacts-unresolvable" in str(self): + raise OSError(64, "The specified network name is no longer available") + return real_resolve(self, *a, **kw) + + monkeypatch.setattr(platform_util.Path, "resolve", _refuse) + + run_dir, _, _ = _escalated_run( + tmp_path, spec_file=str(spec), worktree_path=str(tmp_path / "wt") + ) + monkeypatch.chdir(tmp_path) + + runs.rearm_escalation(run_dir) + + (unreachable,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] + assert unreachable["status"] == "ready-for-dev" + + +def test_rearm_writes_the_project_rooted_spec_when_no_worktree_was_recorded(tmp_path, monkeypatch): + """The `state.project` half of `_task_spec_path` — graded, not merely reachable. + + `engine._finish_inflight` clears `task.worktree_path` while leaving `spec_file` + relative, and `model._serialized_worktree_path` returns it unchanged when + `worktree_path` is falsy, so a state.json legitimately holds a relative spec with + no worktree. Both rows that reach this branch today name a file absent under EITHER + root, so they answer `is_file() -> False` identically whichever root is used and + pass with the anchor reverted to the process cwd. + + The decoy is the grading instrument: it sits at the SAME relative path under the + directory this process actually runs from, so a cwd-anchored resolve has somewhere + plausible to land. + + Ablation: change `_task_spec_root` to `Path(task.worktree_path or "")` and this + reddens twice over — the project spec keeps `status: blocked`, and the decoy's + byte-identity assertion fails behind it. + """ + head = _resolve_repo(tmp_path) + rel = "_bmad-output/specs/6-4-cli-list-command.md" + body = "---\nstatus: blocked\nbaseline_revision: stale-sha\n---\n\n## Intent\n\nx\n" + spec = tmp_path / rel + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text(body, encoding="utf-8") + decoy = tmp_path / "elsewhere" / rel + decoy.parent.mkdir(parents=True, exist_ok=True) + decoy.write_text(body, encoding="utf-8") + untouched = decoy.read_bytes() + + run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel) # worktree_path="" -> the fallback + monkeypatch.chdir(tmp_path / "elsewhere") + + runs.rearm_escalation(run_dir) + + fm = verify.read_frontmatter(spec) + assert fm["status"] == "ready-for-dev" # the project-rooted copy was flipped + assert fm["baseline_revision"] == head # and re-stamped + assert decoy.read_bytes() == untouched # the cwd-rooted decoy is unread + + +@pytest.mark.parametrize( + ("field", "value"), + [("files", 3), ("files", None), ("files", [1, 2]), ("commits", 3), ("commits", None)], +) +def test_rearm_event_notice_survives_a_journal_shape_json_admits(field, value): + """A malformed journal line must not raise out of either surface's `finally`. + + `Journal.entries()` appends `json.loads(line)` with no shape filter, so its + `list[dict[str, Any]]` annotation is a claim about first-party producers rather + than a guarantee — pyright sees `Any` and is satisfied. `", ".join` and `len` are + the only two reads in `rearm_event_notice` that raise on a shape the journal + admits; every sibling is `str()`-wrapped or f-string-interpolated. Both run inside + `cli.cmd_resolve`'s and `TuiApp._do_rearm`'s `finally`, and the TUI has no + `_handle_exception` override, so there a `TypeError` ends the app. + + Note `entry.get("files", [])` does NOT protect against `null`: the key EXISTS, so + the default never applies and `.get` returns `None`. + + Ablation: restore `", ".join(entry.get("files", []))` and `len(entry.get( + "commits", []))` and every row but `("files", [1, 2])` reddens with `TypeError`; + that row reddens too, on `sequence item 0: expected str instance, int found`. + """ + kind = "stale-restore-excluded" if field == "files" else "stale-restore-commits" + notice = runs.rearm_event_notice({"kind": kind, field: value}) + + assert notice is not None + severity, message, _ = notice + assert severity in ("note", "warning") + assert isinstance(message, str) + + +def test_rearm_event_notice_splits_the_flip_skip_on_the_refusal(): + """One kind, two outcomes — and the operator-facing halves must not be swapped. + + `rearm-spec-flip-skipped` is journalled by a re-arm that ABORTED and by one that + completed, and this table reads the journal out of process, with neither the task + nor the tree to re-derive which happened. Before the producer wrote `refused` onto + the record, this row claimed the abort unconditionally: an operator whose re-arm had + SUCCEEDED was told it "was REFUSED" and sent to add a `status:` to a file the + re-drive never opens. The next_step is graded alongside the message because it is + the half that costs the operator time — a remedy aimed at the wrong file. + + Ablation: delete the `if entry.get("refused")` branch and the second leg's + assertions redden; return the refusal's next_step on both and the last one does + alone. + """ + entry = { + "kind": "rearm-spec-flip-skipped", + "spec_file": "wt/specs/s1.md", + "status": "ready-for-dev", + } + _, refused_msg, refused_step = runs.rearm_event_notice({**entry, "refused": True}) + assert "REFUSED" in refused_msg + assert refused_step == "Add a top-level `status:` to the spec, then re-run resolve" + + _, msg, step = runs.rearm_event_notice({**entry, "refused": False}) + assert "NOT refused" in msg + assert "COMMITTED spec" in msg + # nothing to do to THIS file; `rearm-spec-write-unreachable` carries the remedy on + # exactly the legs that still have one, and holds the resume behind it + assert step == "" + + +def test_rearm_holds_the_resume_only_on_the_record_that_proves_a_wedge(): + """The hold is PROOF, not urgency — and it is asked of every kind the table knows. + + `rearm-spec-write-unreachable` is written only once `_committed_spec_status` has + established that the committed spec does not carry the status the re-drive routes + on, so resuming on it is futile rather than risky: step-01 halts blocked on + `unrecognized status in existing story file` and the escalation is spent. Its + next_step already read "commit the corrected spec before resuming" while both + default surfaces resumed in the same breath. + + The advisory kinds must NOT hold. `stale-restore-commits` is the record + `cli._echo_rearm_events`' own docstring calls the one a human must act on, and it + still does not qualify — nothing about it proves the re-drive cannot route, and + holding on a maybe would turn every ordinary degrade into a two-command gesture. + That asymmetry IS the predicate; one that answered True for every warning would be + indistinguishable from no predicate at all. + + Ablation: widen the comparison to `str(kind).startswith("rearm-")` and the three + `rearm-baseline-*` legs redden; return False unconditionally and the first does. + """ + assert runs.rearm_holds_the_resume({"kind": "rearm-spec-write-unreachable"}) is True + for kind in ( + "stale-restore-commits", + "stale-restore-unparseable", + "stale-restore-excluded", + "rearm-baseline-advance-failed", + "rearm-baseline-restamp-skipped", + "rearm-baseline-restamped", + "rearm-spec-flip-skipped", + "story-escalation-resolved", + ): + assert runs.rearm_holds_the_resume({"kind": kind}) is False, kind + # the non-mapping shapes `rearm_event_notice` survives reach this in the SAME walk, + # and it is asked first — a raise here would replace the outcome the operator needs + assert runs.rearm_holds_the_resume(3) is False + assert runs.rearm_holds_the_resume(None) is False + + +def test_rearm_event_notice_ignores_a_non_mapping_entry(): + """A bare scalar on its own journal line is not an entry. + + `journal_entries_or_none` drops non-mappings so the annotation is true for every + caller, and both reads apply the same filter so the `len(before)` watermark stays + exact. This pins the notice's own guard as well, since it is reachable from any + other caller walking raw `Journal.entries()`. + + Ablation: delete the `if not isinstance(entry, dict)` arm and this reddens with + `AttributeError: 'int' object has no attribute 'get'`. + """ + assert runs.rearm_event_notice(3) is None + assert runs.rearm_event_notice(None) is None + + +def test_journal_entries_or_none_drops_a_non_mapping_line(tmp_path): + """The watermark both surfaces diff is a list of MAPPINGS. + + Ablation: return `Journal(run_dir).entries()` unfiltered and this reddens on the + length assertion — the bare `3` survives into the window the surfaces walk. + """ + run_dir = tmp_path / "run" + run_dir.mkdir() + (run_dir / "journal.jsonl").write_text( + '{"kind": "a"}\n3\nnull\n{"kind": "b"}\n', encoding="utf-8" + ) + + entries = runs.journal_entries_or_none(run_dir) + + assert entries is not None + assert [e["kind"] for e in entries] == ["a", "b"] diff --git a/tests/test_runs.py b/tests/test_runs.py index 1f72b9dc..3b1135fa 100644 --- a/tests/test_runs.py +++ b/tests/test_runs.py @@ -2701,6 +2701,60 @@ def _escalated_run(tmp_path, spec_text, *, restore_patch_stale=None, git_project return run.run_dir, spec +# --------------------------------------------------- restamp_code_root + + +@pytest.mark.parametrize("recorded", ["moved", "unchanged", "legacy"]) +def test_restamp_code_root_aims_the_mirror_the_rearm_reads(tmp_path, recorded): + """`rearm_escalation` reads the CODE tree out of the run state (`RunState.code_root`) + and has no `ProjectPaths` to consult, so the surfaces that re-arm BEFORE they resume + have to aim that mirror first. Three rows, because the write and the warning answer + different questions: + + - `moved` — a `repo_root:` edit while the run was paused. The mirror follows, and the + operator is told, because every sha the run already recorded names an object in the + previous tree and nothing here can move them. + - `unchanged` — the ordinary re-arm. No message, and no write at all: a row that + rewrote state.json on every re-arm would make the "durable before the engine + starts" ordering above it meaningless to reason about. + - `legacy` — a state.json written before the field existed reads back `""`. That is a + MISSING value, not a divergent one: it migrates silently, and calling it a move + would fire the warning once on every pre-upgrade run. + + Ablation: drop the `if not moved: return None` arm and `legacy` reddens on the + message; return the message without the `save_state` and `moved` reddens on the + persisted root while the other two rows still pass. + """ + from bmad_loop.journal import STATE_FILE + + run = escalated_run(tmp_path, "r1", story_key="s1") + now = tmp_path / "code" + now.mkdir() + run.state.repo_root = { + "moved": str(tmp_path / "was"), + "unchanged": str(now), + "legacy": "", + }[recorded] + save_state(run.run_dir, run.state) + before = (run.run_dir / STATE_FILE).read_bytes() + + message = runs.restamp_code_root(run.run_dir, now) + + # whatever the row, the tree the re-arm will read is the one the caller is acting in + assert load_state(run.run_dir).code_root == now + rewritten = (run.run_dir / STATE_FILE).read_bytes() != before + assert rewritten is (recorded != "unchanged") + if recorded == "moved": + assert message is not None + assert "the code root in _bmad/bmm/config.yaml has changed" in message + # names NEITHER tree, like resume's: the fact is that the run changed + # repositories, and the paths are the half that puts arbitrary text on a terminal + assert str(now) not in message + assert str(tmp_path / "was") not in message + else: + assert message is None + + _SPEC_WITH_ARR = ( "---\ntitle: t\nstatus: blocked\n---\n\n## Intent\n\nbody\n" "\n## Auto Run Result\n\n- Status: blocked\n\nboom\n" diff --git a/tests/test_runsetup.py b/tests/test_runsetup.py index a537cf89..f98a0e77 100644 --- a/tests/test_runsetup.py +++ b/tests/test_runsetup.py @@ -17,6 +17,7 @@ import dataclasses import shutil import types +from pathlib import Path import pytest @@ -24,7 +25,7 @@ from bmad_loop import policy as policy_mod from bmad_loop import runs, runsetup from bmad_loop.adapters.profile import ProfileError -from bmad_loop.journal import Journal +from bmad_loop.journal import Journal, load_state # A profile overlay carrying the whole launch surface the digest covers. It lives # under .bmad-loop/profiles/, inside the tree every driven session can write. @@ -340,6 +341,101 @@ def _fake_paths(project): ) +class _AcceptingEngine: + """Engine stand-in that CAN be built. + + The sibling `_NeverBuilt` exists because every test around it aborts at + `make_adapters`; the rows below are about what a composition that SUCCEEDS + leaves on disk, so they need the opposite stand-in.""" + + def __init__(self, *args, **kwargs): + pass + + +def _split_root_paths(project): + """`_fake_paths` with the one supported divergence: `repo_root` naming a code + tree that is not the BMAD project dir (`isolation = "none"` plus a `repo_root:` + key; `bmadconfig.worktree_isolation_conflict` refuses the other combination). + + `_fake_paths` leaves the two roots identical — as does the `project` fixture + everywhere else — so without this no composition test can tell a `repo_root` + that was WIRED from one that was hardcoded to `project`.""" + return bmadconfig.ProjectPaths( + project=project, + implementation_artifacts=project / "impl", + planning_artifacts=project / "plan", + repo_root=project / "code", + ) + + +def _accepting_adapters(*_a, **_k): + return {role: None for role in runsetup.ROLES} + + +@pytest.mark.parametrize("run_type", ["run", "sweep"]) +def test_composition_persists_the_code_root(tmp_path, run_type): + """`RunState.repo_root` is written at launch and read back OUT OF PROCESS by + `runs.rearm_escalation`, which has no `ProjectPaths` to consult — so the wiring + from `paths.repo_root` into the state is the whole mechanism, and it is + invisible everywhere the two roots coincide. + + Both composers, because they build the state independently: `compose_run` goes + through `build_run_state` and `compose_sweep` constructs `RunState` inline, so + one being wired says nothing about the other. + + Asserted on the PERSISTED state rather than the in-memory object: a re-arm + reads `state.json` from a different process, so an in-memory-only value would + satisfy an object assertion and still leave the consumer with nothing. + + Ablation: hardcode `repo_root=project` (or drop the argument) at either + composer and that parametrization reddens alone. + """ + paths = _split_root_paths(tmp_path) + assert paths.repo_root != paths.project # the fixture really does diverge + + if run_type == "run": + composed = runsetup.compose_run( + project=tmp_path, + paths=paths, + policy=policy_mod.loads(""), + run_id=RUN_ID, + epic_filter=None, + story_filter=None, + max_stories=None, + stories_on=False, + spec_folder="", + sweep_factory=lambda _trigger, *, started: None, + make_adapters=_accepting_adapters, + engine_cls=_AcceptingEngine, + stories_engine_cls=_AcceptingEngine, + trusted_config_digest="deadbeef", + ) + else: + # NOT `_run_compose_sweep`: that helper bakes in `_fake_paths`, whose two + # roots coincide, so the sweep leg would compose without the divergence and + # the assertion below would hold for the wrong reason. + composed = runsetup.compose_sweep( + project=tmp_path, + paths=paths, + policy=policy_mod.loads(""), + run_id=RUN_ID, + prompting=False, + decisions_only=False, + max_bundles=None, + repeat=None, + max_cycles=None, + trigger="auto", + make_adapters=_accepting_adapters, + sweep_engine_cls=_AcceptingEngine, + trusted_config_digest="deadbeef", + ) + + persisted = load_state(composed.run_dir) + assert persisted.repo_root == str(paths.repo_root) + assert persisted.code_root == paths.repo_root + assert persisted.code_root != Path(persisted.project) + + @pytest.fixture def unwinding(tmp_path): """A project plus a `make_adapters` that fails the way the real one does. diff --git a/tests/test_tui_app.py b/tests/test_tui_app.py index 6b0fb063..37809066 100644 --- a/tests/test_tui_app.py +++ b/tests/test_tui_app.py @@ -4025,6 +4025,494 @@ async def test_escalation_rearm_warns_when_restore_recorded(project, monkeypatch assert any("NOT honored" in n for n in notes) # the drop was surfaced, not silent +async def test_escalation_rearm_surfaces_a_failed_baseline_advance(project, monkeypatch): + """The TUI re-arm RESUMES in the same gesture, so a degrade it does not surface + is a degrade the operator acts on without seeing. + + `cli._echo_rearm_events` prints these to stderr on the other re-arm path; both + records are warn-only by contract (a project that is not a git repo must not + fail re-arm), so a journal line in a scrolling panel was the only trace here. A + failed advance means the re-drive rebuilds against the tree as it stood BEFORE + the resolve — the invisibility #640(b) exists to end, not to relocate to the + other caller. + + Ablation: delete the journal read-back loop in `_do_rearm` and this reddens, + while the plain `re-armed 1` notice still fires. + """ + from bmad_loop import resolve, runs + from bmad_loop.journal import Journal + + calls: list[str] = [] + notes: list[str] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + + def fake_rearm(rd, sk): + Journal(rd).append( + "rearm-baseline-advance-failed", + story_key=sk, + repo=str(rd), + baseline="a" * 40, + error="GitError: not a git repository", + ) + return "ready-for-dev" + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append(str(msg)) or orig_notify(self, msg, **kw), + ) + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: calls == ["20260611-100000-aaaa"]) + assert any("could not advance the re-drive baseline" in n for n in notes) + assert any("re-armed 1" in n for n in notes) # the ordinary notice still fires + + +async def test_escalation_rearm_aims_the_code_root_before_it_rearms(project, monkeypatch): + """Parity with `cli.cmd_resolve`, on the seam that has the same ordering. + + This gesture re-arms and RESUMES in one click, and `runs.rearm_escalation` reads the + code tree out of the run state — so only a process that has just read config.yaml can + tell whether a `repo_root:` edit made while the run was paused moved it. Resume + re-stamps the mirror, but that is downstream of the re-arm here too: without this the + re-arm would advance the attempt baseline in the tree the run has left while the + resumed engine reset and measured in the new one. + + Ablation: delete the `runs.restamp_code_root(...)` call from `_do_rearm` and this + reddens on the stale root; drop the `self.notify(moved, ...)` and it reddens on the + missing warning while the root assertion still passes. + """ + from bmad_loop import resolve, runs + from bmad_loop.journal import load_state, save_state + + calls: list[str] = [] + notes: list[str] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + seen: list = [] + + def fake_rearm(rd, sk): + seen.append(load_state(rd).code_root) + return "ready-for-dev" + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append(str(msg)) or orig_notify(self, msg, **kw), + ) + install_bmad_config(project) + moved = project.project / "moved-code" + moved.mkdir() + cfg = project.project / "_bmad" / "bmm" / "config.yaml" + cfg.write_text( + cfg.read_text(encoding="utf-8") + f"repo_root: '{moved.as_posix()}'\n", encoding="utf-8" + ) + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + state = load_state(run_dir) + state.repo_root = str(project.project / "old-code") + save_state(run_dir, state) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: calls == ["20260611-100000-aaaa"]) + + assert seen == [moved.resolve()] + assert any("the code root in _bmad/bmm/config.yaml has changed" in n for n in notes) + + +async def test_escalation_rearm_refuses_the_isolation_conflict_before_it_mutates( + project, monkeypatch +): + """Parity with `cli.cmd_resolve` on the hoisted refusal, and for the same reason this + surface needed the re-stamp parity above: it re-arms and resumes in ONE click. + + The detached CLI refuses `isolation = "worktree"` beside a `repo_root` override, but + it does so in `_resume_paused_run` — downstream of everything this gesture has + already written. So the re-stamp persisted the unsupported root, `rearm_escalation` + advanced the attempt baseline against it, the operator was toasted "re-armed 1", and + only then did the resumed pane refuse. The story was PENDING by then, and `resolve` + needs an ESCALATED story, so the escalation could not be recovered by re-running it. + + Asserted against the sole producer of the text rather than a literal, matching the + launch guard's row, so a reworded message cannot drift this away from the CLI's. + + Ablation: delete the `conflict is not None` arm from `_do_rearm` and this reddens on + the re-arm that must not happen; move it below the `runs.restamp_code_root(...)` call + and it reddens on the persisted root instead. + """ + from bmad_loop import bmadconfig, resolve, runs + from bmad_loop.journal import load_state, save_state + + calls: list[str] = [] + notes: list[str] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + monkeypatch.setattr( + runs, + "rearm_escalation", + lambda *a, **k: pytest.fail("re-armed under a configuration the run refuses"), + ) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append(str(msg)) or orig_notify(self, msg, **kw), + ) + _split_root_tui_project(project) + expected = bmadconfig.worktree_isolation_conflict( + bmadconfig.load_paths(project.project), "worktree" + ) + assert expected is not None, "the fixture really does carry the conflicting pair" + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + recorded = str(project.project / "old-code") + state = load_state(run_dir) + state.repo_root = recorded + save_state(run_dir, state) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: expected in notes) + + assert not calls # the resume folded into this gesture never fired + assert load_state(run_dir).repo_root == recorded # the mirror was never re-pointed + + +async def test_escalation_rearm_surfaces_the_kinds_it_used_to_drop(project, monkeypatch): + """Every kind the shared table routes reaches this surface — not the three the + TUI's own copy of the chain happened to handle. + + That copy carried `rearm-baseline-*` only and silently dropped the whole + `stale-restore-*` family, including `stale-restore-commits` — the record + `cli._echo_rearm_events`' docstring calls the one a human must act on, and the + one whose whole point is that nothing else will tell them. All of it is + warn-only by contract, so a toast is the only place this path can ever show it, + and this path RESUMES in the same gesture: a dropped record is a degrade the + operator acts on without ever seeing. Routing both surfaces through + `runs.rearm_event_notice` only buys anything if the TUI is graded against the + table's whole vocabulary, so this walks a record of every arm the old copy + missed plus the new spec-flip skip. + + Two renderings, not two tables: the severity map is graded here too (`note` is + Textual's `information`, `warning` stays `warning`), as is the deliberate drop + of `next_step` — its imperative reads "... before resuming" and the resume is + already queued behind this toast. + + Ablation: make `runs.rearm_event_notice` return None for any one of these kinds + and this reddens on that kind's message alone. + """ + from bmad_loop import resolve, runs + from bmad_loop.journal import Journal + + calls: list[str] = [] + notes: list[tuple[str, str]] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + + def fake_rearm(rd, sk): + journal = Journal(rd) + journal.append( + "stale-restore-commits", + story_key=sk, + old_baseline="f" * 40, + commits=["c1", "c2"], + ) + journal.append("stale-restore-excluded", story_key=sk, patch="a.patch", files=["new.txt"]) + journal.append( + "rearm-baseline-restamp-skipped", + story_key=sk, + spec_file="wt/specs/s1.md", + baseline="c" * 40, + ) + journal.append( + "rearm-spec-flip-skipped", + story_key=sk, + spec_file="wt/specs/s1.md", + status="ready-for-dev", + ) + return "ready-for-dev" + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append((str(msg), str(kw.get("severity", "information")))) + or orig_notify(self, msg, **kw), + ) + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: calls == ["20260611-100000-aaaa"]) + + def severity_of(fragment: str) -> str: + hits = [n for n in notes if fragment in n[0]] + assert len(hits) == 1, f"{fragment!r} not surfaced exactly once: {notes}" + return hits[0][1] + + # the one a human must act on — dropped entirely by the pre-table copy + assert severity_of("2 commit(s) sit below the re-drive's new baseline (ffffffffffff..)") == ( + "warning" + ) + assert severity_of("is not a readable file from here") == "warning" + assert severity_of("could not be re-opened to `ready-for-dev`") == "warning" + # `note` maps onto Textual's own channel name, not through unchanged + assert severity_of("excluded the abandoned restore's new files") == "information" + # the CLI's trailing imperative is omitted here: the resume is already queued + assert not any("before resuming" in n[0] for n in notes), notes + assert any("re-armed 1" in n[0] for n in notes) # the ordinary notice still fires + + +async def test_escalation_rearm_holds_the_resume_it_folds_in(project, monkeypatch): + """This surface's whole gesture is re-arm + resume, so the hold has to break it. + + `rearm-spec-write-unreachable` fires only once the re-arm has proven the committed + spec does not carry the status the re-drive routes on — and this path drops the + table's `next_step` precisely because it resumes in the same gesture. That silenced + the one record whose remedy MUST land first in BOTH halves: the imperative was + dropped as moot, and the resume it was warning against happened anyway, mounting a + fresh worktree onto the still-terminal committed spec. + + The re-arm itself is kept — the story is armed and persisted — and the toast names + what the operator can finish from this screen: commit, then resume. The + `rearm-baseline-restamp-skipped` control keeps this a narrowing rather than + "warnings stop resumes": it is a warning on the same walk, and the resume still fires. + + Ablation: drop the `if hold_resume:` arm from `_do_rearm` and the first leg reddens + on `calls == []`, with the resume firing behind the warning it was told to wait for. + Discard `_echo_rearm_events`' return and it reddens the same way. + """ + from bmad_loop import resolve, runs + from bmad_loop.journal import Journal + + calls: list[str] = [] + notes: list[str] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + + def fake_rearm(rd, sk): + Journal(rd).append( + "rearm-spec-write-unreachable", + story_key=sk, + spec_file="wt/specs/s1.md", + status="ready-for-dev", + ) + Journal(rd).append( # a warning on the same walk that must NOT hold the resume + "rearm-baseline-restamp-skipped", + story_key=sk, + spec_file="wt/specs/s1.md", + baseline="c" * 40, + ) + return "ready-for-dev" + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append(str(msg)) or orig_notify(self, msg, **kw), + ) + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: any("not resuming" in n for n in notes)) + + assert calls == [] # the resume this gesture folds in did NOT fire + assert any("re-armed 1" in n for n in notes) # ...while the re-arm itself stands + assert any("commit the corrected spec, then resume this run" in n for n in notes) + # the record that proved it still renders, and its warning sibling did not hold + assert any("land in a worktree the re-drive discards" in n for n in notes) + assert any("is not a readable file from here" in n for n in notes) + + +async def test_escalation_rearm_echoes_residue_when_the_rearm_aborts(project, monkeypatch): + """An aborted re-arm still surfaces what it already journalled — the CLI parity gap. + + `runs._stale_restore_residue` journals BEFORE the re-stamp block that raises + `RearmError`, so on that path the records exist and the operator has to decide what + to do with the tree. `cli.cmd_resolve` echoes them from a `finally`; this surface + used to `return` inside the `except` and drop the whole family — including + `stale-restore-commits`, which `cli._echo_rearm_events`' own docstring calls the one + record a human must act on. The two surfaces had been unified on ROUTING while + still drifting on the abort path, and `docs/FEATURES.md` claimed they could not + drift at all. + + Ablation: move the `self._echo_rearm_events(...)` call out of the `finally` and back + below the `try`, and this reddens — the commits warning never fires — while + `test_escalation_rearm_survives_a_corrupt_journal` still passes. + """ + from bmad_loop import resolve, runs + from bmad_loop.journal import Journal + from bmad_loop.runs import RearmError + + notes: list[str] = [] + calls: list[str] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + + def fake_rearm(rd, sk): + # exactly the real ordering: residue journalled, THEN the abort + Journal(rd).append( + "stale-restore-commits", story_key=sk, old_baseline="f" * 40, commits=["c1"] + ) + raise RearmError("cannot re-stamp baseline_revision on /x/spec.md") + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append(str(msg)) or orig_notify(self, msg, **kw), + ) + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: any("re-arm failed" in n for n in notes)) + # the abort is reported AND the residue it already wrote is surfaced + assert any("commit(s) sit below" in n for n in notes), notes + # ... and an aborted re-arm does not resume the run + assert calls == [], calls + + +async def test_escalation_rearm_survives_a_corrupt_journal(project, monkeypatch): + """An undecodable byte in journal.jsonl costs the echo, never the gesture. + + `_do_rearm` reads the journal twice to diff what the re-arm appended, and before + that echo existed it read it not at all — so `Journal.entries()`' strict UTF-8 + decode would have turned a corrupt journal into a re-arm the operator can no + longer perform. That is strictly worse than the missing echo it was added to + fix, and a regression against the gesture's own history. `runs.journal_entries_or_none` + (shared with `cli.cmd_resolve`) answers `None`, and `_echo_rearm_events` skips the + echo when either end of the diff is unreadable rather than replaying the journal + from zero; the dashboard already reads this same file with `errors="replace"` + everywhere else. + + Ablation: call `Journal(run_dir).entries()` directly in `_do_rearm` and this + reddens — the UnicodeDecodeError escapes into the Textual worker and no + `re-armed 1` notice ever fires. + """ + from bmad_loop import resolve, runs + from bmad_loop.journal import JOURNAL_FILE, Journal + + calls: list[str] = [] + notes: list[str] = [] + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + + def fake_rearm(rd, sk): + Journal(rd).append( + "stale-restore-commits", story_key=sk, old_baseline="f" * 40, commits=["c1"] + ) + return "ready-for-dev" + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + orig_notify = BmadLoopApp.notify + monkeypatch.setattr( + BmadLoopApp, + "notify", + lambda self, msg, **kw: notes.append(str(msg)) or orig_notify(self, msg, **kw), + ) + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + # a real corruption shape: a valid line, then a byte no UTF-8 decoder accepts + (run_dir / JOURNAL_FILE).write_bytes( + b'{"ts": 1.0, "kind": "session-start", "task_id": "t1"}\n\xff\xfe not utf-8\n' + ) + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await _open_review(app, pilot, EscalationModal) + await pilot.click(await ready(pilot, "#act-rearm")) + await until(pilot, lambda: calls == ["20260611-100000-aaaa"]) + # the re-arm ran and the run resumed: the corruption cost only the echo + assert any("re-armed 1" in n for n in notes) + assert not any("re-arm failed" in n for n in notes), notes + assert not any("commit(s) sit below" in n for n in notes), notes + + async def test_escalation_rearm_disabled_without_resolution(project, monkeypatch): monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") _stories_paused_run( diff --git a/tests/test_tui_data.py b/tests/test_tui_data.py index 8798ed6c..65f528a1 100644 --- a/tests/test_tui_data.py +++ b/tests/test_tui_data.py @@ -1126,3 +1126,36 @@ def test_char_style_degrades_unparseable_color_instead_of_raising(): assert style.color is None and style.bgcolor is None assert style.bold and style.underline and not style.italic assert data._char_style(key) is style # fallback is cached like any other + + +def test_story_key_from_task_id_grammar_including_the_generation_suffix(): + """The id grammar this fallback parses, pinned in both directions. + + `_session_task_id` composes `safe_segment(f"{story_key}-{part}-{seq}{gen}")`, and + #705 added `gen` — a `-g` suffix emitted only above generation zero. That + changed the grammar this parser documents, and nothing recorded either half of it. + + The `-g1` row pins a DOCUMENTED LIMITATION, not a desired outcome: the suffix + fails `seq.isdigit()` and the whole id comes back as the story key. It is + unreachable today because every `session-start` has carried `story_key` since #153 + phase 1, so the entries this fallback actually sees predate generations entirely. + It is pinned precisely because that reasoning is an assumption about the CALLER: + widen the fallback to entries that can carry `-gN` and this row is where the + breakage surfaces, instead of a bogus `1-1-a-dev-1-g1` row appearing in the + active-agent view. + + Ablation: drop the `seq.isdigit()` term and the `-g1` row changes answer (it then + peels `g1` as if it were a sequence); change the emitted suffix shape in + `engine._session_task_id` and the last row reddens. + """ + # the ordinary unsuffixed shape: the recorded role is peeled with its seq + assert data._story_key_from_task_id("1-1-a-dev-1", "dev") == "1-1-a" + assert data._story_key_from_task_id("1-1-a-review-12", "review") == "1-1-a" + # a labeled plugin session: role does not match, so one more `-` group goes + assert data._story_key_from_task_id("1-1-a-somelabel-1", "dev") == "1-1-a" + # not the expected shape at all — returned verbatim + assert data._story_key_from_task_id("nonsense", "dev") == "nonsense" + + # generation-suffixed (#705): NOT parsed, returned whole. Unreachable today. + assert data._story_key_from_task_id("1-1-a-dev-1-g1", "dev") == "1-1-a-dev-1-g1" + assert data._story_key_from_task_id("1-1-a-dev-1-g12", "dev") == "1-1-a-dev-1-g12" diff --git a/tests/test_verify.py b/tests/test_verify.py index 57fba0a7..490f5ac0 100644 --- a/tests/test_verify.py +++ b/tests/test_verify.py @@ -12,6 +12,7 @@ _FAIL, _OK, MISSING_TOOL_CMD, + OMIT, UNRESOLVABLE, _file_exists_cmd, fault_read_text, @@ -5059,6 +5060,274 @@ def test_read_frontmatter_ignores_triple_dash_in_value(project): assert fm["title"] == "restore --- review" +# ------------------------------------------- repo_root override (divergent roots) +# +# `isolation = "none"` plus a `repo_root:` key in _bmad/bmm/config.yaml is the ONE +# supported shape where `paths.project` and `paths.repo_root` name different +# directories (`bmadconfig.worktree_isolation_conflict` refuses the other). The +# `project` fixture sets no override, so `repo_root == project` and no pre-existing +# row here can tell the two apart — which is why the wrong-root bug survived. + + +def _repo_root_override(project, tmp_path): + """ProjectPaths for the override: BMAD artifacts under a `project` directory + that is not a checkout, code + git under a separate `repo_root`. + + The session's cwd under this config IS `repo_root` (`Workspace.default` sets + `root = paths.repo_root`), so the dev writer already stamps its baseline there. + Only the readers were anchored on `project`.""" + art = tmp_path / "artifacts-root" + impl = art / "_bmad-output" / "implementation-artifacts" + plan = art / "_bmad-output" / "planning-artifacts" + impl.mkdir(parents=True) + plan.mkdir(parents=True) + return dataclasses.replace( + project, + project=art, + implementation_artifacts=impl, + planning_artifacts=plan, + output_folder=art / "_bmad-output", + repo_root=project.project, + ) + + +def test_verify_dev_measures_proof_of_work_in_the_code_tree(project, tmp_path): + """The baseline is written in `repo_root` (by `Engine._dev_phase`, off + `workspace.root`) and must be READ there too. Anchored on `paths.project` the + gate asked a directory that is not the code checkout about a commit only the + code checkout has. + + Ablation: put the canonical-oid probe back on `paths.project` and this reddens + with "does not match" — `artifacts-root` is not a repo, so the claimed commit + cannot be resolved there. + + The proof-of-work probe deliberately is NOT graded by this row, and cannot be: + `has_changes_since` fails OPEN (`rc != 0 -> return True`), so pointing it at a + non-repo returns "there are changes" and a passing row stays green for the + wrong reason. The refusal row below is what grades it, which is why that one + asserts the exact reason rather than `not out.ok`. + """ + paths = _repo_root_override(project, tmp_path) + write_sprint(paths, {"1-1-a": "review"}) + task = StoryTask(story_key="1-1-a", epic=1) + task.baseline_commit = verify.rev_parse_head(paths.repo_root) + sp = spec_path(paths, "1-1-a") + write_spec(sp, "in-review", task.baseline_commit) + # the session's work lands where the session's cwd is: the CODE tree + (paths.repo_root / "src.txt").write_text("real work\n") + + out = verify.verify_dev(task, paths, dev_result(sp)) + assert out.ok + assert task.spec_file == str(sp) + + +def test_verify_dev_refuses_proof_of_work_only_the_project_tree_holds(project, tmp_path): + """The other half of the same anchor: residue under `project` is not evidence + that anything was implemented, because no session writes code there. + + The assertion is on the exact refusal REASON, not merely on `not out.ok` — but + NOT for the reason a reader might assume. Re-anchoring `has_changes_since` on + `paths.project` does not make the gate fault: `has_changes_since` fails OPEN + (`rc != 0 -> return True`, verify.py), so pointing it at a directory that is not + a git repository reports "there are changes" and the gate PASSES. The exact-reason + assertion is still the right call, for the neighbouring row's sake — that one + cannot grade this probe at all, precisely because the fail-open answer is also + the answer a correct run gives. + + Ablation: re-anchor `has_changes_since` on `paths.project` and this row reddens + on `not out.ok` with `ok=True`. + """ + paths = _repo_root_override(project, tmp_path) + write_sprint(paths, {"1-1-a": "review"}) + task = StoryTask(story_key="1-1-a", epic=1) + task.baseline_commit = verify.rev_parse_head(paths.repo_root) + sp = spec_path(paths, "1-1-a") + write_spec(sp, "in-review", task.baseline_commit) + # a marker ONLY the project tree holds; the code tree is untouched + (paths.project / "marker.txt").write_text("not implementation work\n") + + out = verify.verify_dev(task, paths, dev_result(sp)) + assert not out.ok + assert out.reason == "no changes in worktree since baseline commit" + + +def test_verify_dev_exclude_relpaths_follows_the_root_it_is_given(project, tmp_path): + """The pathspecs handed to git must be relative to the root git is invoked + against. A relpath computed against the other root does not raise — git simply + matches nothing — so the exclusion vanishes silently, which is why this is + pinned rather than left to the gate's behavior. + + Ablation: make the helper ignore `root` (pin `base = paths.project`) and the + second half reddens — the board and spec come back as `project`-relative entries + that name nothing inside the code tree, which is the silent-no-op shape. + """ + paths = _repo_root_override(project, tmp_path) + sp = spec_path(paths, "1-1-a") + sp.write_text("---\nstatus: in-review\n---\n", encoding="utf-8") + + # project-rooted (the default): the board and the spec are both inside it + default = verify.verify_dev_exclude_relpaths(paths, sp, root=paths.project) + assert any(r.endswith("sprint-status.yaml") for r in default) + assert any(r.endswith("spec-1-1-a.md") for r in default) + # code-tree-rooted: neither artifact lives there, so there is nothing to exclude + assert verify.verify_dev_exclude_relpaths(paths, sp, root=paths.repo_root) == () + + +def test_verify_dev_accepts_a_newer_reachable_claim_in_the_code_tree(project, tmp_path): + """The `newer_ok` leg asks its ancestry question in the CODE tree. + + An intervening commit before step-03 stamps `baseline_revision` makes the claim a + descendant of the recorded baseline. That reachability is a fact about the + checkout the code lives in; asked of `paths.project` — which under this override + is not a repository at all — `commit_reachable_above_baseline` reads the Git + failure as False and a correct attempt is refused forever. That is the + burn-every-attempt shape #716 exists to close, and no pre-existing row could see + it: every other ancestry row runs on the `project` fixture where the two roots + are the same object, and both divergent-root rows above claim a baseline EQUAL to + the recorded one, so neither enters this branch. + + Adopting a newer claim sets `include_untracked_proof = False`, so the proof here + is a TRACKED modification rather than a new untracked file. + + Ablation: re-anchor `commit_reachable_above_baseline` on `paths.project` and this + reddens with "does not match". + """ + paths = _repo_root_override(project, tmp_path) + write_sprint(paths, {"1-1-a": "review"}) + task = StoryTask(story_key="1-1-a", epic=1) + task.baseline_commit = verify.rev_parse_head(paths.repo_root) + + # the session commits inside the unit before step-03 stamps its baseline + (paths.repo_root / "prior.txt").write_text("prior work\n") + git(paths.repo_root, "add", "-A") + git(paths.repo_root, "commit", "-q", "-m", "intervening commit in the code tree") + descendant = verify.rev_parse_head(paths.repo_root) + + sp = spec_path(paths, "1-1-a") + write_spec(sp, "in-review", descendant) + (paths.repo_root / "src.txt").write_text("real work\n") + + out = verify.verify_dev(task, paths, dev_result(sp)) + assert out.ok + + +def test_verify_dev_bundle_accepts_an_ancestor_claim_in_the_code_tree(project, tmp_path): + """The `older_ok` leg — #161's bundle-only relaxation — asks the same question in + the same tree. + + `allow_ancestor_baseline` is set at exactly one site (`verify_dev_bundle`), so + this is the only route into `is_ancestor`. Anchored on `paths.project` the call + lands on a non-repository, `is_ancestor` reads any Git failure as False, and a + legitimate bundle adopting a story's older spec baseline is refused. + + Ablation: re-anchor `is_ancestor` on `paths.project` and this reddens with + "does not match". + """ + paths = _repo_root_override(project, tmp_path) + ancestor = verify.rev_parse_head(paths.repo_root) + + # the unit worktree is cut after the story's own baseline + (paths.repo_root / "prior.txt").write_text("unit history\n") + git(paths.repo_root, "add", "-A") + git(paths.repo_root, "commit", "-q", "-m", "unit worktree cut") + + task = StoryTask(story_key="dw-test-bundle", epic=0) + task.baseline_commit = verify.rev_parse_head(paths.repo_root) + sp = paths.implementation_artifacts / "spec-dw-test-bundle.md" + write_spec(sp, "in-review", ancestor) + (paths.repo_root / "src.txt").write_text("real work\n") + + out = verify.verify_dev_bundle(task, paths, dev_result(sp)) + assert out.ok + + +def test_verify_dev_exclude_relpaths_anchors_the_restore_patch_on_the_root(project, tmp_path): + """The latched `restore_patch` follows `root` like every other candidate. + + `base` feeds TWO steps — `resolve_restore_path`'s join and the `relative_to` + that follows — and only a RELATIVE latch can tell them apart. Joined under + `project` but measured against `repo_root`, it is not relative to that root at + all, raises `ValueError`, and drops out of the exclude set silently. A missing + exclusion does not raise — it just stops excluding, letting a restore re-drive + whose session produced nothing pass proof-of-work on the patch file's mere + presence, the hazard this function's own docstring names. + + An ABSOLUTE latch deliberately is not the probe, even though it is what + `cli._resolve_restore_patch` stores: `resolve_restore_path` ignores its base for + an absolute input, so both halves agree however the join is anchored and the row + would pass with the anchor ablated. + + The existing root row passes no `restore_patch`, and both restore-patch rows run + on the `project` fixture where the roots are the same object, so this line was + unpinned in both directions. + + Ablation: pin the `resolve_restore_path` base to `paths.project` and the first + assertion reddens — the latch is no longer relative to the code tree and drops. + """ + paths = _repo_root_override(project, tmp_path) + sp = spec_path(paths, "1-1-a") + sp.write_text("---\nstatus: in-review\n---\n", encoding="utf-8") + (paths.repo_root / "restore.patch").write_text("a saved intent-gap patch\n") + + out = verify.verify_dev_exclude_relpaths(paths, sp, "restore.patch", root=paths.repo_root) + assert "restore.patch" in out + + +def test_verify_dev_stories_roots_its_exclude_on_the_code_tree(project, tmp_path, monkeypatch): + """`_stories_relpaths` is one of the four exclude sources that had to move with + the gate's git root, and it is pinned at the SEAM rather than by outcome — on + purpose. + + Under the supported override the story record and manifest sit outside the code + tree whichever root is used, so both spellings end in "nothing was excluded" and + no outcome assertion can separate them. What the wrong root actually costs is + invisible in a passing gate: a `project`-relative pathspec is resolved by git + against the CODE tree, silently excluding whatever happens to live at that + relative path there. So the contract is the root itself. + + Ablation: pass `paths.project` at the call site and the recorded root reddens. + """ + paths = _repo_root_override(project, tmp_path) + spec_folder = paths.planning_artifacts / "epic-a" + # built by hand rather than via `make_stories_task`, which reads HEAD of + # `paths.project` — under this override that directory is not a checkout + task = StoryTask(story_key="1", epic=1) + task.baseline_commit = verify.rev_parse_head(paths.repo_root) + sp = write_story(spec_folder, "1", "x", "done", task.baseline_commit) + (paths.repo_root / "src.txt").write_text("real work\n") + + seen = [] + real = verify._stories_relpaths + monkeypatch.setattr( + verify, + "_stories_relpaths", + lambda root, folder: (seen.append(root), real(root, folder))[1], + ) + out = verify.verify_dev_stories( + task, paths, dev_result(sp), spec_folder=spec_folder, review_enabled=False + ) + + assert out.ok + assert seen == [paths.repo_root] + # the premise the assertion above rests on: under this override the two roots are + # genuinely different directories. Should the fixture ever collapse them, the + # recorded-root assertion stops separating the two spellings and this reddens. + assert paths.project != paths.repo_root + + +def test_stories_relpaths_follows_the_root_it_is_given(project, tmp_path): + """Same rule for the stories-mode exclude: rooted where git runs.""" + paths = _repo_root_override(project, tmp_path) + spec_folder = paths.implementation_artifacts / "spec-x" + spec_folder.mkdir(parents=True) + assert verify._stories_relpaths(paths.project, spec_folder) == ( + "_bmad-output/implementation-artifacts/spec-x/stories", + "_bmad-output/implementation-artifacts/spec-x/stories.yaml", + ) + # outside the code tree: nothing to exclude there, and no exception + assert verify._stories_relpaths(paths.repo_root, spec_folder) == () + + def test_artifact_relpaths_returns_in_repo_folders(project): """The orchestrator-owned artifact folders, repo-relative posix.""" rels = verify.artifact_relpaths(project) @@ -5139,7 +5408,7 @@ def test_verify_dev_exclude_relpaths_is_file_granular(project): artifact-folder content (deferred-work.md, other stories' specs) is left un-excluded so it can register as real work.""" sp = spec_path(project, "1-1-a") - rels = verify.verify_dev_exclude_relpaths(project, sp) + rels = verify.verify_dev_exclude_relpaths(project, sp, root=project.repo_root) assert "_bmad-output/implementation-artifacts/sprint-status.yaml" in rels assert "_bmad-output/implementation-artifacts/spec-1-1-a.md" in rels assert "_bmad-output/implementation-artifacts" not in rels @@ -5157,9 +5426,11 @@ def test_verify_dev_exclude_relpaths_includes_latched_restore_patch(project): sp = spec_path(project, "1-1-a") patch = project.implementation_artifacts / "attempt.patch" rel = "_bmad-output/implementation-artifacts/attempt.patch" - assert rel in verify.verify_dev_exclude_relpaths(project, sp, str(patch)) - assert rel in verify.verify_dev_exclude_relpaths(project, sp, rel) - assert rel not in verify.verify_dev_exclude_relpaths(project, sp) + assert rel in verify.verify_dev_exclude_relpaths( + project, sp, str(patch), root=project.repo_root + ) + assert rel in verify.verify_dev_exclude_relpaths(project, sp, rel, root=project.repo_root) + assert rel not in verify.verify_dev_exclude_relpaths(project, sp, root=project.repo_root) def test_verify_dev_exclude_relpaths_omits_only_an_uncertain_candidate(project, monkeypatch): @@ -5174,7 +5445,9 @@ def test_verify_dev_exclude_relpaths_omits_only_an_uncertain_candidate(project, patch = project.implementation_artifacts / "attempt.patch" refuse_to_resolve(monkeypatch, project.project, refused_spec) - assert verify.verify_dev_exclude_relpaths(project, refused_spec, str(patch)) == ( + assert verify.verify_dev_exclude_relpaths( + project, refused_spec, str(patch), root=project.repo_root + ) == ( "_bmad-output/implementation-artifacts/sprint-status.yaml", "_bmad-output/implementation-artifacts/attempt.patch", ) @@ -5300,6 +5573,86 @@ def test_verify_dev_baseline_gate_reads_the_skills_baseline_revision_key(project assert verify.verify_dev(task, project, dev_result(sp)).ok +def test_verify_dev_baseline_gate_prefers_the_fresh_revision_over_a_stale_legacy_key(project): + """#716: a spec carrying BOTH keys is what `runs.rearm_escalation` produces — + it inserts `baseline_revision` and never removes a pre-existing + `baseline_commit`. The gate must judge the value the skill just wrote, not the + leftover. + + Ablation: swap `_BASELINE_KEYS` back to ("baseline_commit", "baseline_revision") + and this row reddens with "does not match", which is precisely the attempt this + bug burned — everything the session did was correct. + """ + write_sprint(project, {"1-1-a": "review"}) + task = make_task(project) + sp = spec_path(project, "1-1-a") + write_spec(sp, "in-review", task.baseline_commit, legacy_baseline="0" * 40) + body = sp.read_text() + assert "baseline_revision:" in body and "baseline_commit:" in body # the dual-key shape + (project.project / "src.txt").write_text("real work\n") + + assert verify.verify_dev(task, project, dev_result(sp)).ok + + +@pytest.mark.parametrize("legacy", ["", None]) +def test_verify_dev_baseline_gate_skips_an_unusable_legacy_key(project, legacy): + """An EMPTY (`baseline_commit: ''`) or YAML-null (bare `baseline_commit:`) legacy + key must not shadow the fresh claim. `dict.get`'s default fires only on a MISSING + key, so the empty value used to be selected and read back as "no claim" — which + skips the baseline-match check entirely, on the very spec shape the re-arm writes. + + Ablation: drop the `if value:` / `if raw is None` guards in `auto_dev_baseline_of` + and the gate stops checking (the empty case) or fails on the token "None" (the + null case); either way the pairing below no longer holds. + """ + write_sprint(project, {"1-1-a": "review"}) + task = make_task(project) + sp = spec_path(project, "1-1-a") + (project.project / "src.txt").write_text("real work\n") + + # the fresh key still decides: matching passes ... + write_spec(sp, "in-review", task.baseline_commit, legacy_baseline=legacy) + assert verify.verify_dev(task, project, dev_result(sp)).ok + # ... and a foreign fresh key is still REFUSED (the gate is live, not skipped) + write_spec(sp, "in-review", "0" * 40, legacy_baseline=legacy) + out = verify.verify_dev(task, project, dev_result(sp)) + assert not out.ok and "does not match" in out.reason + + +def test_verify_dev_baseline_gate_refuses_a_stale_revision_beside_a_matching_legacy_key(project): + """The reverse-mismatch row, and a DELIBERATE tightening: `baseline_revision` + wins whenever it is non-empty, so a stale fresh key is refused even though the + legacy key names the right commit. That is the point of a precedence rule — + the stale field cannot override in EITHER direction, and a spec whose two keys + disagree is not silently rescued by whichever one happens to match. + + Ablation: make the reader prefer whichever key matches and this row passes, + which is the "reads as green for the wrong reason" outcome it exists to refuse. + """ + write_sprint(project, {"1-1-a": "review"}) + task = make_task(project) + sp = spec_path(project, "1-1-a") + write_spec(sp, "in-review", "0" * 40, legacy_baseline=task.baseline_commit) + (project.project / "src.txt").write_text("real work\n") + + out = verify.verify_dev(task, project, dev_result(sp)) + assert not out.ok and "does not match" in out.reason + + +def test_verify_dev_baseline_gate_reads_a_legacy_only_spec(project): + """Back-compat: a spec predating the `baseline_revision` rename claims only + `baseline_commit`, and the gate must still read it.""" + write_sprint(project, {"1-1-a": "review"}) + task = make_task(project) + sp = spec_path(project, "1-1-a") + write_spec(sp, "in-review", OMIT, legacy_baseline=task.baseline_commit) + body = sp.read_text() + assert "baseline_revision:" not in body and "baseline_commit:" in body + (project.project / "src.txt").write_text("real work\n") + + assert verify.verify_dev(task, project, dev_result(sp)).ok + + def test_verify_dev_exclude_relpaths_normalizes_dotdot_segments(project): """A spec_path with a lexical '..' hop (as an un-normalized session-reported spec_file could produce) must resolve to the same exclude entry as the plain @@ -5310,9 +5663,9 @@ def test_verify_dev_exclude_relpaths_normalizes_dotdot_segments(project): project.output_folder / "planning-artifacts" / ".." / "implementation-artifacts" / sp.name ) assert messy != sp # genuinely a different (messier) Path object - assert verify.verify_dev_exclude_relpaths(project, sp) == verify.verify_dev_exclude_relpaths( - project, messy - ) + assert verify.verify_dev_exclude_relpaths( + project, sp, root=project.repo_root + ) == verify.verify_dev_exclude_relpaths(project, messy, root=project.repo_root) def test_verify_dev_own_spec_status_flip_via_dotdot_path_is_not_real_work(project): @@ -5338,7 +5691,7 @@ def test_has_changes_since_ledger_content_counts_with_narrow_exclude(project): changes', while a bare own-spec + sprint-status bookkeeping edit still does.""" baseline = verify.rev_parse_head(project.project) sp = spec_path(project, "1-1-a") - exclude = verify.verify_dev_exclude_relpaths(project, sp) + exclude = verify.verify_dev_exclude_relpaths(project, sp, root=project.repo_root) sp.write_text("bookkeeping\n") project.sprint_status.write_text("bookkeeping\n") @@ -6146,3 +6499,47 @@ def test_git_below_floor_honours_the_floor_argument(project, monkeypatch): _fake_git_version(monkeypatch, "git version 2.30.0\n") assert verify.git_below_floor(project.project, (2, 20)) is None assert verify.git_below_floor(project.project, (2, 40)) == "git version 2.30.0" + + +def test_verify_dev_roots_its_exclude_on_the_code_tree(project, tmp_path, monkeypatch): + """The gate's OWN exclude composition, pinned at the seam. + + `_stories_relpaths` has carried a seam pin since this wave landed; the sprint + gate's call did not, and no outcome row can supply one. Under the supported + override the artifact tree is disjoint from the code tree, so a `project`-rooted + exclude yields pathspecs git matches nothing against — and `has_changes_since` + fails OPEN (`rc != 0 -> return True`), so the passing row stays green and the + refusal row reddens for its own unrelated reason. Reverting `root=paths.repo_root` + to `root=paths.project` left the entire suite green before this row existed, which + is how the anchor this wave exists to establish could be silently undone. + + The contract is therefore the ROOT itself, exactly as the stories-mode row states + it: a pathspec relative to the wrong root is not merely wrong, it is SILENTLY + wrong. + + Ablation: pass `root=paths.project` at the call site and the recorded root reddens. + """ + paths = _repo_root_override(project, tmp_path) + write_sprint(paths, {"1-1-a": "review"}) + task = StoryTask(story_key="1-1-a", epic=1) + task.baseline_commit = verify.rev_parse_head(paths.repo_root) + sp = spec_path(paths, "1-1-a") + write_spec(sp, "in-review", task.baseline_commit) + # the session's work lands where the session's cwd is: the CODE tree + (paths.repo_root / "src.txt").write_text("real work\n") + + seen = [] + real = verify.verify_dev_exclude_relpaths + monkeypatch.setattr( + verify, + "verify_dev_exclude_relpaths", + lambda *a, **kw: (seen.append(kw.get("root")), real(*a, **kw))[1], + ) + out = verify.verify_dev(task, paths, dev_result(sp)) + + assert out.ok + assert seen == [paths.repo_root] + # the premise the assertion above rests on: under this override the two roots are + # genuinely different directories. Should the fixture ever collapse them, the + # recorded-root assertion stops separating the two spellings and this reddens. + assert paths.project != paths.repo_root