From f5533170995c13544ed5c356888e12775443e251 Mon Sep 17 00:00:00 2001 From: pigfyy Date: Thu, 22 Oct 2026 19:33:17 -0700 Subject: [PATCH] feat(poteto-mode): seven orchestration and hillclimb lessons from a long multi-agent run - hillclimb step 1: read a readable, permitted reference rule before fitting; earlier fits stay provisional, never blocked. - hillclimb step 2: compare target and reference through one measurement path, with reference cases spanning the workload dimensions. - orchestrate ACCEPTANCE: a question-answering unit is accepted on a ranked list, and an evidenced "no differences found" is a valid result. - orchestrate liveness: after a rate-limit failure, read task_status before relaunching; a rate-limited task can resume and a relaunch puts two writers in one worktree. - orchestrate preferences.md: each order notes what it supersedes and when it expires; superseded orders are rewritten in place, and user orders are removed only with the user's okay. - orchestrate step 2: confirm a verify control skill, or one documented verify command, before the pilot. - poteto-mode Subagents: a costly checkout alone hands its worktree path to a fresh agent instead of justifying reuse of the agent. hillclimb.md and orchestrate.md are declared policy forks in tools/forks.json so the sync check accepts the edits. Co-Authored-By: Claude Opus 5.5 --- plugins/pstack/skills/poteto-mode/SKILL.md | 2 +- .../skills/poteto-mode/playbooks/hillclimb.md | 4 ++-- .../skills/poteto-mode/playbooks/orchestrate.md | 10 ++++++---- tools/forks.json | 14 +++++++++++++- 4 files changed, 22 insertions(+), 8 deletions(-) diff --git a/plugins/pstack/skills/poteto-mode/SKILL.md b/plugins/pstack/skills/poteto-mode/SKILL.md index d79c74f2..db654bef 100644 --- a/plugins/pstack/skills/poteto-mode/SKILL.md +++ b/plugins/pstack/skills/poteto-mode/SKILL.md @@ -99,7 +99,7 @@ Read the leaf skill in full for any principle you apply. Each entry names when i You own every subagent's work. Review the diff and write your own summary, don't pass through what it said. A second opinion is the same prompt against a different model. Agreement is high-signal. -**Fresh subagents by default.** Give new work to a fresh subagent with consolidated scope, meaning the original brief, every later directive, and the prior agent's report and branch. This holds for a fix round, a follow-up, a retry, and the next queue item. Send a follow-up to an existing child (`t3_thread_send` to its `childThreadId`) only when the new work strictly needs state that lives in that agent and is costly to move: its local checkout, its uncommitted changes, or a process it still runs, such as a dev server, a simulator, or a babysit watcher. A stop or hold order to a running agent is not reuse. A role such as a PR owner outlives its agent. Once that agent returns, a fresh agent takes the role's next round. Interrupt-chained resumes silently drop directives, so fire a fresh subagent with consolidated scope rather than trusting a "done" summary. **Stop the abandoned agent first, and confirm it stopped,** when its writes could overlap its replacement's, in the same worktree or the same files. Autopilot-full's audit still replaces a stuck agent at once, without waiting for its stop, per that playbook. A child whose turn completed can still have live work, and it can keep editing after a fresh agent takes its role. `task_cancel` it, which also stops the tasks it delegated, and read `task_status` until the task is terminal with no `hasPendingChildRuns`. A claim about the working tree that `git status` contradicts means it is still running. +**Fresh subagents by default.** Give new work to a fresh subagent with consolidated scope, meaning the original brief, every later directive, and the prior agent's report and branch. This holds for a fix round, a follow-up, a retry, and the next queue item. Send a follow-up to an existing child (`t3_thread_send` to its `childThreadId`) only when the new work strictly needs state that lives in that agent and is costly to move: its uncommitted changes, or a process it still runs, such as a dev server, a simulator, or a babysit watcher. A costly checkout alone, such as a cold build or imported assets, justifies giving a fresh agent the same worktree path, not reusing the agent. A stop or hold order to a running agent is not reuse. A role such as a PR owner outlives its agent. Once that agent returns, a fresh agent takes the role's next round. Interrupt-chained resumes silently drop directives, so fire a fresh subagent with consolidated scope rather than trusting a "done" summary. **Stop the abandoned agent first, and confirm it stopped,** when its writes could overlap its replacement's, in the same worktree or the same files. Autopilot-full's audit still replaces a stuck agent at once, without waiting for its stop, per that playbook. A child whose turn completed can still have live work, and it can keep editing after a fresh agent takes its role. `task_cancel` it, which also stops the tasks it delegated, and read `task_status` until the task is terminal with no `hasPendingChildRuns`. A claim about the working tree that `git status` contradicts means it is still running. **Dispatch on T3.** Every subagent is a `delegate_task` call from the chat that runs the playbook, same model family included, so every child shows up in T3. Give each call its own `clientRequestId`, stable across retries of that call, and keep each returned `taskId`. A child's completion wakes this thread, so end the turn instead of polling, and call `task_status` only when a result is needed mid-turn. `task_cancel` stops a child, and `task_status` confirms it stopped. A child gets only its brief, never your context, so pass absolute paths. `` is the absolute path of the directory that holds this plugin's `skills/` and `agents/`. A persona (`poteto-agent`, `comment-sicko`, `thermo-nuclear-review-subagent`, `thermo-nuclear-code-quality-review-subagent`) is a brief that opens with "Read `/agents/.md` in full and act as it says", and a `poteto-agent` brief also names this file's absolute path. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md b/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md index f59f4a37..50253a0d 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md @@ -4,8 +4,8 @@ Core discipline: one change, one measurement, keep or revert. Never stack untested changes, and never claim a win from code inspection (the **prove-it-works** principle skill). -1. Ground the workload and architecture before choosing the metric. Run the **how** skill over the target, name the realistic workload dimensions that can move the result (data size, history, state, concurrency), and select a case that reproduces the user's complaint. If no case reproduces it, fix the repro instead of hillclimbing. Then fix one metric, the direction that counts as better, and a checkable stop predicate that pairs a target with a floor on attempts so a lucky early win can't end the run (the example "at least 50% better than baseline and at least 10 iterations" is this shape). Use the user's numbers when given, otherwise agree them. -2. Build the measurement harness, prove its sensitivity, then freeze it (the **build-the-lever** principle skill). Run contrasting realistic workloads and confirm the target case reproduces the symptom while easier cases separate as expected. If the harness cannot distinguish them, revise the workload or metric. Vet the harness with the **benchmark-checklist** skill before you freeze it, and make it print its error count and a count of the work done. Once frozen, one repeatable command emits the metric, sampled enough to clear the noise (median of N, not a single run). Record the baseline metric and a green run of the regression gate (the tests that must keep passing) before any change. +1. Ground the workload and architecture before choosing the metric. Run the **how** skill over the target, name the realistic workload dimensions that can move the result (data size, history, state, concurrency), and select a case that reproduces the user's complaint. If no case reproduces it, fix the repro instead of hillclimbing. When the metric matches an external reference whose rule can be read (its source, a binary, a spec) and reading it is permitted, read the rule before you fit to it. A fit kept before the rule is read stays labelled provisional until a reading confirms it, and the label never blocks keeping it. Then fix one metric, the direction that counts as better, and a checkable stop predicate that pairs a target with a floor on attempts so a lucky early win can't end the run (the example "at least 50% better than baseline and at least 10 iterations" is this shape). Use the user's numbers when given, otherwise agree them. +2. Build the measurement harness, prove its sensitivity, then freeze it (the **build-the-lever** principle skill). Run contrasting realistic workloads and confirm the target case reproduces the symptom while easier cases separate as expected. When the metric compares the target with a reference, put both sides through one measurement path, and draw the reference cases from across the workload dimensions named in step 1. If the harness cannot distinguish them, revise the workload or metric. Vet the harness with the **benchmark-checklist** skill before you freeze it, and make it print its error count and a count of the work done. Once frozen, one repeatable command emits the metric, sampled enough to clear the noise (median of N, not a single run). Record the baseline metric and a green run of the regression gate (the tests that must keep passing) before any change. 3. Open the decision log via the **show-me-your-work** skill. A `decision.tsv`, one row per attempt: id, hypothesis, change, before, after, delta, tests, verdict (kept or reverted), note. Read it before each attempt. Keep it out of the tree (gitignored). 4. Ground each hypothesis in the architecture model from step 1, so it names a specific mechanism ("defer X off the boot path because it blocks first paint"), not "try memoizing something". For a perf metric, order hypotheses by the performance mantras in step 2 of the Perf issue playbook (`playbooks/perf-issue.md`). Borrow only their order, not that step's stop rule. 5. Loop, one hypothesis per iteration: diff --git a/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md b/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md index a9816a16..849cc001 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md @@ -22,7 +22,7 @@ Depth stays at coordinator, track, worker. Author the track decomposition per pr Create `orchestrate//` in `${XDG_STATE_HOME:-~/.local/state}/t3-pstack/` (T3 names no agent store). Every file has exactly one writer. Owners publish facts, readers aggregate at read time. Use `bun scripts/orch/orch.ts` for bookkeeping, written below as `orch`, while its canonical plain TSV and JSON stay readable without the CLI. -- `preferences.md` is the standing-orders register: numbered lines, one constraint each (model policy, stack shape and count, verification bar, forbidden paths, escalation policy). Paste it verbatim into every spawn and every resume. Directives decay across resumes, and each dropped one costs a human turn. When you catch yourself restating an instruction, append the line before you act (principle-encode-lessons-in-structure). +- `preferences.md` is the standing-orders register: numbered lines, one constraint each (model policy, stack shape and count, verification bar, forbidden paths, escalation policy). Paste it verbatim into every spawn and every resume. Directives decay across resumes, and each dropped one costs a human turn. When you catch yourself restating an instruction, append the line, or rewrite the order it supersedes, before you act (principle-encode-lessons-in-structure). Each order ends, on the same numbered line, with a note naming what it supersedes and when it expires, such as `(supersedes 12, expires at close)` or `(standing)`. Rewrite a superseded order in place and record its old text in `decisions.tsv`, and remove an order the user wrote only with the user's okay. - `overview.md` is the durable PR and issue DB. Append. Never rewrite wholesale per event. - `units.tsv` has one row per unit: id, track, state, branch, PR, head SHA, brief path. Update rows in place. - `frontier.json` is the computed merge frontier, per Stack safety. @@ -40,7 +40,9 @@ GOAL one sentence, the outcome, executable by a stranger with no chat ac SCOPE paths this unit may write; paths it may not; its exclusive worktree or branch CONTEXT pointers to files and PRs; upstream reports pasted in full when this unit depends on them, because workers cannot see siblings -ACCEPTANCE checkable criteria, one per line +ACCEPTANCE checkable criteria, one per line; a unit built to answer a question is + accepted on a ranked list of what the data says is wrong, and "no + differences found, with evidence" is a valid result VERIFY exact commands or the control-skill path, plus known gotchas TIMEBOX rough cap on runtime; on expiry, return partial findings and stop rather than run on FORBIDDEN no PR retargets or merges, no rebase, no force-push, no fixes outside scope, @@ -59,7 +61,7 @@ A dependency is a context relay, not just ordering. Undeclared upstream context #### Steps 1. **Frame.** State the done predicate as something countable ("all 126 units merged, each ledger-verified `unit-test-verified` or better"). Quantify scope: units, rough effort, expected stacks, and the wall-clock budget. If one agent could finish inside that budget, stop here and run Autonomous run instead. Collapsing must not depend on another document being present. It means do the work directly in this session, plain workers where they help, verification inline, landing as you go, and none of the store, register, or pilot machinery below. Schedule landing against the budget. By roughly 70% of it, stop spawning and land what is verified. Name the tracks per project. A contested decomposition or one-way door goes through the arena skill before the pilot. Present the framing once. Reversible prep proceeds without waiting. -2. **Install the runtime.** Run `orch init`. Open the trail via the show-me-your-work skill, write the standing orders before any spawn, and seed `frontier.json` from existing PRs with `orch frontier set --repo `, adding `--forge origin` when the forge is Origin. +2. **Install the runtime.** Run `orch init`. Before the pilot, create or confirm the project's verify control skill (`/create-verification-skill` makes one). A single documented verify command counts. Open the trail via the show-me-your-work skill, write the standing orders before any spawn, and seed `frontier.json` from existing PRs with `orch frontier set --repo `, adding `--forge origin` when the forge is Origin. 3. **Pilot.** Push one unit through the whole path: brief, worker, verification, stack entry, ledger row, merge. The pilot exists to falsify the brief template, the verify recipe, and the unit size while that costs one agent instead of fifty. Fix the contract from pilot evidence before any fan-out. Scale the pilot to the unit. On programs of near-identical cheap units, the first unit is the pilot, run as a normal unit with its verify command inline, and fan-out starts the moment it lands. The dedicated pilot pipeline (separate verifier agent, audit gate) is for expensive or novel unit shapes, not for clone-units where a serialized pilot has nothing to falsify. 4. **Scale.** Spawn a rolling window of workers up to the in-flight cap, refilling as children finish. Blocking batches pay the slowest child of every batch. Spawn track sub-coordinators only past the one-drain threshold in Roles. Recompute ready work after each drain. Relay upstream reports into downstream briefs. Keep sibling communication upward only. The sampled brief audit runs alongside the wave it samples and stops the next refill on failure, not the current one. 5. **Drain.** Run the queue discipline below at every drain point. @@ -95,7 +97,7 @@ A unit is not done until its output is externalized the moment it lands, never b - Never resume an agent to check on it. A resume restarts an idle agent. Probe read-only: the ledger, `units.tsv`, `gh`, pushed branches, each child's `task_status`. Transcript mtime is not liveness. - A silent death gets a synthetic postmortem row in the inbox (unit, failure mode, last evidence, options). Replan on evidence as it arrives. Never wait for full quiescence. -- Retry by mode: cap-hit or oom, respawn with smaller scope. Network-drop, retry as-is. Tool-error, retry on a different model. Unknown, retry once. Two retries, then abandon the unit and replan around it. +- Retry by mode: cap-hit or oom, respawn with smaller scope. Network-drop, retry as-is. Rate-limit, read the task's `task_status` and relaunch only when it is terminal-failed or has stalled past a bound you set, because a rate-limited task can resume on its own and a relaunch beside it puts two writers in one worktree. Tool-error, retry on a different model. Unknown, retry once. Two retries, then abandon the unit and replan around it. - A zombie that returns hours late reconciles against the current frontier and ledger before anything is accepted. Salvage unique findings through a fresh unit, never a blind merge. - When continued spawning would produce garbage tree-wide (bad upstream output, broken acceptance, dead infra), write a stop line at the top of the standing orders, let in-flight work finish, fix the cause, clear it. - Bound your own infra retries the same way you bound a child's. After a few consecutive tool aborts, stop retrying. Write a terminal handoff to durable state (what is done, where it lives, the exact command to resume) and end the run. diff --git a/tools/forks.json b/tools/forks.json index c85d1d20..e205bec2 100644 --- a/tools/forks.json +++ b/tools/forks.json @@ -2,7 +2,7 @@ "pstack": { "plugins/pstack/skills/poteto-mode/SKILL.md": { "kind": "policy", - "why": "Carries four rules from Michael Denyer's pstack-claude in T3 wording: load a managed platform's skill before deploying, stop an abandoned agent whose writes could overlap its replacement's with task_cancel and confirm with task_status that it stopped, which Autopilot-full's immediate replacement of a stuck agent overrides, keep a review gate closed while no independent reviewer can run, and keep the todolist in .audit/.todo.md when the session has no task tool.", + "why": "Carries four rules from Michael Denyer's pstack-claude in T3 wording: load a managed platform's skill before deploying, stop an abandoned agent whose writes could overlap its replacement's with task_cancel and confirm with task_status that it stopped, which Autopilot-full's immediate replacement of a stuck agent overrides, keep a review gate closed while no independent reviewer can run, and keep the todolist in .audit/.todo.md when the session has no task tool. It also narrows the follow-up rule so a costly checkout alone hands its worktree path to a fresh agent instead of reusing the agent.", "since": "2.0.0-rc.2", "upstream": "not-proposed" }, @@ -12,12 +12,24 @@ "since": "2.0.0-rc.2", "upstream": "not-proposed" }, + "plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md": { + "kind": "policy", + "why": "Has step 1 read a readable, permitted reference rule before fitting to it, with fits kept before that reading labelled provisional but never blocked, and has step 2 put a reference and its target through one measurement path over the full span of workload conditions.", + "since": "2.0.0-rc.19", + "upstream": "not-proposed" + }, "plugins/pstack/skills/poteto-mode/playbooks/opening-a-pr.md": { "kind": "policy", "why": "Carries the rule from Michael Denyer's pstack-claude that every live agent holding a worktree is stopped, and the stop confirmed, before a commit, merge, or deploy from it, and drops the reset --hard between delegates on one branch so that each gets its own worktree.", "since": "2.0.0-rc.2", "upstream": "not-proposed" }, + "plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md": { + "kind": "policy", + "why": "Has step 2 confirm a verify control skill or one documented verify command before the pilot, accepts a ranked list or an evidenced no-difference result from a question-answering unit, gates a rate-limit relaunch on task_status, and gives each standing order a supersedes or expires note, with superseded orders rewritten in place and user-written orders removed only with the user's okay.", + "since": "2.0.0-rc.19", + "upstream": "not-proposed" + }, "plugins/pstack/skills/poteto-mode/playbooks/shipping.md": { "kind": "policy", "why": "Adds --match-head-commit to the GitHub merge in step 4, the head guard from Michael Denyer's pstack-claude, with the current head whose verdict survives step 3's patch-id rule, so a head pushed after that check cannot land, and stops step 5's watcher loop at a merge gate that needs a person.",