Skip to content

feat(poteto-mode): seven orchestration and hillclimb lessons from a long multi-agent run - #44

Closed
pigfyy wants to merge 1 commit into
chhoumann:mainfrom
pigfyy:pstack-orchestration-lessons
Closed

pigfyy wants to merge 1 commit into
chhoumann:mainfrom
pigfyy:pstack-orchestration-lessons

Conversation

@pigfyy

@pigfyy pigfyy commented Oct 23, 2026

Copy link
Copy Markdown

Why

These come from running the orchestrate playbook for about a day on a large personal project: a game-mechanics trainer with dozens of Opus and Codex workers, cross-family reviews, and UE builds that hold a shared lock. Each edit fixes a failure that cost a real retry or a human turn. Each is general, not specific to that project.

What changed

playbooks/hillclimb.md

  • Step 1. When the metric matches an external reference whose rule can be read (source, a binary, a spec) and reading it is permitted, read the rule before fitting to it. A fit kept before that stays labelled provisional, and the label never blocks keeping it.
    • Evidence: movement was fitted piece by piece from recordings before anyone read the reference's code. Reading it resolved several rules the fits had missed.
  • Step 2. When comparing target and reference, put both sides through one measurement path, and draw reference cases from across the workload dimensions named in step 1.
    • Evidence: the two sides of the comparison were once measured with different filters and lag handling, and a review had to block on it.

playbooks/orchestrate.md

  • ACCEPTANCE. A unit built to answer a question is accepted on a ranked list of what the data says is wrong, and "no differences found, with evidence" is a valid result.
  • Liveness, retry by mode. On a rate-limit failure, read task_status, and relaunch only when the task is terminal-failed or stalled past a bound. A rate-limited task can resume on its own, and relaunching beside it put two writers in one worktree.
  • preferences.md. Each order ends, on its numbered line, with what it supersedes and when it expires. Superseded orders are rewritten in place, with the old text recorded in decisions.tsv. A user-written order is removed only with the user's okay.
    • Evidence: an append-only register grew contradictory orders that workers obeyed.
    • The note stays on the numbered line because orch's parser requires numbered lines.
  • Step 2. Before the pilot, create or confirm the project's verify control skill. A single documented verify command counts.

SKILL.md, Subagents

  • "Local checkout" is no longer a reason to reuse an agent. A costly checkout alone, such as a cold build or imported assets, hands its worktree path to a fresh agent. The uncommitted-changes and live-process exceptions and the stop-and-confirm rule are unchanged.

tools/forks.json

  • hillclimb.md and orchestrate.md are declared policy forks with upstream: not-proposed, so tools/sync.mjs --dry-run accepts the edits.
  • The existing SKILL.md entry's why names the narrowed rule.
  • CONTRIBUTING prefers changes to what a skill does to go to cursor/plugins upstream. Six of the seven aren't T3-specific; the task_status one is. If you'd rather carry them upstream, these forks can be dropped once that lands.

Verification

  • tools/sync.mjs --dry-run passes on the patched tree under WSL with bun 1.3.14, CI's version. On Windows it fails even on an unmodified tree, because of mixed path separators.
  • markdownlint reports 0 errors on the three changed Markdown files.
  • Not done: the VERSION bump and the CHANGES.md entry. The fork entries assume 2.0.0-rc.19. Adjust them if the next release differs.

🤖 Generated with Claude Code

…ong multi-agent run

- hillclimb step 1: read a readable, permitted reference rule before fitting;
  earlier fits stay provisional, never blocked.
- hillclimb step 2: compare target and reference through one measurement
  path, with reference cases spanning the workload dimensions.
- orchestrate ACCEPTANCE: a question-answering unit is accepted on a ranked
  list, and an evidenced "no differences found" is a valid result.
- orchestrate liveness: after a rate-limit failure, read task_status before
  relaunching; a rate-limited task can resume and a relaunch puts two
  writers in one worktree.
- orchestrate preferences.md: each order notes what it supersedes and when
  it expires; superseded orders are rewritten in place, and user orders are
  removed only with the user's okay.
- orchestrate step 2: confirm a verify control skill, or one documented
  verify command, before the pilot.
- poteto-mode Subagents: a costly checkout alone hands its worktree path to
  a fresh agent instead of justifying reuse of the agent.

hillclimb.md and orchestrate.md are declared policy forks in
tools/forks.json so the sync check accepts the edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pigfyy

pigfyy commented Oct 23, 2026

Copy link
Copy Markdown
Author

Closing: this was opened by mistake, sorry for the noise. These changes will stay local.

@pigfyy pigfyy closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant