Skip to content

chore(ship): S28 retro: OS-handler URLs banned, eval before merge for verdict changes, run-wait 90, Downloads check, eval exit 1 without runs - #908

Merged
szymdzum merged 2 commits into
mainfrom
chore/ship-retro-s28
Oct 22, 2026
Merged

szymdzum merged 2 commits into
mainfrom
chore/ship-retro-s28

Conversation

@szymdzum

@szymdzum szymdzum commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

S28 retro for the ship skill, plus a fix to the eval harness script. No product code.

Tool bugs

  • Briefs: the forbidden block bans URLs another app opens (mailto:, tel:, sms:, facetime: and others). In S28 a mailto: redirect cell made macOS Chrome, even headless, open the owner's mail app again and again. ship brief --check now refuses a brief that contains such a URL. A scheme named as a word, like the ban itself, doesn't count. Checked against every S28 brief and every open issue's text (those briefs quote the issue): no false refusal.
  • Briefs: the forbidden block left <worktree> open as a placeholder in the design and verifier briefs. I added that line in chore(ship): S27 retro: cloud red-team kit in the skill, verifier foreign in gate 8, snapshot in ~/.cache/ship, FINDINGS, load warning #898.
  • Briefs: every brief runs ls -A ~/Downloads before the first and after the last test or bdg run, not only the tester. In S28 a reviewer's red-commit smoke run left files there.
  • Removed the "snapshot is 12 h old" warning from chore(ship): S27 retro: cloud red-team kit in the skill, verifier foreign in gate 8, snapshot in ~/.cache/ship, FINDINGS, load warning #898. It fired during a session that ran overnight, and a really stale snapshot already shows up as a gate 8 failure.
  • run-wait now defaults to 90 minutes instead of 60. A ×10 repeat of fix: dom fill reads the value again 2 s after filling; a late revert exits 90 (#892) #900 (2 s per dom fill) took longer than 60.
  • ci-wait: a PR waiter that has seen a CI run no longer gives up when one gh poll comes back empty. On this PR it saw "CI queued 4 min", then advised closing and reopening the PR (exit 4).
  • scripts/agent-eval.mjs exits 1 and says why when the budget let no run start. It used to exit 0 with an empty summary.
  • Not changed: "a TIER=1 reviewer brief keeps the red-commit section" doesn't reproduce. The rendered brief has the intended Red commit. None: tier 1… line.

Process

  • Promoted (S27 candidate, S28 found dom fill reports ✓ when the page replaces the value with an unrelated one later (rejection read as formatting) #906 after merge): a PR that changes an action verdict, a hint or a message agents act on runs its area's eval tasks on the branch before merge, next to main (about $0.10 a task).
  • Candidates:
    • first-draft repeat removal: 2 of 3 sessions with no catch;
    • /simplify removal: back to 0, since S28's /simplify found real issues;
    • three new rows: the macOS suite time for a change that slows a frequent command, one yes/no question per AskUserQuestion call, and the "unrelated replacement" row for revert heuristics.
  • log.md: the S28 rows.

Verified

  • Test results:
    • node --test .claude/skills/ship/scripts/ship.test.mjs: 120 pass, with a new test for osHandlerUrls;
    • node --test scripts/agent-eval.test.mjs: 78 pass;
    • eslint clean.
  • A real eval with --budget=0.1 --run-budget=0.5 exited 1 with "no run started (budget: …)". It spent nothing.
  • Rendered implementer, reviewer, tester and verifier briefs: no <worktree> left, and the ban text passes --check.

…, eval before merge for verdict changes, run-wait 90 min, Downloads check in every brief, eval exits 1 when no run started
@szymdzum
szymdzum marked this pull request as ready for review October 22, 2026 15:22
@szymdzum
szymdzum merged commit d614709 into main Oct 22, 2026
15 checks passed
@szymdzum
szymdzum deleted the chore/ship-retro-s28 branch October 22, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant