Skip to content

Stop hera_send from restarting live workers; add worker conduct to hera skill - #1066

Merged
anutron merged 1 commit into
masterfrom
argus/sketch-guides-docs/r1-hera-worker-lessons
Oct 20, 2026
Merged

anutron merged 1 commit into
masterfrom
argus/sketch-guides-docs/r1-hera-worker-lessons

Conversation

@anutron

@anutron anutron commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Why

hera_send from a coordinator ran hera_revive's full check before delivering every message. That check treats "alive, idle for 3s, not parked at a prompt" as stuck and kicks the session (SIGTERM, then a resume via --session-id). That is the normal resting state of a worker between turns, or one waiting on a background job. So every coordinator message to an idle worker restarted it, and the agent's own shutdown killed its in-session background jobs (Bash run_in_background).

Evidence from ~/.argus/daemon.log on 2026-10-18/19: 120 outcome=kicked_stuck revives across 7 orchestrators. Each was followed within about a second by session exited stopped=true exit status 143 pending=true. The sketch-guides-docs video workers lost 20-minute frame renders and encodes at exactly those moments, and their transcripts report "another session restart" seconds after each kick.

What changes

  • Messages only revive dead recipients. hera_send's auto-revive now restarts a recipient only when its session is dead. It never kicks a live one; the message itself wakes an idle agent. A live recipient is reported as the new skipped_alive outcome.
    • New hera.ReviveDeadRole, which shares the dead-restart path with ReviveRole.
    • New mcp.HeraReviveInput.DeadOnly field; the daemon picks the primitive in reviveForInput.
  • Explicit hera_revive keeps its kick, now with an honest description. The tool description, the README table and the hera skill now say plainly that a kick restarts the agent and kills its in-session jobs, so it's a last resort for a truly wedged session.
  • Hera skill (internal/skills/builtin/hera/SKILL.md):
    • corrects §4's false claim that hera_revive is "a safe no-op" on a healthy role;
    • adds a coordinator rule: check progress through team updates or job logs, don't ping;
    • adds worker conduct rules: verification scope (the project's own pre-PR gate wins; otherwise run the narrowest checks and say what only ran in CI), and long jobs run under nohup with a log and an exit-code marker that is checked after any restart.
  • Spec: the OpenSpec change fix-hera-send-no-kick is archived in this PR. It replaces the hera-messaging requirement "auto-revives a dead or stuck recipient" with "auto-revives a dead recipient".
  • Gotchas: new entry in context/knowledge/gotchas/messaging.md, including the follow-up that was not built: an explicit kick could refuse while the agent has live job processes.

Testing

  • New tests:
    • internal/hera: ReviveDeadRole table test, covering a dead session (restarted) and live idle, busy, blocked and coordinator sessions (never kicked).
    • internal/mcp: the send path asks for DeadOnly: true; explicit hera_revive asks for DeadOnly: false.
    • internal/daemon: end-to-end reviveForInput against a real fixture session. A live session keeps the same PID with no pending restart; a dead one is restarted.
  • make pre-pr locally:
    • build, vet, fmt-check and lint-pr all passed.
    • vuln failed on 10 Go standard-library advisories. That check is advisory in CI (continue-on-error).
    • test-cover-gate: three failures in packages this PR doesn't touch, all of which pass when rerun on their own (load flakes per gotchas/ci-gates.md): internal/api TestTailscaleIPFromCLI_AcceptsCGNATOutput, internal/daemon TestHeraRecycleRunner_IsIdle_UsesContentStability, and an internal/tui 200s package timeout.

Frontend parity

hera_send is MCP-only, so there is no REST, TUI, web or macOS surface change. The TUI's Enter-key revive is operator-initiated and unchanged.

🤖 Generated with Claude Code

…ra skill

hera_send's auto-revive reused hera_revive's full gate, whose idle+not-blocked
branch kicked (SIGTERM + resume) every live idle worker a coordinator
messaged, killing the worker's in-session background jobs. The send path is
now dead-only (hera.ReviveDeadRole, skipped_alive for any live session);
explicit hera_revive keeps its kick, documented as a last resort.

The hera skill gains worker verification-scope and long-job-survival rules,
a coordinator check-progress-don't-ping rule, and corrects the claim that
hera_revive is a safe no-op on a healthy role.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

⚠️ Note: Baseline coverage from master branch is not available (artifact may be expired). Showing current coverage for changed files only.

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/drn/argus/internal/daemon 86.96% (+86.96%) 🌟
github.com/drn/argus/internal/hera 68.77% (+68.77%) 🌟
github.com/drn/argus/internal/mcp 87.83% (+87.83%) 🌟

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/drn/argus/internal/daemon/daemon.go 84.11% (+84.11%) 535 (+535) 450 (+450) 85 (+85) 🌟
github.com/drn/argus/internal/daemon/revive.go 88.10% (+88.10%) 42 (+42) 37 (+37) 5 (+5) 🌟
github.com/drn/argus/internal/hera/revive.go 96.77% (+96.77%) 31 (+31) 30 (+30) 1 (+1) 🌟
github.com/drn/argus/internal/mcp/hera.go 82.26% (+82.26%) 789 (+789) 649 (+649) 140 (+140) 🌟
github.com/drn/argus/internal/mcp/server.go 94.00% (+94.00%) 866 (+866) 814 (+814) 52 (+52) 🌟

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/drn/argus/internal/daemon/revive_test.go
  • github.com/drn/argus/internal/hera/revive_test.go
  • github.com/drn/argus/internal/mcp/hera_revive_test.go
  • github.com/drn/argus/internal/mcp/hera_send_revive_test.go

@anutron
anutron merged commit d1564f4 into master Oct 20, 2026
1 check passed
@anutron
anutron deleted the argus/sketch-guides-docs/r1-hera-worker-lessons branch October 20, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant