feat: --max-time, so a stuck build ends itself and says why

[oyvindberg]

Why

Three Windows CI hangs produced no diagnostics. The cause turned out not to be a missing timeout — on the fourth occurrence the 1200s run-bounded.sh bound, the step's timeout-minutes: 25, and the job's 45 were all in place, and all three failed to end the step:

step  Test binary after build   17:20:17 -> null     (47m, never completed)
job   started 17:12:45  destroyed 18:07:45           (55m00s = 45m cap + 10m grace)
logs  BlobNotFound                                   (never uploaded)
telemetry steps  null                                (destroyed with the runner)

GitHub waits for the step's output pipes, not for the process. Raising caps cannot fix that, which is why three of them didn't. And the if: always() telemetry steps are steps — the hung step never yields the runner, so they never run.

What

bleep bounds itself instead:

bleep test --max-time 15m

On expiry it cancels the build, writes a thread dump of the client and the compile server, and exits non-zero. The step then ends normally and the telemetry steps run — diagnostics travel the path that already exists.

Where it deliberately stops: it does not kill the compile server. The client exiting is already enough to end a CI step, and killing a daemon serving other workspaces is collateral nobody asked for.

timeoutAndForget, not timeout

Measured, not reasoned. timeout cancels the fiber then waits for its finalizers, so against an uncancellable hang it does not bound anything. Swapping this one call makes MaxTimeTest's uncancellable case run to completion and return SUCCESS past its deadline — not merely late.

Two silent bugs in ChildProcessDiagnostics

Both made the dump useless exactly where it was needed:

• findJstack looked only under java.home, and only for jstack. The client is a native image with no JDK of its own, so it could never dump a child at all — and with no .exe candidate it found nothing on Windows even in the server, the platform where the hangs happen.
• dumpAll walked only descendants(). A shared compile server is nobody's descendant, so the dump omitted precisely the process doing the stuck work. Verified fixed against a daemon from an earlier run: client pid 73467 dumped server pid 73300, which is not its descendant.

Risk

Off unless asked for — no duration is right for both a laptop and a cold CI runner. That is also why it is per-invocation rather than a setting on the shared server, where whichever client spawned the daemon would impose its choice on everyone else. --max-time 30 is rejected rather than guessed.

CI sets it just under the bound already enforced (19m vs run-bounded.sh's 20m), so it can only convert an existing kill into a clean exit — it cannot fail a run that passes today. run-bounded.sh stays as the outer backstop for what this structurally cannot cover: the client unable to act at all.

Tests

MaxTimeTest is the in-process analogue of the bound-check job, for the reason that job exists — every rung is asserted against a program that really hangs, including that cancellation reaches the program rather than merely abandoning it, and that a failing dump reports both problems instead of replacing one with the other.

Verified end-to-end against a real compile: bound fires, message names the duration, 17KB dump written containing 53 server threads via jstack.

Also regenerates the CLI docs, which were stale for max-cached-workspaces from an earlier merge.

🤖 Generated with Claude Code

[oyvindberg]

Split: the ChildProcessDiagnostics fixes, the stale CLI docs, and the corrected step-duration record moved to #649, which stands on its own merits and is worth merging regardless of what happens here.

This PR is now --max-time alone, and it is deliberately left undecided until we hit the hang again.

Stating the case against it honestly: it rests on a hypothesis we could not confirm — that during the hang the client is alive and merely stuck. 13 consecutive Windows re-runs failed to reproduce, so the mechanism is unproven in production and MaxTimeTest is the only evidence it fires. If the box is wedged, or the client is dead while something else holds the step's pipes, this cannot help.

The case for it does not depend on the hang: a wall-clock bound on a build is a reasonable thing to want for CI, agents and scripts. Decide it on that.

What is well-evidenced either way: no false fires in 13 Windows runs plus build, with 2.7x headroom on the step it bounds.