feat(run): opt-in --stream for live task output (#507)

[0xGaspar]

Implements #507 — opt-in live child output for fledge run.

A correction to the issue's premise

The issue says child stdout/stderr "are not visible until the task exits." That is true for one of the two paths:

• The human-readable path calls Command::status(), which inherits fledge's stdio. Output there was always live and correctly interleaved.
• The --json path calls Command::output(), which buffers both pipes until exit and closes the child's stdin.

So the real defect is narrower and sharper than filed: --json is the mode that goes dark, and it is exactly the mode an agent or CI wrapper uses for a long-running task. --stream targets that path. Recording this so the scope isn't later mistaken for "fledge run was entirely buffered", which it never was.

Decisions

--stream --json streams and captures, mirroring to stderr. Both pipes are teed: bytes go to fledge's stderr as they arrive, and into buffers that fill the envelope's stdout/stderr exactly as before. Mirroring to stdout would interleave child bytes with the envelope and break every | jq consumer; rejecting the combination would deny the feature to the only mode that needs it. The envelope field set and schema_version are unchanged, and stdout/stderr are never silently emptied.

The child also inherits stdin under --stream, so prompts work — a real behavioral difference from the buffered path, captured as an invariant.

Ordering is per-stream, not cross-stream. Each stream is forwarded in order, and write_all on io::Stderr locks per chunk so a chunk is never split by the other. Relative interleaving between stdout and stderr is best-effort — two OS pipes drained by two threads cannot reconstruct the child's true write order. The one design that would preserve it (a single shared fd) makes stdout/stderr inseparable in the envelope, a worse trade. The issue's "forwarded in order" is met per-stream, and the limit is stated in the spec, docs, and code rather than glossed.

Non-TTY: forward anyway. No TTY probe. A flag whose effect silently vanishes in CI is surprising and untestable, and live logs are what a long CI task wants. Output is verbatim — no colour, prefixes, or line framing.

Also: --stream propagates to dependency tasks (an output mode, not a task input), and --stream without --json is an accepted documented no-op so wrappers can pass it unconditionally.

Implementation

pump<R: Read, W: Write> does the tee, generic so it unit-tests against in-memory buffers instead of racing pipes. run_streaming returns StreamedOutput { status, stdout, stderr } and writes through io::stderr() rather than holding a StderrLock for the stream's lifetime, which would starve the other thread until its pipe closed and defeat streaming.

Substantive logic is confined to src/run.rs; src/cli.rs, src/main.rs and src/watch.rs only declare and forward the flag.

Tests — all deterministic, no timing races

Liveness is asserted by the presence or absence of child bytes on fledge's stderr, which distinguishes the modes without racing them.

• Unit: pump capture/mirror byte-equality, partial line (a prompt with no newline), empty input, payload larger than the 8 KiB buffer; run_streaming stream separation, exit-code propagation (exit 7, output before failure retained), and byte-parity with Command::output.
• Integration: buffered stays the default; --stream --json mirrors both streams and still fills the envelope; stdout remains a single JSON document even when the task prints JSON-shaped text; a failing task reports exit_code: 3 with streamed and buffered envelopes asserted equal; deps stream; human-mode no-op keeps the task summary.

Manually verified: a 3s task under --stream --json showed tick-1..3 on stderr at t=1.5s with stdout still empty; an interactive read task produced hello-bruno under --stream versus hello- (stdin closed) buffered.

Verification

cargo test, cargo clippy --all-targets -- -D warnings, cargo fmt --check all green. specsync check exits 0.

Specs: run 6 → 7 with invariants 12–18 and REQ-run-008..013, all four companions updated. Docs: a "Streaming (--stream)" section in the CLI reference covering the intended use for long-running/interactive commands, per the issue's last acceptance criterion. Governance: CHG-0010, verified with 3 requirements evidenced.

Merge note

The introspect snapshot is the one real conflict risk with #505 — adding a flag changes it and #505 regenerates it too. Whoever merges second should re-run INSTA_UPDATE=always cargo test --bin fledge introspect_json_schema_snapshot. The cli.rs/main.rs edits are confined to the Run variant while #505's are in the spec area, so those merge trivially.

Closes #507

🤖 Generated with Claude Code

https://claude.ai/code/session_01KuhwrJF2XFDVHhX7qzDuyy

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[github-actions]

❌ Corvin says...

      _
    <(;\  .oO(oh no...)
     |/(\
      \(\\
      " "\\

"I'm pecking through the errors..."

CI Summary

Check	Status
Dependency Audit	✅ Passed
Integration (3 OS)	❌ skipped
Lint (fmt + clippy)	✅ Passed
Spec Validation	❌ failure
Tests (3 OS)	❌ failure

---

_{Powered by corvid-pet}

[0xLeif]

Review — REQUEST CHANGES

Automated review via Claude Code (/code-review), covering this PR alongside #505 and #504.

CI is currently red on this PR (test (windows-latest), spec-check, trust), and the review also turned up correctness bugs in the --stream implementation independent of that:

Blocker 1: write failure to the mirrored stderr stream discards the captured output/JSON envelope

src/run.rs:276 — pump() returns Err the instant sink.write_all(chunk)? fails, discarding the entire captured buffer even though the child process's real exit status is already known.

Repro: fledge run build --json --stream where fledge's own stderr becomes a pipe/file that stops accepting writes partway through (e.g. a downstream consumer exits early, or a full disk). The error propagates through run_streaming's out_handle.join()?? and execute_task's ? before the run_task envelope is ever built — the caller gets a bare I/O error instead of exit_code/stdout/stderr, with no indication the task actually succeeded.

Blocker 2: documented "stdout stays a single JSON document" guarantee is false for tasks with dependencies

src/cli.rs:203 (and the matching AGENTS.md line) — each recursive execute_task call for a dependency prints its own full JSON envelope to stdout.

Verified: fledge run build --json --stream where build depends on prep prints two concatenated JSON objects on stdout. Feeding that to serde_json::from_str / json.loads fails with a "trailing characters" error — exactly the long-running/multi-step scenario --stream is meant for, contradicting the newly added documented guarantee.

Blocker 3: stdout-pump join failure leaks the stderr-forwarder thread

src/run.rs:326 — in run_streaming, if out_handle.join() errors, the ?/?? short-circuits before err_handle.join() is ever called. err_handle's JoinHandle is dropped un-joined (which detaches, not stops, the thread in Rust), so it keeps reading the child's stderr pipe and writing to io::stderr() after execute_task has already bailed and the CLI is unwinding toward exit.

Verdict

Please fix the Windows test / spec-check / trust failures, then address the three correctness issues above (envelope loss on write failure, the single-JSON-document claim, and the leaked thread on join failure) before re-review.

Superseded by updated review.

[github-actions]

❌ Corvin says...

      _
    <(;\  .oO(oh no...)
     |/(\
      \(\\
      " "\\

"I'm pecking through the errors..."

CI Summary

Check	Status
Dependency Audit	✅ Passed
Integration (3 OS)	❌ skipped
Lint (fmt + clippy)	✅ Passed
Spec Validation	✅ Passed
Tests (3 OS)	❌ failure

---

_{Powered by corvid-pet}

Superseded by updated review.

[github-actions]

❌ Corvin says...

      _
    <(;\  .oO(oh no...)
     |/(\
      \(\\
      " "\\

"Even the dumpster of code seems empty today."

CI Summary

Check	Status
Dependency Audit	✅ Passed
Integration (3 OS)	✅ Passed
Lint (fmt + clippy)	✅ Passed
Spec Validation	❌ failure
Tests (3 OS)	✅ Passed

---

_{Powered by corvid-pet}

[0xLeif]

Approve — the three reds are a GitHub outage, not your code

All three 2026-08-13 blockers are genuinely fixed, and I verified each against source rather than the commit message:

1. Envelope loss on mirror failure — pump now records PumpOutcome::mirror_error, stops echoing, keeps capturing, and the envelope still prints with the child's real exit code.
2. Leaked stderr thread — join_pumps joins both handles unconditionally before evaluating either result.
3. The false "stdout stays a single JSON document" guarantee — corrected in docs rather than by changing behaviour, which is right: one envelope per executed task is pre-existing, and collapsing it would be a breaking contract change.

Each has a named regression test. The Windows failures were root-caused correctly (cmd treats ; as literal) and Windows CI is green.

On the red checks: audit, CodeQL (js-ts) and corvid-pet all died on No server is currently available during the GitHub API outage 17:33–17:42 UTC. audit actually printed {"vulnerabilities":{"found":false,"count":0}} and then failed. These need a re-run, not a fix. Every substantive check — lint, spec-check, test and Integration on all three OSes, trust, CodeQL actions — passed on bf1df42.

One thing to clear: the corvid-pet bot review still says "Spec Validation ❌" and describes commit 640dd2d, not head. spec-check is SUCCESS on head. That stale bot review is likely contributing to BLOCKED — worth dismissing.

Two things I checked that did NOT turn out to be problems, so nobody re-raises them:

• "Unconditional stdin inheritance deadlocks callers" reproduces, but the identical hang already exists on the default human-readable path on main and in every ordinary task runner. Documented at src/cli.rs:203-206, strictly opt-in.
• "The child buffers so you see nothing until exit" — I tested the matrix. Shell loops, Rust, Go, Node and Python-to-stderr all stream live over pipes; only C-stdio-to-stdout and unbuffered-less Python-to-stdout burst at exit. A one-clause docs caveat at most.

I also empirically checked the load-bearing concurrency claim: 2 threads × 2000 × 4096-byte writes through separate io::stderr() handles, 10 runs, zero torn chunks. The documented per-stream atomicity guarantee is real.

Three spec nits, all follow-ups, none blocking:

• specs/run/requirements.md:13 still asserts "exactly one JSON document on stdout" — the claim blocker 3 explicitly retracted. It contradicts REQ-run-010 twelve lines below. Same residue at run.spec.md:183.
• REQ-run-020/021/022 sit as ### under ## Out of Scope, so three normative requirements read as out-of-scope items — and they duplicate REQ-run-008..013.
• run.spec.md has two version-history rows for one change, with row 8 below row 1 in a descending table.

This is the rare follow-up that fixed causes rather than symptoms. The mirror-failure fix draws the right line — a failed echo is advisory, a failed read is fatal because capture would lie — and writes that reasoning down where the next person will find it.

🤖 Reviewed via Claude Code