Skip to content

fix: keep the overflow fixture child out of the profile writer (#275) - #278

Merged
HemSoft merged 1 commit into
mainfrom
t3/fix-metrics-profile-truncation
Oct 27, 2026
Merged

HemSoft merged 1 commit into
mainfrom
t3/fix-metrics-profile-truncation

Conversation

@HemSoft

@HemSoft HemSoft commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #275.

Problem

Post-merge main CI for 7eb98e1 failed only in rust-metrics (run 40427010240). Every instrumented test passed, but strict LLVM profile merging rejected one raw profile. Its retained bytes are a valid v11 header followed by data sections cut off at 4096 of the 21800 declared bytes. The metadata identifies the command_runner test binary.

The scoped-parent fixture ran a finite emit stdout 4194305 child under a 4 MiB cap. The runner fails on the overflow byte and SIGKILLs the process group. The child exits right after writing that byte, and in instrumented builds its exit handler may already be writing the raw profile. The kill can therefore land mid-write. No trace exists from the hosted run, so this PR does not claim that this exact interleaving produced that file. It shows the interleaving is reachable and removes it.

Change

  • Fix (test fixture only): the scoped-parent overflow child is now held. It emits 4 MiB + 1 bytes, then idles until the runner kills it, so it never enters its exit-time profile writer. The 4 MiB cap assertion still fails RED if the cap is ignored: the child would then hit the deadline and return TimedOut instead of Failed. Product runner code, the output cap, deadlines, group cleanup and the other fixture modes are unchanged.
  • Profile-boundary controls (tests/command_runner/profile.rs): compiled only under cfg(coverage) (set by cargo-llvm-cov) on Linux and macOS. Each helper uses the runtime's public __llvm_profile_set_file_object to send its own raw profile through a fopencookie/funopen stream, so every runtime write is a traced boundary. The final byte goes through the real SystemCommandRunner.run_with_options with a 4 MiB cap:
    • written: the profile is fully written before the final byte, then the helper is killed. The profile parses Complete.
    • interrupted: the helper exits normally, and the exit-time writer releases the final byte right after the header write. The kill lands inside the runtime's write, and the file is Truncated against its own header's declared size. This is the deliberate negative control.
    • held: runs the fixture's held() exactly at the cap, so only the deadline ends it (TimedOut). No write may start and no profile may exist. If held() exits, the runner returns Ok and the control fails.
    • Every produced profile is parsed strictly (magic, version 11, value kinds, exact declared section extent). Helper termination is checked with the existing lock/PID released() probe.
  • Workspace unexpected_cfgs now declares cfg(coverage) so clippy -D warnings accepts the gate.

The control helpers send only their own profiles to scratch. Production profiles under llvm-cov-target, coverage scope, thresholds and failure-retention strictness are unchanged.

Evidence (local, macOS aarch64, pinned nightly-2026-10-22 + cargo-llvm-cov 0.9.1)

  • Instrumented command_runner passes:
    • written: events write 0 152 … written, 30632 B, Complete
    • interrupted: write 0 152, release 152, 152 B, Truncated (declared 30632)
    • held: TimedOut, events ["diverted"], no profile
  • RED check: temporarily making held() exit after emitting fails the held control deterministically (None vs Some(TimedOut), exit 101). Restored, it passes.
  • All 80 profiles from the local instrumented runs pass strict llvm-profdata merge -sparse (exit 0).
  • Parser arithmetic cross-check: all 21 accepted hosted Linux raws from the failed run parse Complete, and the rejected raw parses Truncated (4096 of 21800).
  • Gates: cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings (plus provider tests with --cfg coverage), cargo test --workspace --locked, and scripts/tests/test_rust_metrics.py (13 OK).

Acceptance Criteria

  • A deterministic actual-runtime write-boundary control demonstrates reachable interruption through the real output-cap/owned-group-kill path on hosted Linux.
  • The overflow fixture stays alive until runner termination; cap, deadline, cleanup, and existing result assertions remain effective.
  • Strict production profile merging, scope, thresholds, baseline, and product runner behavior remain unchanged. The deliberately interrupted negative control is fixture-owned evidence, not an accepted production profile.

Definition of Done

  • Current-head Codex review clean, zero unresolved threads.
  • All seven CI jobs and all three Package jobs passed at ab7e8caa475713cd9ef020cba96f2bdda7ec41d1.
  • No production UI changed, so rendered UI screenshots are not applicable. No Home, vendor or account operation was used.

Hosted validation

The Linux metrics job ran the controls once:

Control Result Retained profile
written Output overflow, Failed, 55 ms 24,224 bytes, Complete
interrupted Output overflow, Failed, 53 ms; trace ends write 0 152, release 152 152 bytes, Truncated; header declares 24,224
held TimedOut, 1,001 ms; only diverted event No profile write or file

The unchanged strict merge and comparison passed with failures: []. Base 7eb98e1c801c20472cd4944dea8cc7697918bedf and candidate ab7e8caa475713cd9ef020cba96f2bdda7ec41d1 have identical measured production totals: 20,642/23,450 lines, 88.025586%, and 2,945/3,692 branches, 79.767064%; all 1,790 production function measurements remain in scope. The artifact was downloaded and verified against SHA-256 f0fafe947aabc30494717b7b4da73ed1cd3a9c3eabddfeabcefa16c0be718a51.

These controls establish mechanism reachability and remove the fixture's exit/kill race. They do not reconstruct the historical signal sequence of the original 4,096-byte failure. That evidence remains preserved.

The scoped-parent fixture's finite overflow child exited right after its
final byte. The runner kills the group as soon as it reads that byte, so
in instrumented builds the kill could land inside the child's exit-time
LLVM profile write and leave a truncated raw profile.

The overflow child now idles until the runner kills it. Instrumented
builds also run three profile-boundary controls through the real
SystemCommandRunner: a completed write stays valid, a write interrupted
at a runtime write boundary is truncated, and the fixture's held child
never starts a write. Every produced profile is parsed strictly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HemSoft

HemSoft commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T02:03:28.524634Z ab7e8ca Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: ab7e8caa47

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@HemSoft
HemSoft marked this pull request as ready for review October 27, 2026 02:02
@HemSoft
HemSoft merged commit 1690d01 into main Oct 27, 2026
10 checks passed
@HemSoft
HemSoft deleted the t3/fix-metrics-profile-truncation branch October 30, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent truncated command-runner profiles from failing metrics CI

1 participant