Repository navigation
ci: redirect fuzzer stdout to /dev/null - #1881
Merged
Merged
Conversation
Guest output can exhaust runner log storage. Fuzzer diagnostics use stderr. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
andreiltd
requested review from
danbugs,
dblnz,
devigned,
jprendes,
jsturtevant,
ludfjig,
simongdavies,
squillace and
syntactically
as code owners
October 5, 2026 08:44
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The Linux-only redirection safely preserves diagnostics on stderr and command failures.
Review effort: Balanced
Findings: None
What changed in this PR
Suppresses verbose fuzzer and guest stdout to prevent runner log exhaustion while preserving stderr diagnostics and exit status.
Changes:
- Redirects timed fuzzing stdout to
/dev/null.
| File | Description |
|---|---|
.github/workflows/dep_fuzzing.yml |
Suppresses fuzzing stdout in CI. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Benchmark ResultsMeasured commit: kvm / amd (Linux) (❌ 3.79x)Top regressions
Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
kvm / intel (Linux) (❌ 2.42x)Top regressions
Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
mshv3 / amd (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
mshv3 / intel (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
hyperv-ws2025 / amd (Windows) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
hyperv-ws2025 / intel (Windows) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
Reported by |
Member
Author
|
Closes: #1854 |
yoshuawuyts
approved these changes
Oct 5, 2026
jprendes
added a commit
to jprendes/hyperlight
that referenced
this pull request
Oct 6, 2026
A pull request is measured against the daily run at the commit it branched from, found by walking the successful runs of that workflow. Asking GitHub for only the successful ones is what made it answer out of a cold index: that listing came back weeks stale in four of ten tries, stably enough that two replies agreed on it, while the same listing unnarrowed was current every time. The narrowing happens here now. PR hyperlight-dev#1881 branched at `66d5dd3e` on 2 October and was reported against `c3e1d311` from 4 September. Of 114 table cells, 102 held no comparison at all, because most of the reported benchmarks did not exist a month earlier, and two configurations announced a regression of 3.79x for a change that redirects fuzzer output. Reading the same run now finds `66d5dd3e` itself and fills every cell. A baseline that far back reads the same as one next door, so the run says how many commits separate them. Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
jprendes
added a commit
that referenced
this pull request
Oct 7, 2026
A pull request is measured against the daily benchmark run at the commit it branched from. That run was being chosen out of a listing GitHub served stale, so reports compared against commits weeks older than the branch point. PR #1881 branched at `66d5dd3e` on 2 October and was reported against `c3e1d311` from 4 September. **102 of its 114 table cells held no comparison at all**, because most of the reported benchmarks did not exist a month earlier, and two configurations announced a 3.79x regression for a change that redirects fuzzer output. ## Cause `run_at` asked GitHub for the successful runs of the workflow. Narrowing a listing server-side is what leaves it answering out of a cold index: ``` ?status=success newest run 2026-09-06, in 4 of 10 tries no filter newest run 2026-10-05, in 5 of 5 tries ``` The runs it omits are indistinguishable from the ones it returns, both `status=completed conclusion=success`. The existing guard asks until two replies agree on the newest run, which does not help: the stale answer is stable, so consecutive replies agree on it. The narrowing happens locally now, on a `conclusion` field the listing already carries. ## Also here The run reports how many commits separate the baseline from the branch point, read from the comparison it already performs: ``` Pull request 1881 branched at 66d5dd3 Taking benchmarks of 66d5dd3, 0 commits back ``` A baseline a month back reads the same as one next door. This is what would have made the bug visible the first time it happened. ## Verified Re-running the same report against the same run: | | cells | without baseline | verdicts | |:---------|------:|-----------------:|:---------| | before | 114 | 102 (89%) | ❌ 3.79x, ❌ 2.42x, stable x4 | | after | 114 | 0 (0%) | stable x6 | `latest_run_for` still filters by `--branch`, the same class of server-side narrowing. It measured current in 5 of 5 tries, so this leaves it alone. Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Guest output can exhaust runner log storage. Fuzzer diagnostics use stderr.