Skip to content

fix(maul): keep a harness's expectation lines in the fuzz report - #128

Merged
siracozmen01 merged 1 commit into
mainfrom
fix/fuzz-report-expectations
Nov 5, 2026
Merged

siracozmen01 merged 1 commit into
mainfrom
fix/fuzz-report-expectations

Conversation

@siracozmen01

Copy link
Copy Markdown
Contributor

Problem

When a fuzz harness breaks one of its own promises, CI's report cannot say which one. tools/fuzz_target.sh keeps only the sanitizers' FILE:LINE:COL: runtime error: lines and the ERROR: to SUMMARY: block, so a harness's own message is dropped. maul-audio's main went red on 2026-11-05 with fuzz_scene aborting in Expect from CheckRays, and the report showed only the stack: not which of its three checks failed or with what values, and the input stays on the runner by design.

Change

  • fuzz_target.sh also keeps up to five FILE:LINE: expectation failed: ... lines from the part of the log above libFuzzer's first input dump, the same part it already reads.
  • The header comment and docs/internals/maul-conventions.md name that form: a harness's failure message names the broken promise and the values that broke it, never the input's bytes.

A library adopts it by having its harness's check print in that form; nothing else changes, and a harness that prints nothing reports as before.

Verification

  • bash -n passes.
  • A fake target that prints an expectation line, a libFuzzer ERROR:/SUMMARY: block and then a mutation line, a byte dump and Base64: gives a report with the expectation line and the block, and none of the dump.

@siracozmen01 siracozmen01 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 8d73817: ready. No findings. Read, not run.

  • The cut still comes first. head=$(sed -E '/^(MS: [0-9]|0x...,|artifact_prefix=|Base64: )/,$d' "$log") (line 103) deletes from the first line of any dump form to the end. The new grep, like the runtime-error grep and the ERROR-to-SUMMARY sed beside it, reads only <<<"$head" (line 110), never $log. Nothing at or below a dump can reach the report, in a run or in a merge (whose dump has no MS: line, so it is cut at its byte, artifact_prefix= or Base64: line).
  • -m 5 bounds the lines. The pattern ^[^ ]+:[0-9]+: expectation failed: is anchored and differs from the sanitizer form (FILE:LINE:COL: runtime error:), so neither grep takes the other's lines.
  • The merge path needs nothing. A harness's expectation aborts, and libFuzzer's SIGABRT handler prints ==PID== ERROR: libFuzzer: deadly signal and writes a crash file. Either one already fails the merge check (lines 84-86: the new-file test and the ERROR: grep). The merge then falls through to the same report, which now carries the expectation line too, from the same cut.
  • The docs line and the header comment state the form and its limit: the promise and the values, never the input's bytes. The script cannot enforce that, so it rests on each harness's message, which is what the convention line now says.

The body has Problem, Change and Verification.

@siracozmen01
siracozmen01 merged commit b0d09a4 into main Nov 5, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant