Skip to content

fix(report): keep an out-of-range entry point from aborting synthesis - #303

Merged
danielplohmann merged 3 commits into
danielplohmann:masterfrom
r0ny123:upstream-synthesis-entry-point-range
Sep 7, 2026
Merged

fix(report): keep an out-of-range entry point from aborting synthesis#303
danielplohmann merged 3 commits into
danielplohmann:masterfrom
r0ny123:upstream-synthesis-entry-point-range

Conversation

@r0ny123

@r0ny123 r0ny123 commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

The Fuzzing workflow crashed fuzz_synthesis:

error: 'I' format requires 0 <= number <= 4294967295
  File "src/smda/synthesis/PeSynthesizer.py", line 411, in _buildMinimalOptionalHeader
    struct.pack_into("<I", opt, 16, entry_rva)

libFuzzer exited 77 after 116,018 executions having added 26 new units, which is the shape of newly reached coverage finding an old defect rather than of a new one being introduced.

Root cause

_resolveFunctionOffsets caps how far apart a report's functions may sit (MAX_IMAGE_SIZE), and that cap is what keeps every RVA derived from them inside a header field. The entry point is not a function offset, so the cap never sees it: a report may name an oep arbitrarily far from the image being rebuilt, and it reached the header packers unchecked.

Two of the three backends packed it anyway:

backend field width before
PE AddressOfEntryPoint 32-bit RVA raises, losing the whole image
ELF32 e_entry 32-bit silently truncates, entry ends up aimed elsewhere
ELF64 e_entry 64-bit fine
Mach-O LC_MAIN already correct: drops the command when the entry falls outside the text segment

The ELF32 case is the same defect one step quieter. Its bound was checked against the 64-bit space (MAX_ADDRESS_VALUE) on both branches while the 32-bit branch packs <I, so a report with oep = 2**64 - 1 produced e_entry = 0xffffffff and no complaint.

Fix

Both fall back to .text — which is what a report carrying no oep at all already gets, and what the Mach-O synthesizer has always done for an entry outside its text segment. Losing the entry-point field is a much smaller loss than losing the image, and every other out-of-range report input in these modules (sparse string ranges, overlapping IAT slots, oversized section spans) already warns and continues.

An entry past the end of the 64-bit address space still raises. base_addr and oep can each pass deserialization on their own and still sum past the end of the space, which only this derivation sees, and that describes no image at all — there is nothing to approximate.

The relative-or-absolute reading of oep had been copied verbatim into all three backends, so it moves to BinarySynthesizer._resolveEntryPoint and the next backend inherits the bound along with the reading. The reason the value is unbounded is now documented where it is produced rather than in one of the three places that consume it.

Sweep

The class is "a report-derived integer reaching a fixed-width header pack without a width check". The other report integers that reach a header pack — base_addr, binary_size, identified_alignment — were driven to 2**64 - 1 across three formats and both word sizes. No second instance; oep was the only one no span check bounded.

Not from the branch that found it

The failing fuzz run was on an unrelated AArch64 branch that flips one Mach-O config flag, and three independent things say the crash is not that branch's:

  1. That flag is read in exactly one place, the AArch64 candidate manager. fuzz_synthesis never disassembles — it builds a report from JSON and synthesizes — so the flag is unreachable from that target.
  2. The crash reproduces on master with the same traceback at the same line.
  3. Fuzz (fuzz_synthesis) had passed on an earlier run of that same head. Same code, different seeds.

Tests

  • tests/fuzz_regressions/synthesis_entry_point_beyond_rva_space_xored — the minimized reproducer (790 bytes, via fuzzing/minimize.py), pinned per that directory's convention. Verified to fail before the fix and pass after.
  • Eight cases in SynthesisEntryPointTestSuite, four of which fail without the fix: the PE fallback, the ELF32 fallback, and the two _resolveEntryPoint readings. The other four guard what the fix must not change — a valid PE entry is still kept, a 64-bit ELF entry above 4 GiB is still written verbatim, and a report with no oep still aims at the first executable section.

Validation

ruff check / ruff format --check pass, ty check exit 0, full suite green on this branch rebased onto current master (1821 passed, 1 skipped, 2585 subtests), diff coverage 100% on all 24 changed lines against upstream/master.

This is independent of #299 and #300 and merges cleanly against either.

The ty commit at the tip

The last commit on this branch is #302's fix, cherry-picked unchanged.

Master fails Code Quality on its own right now: 572acd7 bumped ty to 0.0.74, and its new unsound-return-statement / unsound-assignment rules become errors under [tool.ty.rules] all = "error". That is 41 errors across 15 files, none of which this PR touches — so every PR opened against master inherits a red check that says nothing about its own diff.

Carrying #302's commit here means this PR's CI reflects only this PR's work. It is the same commit, unmodified, so the moment #302 lands this one is empty and falls out on rebase — there is nothing to untangle later. If you would rather look at this branch without it, merge #302 first and I will drop it.

Checked against ty 0.0.74 itself rather than whatever an older local pin resolves to: on master it exits 1 with 41 errors, on this branch it exits 0 with none.

_resolveFunctionOffsets caps how far apart a report's functions may sit, and that
cap is what keeps every RVA derived from them inside a header field. The entry
point is not a function offset, so the cap never sees it: a report may name one
arbitrarily far from the image being rebuilt, and it reached the header packers
unchecked.

Two of the three backends packed it anyway. A PE optional header holds
AddressOfEntryPoint as a 32-bit RVA, so an oep more than 4 GiB above the image
base reached struct.pack_into and raised, losing the whole image over a field
that already has a perfectly good default. An ELF32 e_entry is 32 bits wide as
well, but its bound was checked against the 64-bit space, so the high half was
truncated away and the entry silently ended up aimed somewhere else.

Both now fall back to .text, which is what a report carrying no oep at all
already gets, and what the Mach-O synthesizer has always done for an entry
outside its text segment. An entry past the end of the 64-bit space still
raises: that describes no image at all, so there is nothing to approximate.

The relative-or-absolute reading of oep was copied into all three backends; it
moves to the shared base so the next backend inherits the bound along with it.

Sweeping the other report integers that reach a header pack -- base_addr,
binary_size, identified_alignment -- over three formats and both word sizes
found no third instance: oep was the only one no span check bounded.

The PE crash was found by the Fuzzing workflow; its minimized reproducer is
pinned under tests/fuzz_regressions/ and fails on the parent commit.
The repository asks for self-documenting code rather than comments. The two
facts worth keeping -- that oep is stored in both absolute and base-relative
form, and that no span check bounds it -- belong in _resolveEntryPoint's
docstring, where they were already stated; the guards themselves are readable
from their conditions and their messages.
@r0ny123
r0ny123 force-pushed the upstream-synthesis-entry-point-range branch from 1cd44a4 to ee3a573 Compare August 30, 2026 03:36
Keep the 0.0.74 bump and satisfy the new unsound-return/assignment/yield
rules with annotations on the public DTO and helper surfaces. Ignore ty
in Dependabot so the next patch release cannot redden master unattended.
@danielplohmann
danielplohmann merged commit 7eac4d0 into danielplohmann:master Sep 7, 2026
26 checks passed
@danielplohmann danielplohmann mentioned this pull request Sep 7, 2026
@r0ny123
r0ny123 deleted the upstream-synthesis-entry-point-range branch September 7, 2026 17:38
danielplohmann added a commit that referenced this pull request Sep 8, 2026
* chore: draft the v4.5.1 release notes

The changelog entry for what has landed since v4.5.0 (6cc5aca): #302, #303,
#304, #305, #306, #307, #308, #310 and #316, plus the Dependabot ty bump
572acd7 that #302 had to answer for.

Held as a draft rather than a release. The version is deliberately NOT
bumped here -- #309, #311 and #312 are reviewed and wanted but waiting on a
rebase, and splitting one coherent accuracy pass across two releases makes
both entries weaker. What this commit does is stop the writeup living in a
scratchpad while that happens.

To finish it: add sections for #309, #311 and #312, bump VERSION in
src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py, and move the
date to the actual release day.

No escaper output changed anywhere in this set -- the only edit to
intel/definitions.py adds two GAP_SEQUENCES entries, which feed padding
detection and not escaping, and testEscaperFingerprint passes unchanged --
so ESCAPER_DOWNWARD_COMPATIBILITY stays at 4.4.5 and
INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs reprocessing.

Figures are the contributors' own except where this session reproduced them,
and the two that were reproduced are stated as measured here: the padding
cut's analysis-time cost (+3.4% claimed, +3.84% measured) and the fixture
movement on rust_pe_gnu_xored (+26 real starts against +8 false, scored
against that fixture's own COFF symbol table rather than the corpus macro
mean).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: fold #318 into the v4.5.1 release notes

#318 landed as 395c88d after this entry was written. Its five corrections go
under Housekeeping: the README's metadata.language claim, the restored
`-> bool`, the ruff-check hook rename, the GAP_SEQUENCES reach note, and the
docstring on the queue-rebuild test.

Keeping this current as things merge is the point of the branch existing, so
it does not go stale between now and the release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: cut v4.5.1

Finishes the draft now that #309, #311 and #312 have landed. Adds their
sections, bumps `VERSION` and `__version__` to 4.5.1, and moves the date to the
release day.

The three new entries: #309's exception-directory refusal of interior gap
candidates on x64, with the chained-versus-primary distinction and the
seeding-is-not-suppression principle that keeps carved records out of it; #311's
four candidate-quality defects, including the corrected entry at `0x40df30` on
`aarch64_static_xored` and the metadata-coverage gate; and #312's ARM64
counterpart, which reconstructs the extent an ARM64 `RUNTIME_FUNCTION` does not
carry and hoists the shared lookup into `common/`.

`USE_MACHO_ADDRESS_REF_CANDIDATES` gets its own **Defaults changed** section. It
is the only default whose value moves in this release, it changes recovery
output on Mach-O input with no action by the caller, and a default going from
off to on inside a follow-ups PR is exactly the kind of thing that gets lost.

The closing movement section is rewritten to describe v4.5.0 -> v4.5.1 rather
than any single PR, and both figures in it were measured on this tree against
independent truth rather than quoted:

  rust_pe_gnu_xored, vs its 2,186 retained COFF .text function symbols
    2,355 -> 2,201 starts, +26 real / -180 false
    recall    92.452% -> 93.641%
    precision 85.817% -> 93.003%

  11-sample ARM64 Mach-O corpus, vs LC_FUNCTION_STARTS (2,056 truth)
    TP 1,787 -> 1,818, FP 1,113 -> 1,107
    recall 86.916% -> 88.424%, no sample losing recall

Two headline figures are marked as the contributor's own because they cannot be
reproduced here at all: the `bti j` 803/0 split, since no bundled fixture holds
a single `bti j` word, and #312's ARM64 PE result, since a scan of all 104
fixture files finds 8 i386 PEs, 4 AMD64, one ReadyToRun image and no `0xAA64`.
Saying which figures are ours and which are theirs is more useful than a uniform
tone of confidence.

`Turtle_5f9cd91d8d1d`'s 12 dropped starts are attributed rather than left as an
unexplained delta: they bisect to #307's inbound-call fix and fall inside the
175 false positives that change already measured, and the sample carries no
LC_FUNCTION_STARTS to score them against individually.

Closes #319

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

* chore: record binweight's serialized type change in the v4.5.1 entry

Left open as a question in yesterday's review and resolved as "yes, it belongs
in the entry". #302 moved `SmdaFunction.binweight`'s class default from int `0`
to float `0.0` while clearing ty 0.0.74's diagnostics, and `binweight` is
serialized through `toDict()`.

The blast radius is small but real: the per-block accumulation already adds
`float(...)`, so every function with at least one block was a float before this
release too. Only a function with no blocks -- a zero-function or error report
-- keeps the class default, and that value now writes as `0.0` where it wrote
`0`. Verified on both trees rather than reasoned about: `SmdaFunction.binweight`
is `0` at 6cc5aca and `0.0` at d111548.

MCRIT stores these reports, so a consumer diffing them byte-for-byte will see
it even though nothing reads the field as an integer. That is exactly the kind
of change that costs someone an afternoon if it is not written down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

* chore: fold the two remaining #317 items into the v4.5.1 entry

#320 and #321 finish the review pass that #318 started, so the release entry
should carry all seven of its corrections rather than five.

Both are behaviour-neutral typing work, but the Housekeeping text says what each
one actually decided rather than listing them as annotations: `blocks` being
typed forces the PIC/OPC hash path to refuse an instruction with no bytes
instead of blanking it, and `extract_strings` could not be narrowed until three
helpers and three `SmdaReport` attributes were declared first.

Behaviour-neutrality is stated with the evidence rather than as a claim -- the
corpus benchmark ran on #320 and reports 0 of 155 files differing across
175,946 functions, which is a stronger statement than the bundled fixtures can
make.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

* chore: format the v4.5.1 entry as a nested list

@r0ny123 asked on #319 whether the changelog could be formatted as a list to
make it easier to read. At ~2,700 words on a single line, this entry was the one
that provoked the question, so it gets the treatment now rather than waiting for
the wider keep-a-changelog adoption tracked in #323.

Deliberately narrow: only the v4.5.1 entry changes shape. The other 192 entries
keep the one-line format, because converting them is a large, low-value rewrite
that loses nuance in translation, and #323 proposes preserving them verbatim
under an `Older releases` heading instead.

Structure is the release summary as the top-level bullet, one sub-bullet per
section, and a further level inside the three sections that cover more than one
change -- Recovery (intel) splits into the switch-table fix, the alignment cut
and the exception-directory refusal; Recovery (AArch64) into the inbound-call
fix, the BTI work and the candidate-quality follow-ups; Housekeeping into the
`ty` bump, the `ruff` pin, the five #318 corrections, the `binweight`
serialization note and the two #317 typing items.

The text is unchanged. Verified mechanically rather than by eye: stripping list
markers and collapsing whitespace gives a string identical to the original entry.
The two exceptions are disclosed rather than silent -- the first sub-bullet of
Recovery (intel) and of Recovery (AArch64) had their opening letter capitalised,
because splitting the section header onto its own line left them starting a
bullet mid-sentence. Housekeeping's opens on a code span and needed nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants