Skip to content

feat(core): function-boundary accuracy from what the image declares - #300

Merged
danielplohmann merged 4 commits into
danielplohmann:masterfrom
r0ny123:accuracy/engine-enhancements
Sep 10, 2026
Merged

feat(core): function-boundary accuracy from what the image declares#300
danielplohmann merged 4 commits into
danielplohmann:masterfrom
r0ny123:accuracy/engine-enhancements

Conversation

@r0ny123

@r0ny123 r0ny123 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Current state

Rebased onto 6240b74 and fully re-measured. Every figure in this description is the re-measured one. The original description carried a table measured before #304/#307/#309/#310/#311 landed, which is what prompted the review request; that table is gone rather than annotated, because most of what it claimed for this branch is now attributable to those merged PRs instead.

  • The rebase itself, the eleven conflict hunks and the two that needed a real decision: rebase and re-measurement.
  • The 260-cell C/C++ matrix rebuilt from source, and the tailcall-gate question resolved with data: follow-ups.

Two things a reviewer should know before reading the diff:

The AArch64 headroom moved. Master's own PPV on the 72-cell AArch64 ELF corpus has gone from 76.676, which this PR originally recorded for it, to 91.497 today. Most of what the old table claimed is now merged. What is left on top of it is 3.450 points of precision and 3,645 false positives, with recall up rather than traded.

One change is a recall trade, and it is the one open decision. Section 5's tailcall-seed gate costs 53 true positives on the AArch64 ELF corpus while removing 580 false positives. By the strict per-change rule this branch set itself, that is a reject. It is kept, for reasons set out in the rebase comment, and the trade largely dissolves once USE_ELF_EH_FRAME_CANDIDATES is on — measured there the gate costs 13 and gains 24, net +11 functions against −583 false positives, because gating the wrong seed lets the deferred FDE pass claim the right start. That flag is untouched here and still defaults off; the measurement is in the follow-up comment. Happy to drop the gate instead — it is a one-line revert.

The short version

SMDA gets better at answering "where does a function start?" — ten engine changes plus one fix to the benchmark workflow, and the engine changes are all variations on one idea: when the binary already tells you something, believe the binary instead of guessing from the bytes.

Compilers write down a lot that the engine was not reading. An ELF says, in .eh_frame, exactly which address range belongs to which routine. It says, in .gcc_except_table, exactly which addresses are exception landing pads. A PE says, in its data directory, exactly where the exception table lives. A COFF header says outright whether the image is 32- or 64-bit and which instruction set it targets. Every one of those was previously either ignored or re-derived from a byte heuristic that gets it wrong on a predictable class of binaries.

Nothing here is a new heuristic. Every rule that refuses a candidate refuses it because a structure the compiler emitted says the address is inside a function, not at the start of one.

The numbers

Measured against compiler symbol tables, on corpora built from source, both trees run back to back on the same machine. Arithmetic macro mean (mean of per-binary rates), master 6240b74 → this branch:

corpus n PPV before → after TPR before → after ΔFP ΔTP
Built C/C++ AArch64 ELF (gcc cross) 72 91.497 → 94.947 97.705 → 98.069 −3,645 +151
Built Rust (gnu targets) 24 79.659 → 87.480 98.237 → 98.361 −1,641 +19
Built Go (pclntab truth) 47 95.361 → 95.626 99.367 → 99.367 −408 0
ARM64 Mach-O (LC_FUNCTION_STARTS) 11 94.281 → 94.499 96.402 → 97.207 −27 +38
Built C/C++, MinGW PE cells 120 bit-identical bit-identical 0 0
ByteWeight msvc10-64 68 bit-identical bit-identical 0 0
ByteWeight msvc10-64, headers stripped 56 bit-identical bit-identical 0 0
Malpedia dumps (.fnmap truth) 57 92.645 → 92.648 98.552 → 98.552 −1 0

The full 260-cell C/C++ matrix, rebuilt from source (gcc 13.3, clang 18.1, mingw 13.2; 260 of 260 cells, 213,706 truth functions): PPV 94.038 → 96.908, TPR 97.001 → 97.074, F1 95.298 → 96.912, FP −12,853, TP +210. That is a fresh paired A/B on a rebuilt corpus rather than the successor to any earlier figure.

No corpus loses recall for the branch as a whole, and five proposals that bought precision by dropping real functions were measured and left out. One change does trade: the tailcall-seed gate in section 5 costs 53 true positives on the 72-cell AArch64 ELF corpus while removing 580 false positives. That corpus was not measured for it originally, it is a reject by the strict per-change rule, and it is discussed in the comments rather than settled here.

The two bit-identical rows are the control, not filler. Go binaries carry no .eh_frame landing pads and Mach-O carries no .eh_frame at all, so if either had moved it would have meant a rule was firing somewhere it had no business firing.

On the malware-dump corpus (malpedia, n=57, .fnmap truth) the branch is level with master: PPV 92.645 → 92.648, TPR 98.552 → 98.552, one false positive apart. Expected — those are packed Windows dumps and most of these rules read ELF unwind data. The one change that does reach them is the container-header fix below, which is worth F1 +0.155 on that corpus when bitness is withheld.

What actually changed, one at a time

1. If the buffer has a header, read the header

Disassembler.declaredArchitecture() and BitnessAnalyzer._declaredBitness().

A memory dump of a mapped image still begins with the headers it was mapped from. Those headers say "x64" or "AArch64" outright. The engine was instead scoring REX.W prefix density to guess bitness, and running a byte probe to guess the instruction set — both of which are good heuristics that are occasionally just wrong, and when they are wrong every block in the report is wrong.

So: if the buffer parses as a PE, ELF or Mach-O and names an instruction set a backend exists for, use it. Otherwise fall through to exactly the same probes as before. A headerless dump or a shellcode blob is unaffected, which is the point — this narrows where the guessing happens, it does not replace it.

A managed PE deliberately still routes to the Intel backend rather than to cil: CLR metadata is addressed by file offset, and a mapped dump no longer has file offsets, so routing on the header alone would lose it.

Worth 93 functions recovered and 64 false positives removed on the malware corpus with bitness withheld.

2. The PE exception table's address is declared, not conventional

BinaryInfo.getExceptionDirectory(), and locateExceptionHandlerCandidates now reads from it.

The engine was finding the x64 exception table by looking for a section literally named .pdata. That is just the name MSVC happens to use. The location is declared in the PE data directory, and a linker may put the table anywhere — the .NET ReadyToRun compiler puts it in .data.

Read the directory; fall back to the .pdata section name only if the image declares no directory entry.

On a ReadyToRun image with 626 declared function starts: 419 → 626 functions, which is 626 of the 626 starts the exception directory declares, with nothing recovered that it does not name. Fixture tests/dotnet_readytorun_pe_xored covers it.

3. A call that never returns is a function boundary (AArch64)

AArch64Backend._callFallthroughFunctionStart, plus a new opens_stack_frame() helper.

If a function's last act is to call something that never returns — abort, _Unwind_Resume, a panic handler — there is no ret after it. Decoding runs straight off the end of the function and into the next one, and the two get merged into one.

The existing checks for this all look for a reason to stop: alignment padding, an existing candidate, a NOP run. When the next function is packed right up against this one with none of those, they all decline.

New check: does the very next instruction open a stack frame? On AArch64 that means sub sp, sp, #imm followed within three instructions by stp x29, x30, [sp, #imm] — allocate a frame, then write the frame record into it. Neither half is conclusive on its own (an alloca does the first, plenty of code does the second), but the pair is: nothing mid-function re-saves the incoming link register into a frame it just created.

ARM64 Mach-O, n=11, as part of the branch: PPV 94.281 → 94.499, TPR 96.402 → 97.207, +38 functions against −27 false positives.

4. A prologue that begins exactly where another prologue ends is not a function

intel/FunctionCandidateManager._opensInsideAnEarlierPrologue.

clang opens a frame with push rbp; mov rbp, rsp and immediately follows it with the callee-saved run push r15; push r14. Both byte sequences are on the seeded prologue list. So the scan finds the real function start, and then finds a second "function" four bytes into the same function's body.

Fix: if a seeded pattern match begins exactly where an earlier seeded pattern ends, and that earlier address is already a candidate, refuse it. (The "already a candidate" condition is what keeps it from firing on a byte coincidence.) This mirrors the existing MSVC hotpatch-pad adjustment right below it.

Across ten corpora: 912 false positives removed, 0 true positives lost.

5. After a bl, the cut is what recovers the function — the seed makes it worse

AArch64Backend._analyzeCallInstruction.

When the AArch64 backend detects a fall-through past a bl, it was doing two things: cutting the caller short there, and seeding the boundary as a tailcall candidate. Turns out the cut is what recovers the next function; the seed then re-books the same address with worse extents than the ordinary candidate machinery would give it.

The seed is now behind RESOLVE_TAILCALLS (default off, same flag the shared engine already gates its tailcall promotion behind). The cut still happens either way.

Re-measured against 6240b74, gate off → gate on: Go n=47, −408 false positives at identical TP; ARM64 Mach-O n=11, −27 false positives, +11 functions; Built C/C++ AArch64 ELF n=72, −580 false positives against −53 functions. That last row is the recall trade noted above.

6. The candidate snapshot is taken before analysis, so it can't be the whole answer

AArch64Backend._isKnownFunctionStart and two sites in the AArch64 candidate manager.

Several checks asked "is this address in getFunctionStartCandidates()?" to decide whether a branch target is a real entry or somebody's interior. But that set is snapshotted before analysis begins, and gap analysis never adds to it. So a function that gap analysis discovered is in code_map and in disassembly.functions — but absent from the candidate set, and therefore indistinguishable from interior code.

Every such check now asks the live function set too.

ARM64 Mach-O n=11: 8 recovered, 0 new false positives. It also lifts the Mach-O fixture's primary pass from 246 to 269 functions, which is why that test's baseline moves — see the note in tests/testMachoFunctionStartCandidates.py.

7. endbr64 is an indirect-branch marker, not a function marker

_seedPrologueMatches(..., refuse_declared_interior=True).

Under -fcf-protection, gcc and clang put endbr64 at every address an indirect branch can land on. That includes every jump-table case label and every exception landing pad — addresses squarely inside a function body. The engine seeded all of them as function starts.

The image says which is which: an FDE in .eh_frame covers exactly one routine, so an endbr64 that is not its own FDE range's start is inside that routine.

This is applied to the endbr64 pattern only, and deliberately: it is the only seeded pattern that names a place a branch can arrive rather than a way a function opens. Every other pattern in the list is a genuine prologue shape and is left alone. The check also resolves the FDE ranges once up front, so an image with no readable .eh_frame pays nothing.

8. A declared landing pad is interior by construction

SmdaConfig.USE_LSDA_LANDING_PADS (new, default on), EhFrameDecoder.decodeEhFrameLandingPads(), and the gap scans of both backends.

.gcc_except_table is the compiler telling the unwinder "if an exception escapes this call site, resume execution here". "Here" is by definition inside a function — that is the whole meaning of the record.

These addresses were being booked as functions because they are the perfect storm for a byte scan: they open with the instruction set's indirect-branch marker (endbr64 on x86, bti on AArch64), and they sit in gaps precisely because nothing in the function branches to them. Every signal a gap scan has says "function start".

So the decoder now reads the LSDA call-site table and collects the declared pads, and both gap scans refuse a candidate at one.

Where the scan resumes is the whole decision. Stepping one instruction past a refused pad lands inside the pad and books that instead — a worse candidate than the one you just refused. Resuming at the end of the FDE that declared the pad passes over that function's body and nothing else. Across the three corpora carrying pads, 0 of 41,215 pads have a declared function start between the pad and that resume point, and 0 sit inside a PLT section.

On AArch64 the rule also has to run in locatePrologueCandidates, not just the gap scan — bti is a recognised entry prologue there, so pads reach the prologue pass directly. On x86 endbr64 is not a prologue shape, so the gap scan is the only path.

Per-corpus attribution for this rule was measured on the pre-rebase tree and is superseded by the re-measurement in the comments; the branch totals are in the table at the top. It also runs faster on pad-heavy images — 3.8% on the two heaviest cells — because the candidates it refuses are candidates nothing then has to analyse.

9. A gap candidate strictly inside a declared range, generally

SmdaConfig.USE_ELF_FDE_INTERIOR_GAPS (new, default on), both gap scans.

Same idea as section 8, wider net. Section 8 refuses only the subset an LSDA names as a landing pad; this refuses any gap candidate strictly inside a range .eh_frame declares, and resumes at that range's end. It is what catches the jump-table case labels a switch emits under -fcf-protection, and it is the single largest precision mechanism measured on this corpus.

Two guards make it safe, and both were found by measuring what it cost without them.

A PLT is exempt. The whole PLT sits under one FDE, so without the exemption the range test reads every stub after the first as interior to the first. That costs 3,457 real functions — on a CET image the gap scan is what recovers the stubs at all.

And the range's own start must already be a recovered function. An FDE can begin in the alignment padding ahead of its function, and then the real entry a few bytes in is interior to nothing. The remaining 35 losses were all of that shape — two per statically linked cell, one of them rt_sigreturn under a signal-frame CIE.

Re-measured against 6240b74, everything else in the branch on, adding one rule at a time:

corpus variant PPV TPR TP FP
AArch64 ELF, n=72 both rules off 91.554 98.045 59,495 6,062
AArch64 ELF, n=72 + USE_LSDA_LANDING_PADS 93.176 98.067 59,513 4,330
AArch64 ELF, n=72 + USE_ELF_FDE_INTERIOR_GAPS 94.947 98.069 59,516 2,484
Rust, n=24 both rules off 82.805 98.237 33,298 6,573
Rust, n=24 + USE_LSDA_LANDING_PADS 86.303 98.321 33,311 5,919
Rust, n=24 + USE_ELF_FDE_INTERIOR_GAPS 87.480 98.361 33,317 5,722

Not an engine change: the benchmark workflow was comparing two machines

.github/workflows/perf_benchmark.yml and .github/workflows/scripts/evaluate_runtime.py.

Worth flagging because it was producing confidently wrong verdicts. The workflow timed the base branch on one runner and the PR branch on another, an hour apart. It once reported a branch 13.18% slower at p = 0.0000 when the only source change in it was AArch64-only code measured on an x86 corpus — a change that cannot have touched a single instruction executed in that benchmark.

Base and PR are now timed interleaved in a single job, leading side rotated per pass, and the reported noise band describes what it actually measures.

What endbr64 was costing, end to end

Rules 7, 8 and 9 all attack the same root cause from different angles, so the useful number is the combined one.

Over six representative C/C++ cells, endbr64 seeded as a function start accounted for 822 false positives. After these three rules it accounts for zero — and true positives on those same cells go up, from 5,695 to 5,718.

Robustness

Both decoders read structures that the analysed file controls, so both are bounded on axes no fixture would naturally exercise.

A 205 KB .eh_frame whose records each named a 64 KB LSDA took 155 seconds to decode, scaling linearly. Each LSDA is now decoded once and memoised, and a per-section budget bounds both the call-site table bytes decoded and the reads that precede them. Same input: 0.047s. 200,000 FDEs with genuinely distinct LSDAs: 2.5s. For scale, the heaviest real image in any of these corpora decodes 31 KB of call-site tables, and libstdc++.so.6 decodes 26 KB.

A decoded landing pad that falls outside the range its own FDE declares is refused, because the format guarantees it cannot. That matters: on one NativeAOT image, LSDA pointers led into arbitrary data whose "headers" parsed cleanly and produced 4,826 fabricated pads. Across four corpora and three system libraries, 43,881 genuine pads are every single one inside their own FDE, and all 4,828 spurious ones are outside.

A second bound came out of review, in 71612d3. The exception-table walk reads one 12-byte record per iteration over a range the image declares, and never polled the analysis budget. It is bounded — a short read ends it — but bounding is not stopping: on an 8 MB buffer declared as one table it walked all 699,050 entries with the deadline already passed, exactly as many as with time still on the clock. Reading the table from the data directory rather than the .pdata extent is what widened the range, so it belongs here. Now 4,096, with the unexpired path unchanged.

Two bundled fixture baselines moved

Both are stated in the tests. One is a correction, one is a genuine cost, and I'd rather flag the second than bury it.

elf_cet_landing_pads_x64 drops four endbr64 addresses. All four are jump-table case labels strictly inside the function the symbol table names dispatch. That is a correction.

aarch64_static drops two mid-function instructions, and one of them — 0x40DF34is a Binary Ninja function start that this branch no longer recovers. The fixture is stripped, so the repo's baseline comes from Binary Ninja rather than from symbols. 0x40DF34 sits inside the FDE at 0x40DDC0, and that FDE really is one unwind range: 0x40DF34 repeats the range's opening minus its prfm prefetch, i.e. it's an alternate entry sharing one frame. The unwinder and Binary Ninja disagree about whether that counts as a function, and this rule follows the unwinder. It is asserted absent in the test rather than quietly deleted from the expected list, so the disagreement stays visible in the source.

What the Evaluate & Report gate reports

On the rebased head every check is green or skipped — Evaluate & Report and Malpedia Benchmark both skip, so there is no red check to explain. The reading below is kept because it is the substance of what that gate found when it did run on this branch, and it is the evidence for the malpedia row in the table above.

That gate compares the recovered function set across 155 malpedia dumps and fails when any file's set changes at all. This branch changes function sets deliberately, so when it runs it fails: three files differ. Its timing half is separately inconclusive — median -1.54%, CI [-2.65%, -0.33%], inside the run-to-run noise band it reports for itself (±3.7%).

The corpus carries no labelled boundaries, so the gate cannot say whether a change is right; it can only say the set moved. Its own artifact classifies each changed address by what the two reports say the address decodes like, and that reading is: 143 likely false positives removed, 48 absorbed into a neighbouring function, 11 split out of one, 3 likely real functions recovered — against 5 addresses that read as a lost function and 6 that read as a new false positive.

Four of the five reads-as-lost are in one Rust ELF and the fifth is a single-instruction address in a Konni dump. Set against 143 removals on the same three files, that is the trade these rules were measured to make, and it is consistent with the malpedia row above being level rather than improved: most of these rules read ELF unwind data, and packed Windows dumps do not carry it.

If the preference is for this gate to stay green, the two new options can ship default-off instead — say the word and I will flip them.

Tests

2,059 pass, 2 skip, 2,593 subtests on the rebased tree. ruff check . and ruff format --check . clean; make typecheck exit 0 with 0 error-level diagnostics, same as master.

New test files: testLsdaLandingPads.py (~40 cases, both architectures plus decoder contract tests), testFdeInteriorGaps.py, testEndbr64FdeInterior.py, testInteriorPrologueSuppression.py, testAArch64NoReturnBoundary.py, testPeExceptionTableDiscovery.py, testDeclaredArchitecture.py.

New fixtures, all built from source and XORed like the rest: elf_cxx_landing_pads_x64_xored (g++ 13.3.0, -O2 -fcf-protection=full, 4 pads), elf_cxx_landing_pads_arm64_xored (aarch64 cross-g++ 13.3.0, -O2 -mbranch-protection=standard, 5 pads all bti j), elf_cet_landing_pads_x64_xored, dotnet_readytorun_pe_xored.

Config

Two new options, both default on, both documented inline in SmdaConfig.py with the measurement that justifies them and the reason:

  • USE_LSDA_LANDING_PADS
  • USE_ELF_FDE_INTERIOR_GAPS

RESOLVE_TAILCALLS (unchanged, default off) now additionally gates the AArch64 bl fall-through seed described in section 5.

Where all the evidence lives

This branch is the enhancements only — engine source, tests, fixtures, and the benchmark workflow fix. Every number above was produced by a benchmark harness, corpora build scripts and a research log that live on a separate branch so they don't add 12,000 lines of tooling to a source change:

👉 accuracy/cloud-research-2026-08 (browse, PR #131)

If you're reviewing this with an agent, point it at that branch — it has the full chain from raw measurement to landed rule.

tools/bench/ — the harness. run.py runs an engine over a corpus, summarize.py re-aggregates and diffs saved result files, metrics.py is the metric. Exact-address-match only; TPR/PPV/F1 per binary; all three aggregations (macro, geometric, micro) written to every result file so a figure is never quietly compared against one computed a different way. An engine that returns nothing scores 0, not "undefined" — a crash should count, not average away. It also drives Ghidra headless through ghidra_scripts/DumpFunctionStarts.java for side-by-side comparison.

tools/bench/build_corpus.py and tools/bench/builders/ — how the corpora are made. Nothing is vendored; the repo carries recipes, not binaries. C/C++ through gcc/clang/MinGW, Go across GOOS/GOARCH and link modes, Rust across targets and profiles and LTO settings, .NET across CIL/ReadyToRun/single-file/NativeAOT, C/C++ through the AArch64 cross compiler including a dedicated -mbranch-protection=standard cell, and ARM64 Mach-O decoded from the fixtures this repo already ships. Ground truth is always the unstripped link's symbol table, go tool nm, assembly metadata, or LC_FUNCTION_STARTS — never another disassembler. Every family writes a manifest.json recording each cell it attempted including failures, so a matrix that quietly shrank can't read like one that passed.

docs/accuracy-research-log.md — the working log, ~3,400 lines, written as the work happened. Every hypothesis, every measurement, every dead end. This is where you'd look to check whether a claim in this PR was measured or assumed.

docs/accuracy-research-report.md — the readable write-up. Section 6 is "measured worse, and measured not worth doing" — five proposals with the numbers that killed them, including two that improved precision substantially but cost recall. Sections 5, 7–11, 19, 20 and 23 are one landed fix each. Section 13 is the ranked remaining agenda with ceilings. Section 17 has where every corpus stands. Section 24 reads the benchmark gate's own artifacts and explains, address by address, why its red is a set change rather than a regression.

docs/paper-replication.md and paper-tables.json — a replication of the origin evaluation with today's engines, including the metric definitions and corpus conventions used, and where the reproduction diverges from the published numbers and why.

Reproducing any figure here is two commands:

tools/bench/build_corpus.py --family native,native-arm64,go,rust,dotnet,macho-arm64 --out "$SMDA_BENCH_GROUNDTRUTH/built"
tools/bench/run.py --corpus native --engine smda --filter all --out results/

Every result file records the SMDA module path it imported from, because putting a second checkout on PYTHONPATH is exactly how you end up measuring the wrong tree and nothing else in the output would prove which one ran.

What is not in here

The .NET CIL backend reports file offsets where every other backend reports virtual addresses, and ReadyToRun native code isn't analysed under default routing. Both are called out in section 13 of the report as maintainer decisions about the report contract rather than things to change unilaterally.

Three research items remain open with next measurements named: Rust precision, the AArch64 tailcall case, and AArch64 recall generally.

USE_ELF_FDE_INTERIOR_GAPS is deliberately narrow here: it is reached only from the gap scan, so it tests self.gap_pointer and candidates from the prologue scan, from branch targets and from the tailcall paths never meet it. Widening where it is consulted is measurably worth more than what it already collects — on master, 5,899 of 6,129 false positives on the AArch64 ELF corpus are strictly interior to a declared FDE and none of the 59,365 true positives is. That is #324, and it is a change to shared candidate admission rather than to the gap scan, so it belongs in its own PR against master rather than bolted onto this one.

@danielplohmann

Copy link
Copy Markdown
Owner

Picking this up now that #309, #311 and #312 are in (68f2cfd, db5eff0, d111548), along with #320 and #321. I have not reviewed the content yet, and I want to explain why rather than let it look like silence: the diff is not readable against today's master, and one thing I did check makes me think it should not be read until it is re-measured.

The measurements are against a baseline that no longer exists

This is the part that matters, and it is not a rebase problem.

The clearest case is the AArch64 row. This PR reports, on the 72-cell built AArch64 ELF corpus:

Built C/C++ AArch64 (gcc cross) | 72 | PPV 76.676 → 80.554

#310 reports the same 72-cell corpus, and it is now merged as 7d01480:

Built C/C++ AArch64 ELF | 72 | PPV 87.596 → 89.186 | Δ FP −2,941

So this PR's post-change figure (80.554) sits about seven points below #310's pre-change figure (87.596) on the same corpus. Whatever the exact provenance, the table cannot be read against current master: the headroom it describes has been substantially closed since it was written, by #304, #307, #309, #310 and #311 — all of which move function-start precision on the same corpora, and several of which refuse candidates for reasons adjacent to this PR's.

I am not claiming the changes here are redundant. Several of them read structures nothing else in the engine reads — .gcc_except_table landing pads and the PE data-directory exception-table location have no overlap with anything merged. But the sizes in that table are no longer attributable, and some of the C/C++ and Rust gains almost certainly now overlap with #304's padding cut and #309's interior-gap refusal.

The Go and Mach-O bit-identical control rows are also worth re-running, for a different reason: #310, #311 and #312 all touched AArch64 candidate discovery after these were measured, so "bit-identical" is a claim about a tree that no longer exists.

The rebase, and what is actually in it

Eleven conflict hunks across five files. Most are mechanical, but not all, so I would rather you resolved them than guess:

file hunks character
common/FunctionCandidateManager.py 2 keep both. You add _eh_frame_fde_ranges, _declared_landing_pads and _plt_ranges where #312 added _pdata_ranges; same constructor and the same init() reset block, no interaction
intel/FunctionCandidateManager.py 3 one real decision. The gap-scan loop now hosts #309's USE_PE_X64_PDATA_INTERIOR_GAPS refusal, and this PR adds USE_LSDA_LANDING_PADS and USE_ELF_FDE_INTERIOR_GAPS into the same loop. All three refuse a gap candidate on declared-structure evidence and all three set gap_pointer to a skip target, so their order changes which one claims an address and therefore how far the scan resumes
the .pdata seeding block is the other one: you change where the table is found (data directory rather than a section named .pdata), while #309/#312 changed how each record is admitted (_admitExceptionRecord, the declared= flag, the is_pe guard) and #312 hoisted the lookup into common/. Complementary changes to the same code, but yours needs re-expressing against the new shape
aarch64/FunctionCandidateManager.py 3 overlaps #311's live-function-set and coverage-gate work
two test files 3 pinned baselines that #311 already moved

Your change 2 (read the declared exception directory instead of trusting the .pdata section name) is one I want independently of the numbers — the ReadyToRun recall result, 66.93% → 100.00%, is exactly the kind of thing the section-name assumption was always going to cost, and tests/dotnet_readytorun_pe_xored is now in master anyway via #305, so that fixture is already bundled.

What I am asking for

  1. Rebase onto current master, dropping the pre-squash copies of fix(ci): address ty 0.0.74 unsound-return checks #302, fix(intel): keep the switch index tied across a relative dispatch's base add #306, fix(aarch64): stop recording a branch target as an inbound call reference #307, docs(docs): say which address space a report's offsets are in #308, perf(intel): refuse gap candidates the exception directory places inside a function #309 and fix(aarch64): read the BTI target type, and resume past a refused pad's block #310 the branch still carries — same pass you did for perf(intel): refuse gap candidates the exception directory places inside a function #309/fix(aarch64): candidate-quality follow-ups to the BTI landing-pad series #311/perf(aarch64): refuse a gap candidate the ARM64 PE unwind data places inside a routine #312 yesterday, which went cleanly. Happy to do it myself if you prefer, but the ordering decision in the gap-scan loop is yours to make rather than mine to guess.
  2. Re-measure on the rebased tree. Only you have the corpora. What would help most is the same table against current master rather than against the old baseline, with the two control rows re-run.

No hurry on either — we are travelling shortly and this is not going into v4.5.1. I would rather it landed measured than landed fast.

One incidental thing, resolved rather than asked: fuzz_buffer was an infra timeout, not a hang. The 55-minute run against the 45-minute cap that failed on this branch's sibling has since re-run green at 2m45s, so the perf changes are cleared of it and it needs no further investigation.

r0ny123 and others added 4 commits September 8, 2026 21:18
Ten source changes that raise function-start accuracy on compiler-built
binaries, each measured against compiler symbol tables on corpora built
from source. Macro means, six corpora, both sides run back to back:

  260 built C/C++   PPV 91.878 -> 94.725  TPR 95.523 -> 95.596
  72 AArch64 ELF    PPV 76.676 -> 80.554  TPR 95.939 -> 95.964
  24 Rust           PPV 78.951 -> 83.608  TPR 97.493 -> 97.617
  4 .NET            PPV 93.589 -> 95.332  TPR 99.461 -> 99.469
  45 Go, 11 ARM64 Mach-O                  bit-identical

No corpus loses recall at any step.

The unifying idea is that the evidence which works is what the image
declares, not what the engine derives. An exception landing pad is where
the personality routine resumes, so it is interior to a function by
construction; a range .eh_frame declares is one routine. Both are facts
the compiler wrote down, and both were previously read as function
entries because they open with the instruction set's indirect-branch
marker - endbr64 under -fcf-protection, bti under -mbranch-protection -
and sit in gaps precisely because nothing branches to them.

The largest single mechanism was endbr64 seeded as a start. Over six
representative C/C++ cells it accounted for 822 false positives; two
rules built on declared evidence take it to zero while true positives
rise from 5,695 to 5,718.

Where a rule refuses a candidate, where the scan resumes decides whether
it costs recall. Stepping one instruction past a refused pad lands
inside the pad and books that instead. Resuming at the end of the
declaring FDE passes over that function's body and nothing else: over
the three corpora carrying pads, 0 of 41,215 have a declared start
between the pad and that end, and 0 sit inside a PLT section.

Two guards were found by measuring what the interior rule cost without
them. A PLT is exempt, because the whole table sits under one FDE and
the range test would otherwise read every stub after the first as
interior, costing 3,457 real functions. And a range's own start must be
a recovered function, because an FDE can begin in the alignment padding
ahead of its function.

The decoders read structures the analysed file controls, so both are
bounded on axes the fixtures cannot exercise: a 205 KB .eh_frame whose
records each named a 64 KB LSDA took 155 seconds before each LSDA was
decoded once and a per-section budget bounded the call-site table bytes
decoded and the reads preceding them. A decoded landing pad outside the
range its own FDE declares is refused - across four corpora and three
system libraries, 43,881 real pads are every one inside their own FDE,
and all 4,828 an unrelated image produced from mis-pointed LSDA data are
outside.

The perf-benchmark workflow now times base and PR interleaved in one job
rather than on separate runners, and its noise band is described as what
it measures. The previous shape reported a branch 13.18% slower at
p = 0.0000 when the only source change was AArch64-only on an x86
corpus; the two sides had run on different machines an hour apart.
_readExceptionTable walks one 12-byte RUNTIME_FUNCTION per iteration
over a range the analysed image declares, and did not poll the
analysis budget. Reading the table from the PE data directory rather
than from the .pdata section extent widened what that range can be:
the directory size is a 32-bit field the image controls and need not
correspond to any section, so a junk value walks the whole image
where the section extent bounded it before.

The walk is still bounded - a short read ends it, so it cannot pass
the bytes that exist - but bounding is not the same as stopping.
Measured on an 8 MB buffer of uniformly nonzero bytes declared as one
table, with the budget already spent: 699,050 entries walked before,
4,096 after, and the unexpired path walks all 699,050 either way. At
the configured MAX_IMAGE_SIZE that is roughly 8.7M entries of work
after the deadline has passed.

Every other loop over an image-declared extent in both candidate
managers already polls, including the AArch64 sibling that reads its
own exception directory the same way; this was the one site that did
not.
Merging master brings the ty bump and the annotations that went with it,
and this branch carries one function master does not, so nothing covered
it: under 0.0.74 the tree had 42 error-level diagnostics where master had
41.

`declaredArchitecture` returns whatever a loader's reader hands back. Those
readers are untyped, so the value arrives as Unknown, and 0.0.74 treats
returning it from a `-> str` signature as unsound rather than as gradual
typing. Coercing is honest: every reader already returns a str, and a value
that somehow is not one fails the membership test and answers "".

Code Quality on this branch is now zero errors under the version it pins,
where before the merge it would have failed the moment it met master.
Four test changes, none of them in the diff's own subject matter.

testInteriorPrologueSuppression's _BufferBinaryInfo implements the parts of
BinaryInfo the seeding scan reads, and danielplohmann#309 gave locateExceptionHandlerCandidates
a _getLiefType() call it does not have. It now answers "OTHER", which is what
BinaryInfo answers for a buffer lief cannot parse and the case the PE-only
branches are guarded for.

The ARM64 landing-pad control had gone vacuous. It switches USE_LSDA_LANDING_PADS
off and asserts a declared pad comes back, so that the assertion above it means
something -- and on current master it does not come back, because all five pads
in that fixture are `bti j` and danielplohmann#310's USE_AARCH64_BTI_TARGET_TYPE refuses a
`bti j` on the word alone. The control now switches that flag off as well and
says why, so it asserts what it always meant to: with both rules off the pads
are booked, with either one on they are not.

The two pinned fixture baselines are re-pinned from a run on the merged tree.
aarch64_static goes 278 -> 276 functions: both readings of the routine at
0x40DDC0's tail are now refused, Binary Ninja's 0x40DF34 and the 0x40DF30 the
gap scan reaches since danielplohmann#311, because the FDE at 0x40DDC0 covers both. The Mach-O
fixture's primary pass goes 256 -> 271, finding 143 of the table's 147 entries
against 128, with the total unchanged at 274 once the table pass runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRvm5qFMW52aWnmVgR5vkD
@r0ny123
r0ny123 force-pushed the accuracy/engine-enhancements branch from 9415216 to 68b2626 Compare September 9, 2026 02:08
r0ny123 pushed a commit to r0ny123/smda that referenced this pull request Sep 9, 2026
…iew replies for posting

Not part of either PR branch. These are the comment bodies and the
description edits produced by the rebase-and-re-measure pass, parked on
this scratch branch so a session with API access to the upstream repo can
fetch and post them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRvm5qFMW52aWnmVgR5vkD
@r0ny123

r0ny123 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto 6240b74 and re-measured. You were right about the table — it was describing a tree that stopped existing around #304 — and right that the gap-scan ordering was the hunk worth not guessing at. There turned out to be a second one you had no way to see from the conflict text, where "keep both" would have quietly undone #311; that one is below too.

The rebase

Dropped the pre-squash copies of #302, #306, #307, #308, #309 and #310, plus one of #311's (test(tests): move the fixture's rationale out of comments into its docstring — master already carries that wording). What is left is the three commits that are actually this PR's — 456b855, the budget poll on the declared-table walk, and the declaredArchitecture return-type fix — plus a fourth, test(tests): repair what the rebase onto current master broke, which is kept separate rather than folded in so what the rebase changed stays readable next to what the PR changed.

Your table of the five files was accurate. Taking them in your order:

common/FunctionCandidateManager.py — kept both. _pdata_ranges / _pdata_range_starts / _pdata_range_reach from #312, then _eh_frame_fde_ranges / _eh_frame_fde_starts / _declared_landing_pads / _plt_ranges, in both the constructor and the init() reset. No interaction, as you said. declaredExceptionRangeContaining stays exactly as #312 left it and ehFrameFdeRanges goes in below it.

intel/FunctionCandidateManager.py — the ordering. Chosen order in the gap-scan loop: #309's .pdata interior refusal, then USE_LSDA_LANDING_PADS, then USE_ELF_FDE_INTERIOR_GAPS.

Two things decide it, and neither is a preference.

The .pdata rule and the two .eh_frame rules can never speak about the same address, because they can never be armed on the same image. _pdata_ranges has exactly two writers in the tree and both are PE-only: on intel it is _admitExceptionRecord under declared=True, reached only from a table the exception directory or a .pdata section named, and the memory-dump path _carveExceptionRecords passes declared=False and records no extents at all; on AArch64 it is the directory walk behind isinstance(lief_binary, lief.PE.Binary), and _carveArm64ExceptionRecords likewise records none. ehFrameFdeRanges() and declaredLandingPads() both return early unless lief reports ELF.Binary. So the first test is a PE test and the other two are ELF tests, and the emptiness check on _pdata_ranges is the cheapest of the three, which is why it goes first.

LSDA before FDE-interior does matter, and in one direction only. declaredLandingPadSkipTarget() is implemented as declaredFdeRangeContaining(addr)[1] — literally the same resume point the interior rule would use — so the two can never disagree about where to resume. They differ in when they fire: the interior rule additionally requires containing[0] in self.disassembly.functions, i.e. that the range's own start has already been recovered. The landing-pad rule needs no such thing, because an LSDA naming an address as a pad is evidence on its own. Putting it first is therefore a strict superset with an identical skip target; putting it second would silently drop the pads whose declaring FDE has not been recovered yet.

intel/FunctionCandidateManager.py — the .pdata seeding block. Re-expressed against #312's shape rather than merged textually. locateExceptionHandlerCandidates now resolves the table address first (getExceptionDirectory(), falling back to a .pdata section only when the directory names none, and only when is_pe), then hands (start, end) to _readExceptionTable, which walks it and calls #309/#312's _admitExceptionRecord unchanged — declared= flag, _isTrustworthyExceptionExtent, chained-record handling, all of it. Your reading of it was right: I locate the table, #309 decides what each record is worth, and the two are orthogonal once the seam is in the right place.

One incidental thing I did not touch: master carries a dangling #: (start, end, is_chained) for every RUNTIME_FUNCTION record the image declares, in intel/FunctionCandidateManager.__init__ with nothing under it — the attribute it documents lives in common/ since #312. I put the new attribute above it rather than below so the merge does not make it look like it documents mine. Left it otherwise alone since it is yours to remove.

aarch64/FunctionCandidateManager.py — kept both, #310 first, and one hunk that had to be dropped.

The prologue predicate now runs is_bti_landing_pad(word) and _isLikelyInteriorBtiCandidate(addr, word) before the LSDA test, per your note: the bti j refusal is an integer compare against a word the loop already has, and the LSDA test decodes .eh_frame. Both continue, so it is purely cost. Inside _isLikelyInteriorBtiCandidate the USE_AARCH64_BTI_TARGET_TYPE block stays first and the three-condition code_map test follows it, same reason.

The gap scan gets the same three rules in the same order as the intel one, and then #310's _endOfRefusedLandingPadRun resume — not this branch's += INSTRUCTION_SIZE, which is the bug #310 fixed.

The hunk that is not "keep both": this branch re-adds

if is_conditional_branch(word):  # a function never opens with a cond branch
    self.gap_pointer += INSTRUCTION_SIZE
    continue

to the AArch64 gap scan, and db5eff0 (#311) deliberately removed it — the removal is the diff hunk, and both fixtures' test comments in master name the addresses it bought. Dropped it.

Worth being precise about the evidence, because the fixtures do not show it. Re-adding the skip changes nothing at all on either bundled fixture — USE_ELF_FDE_INTERIOR_GAPS refuses 0x40DF30 first, and the Mach-O one is untouched either way — so a merge that kept it would have gone green. On the corpora it is not close: measured on the branch with the skip put back,

corpus ΔTPR ΔPPV ΔTP ΔFP
Built C/C++ AArch64 ELF, n=72 −1.531 −2.114 −349 +539
ARM64 Mach-O, n=11 −0.159 −0.165 −4 +4

349 real functions and worse precision, which is #311 measured from the other side. This is the hunk I would most have liked you to see me get wrong, so: the resolution is "drop it", and the number above is why.

SmdaConfig.py and the two test files. Config: kept all three flags, as you said. The two pinned baselines are re-pinned from an actual run on the merged tree, with the reasons rewritten — see below.

The numbers, against 6240b74

Both trees run back to back on the same machine, arithmetic macro mean, --filter all, stock config. Every result file records the smda module path it imported from.

corpus n PPV before → after TPR before → after ΔFP ΔTP
Built C/C++ AArch64 ELF (gcc cross) 72 91.497 → 94.947 97.705 → 98.069 −3,645 +151
Built Rust (gnu targets) 24 79.659 → 87.480 98.237 → 98.361 −1,641 +19
Built Go (pclntab truth) 47 95.361 → 95.626 99.367 → 99.367 −408 0
ARM64 Mach-O (LC_FUNCTION_STARTS) 11 94.281 → 94.499 96.402 → 97.207 −27 +38
Built C/C++, MinGW PE cells 120 bit-identical bit-identical 0 0
ByteWeight msvc10-64 68 bit-identical bit-identical 0 0
ByteWeight msvc10-64, headers stripped 56 bit-identical bit-identical 0 0
Malpedia dumps (.fnmap truth) 57 92.645 → 92.648 98.552 → 98.552 −1 0

Nothing loses recall. That gate still holds, and it is the one thing I would not have wanted to find out the hard way after #304/#309/#310/#311 closed as much headroom as they did.

Malpedia is level to within one false positive, which is what it should be — those are packed Windows dumps and almost every rule here reads ELF unwind data. It is in the table as a control rather than as a result.

Your read on where the overlap would land was right, and the AArch64 row shows both halves of it: master's own PPV on that corpus has gone from the 76.676 this PR recorded for it to 91.497 today, so most of what the old table claimed is now yours, not this branch's. What is left on top of it is still 3.450 points and 3,645 false positives, with recall up rather than traded.

Two rows changed character, and I would rather say so than quietly re-title them

Go is no longer bit-identical. It is now −408 false positives at identical TP and FN. That is not a rule firing where it should not — Go carries no .gcc_except_table and the ELF FDE rule needs a recovered range start it does not have here. It is section 5's tailcall gate, whose effect on Go was always in the PR body (−430 FP) and simply never made it into the headline table's Go row. Attribution below.

The C/C++ row is not the 260-cell population any more, and I could not make it be. The archived corpus behind these figures carries the 120 MinGW-PE cells of that matrix and not the gcc/clang ELF cells, and rebuilding those needs upstream fetches I do not have from here. So the honest statement is the one in the table: on the MinGW PE half, this branch is bit-identical to master, byte for byte. That is consistent rather than disappointing — USE_LSDA_LANDING_PADS, USE_ELF_FDE_INTERIOR_GAPS and the endbr64 interior rule all decode nothing unless lief reports an ELF, so a PE-only C/C++ population is exactly where they should show nothing. It does mean the "260 cells, 91.878 → 94.725" line has no successor I can produce, and I have taken it out of the description rather than restate it with a number I did not measure.

The corpus property the rule rests on re-verifies unchanged, since it is a fact about the images rather than about the engine:

corpus samples with pads declared pads truth starts the FDE-end skip would step over pads inside a PLT
AArch64 ELF 23 12,585 0 0
Rust ELF 8 2,882 0 0

The PE side of change 2, checked rather than assumed

Reading the exception table from the data directory instead of from a section named .pdata is the one engine change here that touches ordinary Windows binaries, so it is worth a control rather than an argument. On ByteWeight msvc10-64 (n=68) and its header-stripped twin (n=56) — 124 PE x64 images, 214,362 truth functions between them — this branch is bit-identical to master on both: same TP, same FP, same FN, to the digit.

That is what it should be. On an MSVC PE the directory and the .pdata section name the same table, and a header-stripped dump has no directory to read, so it still takes the carve path. The change only shows up where the two disagree, which is the ReadyToRun case below.

The ReadyToRun figure, re-measured

66.93% → 100.00% was the old one. On 6240b74 the gap scan already reaches most of that image on its own, so the honest figure is smaller and still the same shape: on tests/dotnet_readytorun_pe_xored, 419 → 626 functions, and 626 of the 626 starts the exception directory declares — recall 100.00%, with nothing recovered that the directory does not name. testEveryDeclaredNativeFunctionIsRecovered pins it.

Sections 8 and 9, re-attributed

The old per-corpus splits between the landing-pad rule and the FDE-interior rule were on the 260-cell C/C++ matrix and on the AArch64 and Rust corpora. The first is gone for the reason above; the other two re-measure like this, everything else in the branch on, adding one rule at a time:

corpus variant PPV TPR TP FP
AArch64 ELF, n=72 both rules off 91.554 98.045 59,495 6,062
AArch64 ELF, n=72 + USE_LSDA_LANDING_PADS 93.176 98.067 59,513 4,330
AArch64 ELF, n=72 + USE_ELF_FDE_INTERIOR_GAPS (branch default) 94.947 98.069 59,516 2,484
Rust, n=24 both rules off 82.805 98.237 33,298 6,573
Rust, n=24 + USE_LSDA_LANDING_PADS 86.303 98.321 33,311 5,919
Rust, n=24 + USE_ELF_FDE_INTERIOR_GAPS (branch default) 87.480 98.361 33,317 5,722

On AArch64 ELF the landing-pad rule alone is −1,732 false positives and +18 true, and the interior rule adds −1,846 and +3 on top of it. On Rust the landing-pad rule alone is −654 false positives and +13 true, and the interior rule adds −197 and +6 on top of it.

The one decision I had to re-open: section 5's tailcall gate

This is the change that gates the AArch64 bl fall-through tailcall seed behind RESOLVE_TAILCALLS. It was measured before #307 changed what addTailcallCandidate does, so I re-measured it rather than carry the old numbers over. Everything else in this branch on, gate off → gate on, against 6240b74:

corpus n ΔPPV ΔTPR ΔFP ΔTP
Built Go (pclntab truth) 47 +0.265 0.000 −408 0
ARM64 Mach-O 11 +0.201 +0.092 −27 +11
Built C/C++ AArch64 ELF 72 +0.359 −0.070 −580 −53

Go with the gate off is byte-identical to master, which is the control that this is the only thing moving that row.

The third line is new, and it is the reason I am flagging this rather than just restating the section. The AArch64 ELF corpus was not measured for this change originally — the write-up only covers Mach-O and Go — and on it the gate is a trade: 580 false positives against 53 real functions. By the strict per-change rule the branch set itself ("a recall drop on any corpus is the reject criterion") that is a reject.

I have kept it, for three reasons, and I would rather you overrule me than have me quietly pick:

  1. F1 is up on all three, and precision is up on all three. The 53 are the only thing pointing the other way.
  2. The branch as a whole still does not lose recall on any corpus — the AArch64 ELF row goes 97.705 → 98.069 with the gate in it. The gate's 53 are more than repaid by the FDE and landing-pad rules on the same corpus.
  3. The gate is what makes both behaviours reachable at all. Today the AArch64 backend seeds these regardless of RESOLVE_TAILCALLS while the shared engine honours it on both of its own tailcall paths, so there is no setting that turns the AArch64 seeding off. With the gate, RESOLVE_TAILCALLS=True gets those 53 back and then some — measured on the branch, AArch64 ELF goes to TPR 98.202 and 59,611 TP (against 98.069 / 59,516 at the default and 98.139 / 59,569 with the seed simply ungated), and ARM64 Mach-O to 97.713 / 2,579. It costs precision to do it, which is presumably why the flag defaults off, but the recall is on a switch rather than gone.

For completeness, since RESOLVE_TAILCALLS also turns on the shared engine's own tailcall promotion, that row is not a pure "ungate" — the ungated-seed-only column is the nogate line in the table above.

Dropping it is a one-line revert in AArch64Backend._analyzeCallInstruction if you would rather have the 53 and the 580 both. Say which and I will push it either way.

Three things the rebase broke that were not conflicts

These all passed on the old base and fail on 6240b74, so I am listing them rather than letting them look like noise in the diff.

testInteriorPrologueSuppression stopped constructing. Its _BufferBinaryInfo stub implements the parts of BinaryInfo the seeding scan reads, and #309 gave locateExceptionHandlerCandidates a _getLiefType() call the stub does not have. It now answers "OTHER", which is what the real BinaryInfo answers for a buffer lief cannot parse, and is the case the PE-only branches are guarded for.

LsdaLandingPadArm64Test.testTheRuleIsOnByDefaultAndTurningItOffShowsWhatItDoes became vacuous — which is your observation, arriving as a red test. That case switches the rule off and asserts a declared pad comes back, so that the assertion above it means something. On 6240b74 it does not come back: all five pads in that fixture are bti j, and USE_AARCH64_BTI_TARGET_TYPE refuses a bti j on the word alone. The control now switches #310's flag off as well, and documents why. So the test asserts what it always meant to — with both rules off the pads are booked, with either one on they are not — instead of asserting a set that is now empty for a reason that has nothing to do with the rule under test.

aarch64_static and the Mach-O fixture are re-pinned from an actual run, not merged textually.

aarch64_static goes 278 → 276 functions (19,882 → 19,735 instructions, 3,499 → 3,476 blocks). The address in the old write-up has moved with #311: the branch refuses both 0x40DF34, where Binary Ninja puts that routine, and 0x40DF30, where the gap scan puts it now that #311 stopped skipping a word opening on a conditional branch. Both sit inside the FDE at 0x40DDC0, which really is one unwind range — 0x40DF34 repeats the range's opening minus its prfm prefetch, so it is an alternate entry sharing one frame. Same disagreement between the unwinder and Binary Ninja as before, one instruction over. Both are asserted absent rather than dropped from the expected list.

The Mach-O fixture's primary pass goes 256 → 271, which is 143 of LC_FUNCTION_STARTS' 147 entries against 128, and the total with the table pass on is unchanged at 274. That shape is the point: what the table adds shrinks by exactly what the primary pass learned to reach on its own.

One thing already in here that overlaps #322

Item 4 of #322 — the corpus benchmark's timing verdict being dominated by between-runner spread — is the same finding as the workflow change in this PR, arrived at from the other end. This branch collapses the base/PR matrix legs back into one job on one runner because of exactly what you describe: three consecutive runs whose PR side produced byte-identical output reported +1.26%, −13.18% and −15.53%, because the PR side drew a different runner each time (sum-of-best 252.83s / 282.21s / 289.04s) while the cached base side stayed frozen at 252.53s across all three. Caching one side is what makes it a measurement from another machine and another hour.

So "pin both sides to one runner" is the option this implements. Not claiming it closes your item — the noise band still derives from within-runner CV, which is the other half of what you wrote — but the runner half is here and measured, and it may save you doing it twice.

Gates

  • full suite: 2059 passed, 2 skipped, 2593 subtests on the rebased tree
  • ruff check . clean, ruff format --check . clean
  • make typecheck: exit 0, 0 error-level diagnostics — same as master, which is also 0
  • the advisory sibling-pair check warns once, on [backends] 1/2: aarch64/AArch64Backend.py changed and intel/X86Backend.py did not. Intentional — the no-return boundary rule reads AArch64 frame-record encodings and has no x86 counterpart in this branch

The branch is accuracy/engine-enhancements again, force-pushed onto 6240b74. Nothing in the diff's substance changed apart from the two AArch64 hunks above; the rest is the same code read against a different tree.

@r0ny123

r0ny123 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Two follow-ups on what I left open, both measured.

The C/C++ row is back, and I was wrong to say it could not be

I withdrew it on the grounds that the archive carries only the 120 MinGW-PE cells and the
gcc/clang ELF half needed fetches I did not have. The first half is true — the archive really
does hold 120 cells and no ELF ones. The second half was me not looking hard enough: the matrix
is reproducible, not merely archived. tools/bench/builders/sources.py fetches all ten
programs from upstream and that path works fine. So I rebuilt the whole thing.

260 of 260 cells, no failures — the original run managed 253, so this one is not averaging
over a matrix that quietly shrank. 213,706 truth functions.

master 6240b74 this branch Δ
PPV 94.038 96.908 +2.870
TPR 97.001 97.074 +0.073
F1 95.298 96.912 +1.614
TP 205,708 205,918 +210
FP 20,841 7,988 −12,853
FN 7,998 7,788 −210

Recall goes up rather than being traded away, and two thirds of the false positives are gone.

One caveat that belongs in the description rather than a footnote: this is a rebuilt corpus,
not the archived one.
gcc 13.3, clang 18.1, mingw 13.2 — not whatever built the original
figures, and truth comes to 213,706 against the old 213,441. So it is not the successor to
91.878 → 94.725 and I am not presenting it as one. It is a fresh paired A/B where both arms
see byte-identical inputs, which is the only property the claim needs.

The tailcall gate: the 53 are not what they looked like

I flagged this as your call. Before handing you a coin flip I went and looked at what the 53
addresses are.

All 53 are .eh_frame FDE starts — every one in .text, 39 preceded by nop padding,
opening with adrp (21), sub (18), b (6), add (4), lsl (3), movz (1). Ordinary
compiled functions the image declares in its own unwind data.

Which points at USE_ELF_EH_FRAME_CANDIDATES, already in this branch and off by default. Its
config comment says why:

Off by default because that is a deliberate baseline move rather than a free win — one bundled
fixture changes — so enabling it belongs with a corpus run, not with a config edit

That corpus run is the one thing I had that the flag did not. Its recorded evidence is 50 locally
built ELF binaries and 11,384 functions; I now have 212 ELF binaries across two architectures. So
rather than argue the gate, here is the flag measured properly. No code changes — a config
override on this branch.

72-cell AArch64 ELF corpus, crossing both:

arm TP FP FN PPV TPR F1
gate off, flag off (master) 59,569 3,064 757 95.108 98.745 96.892
gate on, flag off (this branch) 59,516 2,484 810 95.994 98.657 97.307
gate off, flag on 59,942 2,982 384 95.261 99.363 97.269
gate on, flag on 59,953 2,399 373 96.152 99.382 97.740

And the 260-cell C/C++ matrix, same flag on this branch: +628 true positives for +32 false
positives
, TPR 97.074 → 97.313, PPV 96.908 → 96.898. All of it on the 140 gcc/clang ELF cells;
the 120 MinGW PE cells are bit-identical, which is the control an ELF-only rule has to pass.

The part that settles your question: with the flag on, the gate costs 13 true positives and
gains 24 — net +11, against −583 false positives.
It stops being a recall trade and becomes a
win on both axes. The mechanism is worth naming, because the two changes are not independent: the
deferred FDE pass skips starts already claimed as code, so a wrong tailcall seed vetoes it at
that address. Gating the seed lets the FDE pass claim the right start instead. That is where the
+24 comes from.

So: 40 of the 53 come back from the flag alone, and the remaining 13 are more than paid for by
the 24 the interaction recovers.

I am not asking you to merge any of that here. The flag is untouched in this PR and still
defaults off. This is the corpus run its own comment says it was waiting for, handed over as
data — if you want the default flipped, that is a separate PR against master, and it needs the
bundled AArch64 fixture re-pinned from an actual run, which is exactly the "one bundled fixture
changes" the comment warns about. I am happy to open it, or to leave it with you.

What I would ask either way: the gate's 53 stop being a reason to hesitate over this PR.

Corpus, truth, per-binary predicted sets for all four arms, and the derived address lists are in
r0ny123/smda-eval-data@ff554a7 — enough to recompute any number above by address without
re-running disassembly.

@danielplohmann

Copy link
Copy Markdown
Owner

Reviewed and merging. The rebase and the re-measurement did exactly what was needed, and the two AArch64 hunks you resolved yourself — the gap-scan ordering and dropping the conditional-branch skip — are the two I would have got wrong from the conflict text alone. Flagging the second one with the number that shows the merge would have gone green anyway is the most useful thing in the rebase comment.

What I reproduced here

Two worktrees at 6240b74 and your head 68b2626, module path asserted on every run.

claim here
2059 passed, 2 skipped, 2593 subtests exact, 358s
ruff clean, make typecheck exit 0 with 0 error-level diagnostics confirmed on both trees (264 → 270 warnings, all the pre-existing self.disassembly is None | Unknown pattern)
ARM64 Mach-O n=11: −27 FP, +38 TP exact — TP 1818 → 1856, FP 1107 → 1080 against LC_FUNCTION_STARTS, 2,056 truth, no sample losing recall
ReadyToRun 419 → 626 exact, and the directory declares exactly 626 records

The Mach-O row matters beyond this PR: that corpus is bundled and its truth is inside the images, so unlike #310's bti j split and #312's ARM64 PE result (#322), it is a figure this repository can check. It is now checked.

Two things you did not claim, checked because they are the ones that decide whether the tests mean anything:

  • The new AArch64 encodings are exact. SUB_SP_IMM_MASK/VALUE over all 4,096 imm12 values and STP_FP_LR_OFFSET_MASK/VALUE over all 128 imm7 values, against capstone: 0 disagreements. add, sub sp, sp, #imm, lsl #12, the 32-bit forms, ldp, pre-index, post-index and every wrong-register variant are rejected. FRAME_RECORD_WINDOW behaves as documented, including the None tail.
  • The test set is not vacuous. Your test files against master's source: 4 fail to import and 29 assertions fail across the other 8. Every one of the ten changes has at least one test that fails on master.

And a full A/B over all 30 bundled fixtures. Four move; the other 26 are bit-identical, including every PE. All seven dropped addresses are correct refusals, each checked against the image's own FDE ranges and LSDA tables rather than against the reasoning:

fixture dropped why
elf_cxx_landing_pads_x64 0x1364, 0x153f, 0x15cf all three declared landing pads
elf_cet_landing_pads_x64 0x11a00x11d0 all four strictly inside the FDE [0x1180, 0x11e6), no LSDA on the image
elf_cxx_landing_pads_arm64 0xe24 inside the FDE [0xdd4, 0xe34)
aarch64_static 0x400350, 0x40df30 both inside a declared FDE — see below

Both x64 CET fixtures also gain 0x1020, which is the .plt start and an FDE start of its own. Worth having in the record; it is the PLT exemption doing its job.

The three decisions

The tailcall gate stays. F1 and precision up on all three corpora, the branch as a whole loses recall on none, and your follow-up settles what the 53 are. The argument that decides it is your third one rather than the numbers: today there is no setting that turns the AArch64 seeding off, so the gate is what makes both behaviours reachable at all. Thank you for measuring USE_ELF_EH_FRAME_CANDIDATES properly rather than arguing the gate — that is the corpus run its own comment was waiting for, and the +11-against-−583 interaction is a better answer than either of us had. Flipping that default is its own PR against master, as you say, and it needs the bundled fixture re-pinned; happy for you to open it.

Both new flags stay default-on. The benchmark gate's red is a set change, and its own artifact reads 143 likely false positives removed against 5 that read as a lost function. Default-off would ship the mechanism without the benefit.

The aarch64_static baseline move is accepted. The fixture is stripped, so Binary Ninja was never truth here, and both addresses are mid-function instructions the image's own unwind data covers.

Five findings, all ours to fix

None of them blocks the merge and none is a request. Recording them so the record is complete, and because three are corrections to text this PR ships:

  1. The config comments carry the numbers you retracted. USE_LSDA_LANDING_PADS cites 72 AArch64 ELF cells PPV 76.676 -> 79.172, USE_ELF_FDE_INTERIOR_GAPS cites 79.172 -> 80.554 and 260 cells 94.109 -> 94.725, and the RESOLVE_TAILCALLS block predates fix(aarch64): stop recording a branch target as an inbound call reference #307 (Mach-O n=11: 12 fewer functions and 28 more false positives; Go n=45: 430 more, against the re-measured −27/+11 and −408, with no AArch64 ELF row). You corrected all of these in the PR body; the copies in SmdaConfig.py came along unchanged. A PR body records a conversation, but a config comment is what the next person reads in two years, so these are the ones that matter most. Your re-measured attribution table has what they should say.
  2. getExceptionDirectory matches by substring on the enum's repr"EXCEPTION" not in str(directory.type). The typed idiom is already in the tree twice, including in this PR's own sibling file: aarch64/FunctionCandidateManager.py:238 does lief_binary.data_directory(lief.PE.DataDirectory.TYPES.EXCEPTION_TABLE).
  3. aarch64_static drops 0x400350, not 0x40df34. The test comment and the description both explain 278 → 276 through the Binary Ninja disagreement, but 0x40df34 is not recovered on either tree, so it is not one of the two the fixture lost. The actual pair is 0x400350ldr w3, [sp, #0x90], strictly inside the FDE at [0x400180, 0x400534) — and 0x40df30. That is a plainer story than the one told, and it should be the one in the test.
  4. _opensInsideAnEarlierPrologue is scan-order dependent and nothing pins it. It fires only when the earlier prologue is already a candidate, and DEFAULT_PROLOGUES happens to be scanned before DEFAULT_PROLOGUES_64. Measured: push rbp; mov rbp, rsp + push r15; push r14 refuses the interior match at +4, and the reverse layout does not. Favourable today, unpinned.
  5. One read path in the LSDA decoder is not charged to the budget. The budget is decremented once an LSDA reaches its call-site table length; one that fails earlier still costs a read_va of up to MAX_LSDA_BYTES and charges nothing, so a section naming MAX_RECORDS distinct undecodable LSDAs performs 199,999 such reads. Measured at ~1.2s on top of the ~1s walk — bounded, and not the 155s shape the memoisation fixed, but "the budget gates the read as well as the decode" is stronger than what the code does.

1 to 3 I will fix on master directly rather than send you back here for three comment edits. 4 and 5 join #322, where they belong: both are about how durable our own paths are rather than about anything wrong in this branch.

Two smaller notes for the same record. ehFrameFdeRanges() now filters zero-length FDEs, so USE_ELF_EH_FRAME_CANDIDATES no longer seeds their starts — no bundled ELF fixture carries one (checked all 18), and dropping them is arguably right, but it is a behaviour change the description does not mention. And declaredFdeRangeContaining's early break assumes declared ranges never nest; a nested pair would make it miss the outer one. It fails open, so it costs precision rather than recall, and nested FDEs are not legal DWARF — recording it as an assumption rather than a defect.

On the workflow change: agreed, and it is the runner half of #322's item 4 done properly. The cost is wall clock — the last master run had the two legs at ~8 minutes each in parallel, so six passes in one job is roughly 15 minutes against your new 40-minute timeout. Fits with room.

@danielplohmann
danielplohmann merged commit 50d3289 into danielplohmann:master Sep 10, 2026
28 checks passed
danielplohmann added a commit that referenced this pull request Sep 10, 2026
Three review findings on #300, all in text or in one expression, fixed here rather
than sent back for another round.

The config comments for USE_LSDA_LANDING_PADS and USE_ELF_FDE_INTERIOR_GAPS carried
the per-rule figures measured before #304/#307/#309/#310/#311 landed -- the same
baseline the PR description retracted, because master's own PPV on that corpus had
moved from 76.676 to 91.497 while the branch waited. RESOLVE_TAILCALLS likewise still
described the AArch64 bl fall-through gate as it measured before #307 changed what
addTailcallCandidate does, and omitted the AArch64 ELF corpus entirely, which is the
one row where the gate is a trade. All three now carry the re-measured attribution
against the tree #300 landed on, and the tailcall block says what the 53 lost true
positives are and how USE_ELF_EH_FRAME_CANDIDATES repays them. A description records
a conversation; a config comment is what the next reader has.

getExceptionDirectory matched the data directory with "EXCEPTION" not in
str(directory.type), a substring of the enum's repr that would also claim any future
type whose name contains it. The typed lookup is already the idiom in two other
places, one of them the AArch64 walk over the same directory.

The aarch64_static baseline moved by two starts and the test comment explained one
of them. 0x40DF34 is refused, but it was not recovered before either, so the pair the
fixture actually lost is 0x400350 and 0x40DF30 -- both mid-function instructions
inside a declared FDE, which is a plainer reading than the Binary Ninja disagreement
the comment led with. 0x400350 is now asserted alongside the other two.

Also removes a comment left dangling in intel/FunctionCandidateManager.__init__ by
#312, which hoisted the attribute it documented into common/ where the same text
already sits.

Suite 2060 passed, 1 skipped, 2593 subtests; ruff clean; ty exit 0 with 0
error-level diagnostics.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
r0ny123 added a commit to r0ny123/smda that referenced this pull request Sep 10, 2026
…y a gap one

The .eh_frame rule added in danielplohmann#300 refuses a candidate that opens strictly
inside a range the image declares, and it is reached only from the gap
scan: it tests self.gap_pointer, so candidates from the prologue scan,
from branch targets and from the tailcall paths never meet it. On the
72-cell AArch64 ELF corpus 2,245 of the 2,484 false positives still
standing are interior to a declared range, and none of them is a gap
candidate.

Ask the same question where analysis is about to begin on a candidate
from any source. Both of the gap rule's guards apply unchanged: a PLT is
exempt because the whole table sits under one FDE, and the range's own
start has to be a recovered function because an FDE can begin in the
alignment padding ahead of its own.

A third guard is new, and the corpus is what found it. A declared range
can reach past everything its function's control flow arrives at, and
refusing an address out there discards bytes nothing else claims along
with any reference only those bytes carry. Without it the AArch64 corpus
loses three functions, each the sole target of a call sitting in exactly
that unreached tail. Requiring the owner's own recovered extent to
surround the address costs 163 of the refusals and returns all three.

Measured against compiler symbol tables, no corpus losing a true
positive:

  72 AArch64 ELF cells    PPV 95.994 -> 97.063   -683 FP, TP and FN identical
  140 built C/C++ ELF     PPV 98.903 -> 98.969    -77 FP, TP and FN identical

The 120 MinGW PE cells, 23 Go cells, 11 ARM64 Mach-O cells and all 57
malpedia dumps are bit-identical, which is the control that it reaches
only images carrying an .eh_frame. Rust is bit-identical too, and not
because the rule is inert there: its images decode their ranges and 25
of 26 false positives on the first cell are interior to one, but the
guards decline all of them. Analysis is slightly faster, because a
refused candidate is one nothing then analyses.

Also moves a stray unittest.main() in the test file, which sat above a
test class and so left that class undefined when the file is run
directly.
@danielplohmann danielplohmann mentioned this pull request Sep 10, 2026
danielplohmann added a commit that referenced this pull request Sep 10, 2026
The changelog entry for what has landed since v4.5.1 (4868e19): #300, #325 and
#299. Ten engine changes under feat(core) plus a hot-path pass is a minor-bump
shape, which is why #299 and #300 were re-scoped off v4.5.1 rather than squeezed
into it.

VERSION in src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py bumped
together with the entry, per AGENTS.md.

No escaper output changed anywhere in this set. The only escaping-related edit
memoises escapeBinary results behind a cache keyed on its complete input tuple
and no escaper module is touched, so ESCAPER_DOWNWARD_COMPATIBILITY stays at
4.4.5 and INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs
reprocessing.

Figures are the contributor's own except the ARM64 Mach-O row and the
ReadyToRun result, which were reproduced during the #300 review; the entry says
which is which rather than presenting one table as equally checked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
danielplohmann added a commit that referenced this pull request Sep 10, 2026
The changelog entry for what has landed since v4.5.1 (4868e19): #300, #325 and
#299. Ten engine changes under feat(core) plus a hot-path pass is a minor-bump
shape, which is why #299 and #300 were re-scoped off v4.5.1 rather than squeezed
into it.

VERSION in src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py bumped
together with the entry, per AGENTS.md.

Deliberately still the current changelog format rather than keep-a-changelog.
#323 proposed adopting against an empty Unreleased so the first entries it holds
are written by their authors at merge time; v4.6.0 cannot be that, since #299's
and #300's entries are written here at release time either way. Adopting after
this release lets the inaugural section fill itself from the seven PRs currently
open. What this entry does adopt is the compromise: each topic keeps the
mechanism and its headline figure with the cost, and links the PR carrying the
full measurement and the dead ends -- ~2,200 words against v4.5.1's ~2,790, and
the first entry in the file to use links at all.

No escaper output changed anywhere in this set. The only escaping-related edit
memoises escapeBinary results behind a cache keyed on its complete input tuple
and no escaper module is touched, so ESCAPER_DOWNWARD_COMPATIBILITY stays at
4.4.5 and INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs
reprocessing.

Figures are the contributor's own except the ARM64 Mach-O row and the
ReadyToRun result, which were reproduced during the #300 review; the entry says
which is which rather than presenting one table as equally checked.

Also fixes AGENTS.md:117, which had prescribed a one-line changelog entry since
before v4.5.1 while v4.5.1 and v4.6.0 both use the nested **Topic:** list.
Independent of #323, which would replace that line again on adoption.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
danielplohmann pushed a commit that referenced this pull request Sep 11, 2026
#310 refuses a `bti j` word as a function start and keeps `bti c`, on the
grounds that a jump-only hint marks an indirect branch target inside a
routine where a call hint marks a callable entry. #322 records the figure
behind it as unverifiable here, because no bundled fixture contained a
`bti j` word and every AArch64 fixture was bit-identical across the
change.

The first half of that has since stopped being true: #300 bundled
elf_cxx_landing_pads_arm64_xored, which carries five `bti j` words, and
they do reach the rule. The second half still holds, and now for a
reason worth writing down. All five sit inside exception landing pads,
where the shape test refuses them on its own account and the LSDA rule
refuses them earlier still, so the flag decides nothing on that fixture
and toggling it moves no address.

What the flag needs in order to decide by itself is a `bti j` the shape
test would otherwise accept: after alignment padding, opening a block
that looks like an entry, in an image declaring no landing pads at all.
A raw buffer is exactly that image, so this builds one rather than asking
for a binary that cannot be bundled.

With the flag off the hint word is booked as a function and the block
behind it is not; with it on the hint is refused and the block is
recovered in its place. `bti c`, `bti jc` and a bare `bti` are asserted
unaffected in both settings, which is what makes the case measure the
jump-only split rather than bti handling in general.
r0ny123 added a commit to r0ny123/smda that referenced this pull request Sep 11, 2026
…y a gap one

The .eh_frame rule added in danielplohmann#300 refuses a candidate that opens strictly
inside a range the image declares, and it is reached only from the gap
scan: it tests self.gap_pointer, so candidates from the prologue scan,
from branch targets and from the tailcall paths never meet it. On the
72-cell AArch64 ELF corpus 2,245 of the 2,484 false positives still
standing are interior to a declared range, and none of them is a gap
candidate.

Ask the same question where analysis is about to begin on a candidate
from any source. Both of the gap rule's guards apply unchanged: a PLT is
exempt because the whole table sits under one FDE, and the range's own
start has to be a recovered function because an FDE can begin in the
alignment padding ahead of its own.

A third guard is new, and the corpus is what found it. A declared range
can reach past everything its function's control flow arrives at, and
refusing an address out there discards bytes nothing else claims along
with any reference only those bytes carry. Without it the AArch64 corpus
loses three functions, each the sole target of a call sitting in exactly
that unreached tail. Requiring the owner's own recovered extent to
surround the address costs 163 of the refusals and returns all three.

Measured against compiler symbol tables, no corpus losing a true
positive:

  72 AArch64 ELF cells    PPV 95.994 -> 97.063   -683 FP, TP and FN identical
  140 built C/C++ ELF     PPV 98.903 -> 98.969    -77 FP, TP and FN identical

The 120 MinGW PE cells, 23 Go cells, 11 ARM64 Mach-O cells and all 57
malpedia dumps are bit-identical, which is the control that it reaches
only images carrying an .eh_frame. Rust is bit-identical too, and not
because the rule is inert there: its images decode their ranges and 25
of 26 false positives on the first cell are interior to one, but the
guards decline all of them. Analysis is slightly faster, because a
refused candidate is one nothing then analyses.

Also moves a stray unittest.main() in the test file, which sat above a
test class and so left that class undefined when the file is run
directly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants