feat(core): function-boundary accuracy from what the image declares - #300
Conversation
2c02414 to
9415216
Compare
|
Picking this up now that #309, #311 and #312 are in ( The measurements are against a baseline that no longer existsThis is the part that matters, and it is not a rebase problem. The clearest case is the AArch64 row. This PR reports, on the 72-cell built AArch64 ELF corpus:
#310 reports the same 72-cell corpus, and it is now merged as
So this PR's post-change figure (80.554) sits about seven points below #310's pre-change figure (87.596) on the same corpus. Whatever the exact provenance, the table cannot be read against current master: the headroom it describes has been substantially closed since it was written, by #304, #307, #309, #310 and #311 — all of which move function-start precision on the same corpora, and several of which refuse candidates for reasons adjacent to this PR's. I am not claiming the changes here are redundant. Several of them read structures nothing else in the engine reads — The Go and Mach-O bit-identical control rows are also worth re-running, for a different reason: #310, #311 and #312 all touched AArch64 candidate discovery after these were measured, so "bit-identical" is a claim about a tree that no longer exists. The rebase, and what is actually in itEleven conflict hunks across five files. Most are mechanical, but not all, so I would rather you resolved them than guess:
Your change 2 (read the declared exception directory instead of trusting the What I am asking for
No hurry on either — we are travelling shortly and this is not going into v4.5.1. I would rather it landed measured than landed fast. One incidental thing, resolved rather than asked: |
Ten source changes that raise function-start accuracy on compiler-built binaries, each measured against compiler symbol tables on corpora built from source. Macro means, six corpora, both sides run back to back: 260 built C/C++ PPV 91.878 -> 94.725 TPR 95.523 -> 95.596 72 AArch64 ELF PPV 76.676 -> 80.554 TPR 95.939 -> 95.964 24 Rust PPV 78.951 -> 83.608 TPR 97.493 -> 97.617 4 .NET PPV 93.589 -> 95.332 TPR 99.461 -> 99.469 45 Go, 11 ARM64 Mach-O bit-identical No corpus loses recall at any step. The unifying idea is that the evidence which works is what the image declares, not what the engine derives. An exception landing pad is where the personality routine resumes, so it is interior to a function by construction; a range .eh_frame declares is one routine. Both are facts the compiler wrote down, and both were previously read as function entries because they open with the instruction set's indirect-branch marker - endbr64 under -fcf-protection, bti under -mbranch-protection - and sit in gaps precisely because nothing branches to them. The largest single mechanism was endbr64 seeded as a start. Over six representative C/C++ cells it accounted for 822 false positives; two rules built on declared evidence take it to zero while true positives rise from 5,695 to 5,718. Where a rule refuses a candidate, where the scan resumes decides whether it costs recall. Stepping one instruction past a refused pad lands inside the pad and books that instead. Resuming at the end of the declaring FDE passes over that function's body and nothing else: over the three corpora carrying pads, 0 of 41,215 have a declared start between the pad and that end, and 0 sit inside a PLT section. Two guards were found by measuring what the interior rule cost without them. A PLT is exempt, because the whole table sits under one FDE and the range test would otherwise read every stub after the first as interior, costing 3,457 real functions. And a range's own start must be a recovered function, because an FDE can begin in the alignment padding ahead of its function. The decoders read structures the analysed file controls, so both are bounded on axes the fixtures cannot exercise: a 205 KB .eh_frame whose records each named a 64 KB LSDA took 155 seconds before each LSDA was decoded once and a per-section budget bounded the call-site table bytes decoded and the reads preceding them. A decoded landing pad outside the range its own FDE declares is refused - across four corpora and three system libraries, 43,881 real pads are every one inside their own FDE, and all 4,828 an unrelated image produced from mis-pointed LSDA data are outside. The perf-benchmark workflow now times base and PR interleaved in one job rather than on separate runners, and its noise band is described as what it measures. The previous shape reported a branch 13.18% slower at p = 0.0000 when the only source change was AArch64-only on an x86 corpus; the two sides had run on different machines an hour apart.
_readExceptionTable walks one 12-byte RUNTIME_FUNCTION per iteration over a range the analysed image declares, and did not poll the analysis budget. Reading the table from the PE data directory rather than from the .pdata section extent widened what that range can be: the directory size is a 32-bit field the image controls and need not correspond to any section, so a junk value walks the whole image where the section extent bounded it before. The walk is still bounded - a short read ends it, so it cannot pass the bytes that exist - but bounding is not the same as stopping. Measured on an 8 MB buffer of uniformly nonzero bytes declared as one table, with the budget already spent: 699,050 entries walked before, 4,096 after, and the unexpired path walks all 699,050 either way. At the configured MAX_IMAGE_SIZE that is roughly 8.7M entries of work after the deadline has passed. Every other loop over an image-declared extent in both candidate managers already polls, including the AArch64 sibling that reads its own exception directory the same way; this was the one site that did not.
Merging master brings the ty bump and the annotations that went with it, and this branch carries one function master does not, so nothing covered it: under 0.0.74 the tree had 42 error-level diagnostics where master had 41. `declaredArchitecture` returns whatever a loader's reader hands back. Those readers are untyped, so the value arrives as Unknown, and 0.0.74 treats returning it from a `-> str` signature as unsound rather than as gradual typing. Coercing is honest: every reader already returns a str, and a value that somehow is not one fails the membership test and answers "". Code Quality on this branch is now zero errors under the version it pins, where before the merge it would have failed the moment it met master.
Four test changes, none of them in the diff's own subject matter. testInteriorPrologueSuppression's _BufferBinaryInfo implements the parts of BinaryInfo the seeding scan reads, and danielplohmann#309 gave locateExceptionHandlerCandidates a _getLiefType() call it does not have. It now answers "OTHER", which is what BinaryInfo answers for a buffer lief cannot parse and the case the PE-only branches are guarded for. The ARM64 landing-pad control had gone vacuous. It switches USE_LSDA_LANDING_PADS off and asserts a declared pad comes back, so that the assertion above it means something -- and on current master it does not come back, because all five pads in that fixture are `bti j` and danielplohmann#310's USE_AARCH64_BTI_TARGET_TYPE refuses a `bti j` on the word alone. The control now switches that flag off as well and says why, so it asserts what it always meant to: with both rules off the pads are booked, with either one on they are not. The two pinned fixture baselines are re-pinned from a run on the merged tree. aarch64_static goes 278 -> 276 functions: both readings of the routine at 0x40DDC0's tail are now refused, Binary Ninja's 0x40DF34 and the 0x40DF30 the gap scan reaches since danielplohmann#311, because the FDE at 0x40DDC0 covers both. The Mach-O fixture's primary pass goes 256 -> 271, finding 143 of the table's 147 entries against 128, with the total unchanged at 274 once the table pass runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRvm5qFMW52aWnmVgR5vkD
9415216 to
68b2626
Compare
…iew replies for posting Not part of either PR branch. These are the comment bodies and the description edits produced by the rebase-and-re-measure pass, parked on this scratch branch so a session with API access to the upstream repo can fetch and post them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRvm5qFMW52aWnmVgR5vkD
|
Rebased onto The rebaseDropped the pre-squash copies of #302, #306, #307, #308, #309 and #310, plus one of #311's ( Your table of the five files was accurate. Taking them in your order:
Two things decide it, and neither is a preference. The LSDA before FDE-interior does matter, and in one direction only.
One incidental thing I did not touch: master carries a dangling
The prologue predicate now runs The gap scan gets the same three rules in the same order as the intel one, and then #310's The hunk that is not "keep both": this branch re-adds if is_conditional_branch(word): # a function never opens with a cond branch
self.gap_pointer += INSTRUCTION_SIZE
continueto the AArch64 gap scan, and Worth being precise about the evidence, because the fixtures do not show it. Re-adding the skip changes nothing at all on either bundled fixture —
349 real functions and worse precision, which is #311 measured from the other side. This is the hunk I would most have liked you to see me get wrong, so: the resolution is "drop it", and the number above is why.
The numbers, against
|
| corpus | n | PPV before → after | TPR before → after | ΔFP | ΔTP |
|---|---|---|---|---|---|
| Built C/C++ AArch64 ELF (gcc cross) | 72 | 91.497 → 94.947 | 97.705 → 98.069 | −3,645 | +151 |
| Built Rust (gnu targets) | 24 | 79.659 → 87.480 | 98.237 → 98.361 | −1,641 | +19 |
| Built Go (pclntab truth) | 47 | 95.361 → 95.626 | 99.367 → 99.367 | −408 | 0 |
| ARM64 Mach-O (LC_FUNCTION_STARTS) | 11 | 94.281 → 94.499 | 96.402 → 97.207 | −27 | +38 |
| Built C/C++, MinGW PE cells | 120 | bit-identical | bit-identical | 0 | 0 |
| ByteWeight msvc10-64 | 68 | bit-identical | bit-identical | 0 | 0 |
| ByteWeight msvc10-64, headers stripped | 56 | bit-identical | bit-identical | 0 | 0 |
Malpedia dumps (.fnmap truth) |
57 | 92.645 → 92.648 | 98.552 → 98.552 | −1 | 0 |
Nothing loses recall. That gate still holds, and it is the one thing I would not have wanted to find out the hard way after #304/#309/#310/#311 closed as much headroom as they did.
Malpedia is level to within one false positive, which is what it should be — those are packed Windows dumps and almost every rule here reads ELF unwind data. It is in the table as a control rather than as a result.
Your read on where the overlap would land was right, and the AArch64 row shows both halves of it: master's own PPV on that corpus has gone from the 76.676 this PR recorded for it to 91.497 today, so most of what the old table claimed is now yours, not this branch's. What is left on top of it is still 3.450 points and 3,645 false positives, with recall up rather than traded.
Two rows changed character, and I would rather say so than quietly re-title them
Go is no longer bit-identical. It is now −408 false positives at identical TP and FN. That is not a rule firing where it should not — Go carries no .gcc_except_table and the ELF FDE rule needs a recovered range start it does not have here. It is section 5's tailcall gate, whose effect on Go was always in the PR body (−430 FP) and simply never made it into the headline table's Go row. Attribution below.
The C/C++ row is not the 260-cell population any more, and I could not make it be. The archived corpus behind these figures carries the 120 MinGW-PE cells of that matrix and not the gcc/clang ELF cells, and rebuilding those needs upstream fetches I do not have from here. So the honest statement is the one in the table: on the MinGW PE half, this branch is bit-identical to master, byte for byte. That is consistent rather than disappointing — USE_LSDA_LANDING_PADS, USE_ELF_FDE_INTERIOR_GAPS and the endbr64 interior rule all decode nothing unless lief reports an ELF, so a PE-only C/C++ population is exactly where they should show nothing. It does mean the "260 cells, 91.878 → 94.725" line has no successor I can produce, and I have taken it out of the description rather than restate it with a number I did not measure.
The corpus property the rule rests on re-verifies unchanged, since it is a fact about the images rather than about the engine:
| corpus | samples with pads | declared pads | truth starts the FDE-end skip would step over | pads inside a PLT |
|---|---|---|---|---|
| AArch64 ELF | 23 | 12,585 | 0 | 0 |
| Rust ELF | 8 | 2,882 | 0 | 0 |
The PE side of change 2, checked rather than assumed
Reading the exception table from the data directory instead of from a section named .pdata is the one engine change here that touches ordinary Windows binaries, so it is worth a control rather than an argument. On ByteWeight msvc10-64 (n=68) and its header-stripped twin (n=56) — 124 PE x64 images, 214,362 truth functions between them — this branch is bit-identical to master on both: same TP, same FP, same FN, to the digit.
That is what it should be. On an MSVC PE the directory and the .pdata section name the same table, and a header-stripped dump has no directory to read, so it still takes the carve path. The change only shows up where the two disagree, which is the ReadyToRun case below.
The ReadyToRun figure, re-measured
66.93% → 100.00% was the old one. On 6240b74 the gap scan already reaches most of that image on its own, so the honest figure is smaller and still the same shape: on tests/dotnet_readytorun_pe_xored, 419 → 626 functions, and 626 of the 626 starts the exception directory declares — recall 100.00%, with nothing recovered that the directory does not name. testEveryDeclaredNativeFunctionIsRecovered pins it.
Sections 8 and 9, re-attributed
The old per-corpus splits between the landing-pad rule and the FDE-interior rule were on the 260-cell C/C++ matrix and on the AArch64 and Rust corpora. The first is gone for the reason above; the other two re-measure like this, everything else in the branch on, adding one rule at a time:
| corpus | variant | PPV | TPR | TP | FP |
|---|---|---|---|---|---|
| AArch64 ELF, n=72 | both rules off | 91.554 | 98.045 | 59,495 | 6,062 |
| AArch64 ELF, n=72 | + USE_LSDA_LANDING_PADS |
93.176 | 98.067 | 59,513 | 4,330 |
| AArch64 ELF, n=72 | + USE_ELF_FDE_INTERIOR_GAPS (branch default) |
94.947 | 98.069 | 59,516 | 2,484 |
| Rust, n=24 | both rules off | 82.805 | 98.237 | 33,298 | 6,573 |
| Rust, n=24 | + USE_LSDA_LANDING_PADS |
86.303 | 98.321 | 33,311 | 5,919 |
| Rust, n=24 | + USE_ELF_FDE_INTERIOR_GAPS (branch default) |
87.480 | 98.361 | 33,317 | 5,722 |
On AArch64 ELF the landing-pad rule alone is −1,732 false positives and +18 true, and the interior rule adds −1,846 and +3 on top of it. On Rust the landing-pad rule alone is −654 false positives and +13 true, and the interior rule adds −197 and +6 on top of it.
The one decision I had to re-open: section 5's tailcall gate
This is the change that gates the AArch64 bl fall-through tailcall seed behind RESOLVE_TAILCALLS. It was measured before #307 changed what addTailcallCandidate does, so I re-measured it rather than carry the old numbers over. Everything else in this branch on, gate off → gate on, against 6240b74:
| corpus | n | ΔPPV | ΔTPR | ΔFP | ΔTP |
|---|---|---|---|---|---|
| Built Go (pclntab truth) | 47 | +0.265 | 0.000 | −408 | 0 |
| ARM64 Mach-O | 11 | +0.201 | +0.092 | −27 | +11 |
| Built C/C++ AArch64 ELF | 72 | +0.359 | −0.070 | −580 | −53 |
Go with the gate off is byte-identical to master, which is the control that this is the only thing moving that row.
The third line is new, and it is the reason I am flagging this rather than just restating the section. The AArch64 ELF corpus was not measured for this change originally — the write-up only covers Mach-O and Go — and on it the gate is a trade: 580 false positives against 53 real functions. By the strict per-change rule the branch set itself ("a recall drop on any corpus is the reject criterion") that is a reject.
I have kept it, for three reasons, and I would rather you overrule me than have me quietly pick:
- F1 is up on all three, and precision is up on all three. The 53 are the only thing pointing the other way.
- The branch as a whole still does not lose recall on any corpus — the AArch64 ELF row goes 97.705 → 98.069 with the gate in it. The gate's 53 are more than repaid by the FDE and landing-pad rules on the same corpus.
- The gate is what makes both behaviours reachable at all. Today the AArch64 backend seeds these regardless of
RESOLVE_TAILCALLSwhile the shared engine honours it on both of its own tailcall paths, so there is no setting that turns the AArch64 seeding off. With the gate,RESOLVE_TAILCALLS=Truegets those 53 back and then some — measured on the branch, AArch64 ELF goes to TPR 98.202 and 59,611 TP (against 98.069 / 59,516 at the default and 98.139 / 59,569 with the seed simply ungated), and ARM64 Mach-O to 97.713 / 2,579. It costs precision to do it, which is presumably why the flag defaults off, but the recall is on a switch rather than gone.
For completeness, since RESOLVE_TAILCALLS also turns on the shared engine's own tailcall promotion, that row is not a pure "ungate" — the ungated-seed-only column is the nogate line in the table above.
Dropping it is a one-line revert in AArch64Backend._analyzeCallInstruction if you would rather have the 53 and the 580 both. Say which and I will push it either way.
Three things the rebase broke that were not conflicts
These all passed on the old base and fail on 6240b74, so I am listing them rather than letting them look like noise in the diff.
testInteriorPrologueSuppression stopped constructing. Its _BufferBinaryInfo stub implements the parts of BinaryInfo the seeding scan reads, and #309 gave locateExceptionHandlerCandidates a _getLiefType() call the stub does not have. It now answers "OTHER", which is what the real BinaryInfo answers for a buffer lief cannot parse, and is the case the PE-only branches are guarded for.
LsdaLandingPadArm64Test.testTheRuleIsOnByDefaultAndTurningItOffShowsWhatItDoes became vacuous — which is your observation, arriving as a red test. That case switches the rule off and asserts a declared pad comes back, so that the assertion above it means something. On 6240b74 it does not come back: all five pads in that fixture are bti j, and USE_AARCH64_BTI_TARGET_TYPE refuses a bti j on the word alone. The control now switches #310's flag off as well, and documents why. So the test asserts what it always meant to — with both rules off the pads are booked, with either one on they are not — instead of asserting a set that is now empty for a reason that has nothing to do with the rule under test.
aarch64_static and the Mach-O fixture are re-pinned from an actual run, not merged textually.
aarch64_static goes 278 → 276 functions (19,882 → 19,735 instructions, 3,499 → 3,476 blocks). The address in the old write-up has moved with #311: the branch refuses both 0x40DF34, where Binary Ninja puts that routine, and 0x40DF30, where the gap scan puts it now that #311 stopped skipping a word opening on a conditional branch. Both sit inside the FDE at 0x40DDC0, which really is one unwind range — 0x40DF34 repeats the range's opening minus its prfm prefetch, so it is an alternate entry sharing one frame. Same disagreement between the unwinder and Binary Ninja as before, one instruction over. Both are asserted absent rather than dropped from the expected list.
The Mach-O fixture's primary pass goes 256 → 271, which is 143 of LC_FUNCTION_STARTS' 147 entries against 128, and the total with the table pass on is unchanged at 274. That shape is the point: what the table adds shrinks by exactly what the primary pass learned to reach on its own.
One thing already in here that overlaps #322
Item 4 of #322 — the corpus benchmark's timing verdict being dominated by between-runner spread — is the same finding as the workflow change in this PR, arrived at from the other end. This branch collapses the base/PR matrix legs back into one job on one runner because of exactly what you describe: three consecutive runs whose PR side produced byte-identical output reported +1.26%, −13.18% and −15.53%, because the PR side drew a different runner each time (sum-of-best 252.83s / 282.21s / 289.04s) while the cached base side stayed frozen at 252.53s across all three. Caching one side is what makes it a measurement from another machine and another hour.
So "pin both sides to one runner" is the option this implements. Not claiming it closes your item — the noise band still derives from within-runner CV, which is the other half of what you wrote — but the runner half is here and measured, and it may save you doing it twice.
Gates
- full suite:
2059 passed, 2 skipped, 2593 subtestson the rebased tree ruff check .clean,ruff format --check .cleanmake typecheck: exit 0, 0 error-level diagnostics — same as master, which is also 0- the advisory sibling-pair check warns once, on
[backends] 1/2:aarch64/AArch64Backend.pychanged andintel/X86Backend.pydid not. Intentional — the no-return boundary rule reads AArch64 frame-record encodings and has no x86 counterpart in this branch
The branch is accuracy/engine-enhancements again, force-pushed onto 6240b74. Nothing in the diff's substance changed apart from the two AArch64 hunks above; the rest is the same code read against a different tree.
|
Two follow-ups on what I left open, both measured. The C/C++ row is back, and I was wrong to say it could not beI withdrew it on the grounds that the archive carries only the 120 MinGW-PE cells and the 260 of 260 cells, no failures — the original run managed 253, so this one is not averaging
Recall goes up rather than being traded away, and two thirds of the false positives are gone. One caveat that belongs in the description rather than a footnote: this is a rebuilt corpus, The tailcall gate: the 53 are not what they looked likeI flagged this as your call. Before handing you a coin flip I went and looked at what the 53 All 53 are Which points at
That corpus run is the one thing I had that the flag did not. Its recorded evidence is 50 locally 72-cell AArch64 ELF corpus, crossing both:
And the 260-cell C/C++ matrix, same flag on this branch: +628 true positives for +32 false The part that settles your question: with the flag on, the gate costs 13 true positives and So: 40 of the 53 come back from the flag alone, and the remaining 13 are more than paid for by I am not asking you to merge any of that here. The flag is untouched in this PR and still What I would ask either way: the gate's 53 stop being a reason to hesitate over this PR. Corpus, truth, per-binary predicted sets for all four arms, and the derived address lists are in |
|
Reviewed and merging. The rebase and the re-measurement did exactly what was needed, and the two AArch64 hunks you resolved yourself — the gap-scan ordering and dropping the conditional-branch skip — are the two I would have got wrong from the conflict text alone. Flagging the second one with the number that shows the merge would have gone green anyway is the most useful thing in the rebase comment. What I reproduced hereTwo worktrees at
The Mach-O row matters beyond this PR: that corpus is bundled and its truth is inside the images, so unlike #310's Two things you did not claim, checked because they are the ones that decide whether the tests mean anything:
And a full A/B over all 30 bundled fixtures. Four move; the other 26 are bit-identical, including every PE. All seven dropped addresses are correct refusals, each checked against the image's own FDE ranges and LSDA tables rather than against the reasoning:
Both x64 CET fixtures also gain The three decisionsThe tailcall gate stays. F1 and precision up on all three corpora, the branch as a whole loses recall on none, and your follow-up settles what the 53 are. The argument that decides it is your third one rather than the numbers: today there is no setting that turns the AArch64 seeding off, so the gate is what makes both behaviours reachable at all. Thank you for measuring Both new flags stay default-on. The benchmark gate's red is a set change, and its own artifact reads 143 likely false positives removed against 5 that read as a lost function. Default-off would ship the mechanism without the benefit. The Five findings, all ours to fixNone of them blocks the merge and none is a request. Recording them so the record is complete, and because three are corrections to text this PR ships:
1 to 3 I will fix on master directly rather than send you back here for three comment edits. 4 and 5 join #322, where they belong: both are about how durable our own paths are rather than about anything wrong in this branch. Two smaller notes for the same record. On the workflow change: agreed, and it is the runner half of #322's item 4 done properly. The cost is wall clock — the last master run had the two legs at ~8 minutes each in parallel, so six passes in one job is roughly 15 minutes against your new 40-minute timeout. Fits with room. |
Three review findings on #300, all in text or in one expression, fixed here rather than sent back for another round. The config comments for USE_LSDA_LANDING_PADS and USE_ELF_FDE_INTERIOR_GAPS carried the per-rule figures measured before #304/#307/#309/#310/#311 landed -- the same baseline the PR description retracted, because master's own PPV on that corpus had moved from 76.676 to 91.497 while the branch waited. RESOLVE_TAILCALLS likewise still described the AArch64 bl fall-through gate as it measured before #307 changed what addTailcallCandidate does, and omitted the AArch64 ELF corpus entirely, which is the one row where the gate is a trade. All three now carry the re-measured attribution against the tree #300 landed on, and the tailcall block says what the 53 lost true positives are and how USE_ELF_EH_FRAME_CANDIDATES repays them. A description records a conversation; a config comment is what the next reader has. getExceptionDirectory matched the data directory with "EXCEPTION" not in str(directory.type), a substring of the enum's repr that would also claim any future type whose name contains it. The typed lookup is already the idiom in two other places, one of them the AArch64 walk over the same directory. The aarch64_static baseline moved by two starts and the test comment explained one of them. 0x40DF34 is refused, but it was not recovered before either, so the pair the fixture actually lost is 0x400350 and 0x40DF30 -- both mid-function instructions inside a declared FDE, which is a plainer reading than the Binary Ninja disagreement the comment led with. 0x400350 is now asserted alongside the other two. Also removes a comment left dangling in intel/FunctionCandidateManager.__init__ by #312, which hoisted the attribute it documented into common/ where the same text already sits. Suite 2060 passed, 1 skipped, 2593 subtests; ruff clean; ty exit 0 with 0 error-level diagnostics. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y a gap one The .eh_frame rule added in danielplohmann#300 refuses a candidate that opens strictly inside a range the image declares, and it is reached only from the gap scan: it tests self.gap_pointer, so candidates from the prologue scan, from branch targets and from the tailcall paths never meet it. On the 72-cell AArch64 ELF corpus 2,245 of the 2,484 false positives still standing are interior to a declared range, and none of them is a gap candidate. Ask the same question where analysis is about to begin on a candidate from any source. Both of the gap rule's guards apply unchanged: a PLT is exempt because the whole table sits under one FDE, and the range's own start has to be a recovered function because an FDE can begin in the alignment padding ahead of its own. A third guard is new, and the corpus is what found it. A declared range can reach past everything its function's control flow arrives at, and refusing an address out there discards bytes nothing else claims along with any reference only those bytes carry. Without it the AArch64 corpus loses three functions, each the sole target of a call sitting in exactly that unreached tail. Requiring the owner's own recovered extent to surround the address costs 163 of the refusals and returns all three. Measured against compiler symbol tables, no corpus losing a true positive: 72 AArch64 ELF cells PPV 95.994 -> 97.063 -683 FP, TP and FN identical 140 built C/C++ ELF PPV 98.903 -> 98.969 -77 FP, TP and FN identical The 120 MinGW PE cells, 23 Go cells, 11 ARM64 Mach-O cells and all 57 malpedia dumps are bit-identical, which is the control that it reaches only images carrying an .eh_frame. Rust is bit-identical too, and not because the rule is inert there: its images decode their ranges and 25 of 26 false positives on the first cell are interior to one, but the guards decline all of them. Analysis is slightly faster, because a refused candidate is one nothing then analyses. Also moves a stray unittest.main() in the test file, which sat above a test class and so left that class undefined when the file is run directly.
The changelog entry for what has landed since v4.5.1 (4868e19): #300, #325 and #299. Ten engine changes under feat(core) plus a hot-path pass is a minor-bump shape, which is why #299 and #300 were re-scoped off v4.5.1 rather than squeezed into it. VERSION in src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py bumped together with the entry, per AGENTS.md. No escaper output changed anywhere in this set. The only escaping-related edit memoises escapeBinary results behind a cache keyed on its complete input tuple and no escaper module is touched, so ESCAPER_DOWNWARD_COMPATIBILITY stays at 4.4.5 and INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs reprocessing. Figures are the contributor's own except the ARM64 Mach-O row and the ReadyToRun result, which were reproduced during the #300 review; the entry says which is which rather than presenting one table as equally checked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The changelog entry for what has landed since v4.5.1 (4868e19): #300, #325 and #299. Ten engine changes under feat(core) plus a hot-path pass is a minor-bump shape, which is why #299 and #300 were re-scoped off v4.5.1 rather than squeezed into it. VERSION in src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py bumped together with the entry, per AGENTS.md. Deliberately still the current changelog format rather than keep-a-changelog. #323 proposed adopting against an empty Unreleased so the first entries it holds are written by their authors at merge time; v4.6.0 cannot be that, since #299's and #300's entries are written here at release time either way. Adopting after this release lets the inaugural section fill itself from the seven PRs currently open. What this entry does adopt is the compromise: each topic keeps the mechanism and its headline figure with the cost, and links the PR carrying the full measurement and the dead ends -- ~2,200 words against v4.5.1's ~2,790, and the first entry in the file to use links at all. No escaper output changed anywhere in this set. The only escaping-related edit memoises escapeBinary results behind a cache keyed on its complete input tuple and no escaper module is touched, so ESCAPER_DOWNWARD_COMPATIBILITY stays at 4.4.5 and INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs reprocessing. Figures are the contributor's own except the ARM64 Mach-O row and the ReadyToRun result, which were reproduced during the #300 review; the entry says which is which rather than presenting one table as equally checked. Also fixes AGENTS.md:117, which had prescribed a one-line changelog entry since before v4.5.1 while v4.5.1 and v4.6.0 both use the nested **Topic:** list. Independent of #323, which would replace that line again on adoption. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#310 refuses a `bti j` word as a function start and keeps `bti c`, on the grounds that a jump-only hint marks an indirect branch target inside a routine where a call hint marks a callable entry. #322 records the figure behind it as unverifiable here, because no bundled fixture contained a `bti j` word and every AArch64 fixture was bit-identical across the change. The first half of that has since stopped being true: #300 bundled elf_cxx_landing_pads_arm64_xored, which carries five `bti j` words, and they do reach the rule. The second half still holds, and now for a reason worth writing down. All five sit inside exception landing pads, where the shape test refuses them on its own account and the LSDA rule refuses them earlier still, so the flag decides nothing on that fixture and toggling it moves no address. What the flag needs in order to decide by itself is a `bti j` the shape test would otherwise accept: after alignment padding, opening a block that looks like an entry, in an image declaring no landing pads at all. A raw buffer is exactly that image, so this builds one rather than asking for a binary that cannot be bundled. With the flag off the hint word is booked as a function and the block behind it is not; with it on the hint is refused and the block is recovered in its place. `bti c`, `bti jc` and a bare `bti` are asserted unaffected in both settings, which is what makes the case measure the jump-only split rather than bti handling in general.
…y a gap one The .eh_frame rule added in danielplohmann#300 refuses a candidate that opens strictly inside a range the image declares, and it is reached only from the gap scan: it tests self.gap_pointer, so candidates from the prologue scan, from branch targets and from the tailcall paths never meet it. On the 72-cell AArch64 ELF corpus 2,245 of the 2,484 false positives still standing are interior to a declared range, and none of them is a gap candidate. Ask the same question where analysis is about to begin on a candidate from any source. Both of the gap rule's guards apply unchanged: a PLT is exempt because the whole table sits under one FDE, and the range's own start has to be a recovered function because an FDE can begin in the alignment padding ahead of its own. A third guard is new, and the corpus is what found it. A declared range can reach past everything its function's control flow arrives at, and refusing an address out there discards bytes nothing else claims along with any reference only those bytes carry. Without it the AArch64 corpus loses three functions, each the sole target of a call sitting in exactly that unreached tail. Requiring the owner's own recovered extent to surround the address costs 163 of the refusals and returns all three. Measured against compiler symbol tables, no corpus losing a true positive: 72 AArch64 ELF cells PPV 95.994 -> 97.063 -683 FP, TP and FN identical 140 built C/C++ ELF PPV 98.903 -> 98.969 -77 FP, TP and FN identical The 120 MinGW PE cells, 23 Go cells, 11 ARM64 Mach-O cells and all 57 malpedia dumps are bit-identical, which is the control that it reaches only images carrying an .eh_frame. Rust is bit-identical too, and not because the rule is inert there: its images decode their ranges and 25 of 26 false positives on the first cell are interior to one, but the guards decline all of them. Analysis is slightly faster, because a refused candidate is one nothing then analyses. Also moves a stray unittest.main() in the test file, which sat above a test class and so left that class undefined when the file is run directly.
Current state
Rebased onto
6240b74and fully re-measured. Every figure in this description is the re-measured one. The original description carried a table measured before #304/#307/#309/#310/#311 landed, which is what prompted the review request; that table is gone rather than annotated, because most of what it claimed for this branch is now attributable to those merged PRs instead.Two things a reviewer should know before reading the diff:
The AArch64 headroom moved. Master's own PPV on the 72-cell AArch64 ELF corpus has gone from 76.676, which this PR originally recorded for it, to 91.497 today. Most of what the old table claimed is now merged. What is left on top of it is 3.450 points of precision and 3,645 false positives, with recall up rather than traded.
One change is a recall trade, and it is the one open decision. Section 5's tailcall-seed gate costs 53 true positives on the AArch64 ELF corpus while removing 580 false positives. By the strict per-change rule this branch set itself, that is a reject. It is kept, for reasons set out in the rebase comment, and the trade largely dissolves once
USE_ELF_EH_FRAME_CANDIDATESis on — measured there the gate costs 13 and gains 24, net +11 functions against −583 false positives, because gating the wrong seed lets the deferred FDE pass claim the right start. That flag is untouched here and still defaults off; the measurement is in the follow-up comment. Happy to drop the gate instead — it is a one-line revert.The short version
SMDA gets better at answering "where does a function start?" — ten engine changes plus one fix to the benchmark workflow, and the engine changes are all variations on one idea: when the binary already tells you something, believe the binary instead of guessing from the bytes.
Compilers write down a lot that the engine was not reading. An ELF says, in
.eh_frame, exactly which address range belongs to which routine. It says, in.gcc_except_table, exactly which addresses are exception landing pads. A PE says, in its data directory, exactly where the exception table lives. A COFF header says outright whether the image is 32- or 64-bit and which instruction set it targets. Every one of those was previously either ignored or re-derived from a byte heuristic that gets it wrong on a predictable class of binaries.Nothing here is a new heuristic. Every rule that refuses a candidate refuses it because a structure the compiler emitted says the address is inside a function, not at the start of one.
The numbers
Measured against compiler symbol tables, on corpora built from source, both trees run back to back on the same machine. Arithmetic macro mean (mean of per-binary rates),
master6240b74→ this branch:LC_FUNCTION_STARTS).fnmaptruth)The full 260-cell C/C++ matrix, rebuilt from source (gcc 13.3, clang 18.1, mingw 13.2; 260 of 260 cells, 213,706 truth functions): PPV 94.038 → 96.908, TPR 97.001 → 97.074, F1 95.298 → 96.912, FP −12,853, TP +210. That is a fresh paired A/B on a rebuilt corpus rather than the successor to any earlier figure.
No corpus loses recall for the branch as a whole, and five proposals that bought precision by dropping real functions were measured and left out. One change does trade: the tailcall-seed gate in section 5 costs 53 true positives on the 72-cell AArch64 ELF corpus while removing 580 false positives. That corpus was not measured for it originally, it is a reject by the strict per-change rule, and it is discussed in the comments rather than settled here.
The two bit-identical rows are the control, not filler. Go binaries carry no
.eh_framelanding pads and Mach-O carries no.eh_frameat all, so if either had moved it would have meant a rule was firing somewhere it had no business firing.On the malware-dump corpus (malpedia, n=57,
.fnmaptruth) the branch is level with master: PPV 92.645 → 92.648, TPR 98.552 → 98.552, one false positive apart. Expected — those are packed Windows dumps and most of these rules read ELF unwind data. The one change that does reach them is the container-header fix below, which is worth F1 +0.155 on that corpus when bitness is withheld.What actually changed, one at a time
1. If the buffer has a header, read the header
Disassembler.declaredArchitecture()andBitnessAnalyzer._declaredBitness().A memory dump of a mapped image still begins with the headers it was mapped from. Those headers say "x64" or "AArch64" outright. The engine was instead scoring REX.W prefix density to guess bitness, and running a byte probe to guess the instruction set — both of which are good heuristics that are occasionally just wrong, and when they are wrong every block in the report is wrong.
So: if the buffer parses as a PE, ELF or Mach-O and names an instruction set a backend exists for, use it. Otherwise fall through to exactly the same probes as before. A headerless dump or a shellcode blob is unaffected, which is the point — this narrows where the guessing happens, it does not replace it.
A managed PE deliberately still routes to the Intel backend rather than to
cil: CLR metadata is addressed by file offset, and a mapped dump no longer has file offsets, so routing on the header alone would lose it.Worth 93 functions recovered and 64 false positives removed on the malware corpus with bitness withheld.
2. The PE exception table's address is declared, not conventional
BinaryInfo.getExceptionDirectory(), andlocateExceptionHandlerCandidatesnow reads from it.The engine was finding the x64 exception table by looking for a section literally named
.pdata. That is just the name MSVC happens to use. The location is declared in the PE data directory, and a linker may put the table anywhere — the .NET ReadyToRun compiler puts it in.data.Read the directory; fall back to the
.pdatasection name only if the image declares no directory entry.On a ReadyToRun image with 626 declared function starts: 419 → 626 functions, which is 626 of the 626 starts the exception directory declares, with nothing recovered that it does not name. Fixture
tests/dotnet_readytorun_pe_xoredcovers it.3. A call that never returns is a function boundary (AArch64)
AArch64Backend._callFallthroughFunctionStart, plus a newopens_stack_frame()helper.If a function's last act is to call something that never returns —
abort,_Unwind_Resume, a panic handler — there is noretafter it. Decoding runs straight off the end of the function and into the next one, and the two get merged into one.The existing checks for this all look for a reason to stop: alignment padding, an existing candidate, a NOP run. When the next function is packed right up against this one with none of those, they all decline.
New check: does the very next instruction open a stack frame? On AArch64 that means
sub sp, sp, #immfollowed within three instructions bystp x29, x30, [sp, #imm]— allocate a frame, then write the frame record into it. Neither half is conclusive on its own (an alloca does the first, plenty of code does the second), but the pair is: nothing mid-function re-saves the incoming link register into a frame it just created.ARM64 Mach-O, n=11, as part of the branch: PPV 94.281 → 94.499, TPR 96.402 → 97.207, +38 functions against −27 false positives.
4. A prologue that begins exactly where another prologue ends is not a function
intel/FunctionCandidateManager._opensInsideAnEarlierPrologue.clang opens a frame with
push rbp; mov rbp, rspand immediately follows it with the callee-saved runpush r15; push r14. Both byte sequences are on the seeded prologue list. So the scan finds the real function start, and then finds a second "function" four bytes into the same function's body.Fix: if a seeded pattern match begins exactly where an earlier seeded pattern ends, and that earlier address is already a candidate, refuse it. (The "already a candidate" condition is what keeps it from firing on a byte coincidence.) This mirrors the existing MSVC hotpatch-pad adjustment right below it.
Across ten corpora: 912 false positives removed, 0 true positives lost.
5. After a
bl, the cut is what recovers the function — the seed makes it worseAArch64Backend._analyzeCallInstruction.When the AArch64 backend detects a fall-through past a
bl, it was doing two things: cutting the caller short there, and seeding the boundary as a tailcall candidate. Turns out the cut is what recovers the next function; the seed then re-books the same address with worse extents than the ordinary candidate machinery would give it.The seed is now behind
RESOLVE_TAILCALLS(default off, same flag the shared engine already gates its tailcall promotion behind). The cut still happens either way.Re-measured against
6240b74, gate off → gate on: Go n=47, −408 false positives at identical TP; ARM64 Mach-O n=11, −27 false positives, +11 functions; Built C/C++ AArch64 ELF n=72, −580 false positives against −53 functions. That last row is the recall trade noted above.6. The candidate snapshot is taken before analysis, so it can't be the whole answer
AArch64Backend._isKnownFunctionStartand two sites in the AArch64 candidate manager.Several checks asked "is this address in
getFunctionStartCandidates()?" to decide whether a branch target is a real entry or somebody's interior. But that set is snapshotted before analysis begins, and gap analysis never adds to it. So a function that gap analysis discovered is incode_mapand indisassembly.functions— but absent from the candidate set, and therefore indistinguishable from interior code.Every such check now asks the live function set too.
ARM64 Mach-O n=11: 8 recovered, 0 new false positives. It also lifts the Mach-O fixture's primary pass from 246 to 269 functions, which is why that test's baseline moves — see the note in
tests/testMachoFunctionStartCandidates.py.7.
endbr64is an indirect-branch marker, not a function marker_seedPrologueMatches(..., refuse_declared_interior=True).Under
-fcf-protection, gcc and clang putendbr64at every address an indirect branch can land on. That includes every jump-table case label and every exception landing pad — addresses squarely inside a function body. The engine seeded all of them as function starts.The image says which is which: an FDE in
.eh_framecovers exactly one routine, so anendbr64that is not its own FDE range's start is inside that routine.This is applied to the
endbr64pattern only, and deliberately: it is the only seeded pattern that names a place a branch can arrive rather than a way a function opens. Every other pattern in the list is a genuine prologue shape and is left alone. The check also resolves the FDE ranges once up front, so an image with no readable.eh_framepays nothing.8. A declared landing pad is interior by construction
SmdaConfig.USE_LSDA_LANDING_PADS(new, default on),EhFrameDecoder.decodeEhFrameLandingPads(), and the gap scans of both backends..gcc_except_tableis the compiler telling the unwinder "if an exception escapes this call site, resume execution here". "Here" is by definition inside a function — that is the whole meaning of the record.These addresses were being booked as functions because they are the perfect storm for a byte scan: they open with the instruction set's indirect-branch marker (
endbr64on x86,btion AArch64), and they sit in gaps precisely because nothing in the function branches to them. Every signal a gap scan has says "function start".So the decoder now reads the LSDA call-site table and collects the declared pads, and both gap scans refuse a candidate at one.
Where the scan resumes is the whole decision. Stepping one instruction past a refused pad lands inside the pad and books that instead — a worse candidate than the one you just refused. Resuming at the end of the FDE that declared the pad passes over that function's body and nothing else. Across the three corpora carrying pads, 0 of 41,215 pads have a declared function start between the pad and that resume point, and 0 sit inside a PLT section.
On AArch64 the rule also has to run in
locatePrologueCandidates, not just the gap scan —btiis a recognised entry prologue there, so pads reach the prologue pass directly. On x86endbr64is not a prologue shape, so the gap scan is the only path.Per-corpus attribution for this rule was measured on the pre-rebase tree and is superseded by the re-measurement in the comments; the branch totals are in the table at the top. It also runs faster on pad-heavy images — 3.8% on the two heaviest cells — because the candidates it refuses are candidates nothing then has to analyse.
9. A gap candidate strictly inside a declared range, generally
SmdaConfig.USE_ELF_FDE_INTERIOR_GAPS(new, default on), both gap scans.Same idea as section 8, wider net. Section 8 refuses only the subset an LSDA names as a landing pad; this refuses any gap candidate strictly inside a range
.eh_framedeclares, and resumes at that range's end. It is what catches the jump-table case labels a switch emits under-fcf-protection, and it is the single largest precision mechanism measured on this corpus.Two guards make it safe, and both were found by measuring what it cost without them.
A PLT is exempt. The whole PLT sits under one FDE, so without the exemption the range test reads every stub after the first as interior to the first. That costs 3,457 real functions — on a CET image the gap scan is what recovers the stubs at all.
And the range's own start must already be a recovered function. An FDE can begin in the alignment padding ahead of its function, and then the real entry a few bytes in is interior to nothing. The remaining 35 losses were all of that shape — two per statically linked cell, one of them
rt_sigreturnunder a signal-frame CIE.Re-measured against
6240b74, everything else in the branch on, adding one rule at a time:USE_LSDA_LANDING_PADSUSE_ELF_FDE_INTERIOR_GAPSUSE_LSDA_LANDING_PADSUSE_ELF_FDE_INTERIOR_GAPSNot an engine change: the benchmark workflow was comparing two machines
.github/workflows/perf_benchmark.ymland.github/workflows/scripts/evaluate_runtime.py.Worth flagging because it was producing confidently wrong verdicts. The workflow timed the base branch on one runner and the PR branch on another, an hour apart. It once reported a branch 13.18% slower at p = 0.0000 when the only source change in it was AArch64-only code measured on an x86 corpus — a change that cannot have touched a single instruction executed in that benchmark.
Base and PR are now timed interleaved in a single job, leading side rotated per pass, and the reported noise band describes what it actually measures.
What
endbr64was costing, end to endRules 7, 8 and 9 all attack the same root cause from different angles, so the useful number is the combined one.
Over six representative C/C++ cells,
endbr64seeded as a function start accounted for 822 false positives. After these three rules it accounts for zero — and true positives on those same cells go up, from 5,695 to 5,718.Robustness
Both decoders read structures that the analysed file controls, so both are bounded on axes no fixture would naturally exercise.
A 205 KB
.eh_framewhose records each named a 64 KB LSDA took 155 seconds to decode, scaling linearly. Each LSDA is now decoded once and memoised, and a per-section budget bounds both the call-site table bytes decoded and the reads that precede them. Same input: 0.047s. 200,000 FDEs with genuinely distinct LSDAs: 2.5s. For scale, the heaviest real image in any of these corpora decodes 31 KB of call-site tables, andlibstdc++.so.6decodes 26 KB.A decoded landing pad that falls outside the range its own FDE declares is refused, because the format guarantees it cannot. That matters: on one NativeAOT image, LSDA pointers led into arbitrary data whose "headers" parsed cleanly and produced 4,826 fabricated pads. Across four corpora and three system libraries, 43,881 genuine pads are every single one inside their own FDE, and all 4,828 spurious ones are outside.
A second bound came out of review, in
71612d3. The exception-table walk reads one 12-byte record per iteration over a range the image declares, and never polled the analysis budget. It is bounded — a short read ends it — but bounding is not stopping: on an 8 MB buffer declared as one table it walked all 699,050 entries with the deadline already passed, exactly as many as with time still on the clock. Reading the table from the data directory rather than the.pdataextent is what widened the range, so it belongs here. Now 4,096, with the unexpired path unchanged.Two bundled fixture baselines moved
Both are stated in the tests. One is a correction, one is a genuine cost, and I'd rather flag the second than bury it.
elf_cet_landing_pads_x64drops fourendbr64addresses. All four are jump-table case labels strictly inside the function the symbol table namesdispatch. That is a correction.aarch64_staticdrops two mid-function instructions, and one of them —0x40DF34— is a Binary Ninja function start that this branch no longer recovers. The fixture is stripped, so the repo's baseline comes from Binary Ninja rather than from symbols.0x40DF34sits inside the FDE at0x40DDC0, and that FDE really is one unwind range:0x40DF34repeats the range's opening minus itsprfmprefetch, i.e. it's an alternate entry sharing one frame. The unwinder and Binary Ninja disagree about whether that counts as a function, and this rule follows the unwinder. It is asserted absent in the test rather than quietly deleted from the expected list, so the disagreement stays visible in the source.What the
Evaluate & Reportgate reportsOn the rebased head every check is green or skipped —
Evaluate & ReportandMalpedia Benchmarkboth skip, so there is no red check to explain. The reading below is kept because it is the substance of what that gate found when it did run on this branch, and it is the evidence for the malpedia row in the table above.That gate compares the recovered function set across 155 malpedia dumps and fails when any file's set changes at all. This branch changes function sets deliberately, so when it runs it fails: three files differ. Its timing half is separately inconclusive — median -1.54%, CI [-2.65%, -0.33%], inside the run-to-run noise band it reports for itself (±3.7%).
The corpus carries no labelled boundaries, so the gate cannot say whether a change is right; it can only say the set moved. Its own artifact classifies each changed address by what the two reports say the address decodes like, and that reading is: 143 likely false positives removed, 48 absorbed into a neighbouring function, 11 split out of one, 3 likely real functions recovered — against 5 addresses that read as a lost function and 6 that read as a new false positive.
Four of the five reads-as-lost are in one Rust ELF and the fifth is a single-instruction address in a Konni dump. Set against 143 removals on the same three files, that is the trade these rules were measured to make, and it is consistent with the malpedia row above being level rather than improved: most of these rules read ELF unwind data, and packed Windows dumps do not carry it.
If the preference is for this gate to stay green, the two new options can ship default-off instead — say the word and I will flip them.
Tests
2,059 pass, 2 skip, 2,593 subtests on the rebased tree.
ruff check .andruff format --check .clean;make typecheckexit 0 with 0 error-level diagnostics, same as master.New test files:
testLsdaLandingPads.py(~40 cases, both architectures plus decoder contract tests),testFdeInteriorGaps.py,testEndbr64FdeInterior.py,testInteriorPrologueSuppression.py,testAArch64NoReturnBoundary.py,testPeExceptionTableDiscovery.py,testDeclaredArchitecture.py.New fixtures, all built from source and XORed like the rest:
elf_cxx_landing_pads_x64_xored(g++ 13.3.0,-O2 -fcf-protection=full, 4 pads),elf_cxx_landing_pads_arm64_xored(aarch64 cross-g++ 13.3.0,-O2 -mbranch-protection=standard, 5 pads allbti j),elf_cet_landing_pads_x64_xored,dotnet_readytorun_pe_xored.Config
Two new options, both default on, both documented inline in
SmdaConfig.pywith the measurement that justifies them and the reason:USE_LSDA_LANDING_PADSUSE_ELF_FDE_INTERIOR_GAPSRESOLVE_TAILCALLS(unchanged, default off) now additionally gates the AArch64blfall-through seed described in section 5.Where all the evidence lives
This branch is the enhancements only — engine source, tests, fixtures, and the benchmark workflow fix. Every number above was produced by a benchmark harness, corpora build scripts and a research log that live on a separate branch so they don't add 12,000 lines of tooling to a source change:
👉
accuracy/cloud-research-2026-08(browse, PR #131)If you're reviewing this with an agent, point it at that branch — it has the full chain from raw measurement to landed rule.
tools/bench/— the harness.run.pyruns an engine over a corpus,summarize.pyre-aggregates and diffs saved result files,metrics.pyis the metric. Exact-address-match only; TPR/PPV/F1 per binary; all three aggregations (macro, geometric, micro) written to every result file so a figure is never quietly compared against one computed a different way. An engine that returns nothing scores 0, not "undefined" — a crash should count, not average away. It also drives Ghidra headless throughghidra_scripts/DumpFunctionStarts.javafor side-by-side comparison.tools/bench/build_corpus.pyandtools/bench/builders/— how the corpora are made. Nothing is vendored; the repo carries recipes, not binaries. C/C++ through gcc/clang/MinGW, Go across GOOS/GOARCH and link modes, Rust across targets and profiles and LTO settings, .NET across CIL/ReadyToRun/single-file/NativeAOT, C/C++ through the AArch64 cross compiler including a dedicated-mbranch-protection=standardcell, and ARM64 Mach-O decoded from the fixtures this repo already ships. Ground truth is always the unstripped link's symbol table,go tool nm, assembly metadata, orLC_FUNCTION_STARTS— never another disassembler. Every family writes amanifest.jsonrecording each cell it attempted including failures, so a matrix that quietly shrank can't read like one that passed.docs/accuracy-research-log.md— the working log, ~3,400 lines, written as the work happened. Every hypothesis, every measurement, every dead end. This is where you'd look to check whether a claim in this PR was measured or assumed.docs/accuracy-research-report.md— the readable write-up. Section 6 is "measured worse, and measured not worth doing" — five proposals with the numbers that killed them, including two that improved precision substantially but cost recall. Sections 5, 7–11, 19, 20 and 23 are one landed fix each. Section 13 is the ranked remaining agenda with ceilings. Section 17 has where every corpus stands. Section 24 reads the benchmark gate's own artifacts and explains, address by address, why its red is a set change rather than a regression.docs/paper-replication.mdandpaper-tables.json— a replication of the origin evaluation with today's engines, including the metric definitions and corpus conventions used, and where the reproduction diverges from the published numbers and why.Reproducing any figure here is two commands:
Every result file records the SMDA module path it imported from, because putting a second checkout on
PYTHONPATHis exactly how you end up measuring the wrong tree and nothing else in the output would prove which one ran.What is not in here
The
.NETCIL backend reports file offsets where every other backend reports virtual addresses, and ReadyToRun native code isn't analysed under default routing. Both are called out in section 13 of the report as maintainer decisions about the report contract rather than things to change unilaterally.Three research items remain open with next measurements named: Rust precision, the AArch64 tailcall case, and AArch64 recall generally.
USE_ELF_FDE_INTERIOR_GAPSis deliberately narrow here: it is reached only from the gap scan, so it testsself.gap_pointerand candidates from the prologue scan, from branch targets and from the tailcall paths never meet it. Widening where it is consulted is measurably worth more than what it already collects — on master, 5,899 of 6,129 false positives on the AArch64 ELF corpus are strictly interior to a declared FDE and none of the 59,365 true positives is. That is #324, and it is a change to shared candidate admission rather than to the gap scan, so it belongs in its own PR against master rather than bolted onto this one.