Skip to content

chore: correct the figures and the directory lookup #300 shipped - #325

Merged
danielplohmann merged 1 commit into
masterfrom
chore/300-followups
Sep 10, 2026
Merged

chore: correct the figures and the directory lookup #300 shipped#325
danielplohmann merged 1 commit into
masterfrom
chore/300-followups

Conversation

@danielplohmann

Copy link
Copy Markdown
Owner

Three findings from the review of #300, plus one leftover from #312. All are text or a single expression, so they are fixed here rather than sent back to @r0ny123 for another round.

1. The config comments carried the figures the description retracted

USE_LSDA_LANDING_PADS and USE_ELF_FDE_INTERIOR_GAPS shipped with their per-rule attribution measured before #304/#307/#309/#310/#311 landed:

#   260 built C/C++ cells    PPV 92.623 -> 94.109 ...
#   72 AArch64 ELF cells     PPV 76.676 -> 79.172 ...

That is the same baseline #300's description dropped rather than annotated, because master's own PPV on that corpus had moved from 76.676 to 91.497 while the branch waited. The description was corrected; the copies in SmdaConfig.py came along unchanged.

Replaced with the re-measured attribution from the rebase comment, which is what the rule is worth on the tree it landed on, everything else in #300 on and the rule under test the only thing switched:

flag corpus PPV FP TP
USE_LSDA_LANDING_PADS AArch64 ELF, n=72 91.554 → 93.176 −1,732 +18
Rust, n=24 82.805 → 86.303 −654 +13
USE_ELF_FDE_INTERIOR_GAPS AArch64 ELF, n=72 93.176 → 94.947 −1,846 +3
Rust, n=24 86.303 → 87.480 −197 +6

The 260-cell C/C++ row is gone rather than restated: that matrix was rebuilt and re-measured at branch level only (PPV 94.038 → 96.908 over 213,706 truth functions), so there is no per-rule figure to put back. The comment says so instead of leaving the absence to be noticed.

RESOLVE_TAILCALLS had the same problem one PR further back. It described the AArch64 bl fall-through gate as it measured before #307 changed what addTailcallCandidate does (ARM64 Mach-O n=11: 12 fewer functions and 28 more false positives; Go n=45: 430 more), and it omitted the built C/C++ AArch64 ELF corpus, which is the one row where the gate is a trade rather than a win. It now carries all three re-measured rows including the −580 FP against −53 TP, says that by the branch's own per-change rule that row is a reject, and says why it survives: the 53 are all .eh_frame FDE starts that a wrong tailcall seed vetoes, so with USE_ELF_EH_FRAME_CANDIDATES on the gate costs 13 and gains 24.

A PR description records a conversation. A config comment is what the next reader has.

2. getExceptionDirectory matched a substring of the enum's repr

if "EXCEPTION" not in str(directory.type):

The typed lookup is already the idiom in two other places, one of them aarch64/FunctionCandidateManager.py walking this same directory:

directory = self.getLiefBinary().data_directory(lief.PE.DataDirectory.TYPES.EXCEPTION_TABLE)

Behaviour is unchanged on every bundled fixture, dotnet_readytorun_pe included — it still finds the table the ReadyToRun compiler put in .data, and testPeExceptionTableDiscovery still passes in full.

3. aarch64_static lost two starts and the test named one of them

The comment explained 278 → 276 through the Binary Ninja disagreement over 0x40DF34. That address is refused, but it was not recovered before the change either, so it is not one of the two the fixture lost. The actual pair, checked against the image's own unwind data:

address decodes as declared FDE
0x400350 ldr w3, [sp, #0x90] [0x400180, 0x400534)
0x40DF30 cbz x14, ... [0x40DDC0, 0x40DFCC)

Both are mid-function instructions inside a range the image declares — a plainer story than the one the comment led with, and a better one, since neither reads as an entry on any account. 0x400350 is now asserted absent alongside the other two, and the USE_ELF_FDE_INTERIOR_GAPS comment names both rather than saying "two mid-function instructions".

4. A comment #312 left dangling

#: (start, end, is_chained) for every RUNTIME_FUNCTION record the image declares, sat in intel/FunctionCandidateManager.__init__ with nothing under it since #312 hoisted _pdata_ranges into common/, where the same text already lives in full. #300 added an attribute above it, which left it reading as documentation for _retained_pad. Removed, and _seeded_prologues gets the one-line comment it should have had.

Not fixed here

Two findings from the same pass are going to #322 instead, because both are about how durable our own paths are rather than about anything wrong in #300:

  • _opensInsideAnEarlierPrologue fires only when the earlier prologue is already a candidate, so it depends on DEFAULT_PROLOGUES being scanned before DEFAULT_PROLOGUES_64. Measured: push rbp; mov rbp, rsp + push r15; push r14 refuses the interior match at +4, and the reverse layout does not. Favourable today, pinned by nothing.
  • The LSDA budget is charged only once a table's length field is reached, so an LSDA that fails earlier costs a read_va of up to MAX_LSDA_BYTES for free. Measured at ~1.2s for MAX_RECORDS distinct undecodable pointers, on top of the ~1s walk.

Gates

  • python -m pytest tests/: 2060 passed, 1 skipped, 2593 subtests
  • ruff check . and ruff format --check .: clean
  • make typecheck: exit 0, 0 error-level diagnostics

🤖 Generated with Claude Code

Three review findings on #300, all in text or in one expression, fixed here rather
than sent back for another round.

The config comments for USE_LSDA_LANDING_PADS and USE_ELF_FDE_INTERIOR_GAPS carried
the per-rule figures measured before #304/#307/#309/#310/#311 landed -- the same
baseline the PR description retracted, because master's own PPV on that corpus had
moved from 76.676 to 91.497 while the branch waited. RESOLVE_TAILCALLS likewise still
described the AArch64 bl fall-through gate as it measured before #307 changed what
addTailcallCandidate does, and omitted the AArch64 ELF corpus entirely, which is the
one row where the gate is a trade. All three now carry the re-measured attribution
against the tree #300 landed on, and the tailcall block says what the 53 lost true
positives are and how USE_ELF_EH_FRAME_CANDIDATES repays them. A description records
a conversation; a config comment is what the next reader has.

getExceptionDirectory matched the data directory with "EXCEPTION" not in
str(directory.type), a substring of the enum's repr that would also claim any future
type whose name contains it. The typed lookup is already the idiom in two other
places, one of them the AArch64 walk over the same directory.

The aarch64_static baseline moved by two starts and the test comment explained one
of them. 0x40DF34 is refused, but it was not recovered before either, so the pair the
fixture actually lost is 0x400350 and 0x40DF30 -- both mid-function instructions
inside a declared FDE, which is a plainer reading than the Binary Ninja disagreement
the comment led with. 0x400350 is now asserted alongside the other two.

Also removes a comment left dangling in intel/FunctionCandidateManager.__init__ by
#312, which hoisted the attribute it documented into common/ where the same text
already sits.

Suite 2060 passed, 1 skipped, 2593 subtests; ruff clean; ty exit 0 with 0
error-level diagnostics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📊 SMDA Performance Evaluation Benchmark Results

Generated on: 2026-09-10 13:48:55

Summary

Metric Result
Correctness ✅ PASS — 0 / 155 common file(s) differ
Determinism ✅ PASS — base 3 run(s), PR 3 run(s)
Verdict inconclusive — within run-to-run noise (±3.8%)
Median paired speedup +0.19% (95% CI [-0.78%, +0.82%])
Timing noise band ±3.8%

Per-side Timing Context (best of 3/3 runs per file)

Side Files Functions Median best time/file (s) Sum of best times (s) Throughput estimate (func/s)
base 155 173464 0.7421 289.69 ~599
pr 155 173464 0.7823 292.45 ~593

These context rows sum each file's best observed time across repeated runs; they are normalized comparison estimates, not single-run CI wall-clock times.

Paired per-file timing (positive speedup = PR faster):

Statistic Value
Files compared 155
Median paired speedup +0.19% (95% CI [-0.78%, +0.82%])
Mean speedup -1.31% (95% CI [-2.96%, +0.33%])
Std dev / IQR 10.54% / 8.39%
Wilcoxon signed-rank p 0.7242 (n=155)

ℹ️ base and PR are timed in the same job on the same runner, with their passes interleaved base/PR/PR/base/base/PR so that drift over the job cancels rather than landing on whichever side ran last. The noise band above is therefore run-to-run variation on one machine, which is what it is able to measure; a difference inside it is reported as inconclusive. Correctness and determinism are not timing-based and are unaffected either way.

Determinism (self-check across repeated runs)

Side Runs Files Deterministic Median timing CV
base 3 155 3.8%
pr 3 155 3.0%
Pairwise run matrix (individual run medians, diagnostic)

Diagnostic only: each row compares one raw PR run against one raw base run. The headline verdict above uses paired per-file best-of-runs timings.

Comparison Run PR Files Base Files Common Function Set Matches Med PR (s) Med Base (s) Med Diff (s) Speedup %
pr_0 vs base_0 155 155 155 155/155 0.8422s 0.8152s +0.0270s -3.31%
pr_0 vs base_1 155 155 155 155/155 0.8422s 0.8392s +0.0030s -0.35%
pr_0 vs base_2 155 155 155 155/155 0.8422s 0.8114s +0.0308s -3.80%
pr_1 vs base_0 155 155 155 155/155 0.8750s 0.8152s +0.0598s -7.33%
pr_1 vs base_1 155 155 155 155/155 0.8750s 0.8392s +0.0358s -4.26%
pr_1 vs base_2 155 155 155 155/155 0.8750s 0.8114s +0.0636s -7.84%
pr_2 vs base_0 155 155 155 155/155 0.8842s 0.8152s +0.0690s -8.47%
pr_2 vs base_1 155 155 155 155/155 0.8842s 0.8392s +0.0450s -5.37%
pr_2 vs base_2 155 155 155 155/155 0.8842s 0.8114s +0.0729s -8.98%

@danielplohmann
danielplohmann merged commit e9dfe5f into master Sep 10, 2026
46 checks passed
danielplohmann added a commit that referenced this pull request Sep 10, 2026
The changelog entry for what has landed since v4.5.1 (4868e19): #300, #325 and
#299. Ten engine changes under feat(core) plus a hot-path pass is a minor-bump
shape, which is why #299 and #300 were re-scoped off v4.5.1 rather than squeezed
into it.

VERSION in src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py bumped
together with the entry, per AGENTS.md.

No escaper output changed anywhere in this set. The only escaping-related edit
memoises escapeBinary results behind a cache keyed on its complete input tuple
and no escaper module is touched, so ESCAPER_DOWNWARD_COMPATIBILITY stays at
4.4.5 and INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs
reprocessing.

Figures are the contributor's own except the ARM64 Mach-O row and the
ReadyToRun result, which were reproduced during the #300 review; the entry says
which is which rather than presenting one table as equally checked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
danielplohmann added a commit that referenced this pull request Sep 10, 2026
The changelog entry for what has landed since v4.5.1 (4868e19): #300, #325 and
#299. Ten engine changes under feat(core) plus a hot-path pass is a minor-bump
shape, which is why #299 and #300 were re-scoped off v4.5.1 rather than squeezed
into it.

VERSION in src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py bumped
together with the entry, per AGENTS.md.

Deliberately still the current changelog format rather than keep-a-changelog.
#323 proposed adopting against an empty Unreleased so the first entries it holds
are written by their authors at merge time; v4.6.0 cannot be that, since #299's
and #300's entries are written here at release time either way. Adopting after
this release lets the inaugural section fill itself from the seven PRs currently
open. What this entry does adopt is the compromise: each topic keeps the
mechanism and its headline figure with the cost, and links the PR carrying the
full measurement and the dead ends -- ~2,200 words against v4.5.1's ~2,790, and
the first entry in the file to use links at all.

No escaper output changed anywhere in this set. The only escaping-related edit
memoises escapeBinary results behind a cache keyed on its complete input tuple
and no escaper module is touched, so ESCAPER_DOWNWARD_COMPATIBILITY stays at
4.4.5 and INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs
reprocessing.

Figures are the contributor's own except the ARM64 Mach-O row and the
ReadyToRun result, which were reproduced during the #300 review; the entry says
which is which rather than presenting one table as equally checked.

Also fixes AGENTS.md:117, which had prescribed a one-line changelog entry since
before v4.5.1 while v4.5.1 and v4.6.0 both use the nested **Topic:** list.
Independent of #323, which would replace that line again on adoption.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant