Skip to content

chore: review follow-ups from the open-PR triage pass - #318

Merged
danielplohmann merged 1 commit into
masterfrom
chore/triage-followups
Sep 7, 2026
Merged

chore: review follow-ups from the open-PR triage pass#318
danielplohmann merged 1 commit into
masterfrom
chore/triage-followups

Conversation

@danielplohmann

Copy link
Copy Markdown
Owner

Closes the actionable half of #317 — five small corrections found while reviewing #302 through #311, none of which was worth blocking its PR on or opening a PR for on its own.

No behaviour change: the only src/smda/** edits are one bool() wrapper whose result is identical, and two comments.

What changed

README — the address-space section contradicted itself

It told a consumer the backend is named "in two places, report.architecture and metadata.language in toDict()". The second is wrong in exactly the direction that section exists to prevent:

cil      -> {'.net': 1.0}
dalvik   -> {'dalvik': 1.0}
intel    -> {'c/asm': 0.1, 'delphi': 0.0, '.net': 0.0, 'visualbasic': 0.0, 'go': 0.0, ...}
aarch64  -> {'c/asm': 0.1, 'delphi': 0.0, '.net': 0.0, ...}

metadata.language is a source-language score map. It carries a single decisive entry on the two managed backends and a distribution on the native ones — including a .net score. A consumer following that advice and branching on .net appearing in it can read a native report as managed, and then treat virtual addresses as file offsets, which is the precise mistake the section was added to stop.

The README already stated the score-map contract 130 lines further down ("SmdaReport.metadata.language is always a score map"), so the two halves disagreed. report.architecture is authoritative on all four backends and is now named as the field to branch on. The CilDisassembler docstring added by #308 was already careful here — it says "the language score map" — so only the consumer-facing half needed the correction.

smda_instruction_matches_capstone gets its -> bool back

#302 removed the annotation to satisfy ty 0.0.74's unsound-return-statement. The cause is upstream of the annotation: capstone_instruction is untyped, so .size is Unknown and Unknown * 2 == len(...) infers Unknown. bool() around the return makes the declared type true rather than deleting the declaration.

id: ruff becomes id: ruff-check

At ruff-pre-commit v0.16.x the hook reports itself as ruff (legacy alias). astral renamed it; the alias still resolves and will presumably be dropped in a future major. Verified both hooks run under the new id, and that ruff no longer resolves (No hook with id 'ruff' in stage 'pre-commit'), which is the intended state.

GAP_SEQUENCES gains a note about its reach

#304 added two CS-prefixed GCC nops to it while being titled and measured as a change to the alignment cut. That table is read in seven places outside the cut — five in intel/FunctionCandidateManager (the exact-length and any-length filler tests, the candidate scan, and the gap-length walk) and two in X86Backend — so an encoding added there sharpens all padding-aware discovery. Worth a comment, because it is what makes a measurement over a change to this table hard to attribute to one caller.

One test says what it is for

test_late_tailcall_does_not_rebuild_queue was renamed by #307 from test_new_scored_candidate_does_not_rebuild_queue, and its update.assert_not_called() is now trivially true: the AArch64 addTailcallCandidate no longer records a call reference, so it has no score to change and cannot reach candidate_queue.update. Kept rather than deleted — it is a reasonable guard against reintroducing scoring there — with a docstring saying so, so the next reader does not have to work out whether it is load-bearing.

Two items from #317 deliberately left out

Both turned out to need work upstream of the annotation rather than the annotation itself, which is worth recording as the reason #302 took the shortcut it did. I have added the detail to #317.

Typing SmdaFunction.blocks. Dict[int, List[SmdaInstruction]] is accurate — both population sites build SmdaInstruction objects — and it does restore getInstructionsForBlock's return type. But it surfaces two no-matching-overload errors at SmdaFunction.py:463 and :479, where getPicHashSequence and getOpcHashSequence do "".join(escaped_binary_seqs) over a list built from instruction.bytes, which #302 annotated Optional[str]. A block instruction always carries bytes in practice, so this is latent rather than live — but joining a possibly-None sequence is a real soundness gap and deserves its own change rather than a suppression here.

Narrowing extract_strings. Tuple[str, Any, Any, str] cannot be tightened while read_string, read_go_string and derefs are untyped: all three yield sites then infer tuple[Unknown, Unknown, Unknown, Unknown], so even str in the first position is unprovable and the annotation produces three unsound-yield errors. Typing those three helpers comes first.

Validation

Gate Result
ty check src/smda/ fuzzing/ profiling/ .github/workflows/scripts/ exit 0, zero errors, warning count unchanged
ruff check . All checks passed
ruff format --check . 260 files already formatted
pre-commit run ruff-check --all-files Passed
pre-commit run ruff-format --all-files Passed
full suite 1882 passed, 2 skipped, 2591 subtests

Five small corrections noticed while reviewing #302 through #311, each one
something a merged PR left slightly short of right and none worth a PR of
its own. Collected in #317 as they were found.

`README`: the address-space section told a consumer the backend is named
"in two places, report.architecture and metadata.language". The second is
wrong in the direction the section exists to prevent. metadata.language is
a source-language score map -- a single decisive entry on the managed
backends ({'.net': 1.0}, {'dalvik': 1.0}) but a distribution on the native
ones that carries a .net score -- so branching on .net appearing in it can
read a native report as managed and then treat virtual addresses as file
offsets. The README already stated the score-map contract 130 lines below,
so it contradicted itself. report.architecture is authoritative and is now
named as the field to branch on.

`smda_instruction_matches_capstone` gets its `-> bool` back. #302 dropped
the annotation to satisfy ty 0.0.74's unsound-return-statement, but the
cause is that capstone_instruction is untyped, so `.size` is Unknown and
the comparison infers Unknown. bool() around the return makes the declared
type true instead of removing the declaration.

`.pre-commit-config.yaml` moves to `id: ruff-check`. At ruff-pre-commit
v0.16.x the hook reports itself as "ruff (legacy alias)"; astral renamed it,
and the alias will presumably go in a future major.

`GAP_SEQUENCES` gains a note that it is read in seven places outside the
alignment cut, so an encoding added there changes all padding-aware
discovery rather than one caller -- which is what makes a measurement over
a change to that table hard to attribute.

`test_late_tailcall_does_not_rebuild_queue` says what it is for now that
#307 removed the scoring path it guarded, so a future reader does not have
to work out whether a trivially-true assertion is load-bearing.

Two items from #317 are deliberately not here, because both need work
upstream of the annotation rather than the annotation itself, and #317 now
records what each one runs into:

  - typing SmdaFunction.blocks as Dict[int, List[SmdaInstruction]] is
    accurate and does restore getInstructionsForBlock's return type, but it
    surfaces two no-matching-overload errors where getOpcHashSequence and
    its sibling join instruction.bytes, which is Optional[str]. In practice
    a block instruction always carries bytes, so this is latent rather than
    live -- but it is a real soundness gap and wants its own change.

  - extract_strings' Tuple[str, Any, Any, str] cannot be narrowed while
    read_string, read_go_string and derefs are untyped: every element of
    the yielded tuple infers Unknown, so even `str` in the first position
    is unprovable. Typing those three comes first.

Validation: ty check exit 0 with the error count unchanged at zero and the
warning count unchanged, ruff check and ruff format --check clean, both
pre-commit ruff hooks pass under the new id, full suite 1882 passed, 2
skipped, 2591 subtests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

📊 SMDA Performance Evaluation Benchmark Results

Generated on: 2026-09-07 15:02:43

Summary

Metric Result
Correctness ✅ PASS — 0 / 155 common file(s) differ
Determinism ✅ PASS — base 3 run(s), PR 3 run(s)
Verdict PR is faster
Median paired speedup +16.68% (95% CI [+15.49%, +18.28%])
Timing noise band ±4.3%

Per-side Timing Context (best of 3/3 runs per file)

Side Files Functions Median best time/file (s) Sum of best times (s) Throughput estimate (func/s)
base 155 175976 0.7905 305.44 ~576
pr 155 175976 0.6155 257.20 ~684

These context rows sum each file's best observed time across repeated runs; they are normalized comparison estimates, not single-run CI wall-clock times.

Paired per-file timing (positive speedup = PR faster):

Statistic Value
Files compared 155
Median paired speedup +16.68% (95% CI [+15.49%, +18.28%])
Mean speedup +18.21% (95% CI [+16.65%, +19.79%])
Std dev / IQR 9.88% / 9.41%
Wilcoxon signed-rank p 0.0000 (n=155)

ℹ️ base and PR are timed on separate CI runners, so a small median difference can reflect per-runner hardware variance rather than code. Differences within the timing noise band above are reported as inconclusive; correctness and determinism are unaffected (they are not timing-based).

Determinism (self-check across repeated runs)

Side Runs Files Deterministic Median timing CV
base 3 155 3.7%
pr 3 155 4.3%
Pairwise run matrix (individual run medians, diagnostic)

Diagnostic only: each row compares one raw PR run against one raw base run. The headline verdict above uses paired per-file best-of-runs timings.

Comparison Run PR Files Base Files Common Function Set Matches Med PR (s) Med Base (s) Med Diff (s) Speedup %
pr_0 vs base_0 155 155 155 155/155 0.7980s 0.8477s -0.0498s +5.87%
pr_0 vs base_1 155 155 155 155/155 0.7980s 0.9605s -0.1626s +16.92%
pr_0 vs base_2 155 155 155 155/155 0.7980s 0.8918s -0.0938s +10.52%
pr_1 vs base_0 155 155 155 155/155 0.7811s 0.8477s -0.0666s +7.85%
pr_1 vs base_1 155 155 155 155/155 0.7811s 0.9605s -0.1794s +18.67%
pr_1 vs base_2 155 155 155 155/155 0.7811s 0.8918s -0.1106s +12.40%
pr_2 vs base_0 155 155 155 155/155 0.8027s 0.8477s -0.0450s +5.31%
pr_2 vs base_1 155 155 155 155/155 0.8027s 0.9605s -0.1578s +16.43%
pr_2 vs base_2 155 155 155 155/155 0.8027s 0.8918s -0.0890s +9.98%

@danielplohmann
danielplohmann merged commit 395c88d into master Sep 7, 2026
47 checks passed
@danielplohmann
danielplohmann deleted the chore/triage-followups branch September 7, 2026 15:07
danielplohmann added a commit that referenced this pull request Sep 7, 2026
#318 landed as 395c88d after this entry was written. Its five corrections go
under Housekeeping: the README's metadata.language claim, the restored
`-> bool`, the ruff-check hook rename, the GAP_SEQUENCES reach note, and the
docstring on the queue-rebuild test.

Keeping this current as things merge is the point of the branch existing, so
it does not go stale between now and the release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
danielplohmann added a commit that referenced this pull request Sep 8, 2026
#320 and #321 finish the review pass that #318 started, so the release entry
should carry all seven of its corrections rather than five.

Both are behaviour-neutral typing work, but the Housekeeping text says what each
one actually decided rather than listing them as annotations: `blocks` being
typed forces the PIC/OPC hash path to refuse an instruction with no bytes
instead of blanking it, and `extract_strings` could not be narrowed until three
helpers and three `SmdaReport` attributes were declared first.

Behaviour-neutrality is stated with the evidence rather than as a claim -- the
corpus benchmark ran on #320 and reports 0 of 155 files differing across
175,946 functions, which is a stronger statement than the bundled fixtures can
make.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr
@danielplohmann danielplohmann mentioned this pull request Sep 8, 2026
danielplohmann added a commit that referenced this pull request Sep 8, 2026
@r0ny123 asked on #319 whether the changelog could be formatted as a list to
make it easier to read. At ~2,700 words on a single line, this entry was the one
that provoked the question, so it gets the treatment now rather than waiting for
the wider keep-a-changelog adoption tracked in #323.

Deliberately narrow: only the v4.5.1 entry changes shape. The other 192 entries
keep the one-line format, because converting them is a large, low-value rewrite
that loses nuance in translation, and #323 proposes preserving them verbatim
under an `Older releases` heading instead.

Structure is the release summary as the top-level bullet, one sub-bullet per
section, and a further level inside the three sections that cover more than one
change -- Recovery (intel) splits into the switch-table fix, the alignment cut
and the exception-directory refusal; Recovery (AArch64) into the inbound-call
fix, the BTI work and the candidate-quality follow-ups; Housekeeping into the
`ty` bump, the `ruff` pin, the five #318 corrections, the `binweight`
serialization note and the two #317 typing items.

The text is unchanged. Verified mechanically rather than by eye: stripping list
markers and collapsing whitespace gives a string identical to the original entry.
The two exceptions are disclosed rather than silent -- the first sub-bullet of
Recovery (intel) and of Recovery (AArch64) had their opening letter capitalised,
because splitting the section header onto its own line left them starting a
bullet mid-sentence. Housekeeping's opens on a code span and needed nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr
danielplohmann added a commit that referenced this pull request Sep 8, 2026
* chore: draft the v4.5.1 release notes

The changelog entry for what has landed since v4.5.0 (6cc5aca): #302, #303,
#304, #305, #306, #307, #308, #310 and #316, plus the Dependabot ty bump
572acd7 that #302 had to answer for.

Held as a draft rather than a release. The version is deliberately NOT
bumped here -- #309, #311 and #312 are reviewed and wanted but waiting on a
rebase, and splitting one coherent accuracy pass across two releases makes
both entries weaker. What this commit does is stop the writeup living in a
scratchpad while that happens.

To finish it: add sections for #309, #311 and #312, bump VERSION in
src/smda/SmdaConfig.py and __version__ in src/smda/__init__.py, and move the
date to the actual release day.

No escaper output changed anywhere in this set -- the only edit to
intel/definitions.py adds two GAP_SEQUENCES entries, which feed padding
detection and not escaping, and testEscaperFingerprint passes unchanged --
so ESCAPER_DOWNWARD_COMPATIBILITY stays at 4.4.5 and
INTEL_PIC_HASH_ESCAPE_VERSION at 4.3.5, and no report needs reprocessing.

Figures are the contributors' own except where this session reproduced them,
and the two that were reproduced are stated as measured here: the padding
cut's analysis-time cost (+3.4% claimed, +3.84% measured) and the fixture
movement on rust_pe_gnu_xored (+26 real starts against +8 false, scored
against that fixture's own COFF symbol table rather than the corpus macro
mean).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: fold #318 into the v4.5.1 release notes

#318 landed as 395c88d after this entry was written. Its five corrections go
under Housekeeping: the README's metadata.language claim, the restored
`-> bool`, the ruff-check hook rename, the GAP_SEQUENCES reach note, and the
docstring on the queue-rebuild test.

Keeping this current as things merge is the point of the branch existing, so
it does not go stale between now and the release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: cut v4.5.1

Finishes the draft now that #309, #311 and #312 have landed. Adds their
sections, bumps `VERSION` and `__version__` to 4.5.1, and moves the date to the
release day.

The three new entries: #309's exception-directory refusal of interior gap
candidates on x64, with the chained-versus-primary distinction and the
seeding-is-not-suppression principle that keeps carved records out of it; #311's
four candidate-quality defects, including the corrected entry at `0x40df30` on
`aarch64_static_xored` and the metadata-coverage gate; and #312's ARM64
counterpart, which reconstructs the extent an ARM64 `RUNTIME_FUNCTION` does not
carry and hoists the shared lookup into `common/`.

`USE_MACHO_ADDRESS_REF_CANDIDATES` gets its own **Defaults changed** section. It
is the only default whose value moves in this release, it changes recovery
output on Mach-O input with no action by the caller, and a default going from
off to on inside a follow-ups PR is exactly the kind of thing that gets lost.

The closing movement section is rewritten to describe v4.5.0 -> v4.5.1 rather
than any single PR, and both figures in it were measured on this tree against
independent truth rather than quoted:

  rust_pe_gnu_xored, vs its 2,186 retained COFF .text function symbols
    2,355 -> 2,201 starts, +26 real / -180 false
    recall    92.452% -> 93.641%
    precision 85.817% -> 93.003%

  11-sample ARM64 Mach-O corpus, vs LC_FUNCTION_STARTS (2,056 truth)
    TP 1,787 -> 1,818, FP 1,113 -> 1,107
    recall 86.916% -> 88.424%, no sample losing recall

Two headline figures are marked as the contributor's own because they cannot be
reproduced here at all: the `bti j` 803/0 split, since no bundled fixture holds
a single `bti j` word, and #312's ARM64 PE result, since a scan of all 104
fixture files finds 8 i386 PEs, 4 AMD64, one ReadyToRun image and no `0xAA64`.
Saying which figures are ours and which are theirs is more useful than a uniform
tone of confidence.

`Turtle_5f9cd91d8d1d`'s 12 dropped starts are attributed rather than left as an
unexplained delta: they bisect to #307's inbound-call fix and fall inside the
175 false positives that change already measured, and the sample carries no
LC_FUNCTION_STARTS to score them against individually.

Closes #319

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

* chore: record binweight's serialized type change in the v4.5.1 entry

Left open as a question in yesterday's review and resolved as "yes, it belongs
in the entry". #302 moved `SmdaFunction.binweight`'s class default from int `0`
to float `0.0` while clearing ty 0.0.74's diagnostics, and `binweight` is
serialized through `toDict()`.

The blast radius is small but real: the per-block accumulation already adds
`float(...)`, so every function with at least one block was a float before this
release too. Only a function with no blocks -- a zero-function or error report
-- keeps the class default, and that value now writes as `0.0` where it wrote
`0`. Verified on both trees rather than reasoned about: `SmdaFunction.binweight`
is `0` at 6cc5aca and `0.0` at d111548.

MCRIT stores these reports, so a consumer diffing them byte-for-byte will see
it even though nothing reads the field as an integer. That is exactly the kind
of change that costs someone an afternoon if it is not written down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

* chore: fold the two remaining #317 items into the v4.5.1 entry

#320 and #321 finish the review pass that #318 started, so the release entry
should carry all seven of its corrections rather than five.

Both are behaviour-neutral typing work, but the Housekeeping text says what each
one actually decided rather than listing them as annotations: `blocks` being
typed forces the PIC/OPC hash path to refuse an instruction with no bytes
instead of blanking it, and `extract_strings` could not be narrowed until three
helpers and three `SmdaReport` attributes were declared first.

Behaviour-neutrality is stated with the evidence rather than as a claim -- the
corpus benchmark ran on #320 and reports 0 of 155 files differing across
175,946 functions, which is a stronger statement than the bundled fixtures can
make.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

* chore: format the v4.5.1 entry as a nested list

@r0ny123 asked on #319 whether the changelog could be formatted as a list to
make it easier to read. At ~2,700 words on a single line, this entry was the one
that provoked the question, so it gets the treatment now rather than waiting for
the wider keep-a-changelog adoption tracked in #323.

Deliberately narrow: only the v4.5.1 entry changes shape. The other 192 entries
keep the one-line format, because converting them is a large, low-value rewrite
that loses nuance in translation, and #323 proposes preserving them verbatim
under an `Older releases` heading instead.

Structure is the release summary as the top-level bullet, one sub-bullet per
section, and a further level inside the three sections that cover more than one
change -- Recovery (intel) splits into the switch-table fix, the alignment cut
and the exception-directory refusal; Recovery (AArch64) into the inbound-call
fix, the BTI work and the candidate-quality follow-ups; Housekeeping into the
`ty` bump, the `ruff` pin, the five #318 corrections, the `binweight`
serialization note and the two #317 typing items.

The text is unchanged. Verified mechanically rather than by eye: stripping list
markers and collapsing whitespace gives a string identical to the original entry.
The two exceptions are disclosed rather than silent -- the first sub-bullet of
Recovery (intel) and of Recovery (AArch64) had their opening letter capitalised,
because splitting the section header onto its own line left them starting a
bullet mid-sentence. Housekeeping's opens on a code span and needed nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MDCLUmqGzE5fwJqUWsSStr

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant