Skip to content

Skip HIR hashing when the crate hash comes from encoded metadata [-0.97% non-incr full] - #61

Draft
xmakro wants to merge 1 commit into
upstream-154724-basefrom
svh-from-metadata
Draft

Skip HIR hashing when the crate hash comes from encoded metadata [-0.97% non-incr full]#61
xmakro wants to merge 1 commit into
upstream-154724-basefrom
svh-from-metadata

Conversation

@xmakro

@xmakro xmakro commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Reworked on top of the upstream metadata-based crate hash work (upstream PR 154724, mirrored here as the upstream-154724-base branch). On that base the SVH is an XXH3-128 digest of the encoded metadata bytes plus a supplement fingerprint, and per-owner HIR hashing survives in non-incremental builds for exactly one job: compute_hir_hash sits in that supplement as blanket coverage for HIR-only inputs. This PR drops that job, which lets lowering skip HIR node and attribute hashing in non-incremental builds entirely.

Clean A/B against the base branch (full ThinLTO+jemalloc stage2, instructions:u, full matrix): -0.97% mean on non-incr full measurements over 164 cells, 100 wins / 2 regressions, 72 cells at or under -1%. syn check -2.45%, serde check -2.08%, hyper check -1.96%, cargo check -1.88%, externs check -6.34%. Incremental cells -0.07% mean (the change does not touch incremental hashing; the small residual win is the supplement no longer reducing every owner fingerprint). The only regression is the synthetic large-workspace at +0.43% debug full, where hundreds of tiny crates amplify the per-crate supplement walk.

What changes:

  • needs_hir_hash no longer includes the plain needs_metadata case. Per-owner hashes are still computed for incremental (red/green fingerprinting of lower_to_hir), compilers built with debug assertions (query result fingerprinting), coverage, metrics, proc-macro crates, and -Zmetadata-crate-hash=no.
  • The supplement seeded into the metadata hasher replaces the blanket HIR hash with:
    • for proc-macro crates: the full HIR hash, unchanged. Their metadata is a stub that does not describe the macro implementation, so the encoded bytes cannot stand in for the HIR.
    • for everything else: a hash of just the global_asm! owners (compute_global_asm_hash). Those are the one HIR-only input to codegen that leaves no trace in the encoded bytes — every should_encode_* predicate is off for DefKind::GlobalAsm, so only the def-kind discriminant is encoded regardless of the template — and the template can change without any source file changing, e.g. when a proc-macro reads the environment at expansion time. The owners are re-hashed from their nodes on demand (hash_owner_nodes_ungated), which is free in the common case of zero such items. This is what keeps tests/run-make/proc-macro-global-asm-changes-crate-hash passing without the blanket hash.
  • The supplement composition depends only on the crate type and the flag, never on needs_hir_hash(): a compiler built with debug assertions, or an incremental build, produces the same SVH as a release compiler in a non-incremental build. The previous version of this PR keyed the scheme on needs_hir_hash() and had both warts.

What the SVH deliberately stops covering: HIR-only content that never reaches the encoded bytes and is not a global_asm! body — concretely, bodies of functions whose MIR is not encoded (not generic, not cross-crate-inlinable), in the case where those bodies come out of untracked expansion-time input rather than a source file. Anything that lives in a source file is still covered through the src_hash of files in the encoded source map; anything downstream compilation reads is in the bytes; tracked options are in the supplement's dep_tracking_hash.

tests/ui 21694 passed, tests/run-make 397 passed (including crate-hash-metadata-flag, the three proc-macro crate-hash tests, extern-flag-fun and emit-path-unhashed), tests/incremental 178 passed, 0 failed.

@xmakro xmakro changed the title Compute the crate hash from encoded metadata in non-incremental builds Compute the crate hash from encoded metadata in non-incremental builds [-1.02% non-incr full] Aug 1, 2026
@xmakro xmakro changed the title Compute the crate hash from encoded metadata in non-incremental builds [-1.02% non-incr full] Compute the crate hash from encoded metadata in non-incremental builds [-1.00% non-incr full] Aug 1, 2026
@xmakro
xmakro force-pushed the svh-from-metadata branch from 2baf546 to e8e1760 Compare August 10, 2026 08:05
@xmakro xmakro changed the title Compute the crate hash from encoded metadata in non-incremental builds [-1.00% non-incr full] Skip HIR hashing when the crate hash comes from encoded metadata [-0.97% non-incr full] Aug 10, 2026
@xmakro
xmakro changed the base branch from perf/skip-span-hashing-nonincr to upstream-154724-base August 10, 2026 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant