feat(arena): arena-backed hashconsing behind bumpalo feature (2.16x) - #33
Open
elefthei wants to merge 1 commit into
Open
feat(arena): arena-backed hashconsing behind bumpalo feature (2.16x)#33elefthei wants to merge 1 commit into
bumpalo feature (2.16x)#33elefthei wants to merge 1 commit into
Conversation
Adds `BHConsed`/`BHConsign`, the arena counterpart of `HConsed`/`HConsign`, in a new `arena` module gated on the optional `bumpalo` feature. The existing `Arc` path is untouched. Values are bump-allocated in a caller-owned `bumpalo::Bump`, so handles are `Copy` (`&'bump T` + `u64` uid) rather than refcounted, and the consign keys its table on the interned reference itself, dropping the `T: Clone` bound the `Arc` consign needs. The trade-off is documented prominently: `Bump` never runs destructors, and there are no weak refs, hence no `collect` and no `consign!` analogue (`Bump` is `Send` but not `Sync`). Verified by unit tests (`src/test/arena.rs`), two module doctests, a rayon integration test proving handles are `Send + Sync`, and a new `arena_vs_arc` bench that replays one generated term DAG through both consigns and fails unless the arena wins. Measured, 200k instructions: phase Arc (HConsed) Arena (BHConsed) speedup build 30.989 ms 18.200 ms 1.70x re-intern 22.953 ms 11.956 ms 1.92x traverse 15.506 ms 6.697 ms 2.32x teardown 10.282 ms 0.002 ms 5411.73x total 79.730 ms 36.855 ms 2.16x Assistant-model: Claude Opus
Collaborator
|
Is there a reason that htis uses |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measurements
cargo bench --features bumpalo --bench arena_vs_arc, 200k instructions, min of 5 reps, same generated term DAG replayed through both consigns:HConsed)BHConsed)The bench is a target in this PR, not a one-off: it is
harness = false,required-features = ["bumpalo"], and asserts the wins (total ≥ 2.0x, traverse ≥ 1.5x, teardown ≥ 3.0x), so it exits non-zero if the arena path regresses. Numbers above are from one machine and are not the pass criterion — the exit code is.Only
traverseandteardowncarry thresholds, because those wins are structural: traversal pushes/pops/inserts handles, which is several uncontended atomic RMWs per node on theArcside versus a 16-byte copy on the arena side; teardown freesNArcallocations plusNowned table keys versus dropping one PODHashMapand a few bump chunks.buildandre-internratios are printed but not asserted — real, but smaller and more machine-dependent.Summary
Adds
BHConsed/BHConsign, an arena-backed counterpart toHConsed/HConsign, behind a new optionalbumpalofeature. Values live in a caller-ownedbumpalo::Bump; handles areCopy(&'bump T+u64uid) instead of refcounted. The existingArcpath is untouched — this is an addition, not a change.Two wins beyond raw speed:
T: Clonebound (theArcconsign clones every element to use as its key);Changes
src/arena.rs— new module, compiled only underfeature = "bumpalo".BHConsed<'bump, T>mirrorsHConsed's impls (PartialEq/Eq/Ord/Hashon uid only,Deref,Borrow,Debug,Display) withClone/Copyhand-written to avoid spuriousT: Clone/T: Copybounds.BHConsign<'bump, T, S>providesnew/with_capacity/with_hasher/with_capacity_and_hasher,mk/mk_is_new,contains,len/is_empty/capacity,iter/consed_iter,arena,reserve/shrink_to_fit. Re-exportsbumpaloso callers need not pin a matching version.Cargo.toml— optionalbumpalo = "^3.16"+bumpalo = ["dep:bumpalo"]feature, added to theunstable_docrsaggregate so the module renders on docs.rs.rust-versionuntouched. New[[bench]] arena_vs_arc.src/lib.rs—#[cfg(feature = "bumpalo")] pub mod arena;plus re-exports, and a short crate-doc section. Mentions are plain backticks, not intra-doc links, socargo docstays clean with the feature off.src/test/arena.rs,tests/send_sync.rs— unit tests and a rayon test.benches/arena_vs_arc.rs— the bench above.Caveats
Destructors never run. Values are bump-allocated, so any heap owned by
T(String,Vec<_>, …) is leaked until the arena drops. This is forced rather than incidental: handles areCopyand carry'bump, so no owner could drop a value earlier without leaving handles dangling. Documented prominently in the module header, with the recommendation to use arena-friendly payloads (&'bump str,&'bump [T]allocated viaBHConsign::arena) or to stay on theArcpath when destructors matter.Other differences from the
Arcpath, all documented:collect/collect_to_fit— memory returns when theBumpdrops or resets;consign!analogue:BumpisSendbut notSync, so alazy_staticarena consign is impossible.BHConsignis!Send; individualBHConsedhandles areSend + SyncwhenT: Sync;hash_coll::HConSet/HConMapstayHConsed-only.BHConsed::hashwrites exactly oneu64, soHashSet<Term, hash_coll::hashers::p_hash::Builder>works directly and is already fast. Generalizing those collections overHashConsedis a crate-wide refactor and deliberately out of scope here.Verification
cargo test --workspacecargo test --workspace --all-featurescargo clippy --workspace --all-features#![deny(warnings)])cargo doc --workspace --all-features/cargo doc --workspacecargo fmt --checkcargo bench --features bumpalo --bench arena_vs_arcTwo module doctests double as the user-facing correctness proof: the lambda-calculus one interns
Var(3)twice and assertsfactory.len() == 3,v2.uid() == v3.uid()andstd::ptr::eq(v2.get(), v3.get()), i.e. perfect sharing with each distinct term allocated exactly once. Note its term type deliberately does not deriveClone.benches/traversals.rsis#![feature(test)]and still nightly-only; the new bench isharness = falseand selected explicitly, so nothing here requires nightly and default target selection is unaffected.