Add packed 4-bit SafeTensors weight support - #19
Open
FelixKrall wants to merge 3 commits into
Open
FelixKrall wants to merge 3 commits into
FelixKrall wants to merge 3 commits into
Conversation
Extend the shared external-weight resolver and writer to support packed logical Int4 and Uint4 tensors in versioned SafeTensors archives. Resolve ordinary and packed constants in one memory-mapped traversal, validate names, metadata, storage types, shapes, and lengths, and write archives atomically without an archive-sized output allocation. Document the .webnn external-weight contract and cover packed, mixed, malformed, sanitized-name, manifest, and raw-weight cases.
## Feature additions - Add a canonical `.webnn` format reference covering graph versions, shapes, initializers, node options, multi-output operations, output bindings, and `GraphJson` round trips. - Add a maintained v2 format example and parser → serializer → parser regression coverage. - Document SafeTensors, packed Int4/Uint4 storage, manifest-backed raw weights, sidecar discovery, mmap ownership, and atomic writer behavior. ## Bugfixes - Make the example build workflow reconstruct untracked tensor inputs from the checked-in weight archive. - Update the example workflow to use the current direct JavaScript emitter command. ## Refactors - Consolidate duplicated format material into canonical documentation pages. - Rewrite the README, dynamic-dimension guide, ONNX-lowering guide, and example documentation against current source behavior. - Remove stale model-specific guidance, unsupported quantitative claims, and obsolete agent-oriented files. ## Behavioral impact and compatibility - No library API, file-format, parser, serializer, or runtime behavior changes are intended. - The example build script now works without locally retained raw tensor files but produces the same documented artifacts. - The removed pages are replaced by canonical format and workflow documentation. ## Validation - `cargo fmt --all -- --check` passed. - `cargo test --all-features` passed, including 195 library tests and the format-reference and ResNet conversion integration tests. - `cargo test --no-default-features` passed, including 72 library tests and the format-reference integration test. - Strict Clippy passed for all targets with all features and without default features. - Parse, serialize, structural validation, manifest validation, pack, unpack, and JavaScript-emitter smoke checks passed. - Relative Markdown links, script syntax, line lengths, stale-language searches, and `git diff --check` passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is used and depended on by RustNN #238 and onnx2webnn #5
Summary
Centralize
.webnnexternal-weight reading and writing inwebnn-graph. The resolver now supports versioned packed Int4/Uint4 SafeTensors storage, memory-maps external archives, and shares the same validation rules with a new atomic SafeTensors writer.Feature additions
rustnn.webnn.packed4=1metadata contract.write_external_weights_safetensors, which validates external declarations and writes ordinary and packed tensors through a same-directory temporary file..webnn,@weights, sidecar discovery, dtype mappings, packed-4-bit storage, naming rules, and compatibility guarantees.Bugfixes
Refactors
webnn-graphfor reuse by consumers such as RustNN.Behavioral impact and compatibility
resolve_external_weightsAPI remains compatible.rustnn.webnn.packed4=1and the low-nibble-first layout remain supported.ceil(logical_elements / 2); the.webnndeclaration remains authoritative for their logical dtype and shape.InlineBytes; externally backedGraphJsonstorage is not introduced.Validation
cargo fmt --all -- --check— passed.cargo test external_weights --no-default-features— 13 focused external-weight tests passed.cargo test --all-features— 195 unit tests and the ResNet integration test passed.cargo test --no-default-features— 72 tests passed.cargo clippy --all-targets --no-default-features -- -D warnings— passed.cargo clippy --all-targets --all-features -- -D warnings— reaches a pre-existingclippy::question_markfailure insrc/onnx/shape_inference.rs; no changed external-weight code is implicated.git diff --check— passed.Reviewer focus