Skip to content

perf: runtime AVX2 distance dispatch; fix %zu crashes in error paths - #1

Merged
trvon merged 3 commits into
masterfrom
perf/x86-runtime-dispatch
Sep 23, 2026
Merged

trvon merged 3 commits into
masterfrom
perf/x86-runtime-dispatch

Conversation

@trvon

@trvon trvon commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • perf(distances): runtime AVX2+FMA dispatch for float kernels on x86 — portable builds (no SQLITE_VEC_ENABLE_AVX) ran scalar strict-order loops for float inner product, L2 and cosine. New simd/x86_dispatch.hpp adds AVX2+FMA kernels with per-function target attributes (four accumulators) selected after a one-time CPUID check; baseline TUs stay baseline ISA. Compile-time AVX builds and non-GCC/Clang compilers are unchanged; SQLITE_VEC_DISABLE_X86_DISPATCH opts out. Threadripper 3960X, -O2, 20k rows: 384d inner product 268 → 67 ns (4.0x), L2 3.9x, cosine 3.4x; 768d 2.8–4.3x.
  • fix(sqlite): format size_t error details with %llu in sqlite3_mprintf — SQLite's printf has no z length modifier; %zu parses as its %z (free-after-use string) conversion, so error paths dereferenced a size_t as char* and crashed. Reachable from invalid input (vec0(embedding float[0])) and from corrupt HNSW persistence (dangling entry point). Six call sites fixed; test_overflow and test_persistence_fuzz now pass instead of segfaulting.
  • test(quantization): correct the RaBitQ memory-savings expectation — 384 dims pad to 512 for the FWHT rotation (72 B/vector vs 1536 B FP32 = 21.3x) and 100 vectors carry ~1.7 KB fixed state, so ~17x is correct; the old > 20x (~32x expected) bound failed on every platform.

Tests

  • Full suite on Linux x86 (clang, debug, conan sqlite3): 11/11 (previously 3 failures + HNSW timeout at default multiplier). New dispatch-vs-scalar consistency test covers dims 1–70, 384, 768.

AI usage

  • AI was used; disclosed here. Prepared with an AI coding assistant; needs maintainer review.

Consumer: trvon/yams perf/simd-release-build. Please merge with a merge commit or rebase (not squash) so the submodule SHA YAMS pins stays reachable, or bump the YAMS pointer after merging.

Portable x86-64 builds leave SQLITE_VEC_ENABLE_AVX off, so float inner
product, L2 and cosine ran scalar strict-order loops. Add AVX2+FMA kernels
with per-function target attributes (four accumulators) and select them
after a one-time CPUID check; baseline TUs stay baseline ISA. Compile-time
AVX builds and non-GCC/Clang compilers are unchanged, and
SQLITE_VEC_DISABLE_X86_DISPATCH opts out.

Threadripper 3960X, -O2, pinned core, 20k rows: 384d inner product
268 -> 67 ns (4.0x), L2 3.9x, cosine 3.4x; 768d 2.8-4.3x. A new test checks
the dispatched kernels against the scalar reference for dims 1-70, 384,
and 768.
SQLite's printf has no z length modifier: "%zu" parses as its %z
(free-after-use string) conversion, so every error path that formatted a
size_t dereferenced the integer as a char* and crashed. That turned
invalid input into a segfault instead of an error, e.g. CREATE VIRTUAL
TABLE ... vec0(embedding float[0]) and loading an HNSW index with a
dangling entry point. Use %llu with explicit casts; the existing
test_overflow and test_persistence_fuzz cases now pass instead of
crashing.
384 dims pad to 512 for the FWHT rotation, so each vector costs 64 code
bytes plus 8 bytes of factors (21.3x vs FP32), and with 100 vectors the
fixed centroid/rotation state brings the total to ~17x. The old "> 20x
(~32x expected)" bound ignored padding and fixed cost and failed on every
platform; assert the achievable range instead.
@trvon
trvon merged commit c293456 into master Sep 23, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant