Repository navigation
refactor(semantic): port src/semantic from C to Zig - #166
Merged
ata-sesli merged 1 commit intoNov 11, 2026
Merged
Conversation
1 of 4 tasks
ata-sesli
force-pushed
the
refactor/store-zova-zig
branch
from
November 11, 2026 10:17
273011e to
0687076
Compare
Replace src/semantic/{semantic,ast_profile,rotsq}.c with Zig modules in
libcbm_zig. The C headers (semantic.h, ast_profile.h, rotsq.h) stay the
ABI: each Zig module exports the same symbols, and cbm_zig.zig checks
every pub cbm_* export against its translated header declaration.
- semantic.zig: tokenizer, corpus (IDF, random indexing, co-occurrence),
vector math and the combined score. It reads the pretrained nomic
table through nomic/code_vectors.h, as the C code did.
- ast_profile.zig: the AST structural profile and its string form.
- rotsq.zig: the rotational scalar quantizer.
- fp.zig: mulAdd. The C build contracts `a * b + c` into one fused
multiply-add where the CPU has FMA (clang's -ffp-contract=on); Zig
never fuses on its own. mulAdd fuses at the same places, so the
results match the C build on the same CPU.
Behavior differences from the C code:
- rotsq builds its +/-1 diagonal at compile time instead of on first use
under cbm_once. The values are the same (XXH3 of the dimension index
with the same seed).
Build wiring: Makefile.cbm and build.zig drop the C sources and list the
Zig files; cbm_zig_c.h gains nomic/code_vectors.h, pipeline/worker_pool.h
and the two semantic headers. scripts/extract_nomic_vectors.py names
cbm_zig_c.h as the place that includes the vector table.
Net change: -80 lines.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: professional-tester <250027118+professional-tester@users.noreply.github.com>
Signed-off-by: professional-tester <250027118+professional-tester@users.noreply.github.com>
Co-authored-by: Ata Sesli <atasesli@protonmail.com>
ata-sesli
force-pushed
the
refactor/semantic-zig
branch
from
November 11, 2026 11:24
0bbaa37 to
c0413f8
Compare
1 of 4 tasks
ata-sesli
added a commit
that referenced
this pull request
Nov 11, 2026
The C semantic code never asked for fused multiply-add, but compilers decided for it: Apple clang fused `a * b + c` on arm64, while the Linux release (gcc -std=c11) and the baseline-CPU x86_64 builds did not. The semantic scores therefore differed in the last bit between platforms, which can flip a SEMANTICALLY_RELATED edge whose score sits exactly at the threshold. The Zig port (#166) copied clang's choice through fp.mulAdd, which fuses wherever the CPU has FMA. That kept macOS arm64 unchanged but changed Linux arm64, and the store quantizer (#165) never fused at all. Use one rule everywhere instead: never fuse. Zig does not fuse on its own, so `a * b + c` is written as-is and every platform computes the same scores. fp.zig is removed; its call sites in semantic.zig, rotsq.zig and pass_semantic_edges.zig become plain expressions with the same operand order. Fusing everywhere was not an option: release builds target the baseline x86_64 CPU, which has no FMA instruction, so @mulAdd would become a software fma call in the hot loops. Signed-off-by: ata-sesli <atasesli@protonmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Ports
src/semanticfrom C to Zig, following the pattern of #163. Stacked on the storage-chain port (#165,refactor/store-zova-zig); review #164 and #165 first.semantic.c,ast_profile.candrotsq.cbecomesemantic.zig,ast_profile.zigandrotsq.ziginlibcbm_zig, plus a 28-linefp.zig. No C file and no shim remains insrc/semantic.The C headers do not change.
cbm_zig.zigchecks every exportedcbm_*function against its header declaration at compile time.Net change: 2,236 lines removed, 2,154 added (−82).
Behavior differences
rotsq: the ±1 diagonal is built at compile time instead of on first use throughcbm_once. Same values.fp.mulAddfuses multiply-add at the places where clang's default-ffp-contract=onfused it in the C build, so scores stay bit-identical (see Verification).Build and audit wiring
Makefile.cbmandbuild.zigdrop the three C sources and list the Zig files and headers inCBM_ZIG_SRCS;scripts/check-build-sources.shpasses.cbm_zig_c.haddsnomic/code_vectors.handsemantic/ast_profile.h, and movessemantic/semantic.hto the ported-modules list.scripts/extract_nomic_vectors.py: the hint it prints namedsemantic.c; it now namessrc/cbm_zig_c.h.Verification
Run on Linux x86_64 with the pinned Zova.
At this branch's commit (on top of #165):
libcbm_zig.abuilds for native, x86_64-linux-gnu, aarch64-linux-gnu, x86_64-windows-gnu and aarch64-macos (without Zova).zig fmt --check,scripts/check-build-sources.sh,scripts/security-audit.shandscripts/check-dco.shpass.Before the rebase onto #165 (same semantic files, on top of main
a592846a):scripts/zova-run-tests.shmatch main, including semantic 33, ast_profile 10, pipeline 276, incremental 149 and mcp 176.SEMANTICALLY_RELATEDedges and their scores are byte-identical.zig build cbmcompiles the C sources without-Doptimize, which leaves undefined-behaviour checks in the C code, while the Zig library isReleaseFast. On redis only thesemantic_edgespass changed (9.07 s → 3.23 s).Not run / not verified:
CALLS40,815 or 40,813).make -f Makefile.cbm test,lint-ciandsmoke-test.sh.Checklist
git commit -s)make -f Makefile.cbm test): not run; the Zova-enabled suites listed above passmake -f Makefile.cbm lint-ci): not run🤖 Generated with Claude Code