Allow embedding ced.cpp in another ggml project - #3
Closed
localai-org-maint-bot wants to merge 5 commits into
Closed
localai-org-maint-bot wants to merge 5 commits into
localai-org-maint-bot wants to merge 5 commits into
Conversation
Each triangular mel filter is non-zero over a narrow band of the 257 FFT bins, but the frontend multiplied the full row for every frame. The product now runs over each filter's non-zero band only. The skipped terms are exact zeros, so the output stays bit-identical. The mel frontend runs on the host. With the encoder on a GPU it was most of the latency: on a GB10 it took 101 of 117 ms for ced-base on a 36 s clip. With this change the full classify takes 62 ms. Assisted-by: Claude:claude-opus-5-5 [Claude Code]
The runner always used a static CPU backend and created and freed a
graph allocator for every graph. GPU builds compiled but never used
the GPU.
Each loaded model now owns a backend. It picks the first GPU or
integrated GPU, or the device named in CED_DEVICE ("cpu", "CUDA0",
"Vulkan0", "MTL0"), and falls back to the CPU. One ggml_gallocr is kept
for the model's lifetime. A graph goes through ggml_backend_sched with
a CPU fallback only when the device has no kernel for one of its ops.
On a GPU the weights are uploaded to one device buffer at load. The
tensors the host reads (mel window, filterbank, init_bn stats) keep a
host copy, and the init_bn scale and shift are folded once at load.
classify now runs each chunk as one graph instead of two with a host
round trip in between.
CPU output is bit-identical to the previous code (all 527 probs, four
models, two clips) and CPU speed is unchanged. On GPU the top-5 tags
match the CPU on CUDA (GB10), Vulkan (Radeon 8060S) and Metal (M4).
The GPU matmuls move intermediate activations by up to 1.15e-2, so the
per-stage parity tests use a 2e-2 tolerance off the CPU. The CPU
tolerances and all end-to-end probability checks are unchanged.
Assisted-by: Claude:claude-opus-5-5 [Claude Code]
parakeet.cpp adds ced.cpp as a submodule for scene-sound classification, which needs ced to build inside a host project that already has its own ggml target and dr_wav implementation. Guard the GGML_* cache variable forwarding and the ggml submodule with a CED_TOP_LEVEL / TARGET ggml check so an embedding project's own ggml config and build are left alone. Add CED_EXTERNAL_DR_WAV so the host's dr_wav implementation can be reused instead of compiling a second copy into the same binary, which would otherwise fail to link with duplicate drwav_ symbols. tests/CMakeLists.txt now resolves paths from PROJECT_SOURCE_DIR instead of CMAKE_SOURCE_DIR so the ctest fixtures still resolve when ced is not the top-level project. Also add ced_capi_classify_pcm_probs, which writes every class score in class-index order with no sorting and no allocation, for callers (such as parakeet.cpp's streaming scene detector) that want the raw distribution on every window instead of a sorted top-k. Assisted-by: Claude:claude-sonnet-5 [Claude Code]
examples/cli/CMakeLists.txt still resolved its include dirs from CMAKE_SOURCE_DIR, which is the outermost project's root, not ced's own. CED_BUILD_CLI defaults ON, so any host project doing add_subdirectory(ced) without explicitly turning it off got a build break, the same class of bug the previous commit fixed elsewhere. Switch to PROJECT_SOURCE_DIR. Fixing that path uncovered a second problem: with CED_EXTERNAL_DR_WAV on, libced.a is built without a dr_wav implementation, on the assumption that the embedding host's own binary provides one. ced-cli is not that binary, it is a separate executable built by ced.cpp itself, so it linked with undefined drwav_* references. Give ced-cli its own dr_wav_impl.cpp, compiled in only when CED_EXTERNAL_DR_WAV is set, so it stays a self-contained tool regardless of how the library is configured. Assisted-by: Claude:claude-sonnet-5 [Claude Code]
This was referenced Sep 28, 2026
mudler
added a commit
that referenced
this pull request
Sep 29, 2026
Reuse an existing ggml target when ced.cpp is not the top-level project, add CED_EXTERNAL_DR_WAV so an embedding project can provide the dr_wav implementation, fix ced-cli's include paths when embedded, and add ced_capi_classify_pcm_probs (all class scores, unsorted) for per-window scoring. parakeet.cpp's sound-event support builds on it. Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Contributor
Author
|
Landed on main as 61dec2a (squashed; same change as this branch, verified with a diff). Closing. |
localai-org-maint-bot
added a commit
to mudler/parakeet.cpp
that referenced
this pull request
Sep 29, 2026
third_party/ced.cpp pointed at the head of localai-org/ced.cpp#3, which only existed on its feat/embedding branch. That change landed on ced.cpp main as 61dec2a with an identical tree, so pin main and drop the dependency on the branch. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
mudler
added a commit
to mudler/LocalAI
that referenced
this pull request
Sep 29, 2026
parakeet.cpp #76 moved its ced.cpp submodule from the head of localai-org/ced.cpp#3 (a branch-only commit) to ced.cpp main, where #3 landed with an identical tree. Pin 623a968 so the image builds no longer depend on that branch. Assisted-by: Claude:claude-opus-5-5 [Claude Code]
mudler
added a commit
to mudler/LocalAI
that referenced
this pull request
Sep 29, 2026
parakeet.cpp #76 moved its ced.cpp submodule from the head of localai-org/ced.cpp#3 (a branch-only commit) to ced.cpp main, where #3 landed with an identical tree. Pin 623a968 so the image builds no longer depend on that branch. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
CMakeLists.txt: guards theGGML_*cache variable forwarding, theGGML_NATIVE/GGML_LLAMAFILEdefaults, andadd_subdirectory(third_party/ggml)behind a
CED_TOP_LEVELcheck (is ced.cpp the top-level CMake project?) anda
NOT TARGET ggmlcheck. When ced.cpp is added withadd_subdirectoryby ahost project that already configured its own
ggmltarget, ced reuses itinstead of configuring a second one.
CMakeLists.txt/src/audio_io.cpp: newCED_EXTERNAL_DR_WAVoption(default OFF). When ON, ced does not define
DR_WAV_IMPLEMENTATION, so thehost project can provide the one dr_wav implementation in the final binary
instead of linking two and hitting
multiple definition of drwav_*.tests/CMakeLists.txt: fixture and include paths now resolve fromPROJECT_SOURCE_DIR(ced's own project root) instead ofCMAKE_SOURCE_DIR(the outermost project root), so the ctest targets still find
models/andtests/fixtures/when ced is not the top-level project.include/ced_capi.h/src/ced_capi.cpp: newced_capi_classify_pcm_probs(ctx, samples, n_samples, sample_rate, out, n_out),which writes every class score in class-index order (no sorting, no
allocation) and returns the count written, or
-1on error. Callers thatwant the raw per-class distribution on every window (rather than a sorted
top-k) no longer have to reimplement
do_classify+ memcpy themselves.tests/test_capi.cpp: covers the new function against the existingsorted top-k path (same scores at the same indices), a short output buffer
(
n_out < num_classes), and the null-argument error paths.README.md: documentsCED_EXTERNAL_DR_WAV, the ggml-reuse behavior, andced_capi_classify_pcm_probsin the C-API section.Why
parakeet.cpp is adding ced.cpp as a git submodule for scene-sound tagging.
parakeet.cpp already vendors its own ggml and its own dr_wav, so ced.cpp
needs to build inside that host project without configuring a second ggml
or linking a second dr_wav implementation, and parakeet.cpp's streaming
scene detector needs the full unsorted score vector per window rather than
a sorted top-k.
How to verify
Standalone build is unaffected (Step 7 in the task brief):
Embedding in a host project that already has a
ggmltarget and providesits own dr_wav (Step 8 in the task brief):
Builds clean with no
multiple definition of drwav_*link errors, ggml isconfigured once (by the host project, reused by ced), and the binary prints
EMBED_OK.This is stacked on #2 (the GPU backend):
feat/embeddingbranches fromfeat/gpu-backend, so this diff (againstmain) also carries #2's commitsuntil #2 merges.
🤖 Generated with Claude Code