Skip to content

Allow embedding ced.cpp in another ggml project - #3

Closed
localai-org-maint-bot wants to merge 5 commits into
mainfrom
feat/embedding
Closed

localai-org-maint-bot wants to merge 5 commits into
mainfrom
feat/embedding

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Contributor

What changed

  • CMakeLists.txt: guards the GGML_* cache variable forwarding, the
    GGML_NATIVE/GGML_LLAMAFILE defaults, and add_subdirectory(third_party/ggml)
    behind a CED_TOP_LEVEL check (is ced.cpp the top-level CMake project?) and
    a NOT TARGET ggml check. When ced.cpp is added with add_subdirectory by a
    host project that already configured its own ggml target, ced reuses it
    instead of configuring a second one.
  • CMakeLists.txt / src/audio_io.cpp: new CED_EXTERNAL_DR_WAV option
    (default OFF). When ON, ced does not define DR_WAV_IMPLEMENTATION, so the
    host project can provide the one dr_wav implementation in the final binary
    instead of linking two and hitting multiple definition of drwav_*.
  • tests/CMakeLists.txt: fixture and include paths now resolve from
    PROJECT_SOURCE_DIR (ced's own project root) instead of CMAKE_SOURCE_DIR
    (the outermost project root), so the ctest targets still find
    models/ and tests/fixtures/ when ced is not the top-level project.
  • include/ced_capi.h / src/ced_capi.cpp: new
    ced_capi_classify_pcm_probs(ctx, samples, n_samples, sample_rate, out, n_out),
    which writes every class score in class-index order (no sorting, no
    allocation) and returns the count written, or -1 on error. Callers that
    want the raw per-class distribution on every window (rather than a sorted
    top-k) no longer have to reimplement do_classify + memcpy themselves.
  • tests/test_capi.cpp: covers the new function against the existing
    sorted top-k path (same scores at the same indices), a short output buffer
    (n_out < num_classes), and the null-argument error paths.
  • README.md: documents CED_EXTERNAL_DR_WAV, the ggml-reuse behavior, and
    ced_capi_classify_pcm_probs in the C-API section.

Why

parakeet.cpp is adding ced.cpp as a git submodule for scene-sound tagging.
parakeet.cpp already vendors its own ggml and its own dr_wav, so ced.cpp
needs to build inside that host project without configuring a second ggml
or linking a second dr_wav implementation, and parakeet.cpp's streaming
scene detector needs the full unsorted score vector per window rather than
a sorted top-k.

How to verify

Standalone build is unaffected (Step 7 in the task brief):

rm -rf build && cmake -B build -DCED_BUILD_TESTS=ON && cmake --build build -j 16
ctest --test-dir build
# 100% tests passed, 0 tests failed out of 7

Embedding in a host project that already has a ggml target and provides
its own dr_wav (Step 8 in the task brief):

mkdir -p /tmp/embed && cd /tmp/embed
ln -sfn <path-to-ced.cpp> ced
cat > CMakeLists.txt <<'EOF'
cmake_minimum_required(VERSION 3.18)
project(embed CXX C)
add_subdirectory(ced/third_party/ggml)        # host project's ggml comes first
set(CED_BUILD_CLI OFF CACHE BOOL "" FORCE)
set(CED_EXTERNAL_DR_WAV ON CACHE BOOL "" FORCE)
add_subdirectory(ced)
add_executable(embed main.cpp)
target_link_libraries(embed PRIVATE ced)
EOF
cat > main.cpp <<'EOF'
#define DR_WAV_IMPLEMENTATION
#include "ced/third_party/dr_wav.h"
#include "ced_capi.h"
int main() { return ced_capi_abi_version() == 1 ? 0 : 1; }
EOF
cmake -B b && cmake --build b -j 16 && ./b/embed && echo EMBED_OK

Builds clean with no multiple definition of drwav_* link errors, ggml is
configured once (by the host project, reused by ced), and the binary prints
EMBED_OK.

This is stacked on #2 (the GPU backend): feat/embedding branches from
feat/gpu-backend, so this diff (against main) also carries #2's commits
until #2 merges.

🤖 Generated with Claude Code

Each triangular mel filter is non-zero over a narrow band of the 257
FFT bins, but the frontend multiplied the full row for every frame.
The product now runs over each filter's non-zero band only. The skipped
terms are exact zeros, so the output stays bit-identical.

The mel frontend runs on the host. With the encoder on a GPU it was
most of the latency: on a GB10 it took 101 of 117 ms for ced-base on a
36 s clip. With this change the full classify takes 62 ms.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
The runner always used a static CPU backend and created and freed a
graph allocator for every graph. GPU builds compiled but never used
the GPU.

Each loaded model now owns a backend. It picks the first GPU or
integrated GPU, or the device named in CED_DEVICE ("cpu", "CUDA0",
"Vulkan0", "MTL0"), and falls back to the CPU. One ggml_gallocr is kept
for the model's lifetime. A graph goes through ggml_backend_sched with
a CPU fallback only when the device has no kernel for one of its ops.

On a GPU the weights are uploaded to one device buffer at load. The
tensors the host reads (mel window, filterbank, init_bn stats) keep a
host copy, and the init_bn scale and shift are folded once at load.
classify now runs each chunk as one graph instead of two with a host
round trip in between.

CPU output is bit-identical to the previous code (all 527 probs, four
models, two clips) and CPU speed is unchanged. On GPU the top-5 tags
match the CPU on CUDA (GB10), Vulkan (Radeon 8060S) and Metal (M4).

The GPU matmuls move intermediate activations by up to 1.15e-2, so the
per-stage parity tests use a 2e-2 tolerance off the CPU. The CPU
tolerances and all end-to-end probability checks are unchanged.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
parakeet.cpp adds ced.cpp as a submodule for scene-sound classification,
which needs ced to build inside a host project that already has its own
ggml target and dr_wav implementation.

Guard the GGML_* cache variable forwarding and the ggml submodule with
a CED_TOP_LEVEL / TARGET ggml check so an embedding project's own ggml
config and build are left alone. Add CED_EXTERNAL_DR_WAV so the host's
dr_wav implementation can be reused instead of compiling a second copy
into the same binary, which would otherwise fail to link with duplicate
drwav_ symbols. tests/CMakeLists.txt now resolves paths from
PROJECT_SOURCE_DIR instead of CMAKE_SOURCE_DIR so the ctest fixtures
still resolve when ced is not the top-level project.

Also add ced_capi_classify_pcm_probs, which writes every class score in
class-index order with no sorting and no allocation, for callers (such
as parakeet.cpp's streaming scene detector) that want the raw
distribution on every window instead of a sorted top-k.

Assisted-by: Claude:claude-sonnet-5 [Claude Code]
examples/cli/CMakeLists.txt still resolved its include dirs from
CMAKE_SOURCE_DIR, which is the outermost project's root, not ced's own.
CED_BUILD_CLI defaults ON, so any host project doing add_subdirectory(ced)
without explicitly turning it off got a build break, the same class of
bug the previous commit fixed elsewhere. Switch to PROJECT_SOURCE_DIR.

Fixing that path uncovered a second problem: with CED_EXTERNAL_DR_WAV on,
libced.a is built without a dr_wav implementation, on the assumption that
the embedding host's own binary provides one. ced-cli is not that binary,
it is a separate executable built by ced.cpp itself, so it linked with
undefined drwav_* references. Give ced-cli its own dr_wav_impl.cpp,
compiled in only when CED_EXTERNAL_DR_WAV is set, so it stays a
self-contained tool regardless of how the library is configured.

Assisted-by: Claude:claude-sonnet-5 [Claude Code]
Brings in the squash-merged GPU backend (#2) this branch was stacked on. The only conflict was the README build section; kept this branch's side, which has #2's text plus the add_subdirectory note.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
mudler added a commit that referenced this pull request Sep 29, 2026
Reuse an existing ggml target when ced.cpp is not the top-level
project, add CED_EXTERNAL_DR_WAV so an embedding project can provide
the dr_wav implementation, fix ced-cli's include paths when embedded,
and add ced_capi_classify_pcm_probs (all class scores, unsorted) for
per-window scoring. parakeet.cpp's sound-event support builds on it.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
@localai-org-maint-bot

Copy link
Copy Markdown
Contributor Author

Landed on main as 61dec2a (squashed; same change as this branch, verified with a diff). Closing.

localai-org-maint-bot added a commit to mudler/parakeet.cpp that referenced this pull request Sep 29, 2026
third_party/ced.cpp pointed at the head of localai-org/ced.cpp#3,
which only existed on its feat/embedding branch. That change landed on
ced.cpp main as 61dec2a with an identical tree, so pin main and drop
the dependency on the branch.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
mudler added a commit to mudler/LocalAI that referenced this pull request Sep 29, 2026
parakeet.cpp #76 moved its ced.cpp submodule from the head of
localai-org/ced.cpp#3 (a branch-only commit) to ced.cpp main, where
#3 landed with an identical tree. Pin 623a968 so the image builds no
longer depend on that branch.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
mudler added a commit to mudler/LocalAI that referenced this pull request Sep 29, 2026
parakeet.cpp #76 moved its ced.cpp submodule from the head of
localai-org/ced.cpp#3 (a branch-only commit) to ced.cpp main, where
#3 landed with an identical tree. Pin 623a968 so the image builds no
longer depend on that branch.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants