refactor(codegen): migrate bb.js BB API codegen to ipc-codegen#23316
Open
charlielye wants to merge 4 commits into
Open
refactor(codegen): migrate bb.js BB API codegen to ipc-codegen#23316charlielye wants to merge 4 commits into
charlielye wants to merge 4 commits into
Conversation
charlielye
force-pushed
the
cl/ipc-codegen-migrate-bb-wsdb
branch
6 times, most recently
from
f8c3cd3 to
f5f4f12
Compare
charlielye
force-pushed
the
cl/ipc-codegen-migrate-bb-wsdb
branch
from
f5f4f12 to
905a0cb
Compare
charlielye
force-pushed
the
cl/ipc-codegen-migrate-bb-wsdb
branch
4 times, most recently
from
47c4084 to
d842be8
Compare
charlielye
requested review from
IlyasRidhuan,
MirandaWood and
jeanmon
as code owners
6 tasks
charlielye
force-pushed
the
cl/ipc-codegen-migrate-bb-wsdb
branch
from
910e425 to
dd3a119
Compare
charlielye
force-pushed
the
cl/ipc-codegen-migrate-bb-wsdb
branch
4 times, most recently
from
f172910 to
4e82660
Compare
…te cbind/
Move bb.js's BB API and aztec-wsdb's IPC bindings from the legacy
barretenberg/ts/src/cbind/ generator to ipc-codegen. Each consumer now
owns the JSON schema it depends on (committed next to the C++ server
that defines the wire format) and invokes ipc-codegen with that local
path — ipc-codegen has no knowledge of any specific service.
Schemas (committed):
- barretenberg/cpp/src/barretenberg/bbapi/bb_schema.json
- barretenberg/cpp/src/barretenberg/bbapi/bb_curve_constants.json
- barretenberg/cpp/src/barretenberg/wsdb/wsdb_schema.json
Generation entrypoints:
- TS + Rust: barretenberg/ts/scripts/generate.sh, invoked by `yarn generate`.
Passes each consumer's schema path explicitly.
- C++ WSDB: barretenberg/cpp/src/barretenberg/wsdb/CMakeLists.txt declares
an add_custom_command that runs ipc-codegen at build time, with ninja's
dependency graph ensuring generated files exist before wsdb_ipc_client
compiles.
WSDB C++ client (barretenberg/cpp/src/barretenberg/wsdb_client/
wsdb_ipc_merkle_db.{hpp,cpp}): lifted from cl/more_ipc_v2 — adds inline
wire<->domain conversion helpers (fr_to_wire / fr_from_wire / fr_vec_from_wire
/ revision_to_wire / tree_id_to_wire) and threads them through every call
site. Public interface unchanged. The aztec-wsdb server binary is untouched
on next — its hand-written SERIALIZATION_FIELDS still define the canonical
wire format, and both cbind and ipc-codegen emit byte-compatible clients
against the schema extracted from that server.
barretenberg-rs: switch the BB API consumer from the legacy hand-written
types/backends to ipc-codegen output, dropping ~280 LOC of duplicated
backend/error/types/ffi scaffolding. iOS/embedded FFI build path preserved
via the codegen-emitted FfiBackend (byte-identical to the deleted version).
Deletions:
- barretenberg/ts/src/cbind/ (legacy multi-language codegen)
- barretenberg/ts/src/aztec-wsdb/generate.ts (driver for the deleted cbind/)
- yarn-project/world-state/src/native/ipc_world_state_instance.ts (dead
code on next; the IPC stack re-adds it against the new generated TS
shape during its cutover)
- barretenberg/rust/barretenberg-rs/src/{backend,error,types}.rs and
backends/ffi.rs (superseded by ipc-codegen output)
- barretenberg/cpp/src/barretenberg/wsdb/wsdb_ipc_client_generated.{cpp,hpp}
(legacy cbind-emitted, superseded by ipc-codegen output under generated/)
The WSDB IPC-backed LowLevelMerkleDBInterface lived at top-level
barretenberg/wsdb_client/ but it is purely an AVM-simulation primitive:
- Implements bb::avm2::simulation::LowLevelMerkleDBInterface.
- Its only sibling impls (HintingRawDB, PureRawMerkleDB,
MockLowLevelMerkleDB) all live under vm2/simulation/{lib,testing}/ in
the bb::avm2::simulation namespace.
- It already linked vm2_sim; the top-level placement was the outlier.
Now under barretenberg/cpp/src/barretenberg/vm2/simulation/dbs/, in
namespace bb::avm2::simulation. No consumers reference it yet (the IPC
stack cutover wires it in later), so this is a zero-blast-radius rename.
Kept as its own STATIC library (`wsdb_ipc_merkle_db`) rather than folded
into vm2_sim so nodejs_module / wasm builds don't pull in the IPC stack
they never use. The vm2/CMakeLists.txt globs that build vm2_sim and vm2
both explicitly exclude this source path; the library target is gated on
NOT(FUZZING) AND NOT(WASM) like the underlying wsdb_ipc_client.
charlielye
force-pushed
the
cl/ipc-codegen-migrate-bb-wsdb
branch
from
4e82660 to
783c0c3
Compare
… only
PipeBackend (stdin/stdout to a spawned bb) is superseded by the codegen's
two supported transports:
- FfiBackend — links libbb-external, in-process `bbapi` calls
(mobile / embedded consumers).
- ipc_runtime::IpcClient — UDS or MPSC-SHM to a separately-spawned bb
(dev/test / desktop hosts).
The Backend impl for ipc_runtime::IpcClient lives in the regenerated
generated/backend.rs, so the crate now depends on ipc-runtime directly.
Also clears out two orphan files (src/api.rs, src/generated_types.rs)
that claimed `AUTOGENERATED - DO NOT EDIT` but were never declared as
modules in lib.rs and weren't being regenerated.
Cargo.toml fallout:
- drop `native` and `async` features
- drop `tokio`, `nix`, `tracing` (all only used by pipe.rs)
- workspace `Cargo.toml`: drop the same trio from workspace.dependencies
Tests:
- delete tests/src/{blake2s,pedersen,poseidon,pipe_test,utils}.rs
(the parallel ffi/* tests cover the same surface)
- keep debug_msgpack.rs (backend-agnostic wire-format dump)
- regenerated FFI imports: `backends::FfiBackend` -> `FfiBackend`
Verified: cargo test -p barretenberg-tests --features ffi --release
passes 60 tests; --no-default-features build clean.
…d bb.js server emit Gap-closure from the post-migration audit: wsdb_ipc_server.cpp: - Replace the local `.shm`/`.sock` if/else + `IpcServer::create_*` calls with `ipc::make_server(input_path, opts)`. - Replace `setup_parent_death_monitoring()` + the SIGTERM/SIGINT/SIGBUS/ SIGSEGV `signal()` block with `ipc::install_default_signal_handlers(*server)`. Both already live in ipc-runtime; the local copies duplicated them. - ~85 LOC removed. bb.js TS generation: - bb.js is a pure client (the bb binary is the server), so the `--server` emit was producing two unreferenced files (`cbind/generated/server.ts`, `cbind/generated/ipc_server.ts`). Drop the flag. The wsdb dispatch lambda is unchanged: switching it to the codegen's per-method `serve()` entrypoint requires aligning wire (codegen) and internal (wsdb_commands.hpp) type hierarchies first — separate refactor.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stack A PR2 of the two-stack codegen migration (see `/mnt/user-data/charlie/.claude/plans/glittery-snuggling-horizon.md`).
Swaps the bb.js core BB API binding producer from `barretenberg/ts/src/cbind/generate.ts` (legacy schema-visitor) to the new `ipc-codegen/` package introduced in #23314. The output landing place (`barretenberg/ts/src/cbind/generated/`) and the exported names (`AsyncApi`, `SyncApi`, `ChonkProof`, etc.) are unchanged, so consumers in `barretenberg/ts/src/barretenberg/` and `bbapi/` compile without modification.
What this lands
Out of scope (deferred)
Local verification
Test plan