jit: daslang -lib emits a C-ABI library - #3927
Open
aleksisch wants to merge 5 commits into
Open
Conversation
aleksisch
force-pushed
the
aleksisch/standalone-jit
branch
12 times, most recently
from
September 9, 2026 16:38
c2233ed to
82adf41
Compare
aleksisch
marked this pull request as ready for review
September 10, 2026 22:40
aleksisch
force-pushed
the
aleksisch/standalone-jit
branch
9 times, most recently
from
September 11, 2026 12:26
064c115 to
21620d3
Compare
aleksisch
force-pushed
the
aleksisch/standalone-jit
branch
2 times, most recently
from
September 11, 2026 12:34
642f7ef to
42685b7
Compare
src/builtin/jit_runtime.cpp is not a module - it is the set of extern "C" symbols jitted and LLVM-AOT-emitted code links against, plus the address takers the jit module binds. module_jit.cpp keeps the das module: bindings, linker invocation and emit orchestration.
aleksisch
force-pushed
the
aleksisch/standalone-jit
branch
from
September 11, 2026 12:42
42685b7 to
11de83c
Compare
… field's module A member-pointer expression spelled its structure with the module's own name rather than aotModuleName, so a namespaced module - every standalone context - emitted `&Node::next` against an undeclared Node. A raw `?` in an emitted literal read as a trigraph. dumpAnnotationInfo clears its name table before writing rather than after. A used structure keeps its field types' modules alive: a handled field is the only thing reaching the module that declares it, and dropping that module dropped its aot_require include. A standalone context emits its own module's structures whether or not code reaches them, so an unreferenced one is walked too.
`daslang -aot script.das out.cpp -aot-macros` core dumped whenever the module graph
carried a dasbind [extern], which covers every dasLLVM binding:
Should be unreachable. We handled it in transformCall. Failed on: c_strlen.
-aot-macros sets export_all, which force-marks every function of every module and skips
removeUnusedSymbols, so an [extern] declaration - which has no body - reached simulate.
transformCall rewrites every CALL to a bound extern, which is what matters; simulating
the function itself is harmless because nothing calls it. The base annotation already
returns nullptr, so the override goes.
The generated .cpp held a C++ class whose methods call the AOT functions directly. It now also emits one `extern "C"` entry point per export, wrapping the method beside it, plus an opaque handle and <prefix>_create / _destroy / _last_error - so one pair serves a C++ host and a C host at once. daslib/c_api_header owns which signatures can cross, how each type is spelled and the layout asserts; aot_standalone owns the C++ spellings. Function.flags.exports is the only selection truth, which is what licenses skipping an unspellable signature with a warning while a function that ASKED for C and cannot cross is an error. The .cpp carries the header's text rather than including it: the header is the host's copy to move or edit, and the implementation must not break with it. The context namespace takes a ctx_ prefix - a name from the file stem lands inside namespace das, where cast.das redeclared das::cast - while the C prefix drops keyword escaping, since every emitted name suffixes it. Generated files are named for the input file, not the module it declares. The fixed API is defined even when nothing is exported, and what das owns on the heap crosses as an opaque handle a host hands back.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
It is large ABI-level compiler/codegen work (thunk emission, C-header layout asserts, refactored exe/lib startup) whose correctness hinges on subtle alignment and error-boundary behavior that warrants human review despite its extensive tests.
Pull request overview
This PR adds a new daslang -lib emission mode that compiles a daslang script into a native, C-ABI library plus a generated C header, so a host that only needs to call one script can do so across a plain ABI boundary (including from C) without embedding the compiler or hand-compiling standalone-AOT C++. Function selection flows through a single bit, Function.flags.exports, set either per-function by the new [export_c] macro or wholesale by -lib-export-all (policies.export_public_functions), so the interpreter, AOT, and header emitter cannot drift. Each thunk runs under the context's catch boundary (panics return zero and surface via <P>_last_error), and <P>_destroy runs [finalize] itself. The change also refactors the shared exe/lib startup helpers and adds two opt-in standalone "sweeps" (AOT -ctx and JIT -lib) over the test_aot corpus.
Changes:
- New
-lib/-lib-export-allCLI mode inutils/daslang/main.cpp(with a-outputrequirement guard), plus the LLVM thunk/entry-point + C-header emitters and an[export_c]selection macro. - Refactor of the JIT tool's CoP setup into
jit_setup_cop, batch--lib-output-dir/--lib-export-allclargs, and unifiedwrite_artifact/artifact_pathreplacing the per-kind exe/dll paths. - New opt-in
tests/standalone-sweepharness (C and JIT drivers, CMake generators) andjit_libtest fixtures.
File summaries
| File | Description |
|---|---|
| utils/daslang/main.cpp | Adds JitMode::Library, -lib/-lib-export-all parsing, jit_lib option injection, help text, and the -output requirement guard. |
| utils/internal/jit/main.das | Extracts jit_setup_cop; adds batch --lib-output-dir/--lib-export-all clargs and platform-aware shared-lib suffix for the log line. |
| utils/aot/main.das | Threads -bindings/-bindings_library into standalone_aot; enforces one -ctx input when -bindings is set. |
| utils/CMakeLists.txt | Adds daslib/c_api_header.das to the watchdog standalone-AOT source list. |
| utils/watchdog/main.cpp | Updates to the renamed ctx_main::Standalone namespace. |
| tutorials/integration/cpp/20_standalone_context.cpp | Updates to the renamed ctx_standalone_context::Standalone namespace. |
| tests/standalone-sweep/CMakeLists.txt | New opt-in sweep targets over the test_aot corpus with a filter knob. |
| tests/standalone-sweep/sweep_jit.cmake | JIT-tier sweep: batch-emit libraries, then load/drive each via a generated host. |
| tests/standalone-sweep/sweep_contexts.cmake | Generates the X-macro context list/header for the C++ driver. |
| tests/standalone-sweep/sweep_driver.c | C driver exercising every emitted context through its C entry points. |
| src/ast/ast_module.cpp, src/ast/ast_parse.cpp | Remove the describe_pending_dynamic_modules() forward decl (now in dyn_modules.h); leaves a stale comment. |
Review details
- Files reviewed: 97/98 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The JIT could produce an exe and an AOT object; a host that only calls one script had neither. -lib writes a shared library (or an archive under --jit-lib-static) plus a C header, and links no daslang API into the host. The C surface is daslib/c_api_header's, so a JIT library and a standalone context declare the same API and skip the same signatures. Which functions cross is Function.flags.exports, read three ways: [export_c] alone by default, the bit however it was set under --jit-lib-export-marked, and every public function under -lib-export-all, which is a policy because MarkSymbolUse consumes it before infer. Codegen stays in LlvmJitMode.EXE, so inject_main's startup is shared. Each export gets an extern "C" thunk that packs its arguments into a frame and hands it to a trampoline through jit_lib_invoke_guarded, so a das panic reaches the caller as a zero return with its text in <P>_last_error, never as an unwind through C. Nothing in the emit path panics: the artifact writers return whether they wrote and run_jit collects it. A library initializes the modules it registers - the host's own runtime is already initialized, but an uninitialized module carries no functions, which an extern lookup reported as a missing function in a module that plainly existed. Two opt-in sweeps drive the corpus test_aot covers, one per backend: the C++ tier compiles and launches every context in one process, the JIT tier emits each library and loads it back. ARCHITECTURE_LIB.md#lib-runtime-scope records what a library's runtime guarantees.
aleksisch
force-pushed
the
aleksisch/standalone-jit
branch
from
September 11, 2026 13:39
11de83c to
71df22a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A host that only needs to call one daslang script had two options, and both asked
too much. The C++ API embeds the compiler. Standalone AOT emits C++ source the host
has to compile itself, and the result is a C++ class. Neither works when the host
is C, or when it wants a plain ABI boundary.
daslang -lib script.das -output build/scriptnow writes a native library and theC header a host calls it through.
--jit-lib-staticgives an archive instead. Whichfunctions cross is your choice:
[export_c]marks them one at a time, and-lib-export-alloffers every public function of the entry module whose signature Ccan spell, naming the ones it skips. Selection reads one bit,
Function.flags.exports,so the interpreter, AOT and the header emitter cannot drift apart. Scalars,
string,pointers and enums cross by value; structures and the
float2..uint4/rangefamilies cross as
const T *and return through a trailing out pointer. Everydeclared structure carries a size assert and one offset assert per field, so a host
built for another target fails to compile rather than misreading memory.
A library is called by code that cannot catch anything, which shapes the entry
points. Every thunk runs its body under the context's catch boundary: a das panic
returns zero, leaves the out param untouched, and reports through
<P>_last_error(ctx).<P>_createworks on any thread, because daslang'senvironment is thread-local while the registration behind it happens once. A second
daslang runtime in one process declines with a null instance instead of aborting.
And
<P>_destroyemits the program's[finalize]calls itself, because a jittedSimFunction carries no shutdown flag for the runtime's own drain to find.
Where to look:
modules/dasLLVM/daslib/llvm_lib.dasis the entry and thunk emitter,daslib/c_api_header.dasdecides what can cross and writes the header, and the fouremit_standalone_*helpers inllvm_exe.dasare the exe startup both artifacts nowshare. The riskiest spot is the thunk ABI: a structure result is built in an alloca
of its own alignment, never in a frame field, because an in-struct
[N x i8]slotaligns to 1 while the body stores through it at the structure's real alignment.