forkrun v3.5.2 - #553
Merged
Merged
forkrun v3.5.2#553
Conversation
ENGINE_KNOWN_FLAGS is now FORKRUN_CTX_FLAG_RAW; ring_call dispatches raw-first (raw > stdin-feed arm (W-STDIN placeholder) > argv tokenize). Raw plugins get a borrowed zero-copy window in ctx->reserved[0] via a persistent per-worker MAP_SHARED mmap (lazy-map, mremap growth), with fixed-args-only argv. Raw requires v2 (v1 masked + dlopen warning). Docs: C_PLUGIN.md section 4 + twins; lock-in tests T-RAW-1..7.
The bash JIT exports FORKRUN_C_STDIN=1 for -C + (-s | -b) — the entire bash-side change, riding FORKRUN_EXTRA_VARS — and ring_call fills arm 2 of the dispatch W-RAW established: FLAG_RAW > stdin mode > argv. Stdin batches skip tokenization (argv = fixed args only) and arrive on fd 0 as an EOF-terminated stream. Tier split mirrors external -s: fitting batches feed synchronously; larger batches fork a SIGCHLD-shielded feeder (the ring_exec pattern verbatim) that splices concurrently. The child _exits, ignores SIGPIPE, and scrubs the mask-hazard fds via targeted close (death-pipe write end through the fd_worker_w array walk, FD_TRAP_ACK_W, fd_fallow_w) — no new protocol, no /proc opens. Failure semantics from process lifecycle (short-read retry, parent-side fail on dead feeder, orphan EPIPE-exit on worker death). Old -s/-b-ignored warning removed; help text updated. Docs: C_PLUGIN.md section 5 + twins; lock-in tests T-STDIN-1..9.
N1: drop the dead double-close of pfd[0] in the feeder child — the scrub's pipe_r close owns it. N2: stdin suite keeps plugin .c sources for post-failure debugging (binaries/outputs still cleaned), matching the raw suite. N3: both suites documented as standalone scripts (test_c_plugins_rigorous.sh precedent). N4: T-STDIN-1 comment made precise (ctx-less 2-arg form, distinct from use_ctx = 1). No behavioral changes; T-RAW and T-STDIN re-verified green.
Re-purposes the v1.3 Stage 0 gate as the project's evidence layer per docs_port/STAGE0_AMENDMENT.md (proceed decision unconditional; harness sharpens feedback). benchmarks/python/: seeded input generators (JSONL 1M, int32 tokens 100M, transform 10M, endless pipe stream), per-niche incumbents (mp-Pool, futures, torch-probe with UNMEASURED path, GNU parallel, split), forkrun-side C plugins compiled at harness time (argv/raw/tensor-burn/segv/drain/probes), whole-tree RSS sampler, engine-capability gating (RAW/stdin probes; UNMEASURED rows when the shipped blobs predate v3.5.2), fault-isolation legs, CSV+MD tables, crossover data, rental-day runner (documented, unexecuted). results/: 27 measured rows on i9-7940X single-socket (JSONL to 10.6x, crossover 10-100us, tensor substrate ceiling, flat streaming RSS, 4-config fault tables) + narrative report with NOT-prove section.
…15977932 🤖 Auto-Build: Update Ring Loadables
…17625705 🤖 Auto-Build: Update Ring Loadables
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Sorry @jkool702, your pull request is larger than the review limit of 150,000 diff characters
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
forkrun v3.5.2 — C-plugin delivery contract + port infrastructure
Features: raw window (FLAG_RAW) and stdin (-s/-b with -C) delivery for
C plugins — the delivery contract is complete (argv / stdin / raw window,
precedence: FLAG_RAW > stdin > argv). Substrate config boundary (Stage 1:
engine consumes fr_config_t, not env vars per call; WorkerBatchState↔fr_state_t
aligned). IDL scaffolding (Stage 3.0: 41-function schema, generator, ctypes
definitions, CI check — annotation-only, no runtime changes). FFI boundary
measurement (Stage 2: 1.7µs worst-case per claim call vs 10-100ms budget —
the boundary is free). Stage 0 evidence table (JSONL 9-10.6× over Pool, RSS
92× better, fault resilience vs all incumbents). P1 residual documented.
Verification: proportionate to delta. Comprehensive + benchmark suites on UMA
and NUMA. Both sanitizers on NUMA. QEMU aarch64 both topologies. All new
test files. Full matrix deferred to v3.5.3 (next engine-change release).
No bug fixes. No breaking changes.
Full notes + leg logs + gate record: docs_port/, benchmarks/python/results/