Repository navigation
Preserve native DeepSeek snapshots when admission is rejected - #54
beastllama wants to merge 1 commit into
Conversation
|
Sorry for the slow reply, and thank you for your patience. Thank you @beastllama for a precise root cause and an honest limitations list. In one line: prompt-cache lookup reserved room for future saves by evicting snapshots even when the backend then refused the placement, so a refused request still cost a reusable warm snapshot on native DeepSeek-V4.1; this PR defers that reservation to the existing The failing-then-passing regression on the exact scenario, the success control proving admitted work still evicts exactly one snapshot at the same budget, and the pre-existing test panic reproduced on the unmodified base are all the right evidence. We are reviewing it, including a full native build on our side since your WSL environment could not run one, and will follow up in this thread. |
|
CI never started on this head: the runs we approved on GitHub stayed queued with no jobs, and GitHub refuses to cancel or re-run them. We are closing and reopening this PR once to start fresh runs. Nothing about the PR changes. Sorry for the noise. |
|
Thank you again @beastllama. GitHub's CI never started on this head, so we ran the same checks ourselves on |
Root cause: prompt-cache lookup evicts snapshots to reserve future saves before the backend accepts placement, even for native DeepSeek-V4.1 whose drops reach followers immediately. Evidence: the CPU regression on
e100ee8consumes aContextFullrefusal without another prefill, leaves the request queued beside the existing stream, and nevertheless evicts one reusable snapshot.This change lets native DSV41 defer reservation to the existing budget-enforcing
keepBornpath.Rules.reserve_before_passdefaults totrue; native DSV41 alone opts out throughLoaded.snaps. Families whose peers receive drops with the next request retain their existing reservation behavior. No cache budget, admission capacity, arithmetic or wire protocol changes.The deterministic LaneHost regression checks that refused placement preserves both warm snapshots. Its success control checks that admitted work still evicts exactly one snapshot, saves its mark, and stays at the same 128-byte fixture budget.
Validation with the repository-pinned Zig 0.17.0, CPU only:
expected 0, found 1evictions).zig fmt --check, andgit diff --checkpass.Reproduction commands from the repository root (
zigmust be 0.17.0):Local limitations: the broader engine API suite reached 42/179 before an integer-overflow panic in the existing
prompt_imprint_fault_testallocation-failure test; that same panic was independently reproduced on an unmodifiede100ee8archive. The native build runner failed before source compilation withchecking cache failed: manifest_stat Unexpectedunder the available WSL environment, including with isolated Linux source/cache directories. A complete native build and full host suite are therefore not claimed passed. GPU/model qualification was not run for this public-branch port; no serving process was changed and no performance gain is claimed by these CPU fixtures.