Kev-0.8B on Core ML: fused decisions + Guess Who demo - #19
Merged
Merged
Conversation
…n path QwenBPETokenizer: Qwen byte-level BPE in Swift (NFC, added-token split, Qwen pre-tokenizer regex, GPT-2 byte map, merge ranks); matches HF tokenizers on 3,694 texts / 522,084 tokens. KevManager: Kev's row form (state + one question branch per Core ML call) over the L<length>_K<options> buckets; questions formatted like kev.api.to_record, <|x|> escaped, state capped at Kev's MAX_STATE 384 for evaluation. Swift == Python Core ML on 191 questions, 0 top-answer flips. KevFastManager (macOS 15 / iOS 18): one fused_S*_P* function call per request from the weight-shared multifunction KevFused package. The state and every question run together; questions are packed end to end in 128-token lanes and each restarts from the state (segment mask). Questions over a lane or with more than 16 options fall back to rows. 191 questions, 0 flips, max |dp| 0.004. M5 Pro, warmed, Kev's serving_bench cases vs Kev's MLX (new state): 2 questions, short state 18.1 ms (MLX 26.6) 6 questions, short state 37.8 ms (MLX 51.5) 5 questions, 370-token state 64.1 ms (MLX 69.6) KevCheck: tokenizer, parity, serving, fast-parity, fast-serving modes.
KevGuessWhoDemo: a self-playing Guess Who over 80 real Wikipedia people (DBpedia-14 test split: artists, athletes, politicians). Kev reads each bio once and answers all 12 yes/no questions in one fused Core ML call per card, then plays turns (question closest to a 50/50 split of the cards still up) until the hidden person, chosen among cards with a unique answer signature, is left. Header metrics (last / median call, decisions per second, total), Pause (Space) / Reset (Cmd-R). Inference runs back to back off the main actor and streams into the UI; rendering the grid no longer delays calls (118 -> ~40 ms per card wall). demo.sh opens a Terminal with asitop above and the colored model log below. M5 Pro: ~38 ms median per card (12 decisions), 960 decisions in 3.0-3.5 s. KevFastManager: buckets are discovered from the package's function names (MLModelAsset.functionNames), so a package with any subset of fused functions works; warm() now runs each function once. A function left idle pays a 0.3-0.8 s re-setup on its next call (not input dependent), so the demo re-warms before each scan. DBpediaSample: optional class filter; pages requests at the dataset viewer's 100-row limit.
Reset crashed the app: cancelling a game ends its result stream without an
error, so the old game went on into its turns reading the new wall's
unscanned cards (nil unwrap in split). The game now stops on cancellation
after re-warming and after the scan, and turn logic no longer force-unwraps.
Buttons are not keyboard-focusable, so Space only toggles Pause.
Header: controls and fixed-width metrics on the first row, the question on
a full-width second row, so a changing status never moves anything. The app
opens idle ("Ready — press Play"); Play starts the first game.
The demo looped games forever. Play now deals one wall, plays it to the solve and stops on the result; Play or Reset deals the next. Reset also clears a pause. The game counter is gone from the sidebar.
Play runs up to 4 games (3 s apart) and stops on the last result; Reset starts a fresh set. The sidebar shows the game's place in the set.
KevModelStore fetches the pinned FluidInference/kev-0.8b-coreml snapshot (8fa70891: fused package, L512_K16 + L1024_K80 row buckets, embedding table, tokenizer), checksummed once per revision. The Guess Who demo uses it unless a local model directory is given. The embedding table ships once, so both managers look for the bucket folder that has it instead of the first one. KevPackingTests: lane placement, grouping by packed length and readouts, yes/no rendering, and the pinned asset list.
KevFastManager no longer loads the row fallback models up front; each row bucket loads on its first question (KevManager.load(eager:), default eager). A workload that never falls back holds only the fused package: peak memory footprint 0.67 GB on the 80-bio Guess Who scan. Packages are compiled once and the .mlmodelc kept beside them, so launches after the first skip the compile (HF snapshot: warm 24 s -> 11 s). KevCheck bios: timed yes/no workload through the fused path, for comparison with Kev's PyTorch model (mobius bench_original_bios.py).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Kev-0.8B (Qwen3.5 backbone) on the GPU through Core ML.
QwenBPETokenizer: Qwen byte-level BPE in Swift, identical to HF tokenizers on 3,694 texts / 522k tokens.KevManager: Kev's row form, one question per call (fallback path).KevFastManager: one fused call per request. The state and all questions run together, questions packed end to end under a segment mask so each restarts from the state. Buckets are read from the package's function names.KevModelStore: pinned, checksummed snapshot of FluidInference/kev-0.8b-coreml @8fa70891.KevGuessWhoDemo: self-playing Guess Who over 80 Wikipedia people (DBpedia-14), 12 yes/no answers per bio in one call;demo.shadds a terminal with asitop and the model log.KevCheck: tokenizer / parity / serving checks.DBpediaSamplegains a class filter and pages at the 100-row API limit.Fidelity: fused fp16 functions vs the fp16 row packages, 191 questions, 0 top-answer flips (max |Δp| 0.004). The row packages reproduce Kev's published transfer-v4 / decision-v7 numbers (3 flips in 2,232 questions vs Kev fp32).
Speed, M5 Pro, Kev's
serving_bench.pyshapes (new state), Core ML vs Kev's MLX server: 2 q short 18.1 vs 26.6 ms, 6 q short 37.8 vs 51.5 ms, 5 q / 370-token state 64.1 vs 69.6 ms. Conversion: FluidInference/mobiusmodels/computer-use/kev-0.8b/coreml.🤖 Generated with Claude Code