Add Cua-S1-4B-0.2 Core ML runtime - #12
Open
Alex-Wengg wants to merge 2 commits into
Open
Alex-Wengg wants to merge 2 commits into
Alex-Wengg wants to merge 2 commits into
Conversation
Swift host for the mobius Cua-S1-4B conversion (Qwen3.5-4B + Cua LoRA, text and multimodal): Qwen3.5 byte-level BPE tokenizer, FourBModel prompt and chat template, host embedding gather, M-RoPE, four GPU decoder parts, and the vision host side (smart_resize, bicubic AA, merge-window patches, pos-embed, 2D rope). Pinned, SHA-256 checked download from FluidInference/cua-s1-4b-coreml; prewarm() for the ~100 s first-call GPU specialization; FluidUseCuaS1 parity CLI. A contiguous copy between decoder parts avoids an MPSGraph stride assert when a GPU output array is fed straight into the next model. Parity vs the fp32 reference (38 tasks): text fp16 38/38 (max dp 0.0053), gptq 38/38 (0.133), multimodal 38/38 (0.135); chat string and token ids match the Python reference 38/38. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
117 files, 26.3 GB. FluidUseCuaS1 parity --models hub downloads through the pinned store: gptq text 3.8 GB, prewarm 82 s, parity 38/38 (max dp 0.133), identical to the local build. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Alex-Wengg
marked this pull request as ready for review
September 24, 2026 06:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
CuaS1FourBManager, which runs Cua-S1-4B-0.2 (Qwen3.5-4B plus Cua's LoRA adapters) on the GPU. Each decision is one forward pass that scores a closed list of(element, action)options, from either an accessibility tree (.text) or a screenshot (.multimodal). Conversion: FluidInference/mobius#104. Models: FluidInference/cua-s1-4b-coreml.FourBModelprompt through the chat template. It builds the input embeddings from the table, computes the M-RoPE positions, and does the vision preprocessing: resize, bicubic AA, patching, position table and 2D rope. It then runs the four decoder parts.""(fp16),w8, andgptq(2.6 GB, text only).prewarm()pays the one-time GPU specialization cost (~100 s the first time a 4B graph runs).FluidUseCuaS1 parityreruns the fixture check against a local mobius build.Parity against the fp32 reference, 38 tasks: the chat string and token ids match 38/38. Argmax agrees 38/38 for text fp16 (max Δp 0.0053), gptq (0.133) and multimodal (0.135), the same as the Python Core ML path. On GUI-360 text (613 tasks), fp16 and gptq both score 85.5%.
Models pinned at
735de16e.FluidUseCuaS1 parity --models hub --variant gptqdownloads 3.8 GB through the pinned store and reproduces the 38/38 parity.Notes for review:
🤖 Generated with Claude Code