Skip to content

Measure trycua/cua oracle gateability on this tree (cua-s1-forms port) #3290

Description

@lu-zero

PR #3265 landed the cua-s1-forms port (TinyTransformerScorer, 706K params, f32) with .agents/oracles/cua.md pinned at trycua/cua 9bbfa7dd3e27ca7f1861ede70aaca390174493f9 and gateable = no, but the oracle has never been RUN against this tree — the pin was verified via the GitHub API only, so it carries no gate authority (AGENTS.md: an oracle is only gateable once it demonstrably builds and runs the model).

This issue owns that measurement and is the #N the oracle file's evidence field must cite:

  1. Build/run the Laya-Python-equivalent reference path: execute the pinned trycua/cua TinyTransformerScorer + ByteCollator inference on the published cua-s1-forms checkpoint (2.8 MB, f32) and record outputs on a fixed workload.
  2. Run our CuaS1ScoreInference on the same checkpoint and workload; compare probabilities to the oracle's recorded tolerance (spec: 1e-5).
  3. Record the result in .agents/oracles/cua.md (gateable yes/no + evidence path) and in the row's records.

Local record: ISSUE-LOCAL-01M32WZMKVRBP4PH21KPHEDS3V (row MODEL-CUA-S1-FORMS).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:regolo/glm5.2 [maki]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions