evals: wire @corbits/evals + first dispatch-routing eval

Add @corbits/evals@0.1.1 as a devDependency (LGPL-2.1-only; deps
@corbits/system-one ^0.3.0) and land the first eval built on it
(CL-10452): dispatch-routing exercises the real dispatch director
system prompt (agents/dispatch/src/prompt.ts) against 9 brand-new
routing requests via evaluate() with mentions/avoids/under checks and
per-check miss allowances. Run: bun run eval:prompts-dispatch.

Author: TheGreatAxios

.gitignore

@@ -25,6 +25,7 @@ worktrees
 evals/capability/results/*
 !evals/capability/results/baseline-0286.json
 evals/public/results/*
+evals/prompts/dispatch-routing/last.json
 local-harness-bench/
 subagents/
 

bun.lock

@@ -0,0 +1,75 @@
+# Dispatch routing eval (first `@corbits/evals` eval)
+
+First eval wired through the published [`@corbits/evals`](https://github.com/corbitsdev/corbits-evals#readme)
+package (CL-10452). It exercises the **real dispatch director system prompt**
+(`agents/dispatch/src/prompt.ts`, exported by `@corbits/code-agent-dispatch`)
+against **brand-new routing requests**: each case is a user request that the
+dispatch director must route to one specialist.
+
+## Run
+
+```sh
+bun scripts/eval-prompts-dispatch.ts
+# or
+bun run eval:prompts-dispatch
+```
+
+The run calls a model through local Ollama by default (matching the package's
+own examples). Override with env:
+
+```sh
+EVAL_MODEL=tev1:0.8b bun scripts/eval-prompts-dispatch.ts
+EVAL_OLLAMA_URL=http://localhost:11434 EVAL_MODEL=qwen3:0.6b bun scripts/eval-prompts-dispatch.ts
+```
+
+## What it measures
+
+The run composes the dispatch card **verbatim** + the case request + a fixed
+reply-format stanza (harness glue so the routing decision is measurable), then
+asks the model which specialist it would spawn. Checks:
+
+| Check     | Helper      | Passes when                                             |
+| --------- | ----------- | ------------------------------------------------------- |
+| `routes`  | `mentions`  | the reply names the expected specialist                 |
+| `noDiy`   | `avoids`    | the reply does not volunteer to do the work itself      |
+| `fast`    | `under`     | the run finishes within 120s                            |
+
+## Cases
+
+`cases.jsonl` — 9 brand-new routing prompts. Each case:
+
+```json
+{
+  "name": "map-retry-backoff",
+  "input": "Where does the repo handle retry backoff? Find every call site and sketch the flow between them.",
+  "expect": { "route": "explorer", "decoys": ["coder", "planner"] }
+}
+```
+
+`expect.route` is the correct specialist; `expect.decoys` lists the likely
+wrong picks for humans scanning the report.
+
+## Gates
+
+Per check, `allow` in `scripts/eval-prompts-dispatch.ts`:
+
+- `routes: { miss: 6, drop: 0 }` — the gate trips only when fewer than 3 of 9
+  cases route correctly. Calibrated to the local smoke model (`qwen3:0.6b`,
+  observed 3/9); **tighten when running a capable model** (e.g. `miss: 1`).
+- `noDiy: { miss: 2 }`
+- `fast: 0`
+
+`compare` + `save` point at `last.json`: each run compares drops against the
+previous stored run, then replaces it. Until a capable-model baseline is
+stored, drops are not gated.
+
+## Integration notes
+
+- `@corbits/evals@0.1.1` is a devDependency (LGPL-2.1-only — see the PR body).
+- The package's `evaluate()` is the whole harness: run, cases, checks, and
+  per-check miss/drop allowances in one call. It rejects with `EvalFailure`
+  when a check breaks its allowance; `.report.markdown()` prints the summary.
+- Jev (LLM-as-judge string/`ask` checks) needs the `@corbits/system-one`
+  gateway; this eval uses code checks only so it runs offline.
+- Trajectory scoring (`ranSteps` / `@corbits/evals/workflow`) is the
+  CL-10452 follow-up: this first eval uses a plain function run.

evals/prompts/dispatch-routing/README.md

@@ -0,0 +1,9 @@
+{"name":"map-retry-backoff","input":"Where does the repo handle retry backoff? Find every call site and sketch the flow between them.","expect":{"route":"explorer","decoys":["coder","planner"]}}
+{"name":"plan-offline-mode","input":"We need offline mode. Write a build plan with phases, risks, and acceptance criteria.","expect":{"route":"planner","decoys":["coder","explorer"]}}
+{"name":"fix-token-refresh","input":"There's a bug in the login flow: the token refresh silently fails. Fix it and add a regression test.","expect":{"route":"coder","decoys":["explorer","reviewer"]}}
+{"name":"review-ship-diff","input":"Review the attached diff for correctness and edge cases before we ship it.","expect":{"route":"reviewer","decoys":["coder"]}}
+{"name":"design-settings-cramped","input":"The settings screen looks cramped. Update the design tokens and spacing rules.","expect":{"route":"designer","decoys":["coder","explorer"]}}
+{"name":"doc-storage-layer","input":"Write the architecture doc for the new storage layer, covering the write path and failure modes.","expect":{"route":"shakespeare","decoys":["explorer","planner"]}}
+{"name":"audit-permissions-boundary","input":"Audit the new permissions boundary for trust issues and secret exposure.","expect":{"route":"warden","decoys":["reviewer"]}}
+{"name":"probe-latency-percentiles","input":"Measure latency percentiles for the new model endpoint across 200 requests and report the distribution.","expect":{"route":"prober","decoys":["qa-lead","reviewer"]}}
+{"name":"qa-onboarding-run","input":"Run the CLI through the onboarding flow end to end and show it works with a pass/fail record.","expect":{"route":"qa-lead","decoys":["prober","coder"]}}

evals/prompts/dispatch-routing/cases.jsonl

@@ -41,6 +41,7 @@
     "check": "bun run lint && bun run typecheck && bun run check:dead-exports && bun run build && bun run check:projects-dir-guard",
     "start": "bun run build && bun ./dist/index.js",
     "eval:capability": "bun scripts/eval-capability.ts",
+    "eval:prompts-dispatch": "bun scripts/eval-prompts-dispatch.ts",
     "eval:public-swe-one": "bun scripts/eval-public-swe-one.ts",
     "tui:smoke": "bun ./src/tui/smoke.ts"
   },
@@ -115,6 +116,7 @@
     "@corbits/code-agent-explorer": "workspace:*"
   },
   "devDependencies": {
+    "@corbits/evals": "^0.1.1",
     "@intx/inference-testing": "0.3.0",
     "@types/bun": "1.3.9",
     "oxfmt": "^0.67.0",