[andrewklatzke]

Summary

• Add @launchdarkly/ai-typesafe, a judge handler that sends a variation's classifiers to Jev in one call.
• Each label becomes its own judgeResults entry, tracked on that label's eventKey, with the selected label as the reason.
• A score is the expected rubric index divided by one less than the number of levels, then clamped to 0–1.

https://launchdarkly.atlassian.net/browse/AIC-3534

Test plan

• vitest for @launchdarkly/ai-typesafe and the client judge, schema, and conversation tests
• Run a TypeSafe judge variation and confirm one result per label, with the label value in response

Made with Cursor

[cursor]

Stale Bugbot comment from a previous run.

[jeffdupont]

Reviewed together with python-ai-sdk #149 and ai-sdks-monorepo #49. The handler logic matches the spec and the Python side: question extraction, the state, the score mapping (I checked the @typesafe-ai/sdk 0.6.0 types: legend keys start at 0, so dividing by levels - 1 is right), the reason selection, and per-label tracking. typesafe 12/12 and client 450/450 pass on this head. It conflicts with main (judges.ts, the release-please files, yarn.lock).

1. Must fix: after a rebase, client CI fails on the root-exports snapshot. I merged this into current main (resolving judges.ts by adding userInput to BuildJudgeTasksArgs). src/__tests__/skills.test.ts › "the package root exports exactly the snapshotted values" then fails, with + "FORMATTING_INSTRUCTIONS": 1 failed, 1405 passed. The new root export is permanent public API once 1.0 ships. Python keeps it internal (launchdarkly_ai_server.judge_scoring). Either make it public on purpose (snapshot, the approved API list in ai-sdks-monorepo api-surface/typescript, and a matching Python root export), or keep it off the root export in both languages.

2. Should fix: createTypesafeHandler reports no helper usage. TESTING.md §3.27 makes reportUsage(...) the first statement of every public factory, and every factory has an ID in api-surface/usage-ids.txt. Every handler on main does this (for example openai-messages/src/handler.ts:451). Without it, TypeSafe judges are missing from the helper-usage data. Needs a typesafe.createTypesafeHandler ID in monorepo #49 too.

The examples/judge-worker.ts change (track each metrics entry, otherwise evaluationMetricKey) is the right pattern for deferred judges. Python #149 needs the same guidance; I left that comment there.

[cursor]

Stale Bugbot comment from a previous run.

[jeffdupont]

Matches the §1.12 contract and ai-typesafe.txt exports, and parity with the Python PR looks good. Same note as there: judges.ts:212/421 add input to every online judge, and TESTING.md:1476 says they get only the two vars, so that line probably needs updating in #49. Also heads up that #49 still pins typescript at 7b48c71, one behind this head.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit a0fc87b. Configure here.}