Repository navigation
feat(evaluations)!: promote Evaluations from Code to core - #235
Merged
Merged
Conversation
Export the Evaluations from Code names from launchdarkly_ai_server. Remove launchdarkly_ai_server.experimental.evaluations. Report the usage helper as client.initEvaluations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
donei003
force-pushed
the
feat/promote-evaluations-to-core
branch
from
October 28, 2026 17:02
f3586b7 to
43ecea8
Compare
5 of 6 tasks
…ons-to-core # Conflicts: # packages/client/tests/test_evaluations_run.py
…ons-to-core # Conflicts: # packages/client/api-report/launchdarkly_ai_server.experimental.evaluations.txt
…ons-to-core # Conflicts: # packages/client/README.md
aknight-ld
approved these changes
Oct 28, 2026
…ons-to-core # Conflicts: # packages/client/api-report/launchdarkly_ai_server.experimental.evaluations.txt
9 tasks done
donei003
added a commit
to launchdarkly/js-ai-sdk
that referenced
this pull request
Oct 29, 2026
Promotes Evaluations from Code from `@launchdarkly/ai-server/experimental` to the package root. This follows the Agent Skills promotion in #159. > **Draft.** This PR releases together with launchdarkly/python-ai-sdk#235. A Jira ticket is not linked yet. ## Migration Change the import path. The names, signatures and behavior do not change. ```ts // before import { EvalTool, initEvaluations } from '@launchdarkly/ai-server/experimental'; // after import { EvalTool, initEvaluations } from '@launchdarkly/ai-server'; ``` Use `@launchdarkly/ai-node` in place of `@launchdarkly/ai-node/experimental` in the same way. This is a breaking change (`!`). It bumps the minor version while the package is 0.x. ## What changed - **Root exports.** The 17 Evaluations from Code names (values and types) are exported from the `@launchdarkly/ai-server` root with the same spelling. No name collides with a current root export. `@launchdarkly/ai-node` gets them through its existing `export *`. - **Experimental entry point.** `src/experimental.ts` keeps its header and exports nothing. The `./experimental` subpath stays in both packages, so a future experimental feature has a place to go. An old import now gives a missing name, not a missing module. - **Usage ID.** `initEvaluations` now reports `client.initEvaluations`, not `experimental.initEvaluations`. Queries that count the old ID must also count the new one. - **API surface.** The names are added to `api-surface/ai-server.txt` as `+` and marked `-` in `api-surface/ai-server.experimental.txt`. Both API Extractor reports are regenerated. The experimental report is now empty. - **API report warnings.** The root report now has 12 `ae-forgotten-export` warnings, up from 7. The new ones are evaluations types that are not exported themselves, for example `EvalToolImplementation`, `ScorerFn` and `Transport`. The experimental report had the same warnings before, so nothing new is exposed. - **Changelog.** The three `evaluations` rows are removed from `changelog-sections`, so `feat(evaluations):` lands under Features. The `AGENTS.md` scope examples now use a placeholder scope. - **Docs.** Both READMEs and both `agents.md` files no longer call Evaluations from Code experimental. The generated export blocks in `packages/client/agents.md` are regenerated from `index.ts` and `experimental.ts`. The README anchor changed from `#evaluations-from-code-experimental` to `#evaluations-from-code`. ## Tests - The package export tests check that the experimental entry point exports no value, that the root snapshot includes the evaluations values, and that every evaluations name is in the root API report and none is in the experimental one. - The ai-node re-export test checks that the evaluations values are on the root as the same references and not on `/experimental`. - The evaluations tests import from the root. `sdk-usage.test.ts` expects `client.initEvaluations`. - The integration fixtures check that `initEvaluations` is on both roots and absent from both `/experimental` subpaths, at run time (CJS and ESM) and at type-check time. ## Checks - [x] `yarn code:check`, `yarn build`, `yarn typecheck`: pass - [x] `yarn test`: pass (client: 1929 passed, 11 skipped) - [x] `yarn test:integration`: 36 passed - [x] `yarn api:surface`, `yarn test:api-surface`, `yarn test:api-breaking`, `yarn lint:pkg`, `yarn api:check`: pass - [x] `node scripts/api-breaking.mjs pr --base origin/main`: no breaking changes to the public API - [ ] Run one evaluation end to end against staging with the root import ## Conflicts The open PR #175 edits evaluations code. Merge it first, or rebase it after this PR. ## Related PRs Releases together with: - launchdarkly/python-ai-sdk#235 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Evaluations from Code** is promoted from `@launchdarkly/ai-server/experimental` (and `@launchdarkly/ai-node/experimental`) to the **package root**, mirroring the earlier Agent Skills promotion. Consumers should import names like `initEvaluations`, `EvalTool`, `Judge`, and `Scorer` from `@launchdarkly/ai-server` or `@launchdarkly/ai-node` instead of the `/experimental` subpath; behavior and signatures are unchanged. > > The experimental barrel now exports nothing (`export {}`), while the `./experimental` subpath remains for future features. Public API surface lists, API Extractor reports, and `release-please-config.json` are updated accordingly, and `initEvaluations` reports usage as **`client.initEvaluations`** rather than `experimental.initEvaluations`. Docs, `AGENTS.md`, and tests (including CommonJS/webpack integration fixtures) assert evaluations live on the root only. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 591d138. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/launchdarkly/js-ai-sdk/pull/181" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Merged
jeffdupont
added a commit
that referenced
this pull request
Oct 30, 2026
Steps 1 and 2 of [AIC-3672](https://launchdarkly.atlassian.net/browse/AIC-3672). The JS half is AIC-3673. ## What this does - **`release-as: 1.0.0`** on all 14 packages in `release-please-config.json`. Without it, release-please keeps bumping 0.x minors, so the next release PR would ship `launchdarkly-ai-server` 0.6.0 instead of 1.0.0. - **Internal pins.** Every package that depends on `launchdarkly-ai-server` now requires `>=1.0.0,<2`. release-please doesn't update these for Python. #218 (merged) set them to `>=0.6,<2` so either release order resolves; this tightens them to the 1.0 line. - **`uv.lock`** needs no change: uv records workspace dependencies without a version specifier. `uv lock --check` passes. `changelog-sections` is left alone. #235 removes the evaluations Experimental rows when it promotes Evaluations from Code to core. ## Merge this last Merge this after the PRs below. Anything merged before the release PR (#231) merges ships in 1.0.0, and a `!` change merged after 1.0 forces 2.0. - [x] #218 fix(AIC-3598)!: packaging and public types (merged 29 Oct) - [x] #235 feat(evaluations)!: promote Evaluations from Code to core (merged 29 Oct). After it merges, nothing in `launchdarkly_ai_server.experimental` is public. - [x] #237 fix(claude-agents)!: honour max_tokens and disable thinking by default (merged 29 Oct) - [ ] #136 feat(AIC-3439): wildcard handlers accept only named providers (approved; conflicts with main, needs a merge of main) - [x] #234 fix(AIC-3568)!: give the barrel core's `__all__` (merged 29 Oct) - [ ] #246 fix(AIC-3592): make native LangGraph hand-offs complete (spec launchdarkly/ai-sdks-monorepo#118) - [ ] #248 fix(AIC-3573): never send a NativeTool to the model as a function tool (spec launchdarkly/ai-sdks-monorepo#116) - [ ] #240 feat(client): add result variant and value to the feature_flag span event (approved, mergeable) - [ ] #251 fix(AIC-3716): build OpenAI native graph node tools with FunctionTool (review required; touches the same loop as #248) #241 (Shadow Requests) is closed; 1.0 ships with nothing experimental. ## After merging 1. Confirm the regenerated #231 lists 1.0.0 for all 14 packages, then merge it. 2. Install each package from PyPI into an empty venv and import its public entry points (AIC-3672 step 5). 3. Open a PR removing `release-as` straight away, or every later release PR proposes 1.0.0 again. ## Checks - `release-please-config.json` parses as JSON, with `release-as: 1.0.0` on all 14 packages. The first push wrote the lines with literal backslashes; fixed in d720896. - `make test`: 3,711 passed, 18 skipped (before #175 and #236 merged; CI is the current check) (with the parent checkout's `.env` LD URIs blanked; see AIC-3593) - `uv lock --check` passes 🤖 Generated with [Claude Code](https://claude.com/claude-code) [AIC-3672]: https://launchdarkly.atlassian.net/browse/AIC-3672?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Prepares the **1.0.0 GA** cut for all 14 Python packages by teaching release-please to release **1.0.0** instead of another **0.x** bump. > > Adds **`release-as: "1.0.0"`** on every entry in `release-please-config.json` (core `launchdarkly-ai-server`, barrel, and all handler packages). > > Tightens **internal PyPI pins** from `launchdarkly-ai-server>=0.6,<2` to **`>=1.0.0,<2`** in each handler `pyproject.toml` and in `packages/ai` (including **`otel`** / **`ld`** extras). Published wheels will require the 1.0 server line once those versions ship. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ae9f2cc. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: andrewklatzke <aklatzke@launchdarkly.com>
jeffdupont
pushed a commit
that referenced
this pull request
Oct 30, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>launchdarkly-ai-server: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-server-0.5.0...launchdarkly-ai-server-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **client:** init_client raises when the LaunchDarkly client did not initialize ([#254](#254)) * **evaluations:** route generation and judges to a scoped wildcard handler only for its named providers, and accept two scoped wildcards of one mode when their lists differ * **AIC-3439:** let wildcard handlers accept only named providers * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **evaluations:** promote Evaluations from Code to core ([#235](#235)) * **client:** capture conversation content by default ([#213](#213)) * **evaluations:** remove default pass rate threshold ([#238](#238)) * **evaluations:** set default pass rate threshold ([#226](#226)) * **client:** spans carry launchdarkly.project_id instead of highlight.project_id * **AIC-3687:** rename create_openai_agent_handler to create_openai_agents_handler ([#225](#225)) * **skills:** rename FDv2SkillStore to LaunchDarklySkillStore ([#223](#223)) ### Features * **AIC-3439:** let wildcard handlers accept only named providers ([454a4de](454a4de)) * **AIC-3687:** rename create_openai_agent_handler to create_openai_agents_handler ([#225](#225)) ([a7cdbc0](a7cdbc0)) * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) * **client:** point judge spans at the run they graded ([#253](#253)) ([863d423](863d423)) * **evaluations:** promote Evaluations from Code to core ([#235](#235)) ([0bc43a5](0bc43a5)) * **evaluations:** remove default pass rate threshold ([#238](#238)) ([882d099](882d099)) * **evaluations:** route generation and judges to a scoped wildcard handler only for its named providers, and accept two scoped wildcards of one mode when their lists differ ([454a4de](454a4de)) * **evaluations:** set default pass rate threshold ([#226](#226)) ([7ee368c](7ee368c)) * **skills:** rename FDv2SkillStore to LaunchDarklySkillStore ([#223](#223)) ([11f2c0b](11f2c0b)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) * **AIC-3601:** stop graph routing text inviting a router to answer for its target ([#230](#230)) ([27b8d09](27b8d09)) * **client:** init_client raises when the LaunchDarkly client did not initialize ([#254](#254)) ([6e66c00](6e66c00)) * **client:** stamp launchdarkly.project_id on the telemetry resource ([#229](#229)) ([5347558](5347558)) * **evaluations:** copy inline tool schemas and name both sources on a key collision ([#227](#227)) ([bfbdd4c](bfbdd4c)) * **evaluations:** send criterion errors as errorMessage and skip criteria for rows that did not generate ([#228](#228)) ([8b05b56](8b05b56)) * **graph:** report a node's own judge scores on its node_done stream event ([#236](#236)) ([2f3f9ee](2f3f9ee)) ### Documentation * **AIC-3692:** say which project id attribute the telemetry resource carries ([#239](#239)) ([64eff43](64eff43)) * state that Agent Skills and Evaluations from Code need 1.0.0 ([#191](#191)) ([fe3ef08](fe3ef08)) </details> <details><summary>launchdarkly-ai-python: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-python-0.4.0...launchdarkly-ai-python-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **evaluations:** promote Evaluations from Code to core ([#235](#235)) ### Features * **evaluations:** promote Evaluations from Code to core ([#235](#235)) ([0bc43a5](0bc43a5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) ### Documentation * state that Agent Skills and Evaluations from Code need 1.0.0 ([#191](#191)) ([fe3ef08](fe3ef08)) </details> <details><summary>launchdarkly-ai-claude-agents: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-claude-agents-0.3.0...launchdarkly-ai-claude-agents-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **claude-agents:** honour the config's max_tokens and disable thinking unless the config asks ([#237](#237)) * **client:** capture conversation content by default ([#213](#213)) * **AIC-3685:** make the internal provider helpers private ([#224](#224)) * **skills:** rename FDv2SkillStore to LaunchDarklySkillStore ([#223](#223)) ### Features * **AIC-3685:** make the internal provider helpers private ([#224](#224)) ([9afdb94](9afdb94)) * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) * **skills:** rename FDv2SkillStore to LaunchDarklySkillStore ([#223](#223)) ([11f2c0b](11f2c0b)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) * **claude-agents:** honour the config's max_tokens and disable thinking unless the config asks ([#237](#237)) ([a7ded1b](a7ded1b)) </details> <details><summary>launchdarkly-ai-claude-messages: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-claude-messages-0.3.0...launchdarkly-ai-claude-messages-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) * **AIC-3735:** send Claude Messages sampling parameters through extra_body ([#256](#256)) ([c6d0376](c6d0376)) </details> <details><summary>launchdarkly-ai-openai-agents: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-openai-agents-0.3.0...launchdarkly-ai-openai-agents-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) * **AIC-3685:** make the internal provider helpers private ([#224](#224)) * **AIC-3687:** rename create_openai_agent_handler to create_openai_agents_handler ([#225](#225)) ### Features * **AIC-3685:** make the internal provider helpers private ([#224](#224)) ([9afdb94](9afdb94)) * **AIC-3687:** rename create_openai_agent_handler to create_openai_agents_handler ([#225](#225)) ([a7cdbc0](a7cdbc0)) * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) * **AIC-3716:** build OpenAI native graph node tools with FunctionTool ([#251](#251)) ([b859b67](b859b67)) * **AIC-3726:** realign three usage IDs with the 1.0 factory names ([#252](#252)) ([1c4d904](1c4d904)) </details> <details><summary>launchdarkly-ai-vercel-agents: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-vercel-agents-0.2.0...launchdarkly-ai-vercel-agents-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> <details><summary>launchdarkly-ai-litellm-agents: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-litellm-agents-0.2.0...launchdarkly-ai-litellm-agents-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> <details><summary>launchdarkly-ai-openai-messages: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-openai-messages-0.3.0...launchdarkly-ai-openai-messages-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) * **AIC-3726:** realign three usage IDs with the 1.0 factory names ([#252](#252)) ([1c4d904](1c4d904)) </details> <details><summary>launchdarkly-ai-vercel-messages: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-vercel-messages-0.2.0...launchdarkly-ai-vercel-messages-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> <details><summary>launchdarkly-ai-litellm-messages: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-litellm-messages-0.2.0...launchdarkly-ai-litellm-messages-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> <details><summary>launchdarkly-ai-langchain-agents: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-langchain-agents-0.3.0...launchdarkly-ai-langchain-agents-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **evaluations:** route generation and judges to a scoped wildcard handler only for its named providers, and accept two scoped wildcards of one mode when their lists differ * **AIC-3439:** let wildcard handlers accept only named providers * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **AIC-3439:** let wildcard handlers accept only named providers ([454a4de](454a4de)) * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) * **evaluations:** route generation and judges to a scoped wildcard handler only for its named providers, and accept two scoped wildcards of one mode when their lists differ ([454a4de](454a4de)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3590:** build Python langchain-agents with langchain.agents.create_agent ([#175](#175)) ([8218626](8218626)) * **AIC-3592:** make native LangGraph hand-offs complete ([#246](#246)) ([bfb738f](bfb738f)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> <details><summary>launchdarkly-ai-langchain-messages: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-langchain-messages-0.3.0...launchdarkly-ai-langchain-messages-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **evaluations:** route generation and judges to a scoped wildcard handler only for its named providers, and accept two scoped wildcards of one mode when their lists differ * **AIC-3439:** let wildcard handlers accept only named providers * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **AIC-3439:** let wildcard handlers accept only named providers ([454a4de](454a4de)) * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) * **evaluations:** route generation and judges to a scoped wildcard handler only for its named providers, and accept two scoped wildcards of one mode when their lists differ ([454a4de](454a4de)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3573:** never send a NativeTool to the model as a function tool ([#248](#248)) ([73caeae](73caeae)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) * **AIC-3726:** realign three usage IDs with the 1.0 factory names ([#252](#252)) ([1c4d904](1c4d904)) </details> <details><summary>launchdarkly-ai-opencode-agents: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-opencode-agents-0.3.0...launchdarkly-ai-opencode-agents-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> <details><summary>launchdarkly-ai-typesafe: 1.0.0</summary> ## [1.0.0](launchdarkly-ai-typesafe-0.2.0...launchdarkly-ai-typesafe-1.0.0) (2026-10-30) ### ⚠ BREAKING CHANGES * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) * **client:** capture conversation content by default ([#213](#213)) ### Features * **client:** add result variant and value to the feature_flag span event ([#240](#240)) ([040d487](040d487)) * **client:** capture conversation content by default ([#213](#213)) ([77a7cf5](77a7cf5)) ### Bug Fixes * **AIC-3568:** give the barrel core's __all__, and drop private-spec references ([#234](#234)) ([88e57e6](88e57e6)) * **AIC-3598:** make Python packaging and public types GA-ready ([#218](#218)) ([5860b40](5860b40)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > This is a **Release Please** cut that moves every monorepo package to **1.0.0** and records the GA release in changelogs. It updates `.release-please-manifest.json`, each package’s `pyproject.toml` / `__version__`, and prepends **1.0.0** notes to every `CHANGELOG.md`. > > Handler and barrel packages now require **`launchdarkly-ai-server>=1.0.0,<2`**, so installs must upgrade the core client in lockstep with adapters. > > There is **no new application logic** in this diff—only release metadata. The changelog text still matters for reviewers: it summarizes breaking work already on `main`, including **Evaluations from Code promoted to core**, **conversation content captured by default**, **`init_client` failing when LD init fails**, telemetry **`launchdarkly.project_id`**, renames like **`LaunchDarklySkillStore`** and **`create_openai_agents_handler`**, tightened **wildcard handler routing**, and **GA packaging / public `__all__`** cleanup. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 2e904f7. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes Evaluations from Code from
launchdarkly_ai_server.experimental.evaluationsto the package root. This follows the Agent Skills promotion in #210.Migration
Import Evaluations from Code names from
launchdarkly_ai_serverinstead oflaunchdarkly_ai_server.experimental.evaluations. The names, signatures and behavior do not change.This is a breaking change (
!). It bumps the minor version while the package is 0.x.What changed
experimental/evaluations.pyexported are now inlaunchdarkly_ai_server.__all__, with the same spelling. No name collides with a current root export.launchdarkly-ai-pythongets the names through its existingimport *.experimental/evaluations.pyis deleted.experimental/__init__.pystays as a docstring only, so a future experimental feature has a place to go.init_evaluationsnow reportsclient.initEvaluations, notexperimental.initEvaluations. Queries that count the old ID must also count the new one.make api-report. The names are inlaunchdarkly_ai_server.txt, andlaunchdarkly_ai_server.experimental.evaluations.txtis deleted.evaluationsrows are removed fromchangelog-sections, sofeat(evaluations):lands under Features. TheAGENTS.mdscope examples now use a placeholder scope.launchdarkly-ai-pythonREADME,packages/client/agents.mdandAGENTS.mdno longer call Evaluations from Code experimental. The README heading anchor changed from#run-an-evaluation-from-code-experimentalto#run-an-evaluation-from-code.Tests
TestPackageExportschecks that every evaluations name is on the root and thatexperimental.evaluationsno longer imports.test_public_surface.pyadds the 14 names to the root snapshot.test_reexports.pychecks that the barrel re-exports the names.test_core_does_not_import_experimental_codenow checks onlylaunchdarkly_ai_server.experimental, because the root loadsevaluations. A new test checks thatlifecycleimports no evaluations module.test_sdk_usage.pyexpectsclient.initEvaluations.Import time
import launchdarkly_ai_servernow loads theevaluationspackage. Measured withpython -X importtime, it adds about 33 ms to a root import of about 130 ms. The largest part isevaluations.api.Checks
make lint,make format-check,make typecheck,make api-report-check: passpython -m scripts.api_breaking pr --base origin/main: no breaking changes to the public APImake test: 3640 passed, 18 skippedConflicts
The open evaluations PRs #228, #227, #143, #135 and #92 edit the same test files. Merge them first, or rebase them after this PR.
Related PRs
Releases together with:
🤖 Generated with Claude Code
Note
Overview
Evaluations from Code moves from
launchdarkly_ai_server.experimental.evaluationsto thelaunchdarkly_ai_serverpackage root (and throughlaunchdarkly-ai-pythonre-exports). The experimental shim is removed; callers should importinit_evaluations,EvalTool,Judge,Scorer, and the rest from the root instead.The public API contract shifts accordingly: generated
launchdarkly_ai_server.txtnow lists the evaluations surface, andlaunchdarkly_ai_server.experimental.evaluations.txtis deleted.init_evaluationsusage telemetry changes fromexperimental.initEvaluationstoclient.initEvaluations.Release/changelog policy is updated:
evaluationsis no longer an Experimental scope inrelease-please-config.json, andAGENTS.md/ READMEs describe Evaluations (and Skills) as core, with docs and examples using root imports and a non-experimental README anchor.Tests and snapshots (
test_public_surface,TestPackageExports,test_reexports, lifecycle import guards) enforce the new export layout and thatexperimental.evaluationscannot be imported.Reviewed by Cursor Bugbot for commit 3ee3612. Bugbot is set up for automated code reviews on this repo. Configure here.