Repository navigation
fix(claude-agents)!: switch the built-ins off by default and load no host settings - #201
Conversation
…host settings A run with no NativeTool attached now sends tools=[], so the Claude Code built-ins (Read, Bash, Skill and the rest) are off unless a config attaches them. 50e82fd omitted the key instead, which gave every run the full built-in set. TypeScript has sent tools=[] since its first release. TESTING.md §2.x.2. create_claude_agents_handler takes cwd and setting_sources, forwarded to every query() on both paths. setting_sources not given sends [], because the Agent SDK reads None as "load every source": the host's plugins, skills, hooks, permission rules and CLAUDE.md reached every run (AIC-3584). cwd is sent only when given. to_claude_agents also sends setting_sources=[]. §2.x.1. BREAKING CHANGE: runs no longer get Claude Code built-ins unless the config attaches them as NativeTool sentinels, and no longer load host Claude Code settings unless setting_sources names them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Two things so this matches js #135 (launchdarkly/js-ai-sdk#135), which is now approved. I'm asking spec #59 to make the same change. 1. The wrappers and the native adapter should take
2. Add a migration note to the README. |
…t-options # Conflicts: # packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py
…native adapter Matches launchdarkly/js-ai-sdk#135. claude_agents(), claude_graph() and to_claude_agents() hard-coded setting_sources=[], so a wrapper user could not opt back in to host settings, and these signatures freeze at 1.0. - claude_agents and claude_graph take cwd / setting_sources keyword-only, build the handler with them, and pass neither on to config() / graph(). - to_claude_agents reads both from opts and sends _host_query_options(...) to every node's query(), root and sub-agents, in place of the hard-coded []. - The README gains a migration note: [] also drops the settings-file env (ANTHROPIC_BASE_URL, CLAUDE_CODE_USE_BEDROCK / _VERTEX, proxies), apiKeyHelper and model, so a Bedrock, Vertex or gateway deployment configured through ~/.claude/settings.json stops authenticating. - agents.md §2a describes the model-parameter allowlist from main instead of "forwards none of model.parameters". - API report regenerated for the new signatures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
left a comment
There was a problem hiding this comment.
Approved after the GA 1.0 PR audit. Every earlier review point is addressed and the API report is regenerated. The CI failure is the flaky opencode boot-timeout test, so I'm rerunning it.
…riation (#208) **Stacked on #201 (host options and built-in tool set).** Review that one first. This PR's base is its branch. A skill attached to an agent-mode variation never reached a model run through `create_claude_agents_handler`. #201's `tools=[]` removes the `Skill` tool, and nothing hands the CLI the skill files. `create_claude_agents_skills_handler(*, capture_content=None, cwd=None, setting_sources=None, skills_temp_dir=None)`, from the new `launchdarkly_ai_claude_agents.experimental.skills` module, fixes that. On every invoke and stream it: 1. Writes `skill_refs(config)` with `write_skills`, through the store configured with `set_skill_store`, into `mkdtemp(<skills_temp_dir>/ld-skills-)/launchdarkly/skills`. Pinned versions are honored. 2. Passes `plugins=[{"type": "local", "path": <dir>/launchdarkly}]` and `skills=["launchdarkly:<key>", …]`, and appends `Skill` to `tools` (not to `allowed_tools`). 3. Removes the directory when the run ends, however it ends. - **One directory per run.** There's no shared root, no watcher and no prune. Detaching a skill from a variation takes effect on the next run, and concurrent runs never share files. - **Fail closed.** If the reconcile isn't `ok` (no store, store not yet initialized, a skill withheld by verification, or an absent pin), the run raises before `query()` and names each skill. - **No-skills path.** A variation with no skills sends exactly the core handler's options and needs no store. - **`setting_sources` is never `None`.** With `skills` set and `setting_sources=None`, `claude-agent-sdk` sends `--setting-sources=user,project`. The core handler already always sends it (`[]` unless given), and this handler inherits that. - **Telemetry.** The tool hooks are installed whenever `Skill` is in `tools`, so a `Skill` call gets an `execute_tool Skill` span, with no `$ld:ai:tool_call` event. - **Stage boundary.** Core reaches this code only through a private `_prepare` keyword argument on `_create_claude_agents_handler`. It is not an option of the public factory. The package root doesn't import the experimental module, and a test checks that in a fresh interpreter. - **Not covered:** `claude_agents()`, `claude_graph()` and `to_claude_agents()`. The README says so. ## Differences from TypeScript - **No `skipMcpDiscovery`.** `claude-agent-sdk`'s `SdkPluginConfig` has no such key, so the plugin entry carries only `type` and `path`. The directory holds no MCP config, so there is nothing to discover. - **Synchronous removal.** The directory is removed with a synchronous `shutil.rmtree`, so a cancellation during teardown can't interrupt it. A failed removal is logged and doesn't replace the run's outcome. ## Tests - A new `test_skills.py` (49 tests) uses a mocked `query()` with a real `InMemorySkillStore`, a real filesystem (`tmp_path` as `skills_temp_dir`) and a real OTel provider. Each spec bullet runs on both invoke and stream where it applies. - `test_sdk_usage.py` adds the `experimental.createClaudeAgentsSkillsHandler` row. - Mutation-checked: making the cleanup a no-op fails 9 tests. Dropping `Skill` from the hook-install condition fails the span tests. Checks: - `uv run pytest`: everything passes except `opencode-agents` `test_a_fake_opencode_is_started_spoken_to_and_stopped`, the macOS `/private/var` temp-path failure already on `main`. - `make lint`, `make format-check` and `make typecheck` are clean. ## Follow-ups in the monorepo - The A.18 Python rows and an `EXPERIMENTAL.md` row for Python. - `api-surface/python/launchdarkly_ai_claude_agents.experimental.skills.txt`. - `python` on `experimental.createClaudeAgentsSkillsHandler` in `usage-ids.txt`. **Not verified yet:** a live run where a model actually loads the skill. **Spec:** TESTING.md §2.x "Claude Agents Skills Integration" and A.18, in launchdarkly/ai-sdks-monorepo#59. The TypeScript side is launchdarkly/js-ai-sdk#136. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds **experimental** `create_claude_agents_skills_handler()` under `launchdarkly_ai_claude_agents.experimental.skills` so Claude Agent runs get the **skills attached to their AI Config variation**. The core `create_claude_agents_handler()` path is unchanged and still ignores variation skills. > > On each invoke/stream with skills, the handler writes skill files from `set_skill_store` into a **per-run temp directory**, passes it to `query()` as a local `launchdarkly` plugin with qualified `launchdarkly:<key>` skill names, enables the **`Skill`** built-in (via new internal `_prepare` / `QueryPreparation` on `_create_claude_agents_handler`), and **removes the directory** in a final cleanup on success, failure, cancel, or stream abandonment. Reconcile failures **fail closed** before `query()`; empty skills behave like the core handler. > > Docs, API report, usage tracking for `experimental.createClaudeAgentsSkillsHandler`, and a large `test_skills.py` suite cover materialization, concurrency, telemetry (`execute_tool Skill`), and that `claude_agents` / graph helpers are **not** wired yet. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ac743e0. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Jeff Dupont <jdupont@launchdarkly.com>
🤖 I have created a release *beep* *boop* --- <details><summary>launchdarkly-ai-server: 0.5.0</summary> ## [0.5.0](launchdarkly-ai-server-0.4.0...launchdarkly-ai-server-0.5.0) (2026-10-27) ### ⚠ BREAKING CHANGES * **skills:** promote Agent Skills to core ([#210](#210)) * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Experimental * **evaluations:** export each offline row as one trace ([#170](#170)) ([fc241df](fc241df)) * **evaluations:** normalize text behaviors ([7cee8c8](7cee8c8)) ### Features * **skills:** export the Claude Agents skills handler from the package root ([#217](#217)) ([5d46dbb](5d46dbb)) * **skills:** promote Agent Skills to core ([#210](#210)) ([715815f](715815f)) ### Bug Fixes * **client:** show a judge an output of 0 or false, and render non-string values as JSON ([7cee8c8](7cee8c8)) * **client:** strip whitespace from LD_BASE_URI, LD_STREAM_URI and LD_EVENTS_URI in init_client ([1629123](1629123)) * **judges:** a judge config's outputFormat must not reach the provider ([#86](#86)) ([d8b893d](d8b893d)) * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) * **skills:** coalesce a watcher's debounce window into one reconcile ([#215](#215)) ([b214bbd](b214bbd)) * **skills:** default FDv2SkillStore endpoints from LD_BASE_URI and LD_STREAM_URI ([1629123](1629123)) </details> <details><summary>launchdarkly-ai-python: 0.4.0</summary> ## [0.4.0](launchdarkly-ai-python-0.3.0...launchdarkly-ai-python-0.4.0) (2026-10-27) ### ⚠ BREAKING CHANGES * **skills:** promote Agent Skills to core ([#210](#210)) ### Features * **skills:** promote Agent Skills to core ([#210](#210)) ([715815f](715815f)) </details> <details><summary>launchdarkly-ai-claude-agents: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-claude-agents-0.2.6...launchdarkly-ai-claude-agents-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * **skills:** promote Agent Skills to core ([#210](#210)) * **claude-agents:** switch the built-ins off by default and load no host settings ([#201](#201)) * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Features * **skills:** export the Claude Agents skills handler from the package root ([#217](#217)) ([5d46dbb](5d46dbb)) * **skills:** promote Agent Skills to core ([#210](#210)) ([715815f](715815f)) ### Bug Fixes * **claude-agents:** switch the built-ins off by default and load no host settings ([#201](#201)) ([f99d538](f99d538)) * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-claude-messages: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-claude-messages-0.2.5...launchdarkly-ai-claude-messages-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-openai-agents: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-openai-agents-0.2.6...launchdarkly-ai-openai-agents-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-vercel-agents: 0.2.0</summary> ## [0.2.0](launchdarkly-ai-vercel-agents-0.1.1...launchdarkly-ai-vercel-agents-0.2.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-litellm-agents: 0.2.0</summary> ## [0.2.0](launchdarkly-ai-litellm-agents-0.1.1...launchdarkly-ai-litellm-agents-0.2.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-openai-messages: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-openai-messages-0.2.5...launchdarkly-ai-openai-messages-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-vercel-messages: 0.2.0</summary> ## [0.2.0](launchdarkly-ai-vercel-messages-0.1.1...launchdarkly-ai-vercel-messages-0.2.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-litellm-messages: 0.2.0</summary> ## [0.2.0](launchdarkly-ai-litellm-messages-0.1.1...launchdarkly-ai-litellm-messages-0.2.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-langchain-agents: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-langchain-agents-0.2.6...launchdarkly-ai-langchain-agents-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-langchain-messages: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-langchain-messages-0.2.5...launchdarkly-ai-langchain-messages-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-opencode-agents: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-opencode-agents-0.2.0...launchdarkly-ai-opencode-agents-0.3.0) (2026-10-27) ### ⚠ BREAKING CHANGES * **opencode-agents:** honor AI Config tools ([#216](#216)) * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Experimental * **evaluations:** normalize text behaviors ([7cee8c8](7cee8c8)) ### Features * **opencode-agents:** honor AI Config tools ([#216](#216)) ([b5a63ec](b5a63ec)) ### Bug Fixes * **client:** show a judge an output of 0 or false, and render non-string values as JSON ([7cee8c8](7cee8c8)) * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> <details><summary>launchdarkly-ai-typesafe: 0.2.0</summary> ## [0.2.0](launchdarkly-ai-typesafe-0.1.1...launchdarkly-ai-typesafe-0.2.0) (2026-10-27) ### ⚠ BREAKING CHANGES * handlers now forward the AI Config's model parameters to the provider, limited to each handler's list in TESTING.md 1.12. Configs that set parameters which used to be ignored now take effect. ### Bug Fixes * pass AI Config model parameters through to every provider handler ([#107](#107)) ([e3afb3f](e3afb3f)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > This is a **Release Please** cut that bumps versions across the monorepo (notably `launchdarkly-ai-server` **0.5.0**, `launchdarkly-ai-python` **0.4.0**, and matching minor bumps on provider adapters). The edits are manifest entries, `pyproject.toml` / `__version__` strings, and new **CHANGELOG** sections for 2026-10-27. > > The release notes capture several **breaking** runtime changes already on `main`: **Agent Skills** move from experimental to core (`launchdarkly_ai_server.skills`), **AI Config model parameters** are now forwarded to every provider handler (configs that previously ignored them will behave differently), **Claude Agents** defaults built-in tools off and stops loading host settings, and **OpenCode Agents** now honor AI Config tools. The core client also documents judge fixes (`outputFormat` not sent to providers), URI trimming in `init_client`, skills watcher/store fixes, and experimental evaluation trace export. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 4138892. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Summary
This is the Python side of the core half of launchdarkly/ai-sdks-monorepo#59 (TESTING.md §2.x "Claude Agents Handler — host options and built-in tool set"). The TypeScript side is launchdarkly/js-ai-sdk#135.
tools=[]when a config attaches noNativeTool(§2.x.2). This reverses 50e82fd. With the key omitted, every Python run got the full Claude Code built-in set (Read,Bash,Skilland the rest), and the read-only tools in it need no permission. A config now turns a built-in on by attaching its sentinel, which is what TypeScript has always done. User-defined tools still go through the in-process MCP server andallowed_tools, whatevertoolsholds.create_claude_agents_handler(*, capture_content=None, cwd=None, setting_sources=None)(§2.x.1). Both options are forwarded to everyquery()on the blocking and streaming paths.setting_sourcesnot given sends[]. The Agent SDK readsNoneas "load every source", which put the host's plugins, skills, hooks, permission rules andCLAUDE.mdinto every run (AIC-3584).cwdis sent only when given.capture_content=Nonestill followsconfigure().model.parameters, and both keys are on the excluded list. A test pins that.claude_agents()andclaude_graph()takecwd/setting_sourceskeyword-only and build the handler with them. Neither reachesconfig()/graph().to_claude_agents()reads them fromoptsand sends_host_query_options(...)to every node'squery(), root and sub-agents. It already senttools=[]with no native tools, and a test now pins that too.setting_sources=[]and nocwd.[]also drops the settings-fileenv(ANTHROPIC_BASE_URL,CLAUDE_CODE_USE_BEDROCK/_VERTEX, proxy variables),apiKeyHelperandmodel. A deployment that reaches Bedrock, Vertex or a gateway through~/.claude/settings.jsonneedssetting_sources=["user"]or those variables in the process environment.Breaking
Two changes in runtime behaviour:
setting_sources.This needs to land before the 1.0 freeze, because changing either default after 1.0 would be breaking.
Tests
TestBuiltinsSurviveAnEmptyToolListis replaced. Its "omits the key" test asserted the behaviour this PR reverses.TestHostOptionsAndBuiltinToolSetadds nine tests, each run on both the invoke and stream paths.TestToClaudeAgentsHostIsolationadds four, including root and leaf for the default and for given values.TestConvenienceExportHostOptionsandTestClaudeGraphHostOptionsadd two each: the default, and given values reachingquery()but notconfig()/graph().Checks:
uv run pytest: every claude-agents test passes. The one failure in the full suite isopencode-agentstest_a_fake_opencode_is_started_spoken_to_and_stopped, a macOS/private/varvs/vartemp-path mismatch already onmain.make lint,make format-checkandmake typecheckare clean.🤖 Generated with Claude Code
Note
Overview
Breaking: Python Claude agent runs no longer inherit the Agent SDK’s implicit defaults for built-in tools or host Claude Code settings.
Runs now pass
tools=[]when the flag config attaches noNativeToolsentinel, which turns off Claude Code built-ins (Read,Bash, etc.) unless you explicitly enable them viaClaudeRead,ClaudeBash, and similar intool_handlers. User-defined tools still use the in-process MCP server unchanged.Host isolation:
create_claude_agents_handler,claude_agents,claude_graph, andto_claude_agentsgain app-onlycwdandsetting_sources(via_host_query_options), forwarded on everyquery()(invoke, stream, and graph nodes). Defaultsetting_sources=[]stops loading~/.claude/settings.json, projectCLAUDE.md, plugins, hooks, and settings-fileenv(Bedrock/Vertex gateways may break until you opt in or set process env). Flagmodel.parameterscannot override these options.README/agents.md document migration; tests replace the old “omit
tools” expectation and add coverage for host options and built-in tool sets across invoke/stream/graph paths.Reviewed by Cursor Bugbot for commit bcbde98. Bugbot is set up for automated code reviews on this repo. Configure here.