Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,10 +140,11 @@ The client manages a singleton connection to LaunchDarkly and the associated tel

| Export | Description |
|---|---|
| `init_client(options?)` | Auto-discovers and initializes `launchdarkly-server-sdk` (optional dep, loaded via `importlib`). Optional — the first AI API call triggers lazy init when `LD_SDK_KEY` is set. Accepts optional overrides for SDK key, base URIs, service name, environment, and OTLP endpoint. Returns `Awaitable[LDClientInterface]`. On every successful path, including the already-initialized path, flushes `$ld:ai:sdk:info` for any LaunchDarkly AI packages that have not yet reported. |
| `init_client(options?)` | Auto-discovers and initializes `launchdarkly-server-sdk` (optional dep, loaded via `importlib`). Optional — the first AI API call triggers lazy init when `LD_SDK_KEY` is set. Accepts optional overrides for SDK key, base URIs, service name, environment, and OTLP endpoint. Returns `Awaitable[LDClientInterface]`. On every successful path, including the already-initialized path, flushes `$ld:ai:sdk:info` for any LaunchDarkly AI packages that have not yet reported. Does not take behavior defaults. |
| `init_client(client=...)` | **BYOC overload** — accepts a pre-initialized `LDClientInterface`. Stores it directly without calling the SDK. Flushes pending `$ld:ai:sdk:info` events. |
| `configure(*, options)` | Sets process-wide behavior defaults on the lifecycle singleton. `options` is a frozen `SDKOptions` (`capture_content: bool = False`). Keyword-only. Handlers read it at invocation. An explicit factory or wrapper `capture_content` overrides it. Replaceable until the first AI invocation; an identical value is allowed after. Not read from request data. |
| `get_client()` | Returns the initialized `LDClientInterface`. Throws if initialization has not completed. |
| `shutdown()` | Flushes all pending events and telemetry, then closes the client. Must be awaited before the process exits. Clears sdk-info reporting so a later client reports again. |
| `shutdown()` | Flushes all pending events and telemetry, then closes the client. Must be awaited before the process exits. Clears sdk-info reporting so a later client reports again. Restores the built-in SDK defaults. |
| `register_ai_sdk_package(name, version)` | Records a LaunchDarkly AI package identity. Handler and convenience packages call this at import time. |

### Core Data Types
Expand Down Expand Up @@ -554,9 +555,12 @@ providers out of three. `SpanUsage` is the type that means the folding is alread

#### Content is off by default

Every factory takes `capture_content: bool = False`. Guard at the
call site as well as inside the helper: the helper's guard makes a forgotten call site harmless, and
the call site's guard avoids serialising JSON that would then be discarded, once per turn, in a loop.
Every factory takes `capture_content: bool | None = None`. `None` reads `configure()` at
invocation, which is off until a process sets `SDKOptions(capture_content=True)`. An explicit
`True` or `False` overrides that default. Guard at the call site as well as inside the helper: the
helper's guard makes a forgotten call site harmless, and the call site's guard avoids serialising
JSON that would then be discarded, once per turn, in a loop. Resolve the bool when the handler
runs, not when it is constructed.

#### Finish reasons have three mechanisms, not one

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -519,7 +519,7 @@ than inside it. A turn that reads 19,971 tokens from cache and writes 3,580 more
Prompts, model output, tool arguments and tool results are personal data, so a span carries only
metadata unless you ask for more: models, token counts, timings, tool names.

Pass `capture_content=True` to a handler factory to include the conversation:
Pass `capture_content=True` to a handler factory to include the conversation, or turn it on for every handler from process startup. See [Process-wide defaults](packages/client/README.md#process-wide-defaults).

```python
from launchdarkly_ai_claude_messages import create_claude_messages_handler
Expand Down
14 changes: 9 additions & 5 deletions TELEMETRY-CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -394,11 +394,15 @@ error for an abandoned stream, so two dashboards would describe the same run dif

## 7. Content capture

Every handler factory takes `capture_content: bool = False`.

The default is off. Conversation content is personal data. A run emits only metadata, meaning
models, token counts, timings and tool names, until a caller asks for more. Turning it on sends the
text of every request and response to whatever collector the SDK points at.
Every handler factory takes `capture_content: bool | None = None`.

`None` reads the process-wide default at invocation (`configure(options=SDKOptions(...))`), which
is off. `True` or `False` overrides that default for the handler, and an explicit `False` wins
over a configured `True`. A convenience wrapper's `capture_content` is the per-call override: when
the caller passes it, that value is what the handler is built with; when they omit it, the wrapper
passes `None` and the process default applies. Conversation content is personal data. A run emits
only metadata, meaning models, token counts, timings and tool names, until capture is turned on.
Turning it on sends the text of every request and response to whatever collector the SDK points at.

Guard twice, on purpose. The helper returns without writing when capture is off, which makes a
forgotten call site harmless. The call site also checks before building the argument, which avoids
Expand Down
2 changes: 1 addition & 1 deletion packages/claude-agents/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with
> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler.
> Conversation content is off unless the caller passes `capture_content=True`.
> Conversation content is off unless the caller passes `capture_content=True`, or the process sets it. See [Process-wide defaults](../client/README.md#process-wide-defaults).
> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code.


Expand Down
2 changes: 1 addition & 1 deletion packages/claude-agents/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "launchdarkly-ai-claude-agents"
version = "0.2.5"
requires-python = ">=3.12"
dependencies = [
"launchdarkly-ai-server",
"launchdarkly-ai-server>=0.3.1",
"opentelemetry-api>=1.25",
"anthropic>=0.40",
"claude-agent-sdk>=0.2",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
set_output_content_attributes,
set_tool_call_content_attributes,
)
from launchdarkly_ai_server.lifecycle import effective_capture_content

from ._version import PACKAGE_NAME, __version__
from .spans import (
Expand Down Expand Up @@ -504,18 +505,25 @@ def _build_query_options(
# ---------------------------------------------------------------------------


def create_claude_agents_handler(*, capture_content: bool = False) -> ProviderHandler:
def create_claude_agents_handler(
*, capture_content: bool | None = None
) -> ProviderHandler:
"""Creates a ``ProviderHandler`` for Anthropic's Claude via the claude-agent-sdk.

Set *capture_content* to put prompts, model output, tool arguments and tool results on the
emitted spans. It defaults to off. See TELEMETRY-CONTRACT.md section 7.
emitted spans. ``None`` (the default) reads the process-wide default at invocation,
which is off until ``configure`` sets it. ``True`` or ``False`` overrides that default
for this handler. See TELEMETRY-CONTRACT.md section 7.
"""
report_usage("claude-agents.createClaudeAgentsHandler", PACKAGE_NAME, __version__)
return _create_claude_agents_handler(capture_content=capture_content)


def _create_claude_agents_handler(*, capture_content: bool = False) -> ProviderHandler:
def _create_claude_agents_handler(
*, capture_content: bool | None = None
) -> ProviderHandler:
"""Non-reporting :func:`create_claude_agents_handler`, used by this package's wrappers."""
content_override = capture_content

async def _call_impl(
config: AiConfigRep,
Expand All @@ -524,6 +532,8 @@ async def _call_impl(
variables: dict[str, Any] | None = None,
history: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
# None follows configure(); an explicit bool overrides it. Read when the call runs.
capture_content = effective_capture_content(content_override)
th = tool_handlers or {}
vs = variables or {}

Expand Down Expand Up @@ -699,14 +709,14 @@ def _stream_impl(
tool_handlers or {},
variables or {},
history,
capture_content=capture_content,
capture_content=effective_capture_content(content_override),
)

return _create_handler(
("Anthropic", "agent"),
_call_impl, # type: ignore[arg-type]
_stream_impl, # type: ignore[arg-type]
capture_content=capture_content,
capture_content=content_override,
)


Expand Down Expand Up @@ -896,7 +906,7 @@ def claude_agents(
"""Convenience wrapper: creates a handler and calls config(...).invoke()."""
report_usage("claude-agents.claudeAgents", PACKAGE_NAME, __version__)
variables = kwargs.pop("variables", None)
capture_content = kwargs.pop("capture_content", False)
capture_content = kwargs.pop("capture_content", None)
return _config(
key=config_key,
handler=_create_claude_agents_handler(capture_content=capture_content),
Expand Down
25 changes: 25 additions & 0 deletions packages/claude-agents/tests/test_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -903,6 +903,31 @@ async def test_content_present_when_enabled(
assert "gen_ai.output.messages" in chat.attributes
assert "gen_ai.system_instructions" in root().attributes

async def test_process_default_is_read_on_the_streaming_path(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
from launchdarkly_ai_server import SDKOptions, configure
from launchdarkly_ai_server.lifecycle import _reset_for_testing

_reset_for_testing()
try:
configure(options=SDKOptions(capture_content=True))
monkeypatch.setattr(
handler_mod,
"query",
_fake_query(
[stream_event("Hi"), assistant_message(), result_message("Hi")]
),
)
await _collect(
await create_claude_agents_handler().stream(BASE_CONFIG, "q", {}, {})
)
assert any(
"gen_ai.input.messages" in (span.attributes or {}) for span in spans()
)
finally:
_reset_for_testing()


# ---------------------------------------------------------------------------
# Errors, and the failure path — section 6
Expand Down
2 changes: 1 addition & 1 deletion packages/claude-messages/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with
> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler.
> Conversation content is off unless the caller passes `capture_content=True`.
> Conversation content is off unless the caller passes `capture_content=True`, or the process sets it. See [Process-wide defaults](../client/README.md#process-wide-defaults).
> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code.


Expand Down
2 changes: 1 addition & 1 deletion packages/claude-messages/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "launchdarkly-ai-claude-messages"
version = "0.2.4"
requires-python = ">=3.12"
dependencies = [
"launchdarkly-ai-server",
"launchdarkly-ai-server>=0.3.1",
"opentelemetry-api>=1.25",
"anthropic>=0.40",
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
set_tool_call_content_attributes,
to_semconv_finish_reason,
)
from launchdarkly_ai_server.lifecycle import effective_capture_content

from ._version import PACKAGE_NAME, __version__
from .spans import (
Expand Down Expand Up @@ -351,14 +352,18 @@ def _is_coroutine(fn: Any) -> bool:
_MAX_STEPS = 10


def create_claude_messages_handler(*, capture_content: bool = False) -> ProviderHandler:
def create_claude_messages_handler(
*, capture_content: bool | None = None
) -> ProviderHandler:
"""Creates a ``ProviderHandler`` for Anthropic Claude (messages API).

Requires ``anthropic`` to be installed as a peer dependency.

Set *capture_content* to put prompts, model output, tool arguments and tool results on the
emitted spans. It defaults to off. Conversation content is PII, so a run emits only metadata,
meaning models, token counts, timings and tool names, until a caller asks for more. Turning this
emitted spans. ``None`` (the default) reads the process-wide default at invocation,
which is off until ``configure`` sets it. ``True`` or ``False`` overrides that default
for this handler. Conversation content is PII, so a run emits only metadata, meaning
models, token counts, timings and tool names, until capture is turned on. Turning this
on sends the text of every request and response to whatever collector the SDK is pointed at.
"""
report_usage(
Expand All @@ -368,9 +373,10 @@ def create_claude_messages_handler(*, capture_content: bool = False) -> Provider


def _create_claude_messages_handler(
*, capture_content: bool = False
*, capture_content: bool | None = None
) -> ProviderHandler:
"""Non-reporting :func:`create_claude_messages_handler`, used by this package's wrappers."""
content_override = capture_content
import importlib

anthropic_mod = importlib.import_module("anthropic")
Expand All @@ -383,6 +389,8 @@ async def _call_impl(
variables: dict[str, Any] | None = None,
history: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
# None follows configure(); an explicit bool overrides it. Read when the call runs.
capture_content = effective_capture_content(content_override)
th = tool_handlers or {}
vs = variables or {}

Expand Down Expand Up @@ -466,14 +474,14 @@ def _stream_impl(
tool_handlers or {},
variables or {},
history,
capture_content=capture_content,
capture_content=effective_capture_content(content_override),
)

return _create_handler(
("Anthropic", "messages"),
_call_impl, # type: ignore[arg-type]
_stream_impl, # type: ignore[arg-type]
capture_content=capture_content,
capture_content=content_override,
)


Expand Down Expand Up @@ -722,7 +730,7 @@ def claude_messages(
# asking for content on spans got a TypeError instead of content.
report_usage("claude-messages.claudeMessages", PACKAGE_NAME, __version__)
variables = kwargs.pop("variables", None)
capture_content = kwargs.pop("capture_content", False)
capture_content = kwargs.pop("capture_content", None)
return _config(
key=config_key,
handler=_create_claude_messages_handler(capture_content=capture_content),
Expand Down
29 changes: 27 additions & 2 deletions packages/claude-messages/tests/test_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1653,6 +1653,28 @@ async def test_emits_no_content_by_default_on_the_streaming_path(
assert [k for k in span.attributes if k.startswith("gen_ai.prompt")] == []
assert [n for n, _ in span.events if n.startswith("gen_ai.content")] == []

async def test_process_default_is_read_on_the_streaming_path(
self, mock_anthropic: MagicMock
) -> None:
from launchdarkly_ai_server import SDKOptions, configure
from launchdarkly_ai_server.lifecycle import _reset_for_testing

_reset_for_testing()
try:
configure(options=SDKOptions(capture_content=True))
self._patch_stream(mock_anthropic, ["hi"])
ctx, rec = _recording()
from launchdarkly_ai_claude_messages import create_claude_messages_handler

with ctx:
async for _ in await create_claude_messages_handler().stream(
CONFIG, "q"
):
pass
assert any("gen_ai.input.messages" in span.attributes for span in rec.spans)
finally:
_reset_for_testing()


# ---------------------------------------------------------------------------
# History parameter
Expand Down Expand Up @@ -1824,8 +1846,11 @@ def test_capture_content_reaches_the_factory(self) -> None:
# And it must not have been forwarded to config(), which does not accept it.
assert "capture_content" not in seen["config_kwargs"]

def test_defaults_to_off(self) -> None:
assert self._run()["capture_content"] is False
def test_omitted_capture_content_defers_to_the_process_default(self) -> None:
assert self._run()["capture_content"] is None

def test_explicit_false_reaches_the_factory(self) -> None:
assert self._run(capture_content=False)["capture_content"] is False


class TestToolSpanNeverLeaks:
Expand Down
19 changes: 18 additions & 1 deletion packages/client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,10 +211,11 @@ asyncio.run(main())

| Export | Description |
|---|---|
| `configure(options=...)` | Set process-wide behavior defaults. Keyword-only. Separate from `init_client`. |
| `init_client(options?)` | Auto-discover and initialize `launchdarkly-server-sdk`. Optional — the first AI API call triggers lazy init when `LD_SDK_KEY` is set. Returns `Awaitable[LDClientInterface]`. |
| `init_client(client=...)` | **BYOC overload** — accept a pre-initialized `LDClientInterface`. Skips SDK auto-discovery. |
| `get_client()` | Return the initialized `LDClientInterface`. Raises if `init_client` has not completed. |
| `shutdown()` | Flush all events and telemetry, then close the client. Await before process exit. |
| `shutdown()` | Flush all events and telemetry, then close the client. Await before process exit. Restores the built-in SDK defaults. |
| `inspect_config(key, context)` | Read an AI Config variation without invoking the model. Never raises. Returns `{"enabled", "config", "meta"}`. |

`init_client` reads these option keys. Each overrides the environment variable of the same
Expand All @@ -231,6 +232,22 @@ warning and ignored.
| `serviceName` | `LD_SERVICE_NAME` | OTel `service.name` resource attribute |
| `environment` | `LD_ENVIRONMENT` | `deployment.environment` resource attribute |

### Process-wide defaults

`configure` sets behavior for the process. It does not initialize the LaunchDarkly client; call `init_client` for that. Handlers built without an explicit `capture_content` read this value when they run, so one startup call covers every handler. An explicit `True` or `False` on a handler factory or convenience wrapper overrides it. Call `configure` from process startup, not from request data. It can be replaced until the first AI invocation; after that only the same options are accepted. `shutdown()` restores the built-in default, which is off.

```python
from launchdarkly_ai_server import SDKOptions, configure, init_client, shutdown

async def main():
configure(options=SDKOptions(capture_content=True))
await init_client({"sdkKey": "sdk-...", "serviceName": "my-service"})
# create_*_handler() with no capture_content argument follows the default above.
await shutdown()
```

`SDKOptions` is the extension point for further process-wide defaults.

### Give the evaluation tools

Pass `tools` to `run()` as a list of `EvalTool`. Construct one to define a tool in code. Await `evals.tools.get()` to use a tool that already exists in your project's AI library, which reads the tool and pins its version at that point. One list can hold both kinds.
Expand Down
1 change: 1 addition & 0 deletions packages/client/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -867,6 +867,7 @@ When `enabled` is `False`, `config` is always `None`. When `enabled` is `True` b

## Lifecycle Invariants

- **Process defaults.** `configure(options=SDKOptions(capture_content=True))` sets behavior defaults for the process. Call it once at startup, not from request data, and not via `init_client`. Handlers omit `capture_content` to follow it. `shutdown()` restores the built-in defaults.
- **Lazy initialization.** Importing the package does not initialize the LD client. The first API call that needs LaunchDarkly calls `init_client()` internally when `LD_SDK_KEY` is set.
- **Explicit initialization — SDK path.** `await init_client(options?)` dynamically imports `launchdarkly-server-sdk` at runtime (optional peer dep). If the package is not installed it raises with a clear message.
- **Explicit initialization — BYOC path.** `await init_client(client)` accepts any pre-initialized object that satisfies `LDClientInterface` — this is the path for custom or edge environments whose SDK has different init semantics.
Expand Down
Loading
Loading