[apucacao]

Summary

Runs made through the turnkey SDK don't show up on an AI Config's monitoring traces tab. Turnkey writes the feature_flag span event without feature_flag.result.value. Since launchdarkly/observability#2553, a trace search for a flag key only scans hours where a feature_flag event with a value was recorded, so turnkey runs are never found. The LaunchDarkly observability plugins always send the value (the SDK tracing hook with includeValue: true).

This adds two attributes to the event setLdSpanAttributes writes:

• feature_flag.result.variant: the variation key. The OpenTelemetry feature-flag conventions make it conditionally required when the provider has variants.
• feature_flag.result.value: with content capture on, the evaluated variation, serialized with JSON.stringify exactly as TracingHook does. With capture off, the variation key, JSON-encoded like any flag value (for example "opus"), so the prompt is never sent unless capture is on.

What changed

• extractVariation returns the raw variation it already receives from client.variation. invoke, stream, inline judges and graph nodes pass it to handlers as variables.__ldValue, next to __ld.
• Deferred judges get the value too. JudgeTask has a new optional field, judgeValue: the judge variation exactly as the LaunchDarkly SDK returned it. buildJudgeTasks fills it, and runJudge passes it on under the same capture check as the inline path. It is plain JSON, so a task still survives structured clone, and it adds no new kind of content, since the task already carries judgeConfig and actualOutput.
• The launchdarkly.graph root span gets the value too. graph().invoke() and graph().stream() share one code path, which now passes the graph flag's raw variation to the root span. The root has no single handler, so it follows the process-wide content capture default, which is off today. With the default off, the root keeps the JSON-encoded variation key.
• The value is deliberately not added to TrackData. TrackData is the payload of every $ld:ai:* track event, and the value includes the prompt.
• The core passes the raw variation only to handlers with captureContent on. executeAndTrack and executeAndStream set __ldValue only when the handler's captureContent is true, the same check judge reasoning uses, so the raw config never reaches a capture-off handler. That check, and the graph root's, go through one function, captureContentFor, which returns a handler's own captureContent or the process-wide default when there is no handler or it states none. This uses the existing captureContent handler option; there is no new setting.
• setLdSpanAttributes writes the variant when the variation key is non-empty. For the value it writes the raw variation's JSON when __ldValue is present, otherwise the JSON-encoded variation key, and leaves it off when there is neither (native-graph spans).
• The only public API change is the optional JudgeTask.judgeValue field. setLdSpanAttributes keeps its signature and the handlers are unchanged.
• AGENTS.md describes the new attributes, including the deferred judge and graph root cases.

vercelEvaluate has no raw variation at hand, so its value is always the variation key.

Testing

• New tests: the core passes __ldValue only to a capture-on handler, for both executeAndTrack and executeAndStream; the value is JSON.stringify of the raw variation when present, the JSON-encoded variation key otherwise, and absent with neither; __ldValue reaches the handler and stays out of TrackData and track payloads.
• New tests: a JudgeTask built from a judge variation carries it as judgeValue through structured clone, and its runJudge run writes the raw variation as feature_flag.result.value with capture on and the variation key with capture off. The launchdarkly.graph root writes the variation key with the default off and the raw graph variation with it on, for both invoke and stream.
• Updated the four handler tests that assert the root event's exact attributes (claude-agents, langchain-agents, langchain-messages, openai-agents).
• Ran the handler, graph, span, tracking and utils tests in every package, the full client test suite, tsc --noEmit in every package, api:check, api:surface and biome.
• End to end: with this build installed, a real agent run's trace appeared on the AI Config monitoring tab on catamorphic, where it hadn't before.

The cross-SDK spec change is in launchdarkly/ai-sdks-monorepo#113.

---

Note

Overview
Turnkey SDK runs were missing from AI Config monitoring traces because the root feature_flag event lacked feature_flag.result.value, which observability now requires to correlate traces with a flag. This PR adds feature_flag.result.variant (served variation key, omitted when empty) and feature_flag.result.value on the event setLdSpanAttributes writes, aligned with LaunchDarkly OTel TracingHook serialization.

extractVariation now returns the raw SDK value, threaded through invoke/stream, graph nodes, and judges as optional evaluatedValue. The core exposes it to handlers only as variables.__ldValue when captureContentFor(handler) is true (handler captureContent or process default); otherwise the event gets JSON-encoded variation key only. Raw variation never lands in TrackData or track payloads. Deferred judges store it on optional JudgeTask.judgeValue; the launchdarkly.graph root uses the graph flag raw value only when the process-wide capture default is on.

captureContentFor centralizes that gate. Handler package tests and AGENTS.md are updated; the only public API addition is JudgeTask.judgeValue.

^{Reviewed by Cursor Bugbot for commit 869636a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[jeffdupont]

Looks good, and it matches the updated #113 spec. Gating on handler.captureContent in executeAndTrack / executeAndStream keeps the raw config away from capture-off handlers, and since createHandlerInternal (utils.ts:74) sets that property for every handler, it is a single check. The fallback also closes the vercelEvaluate / JudgeTask gap from the first version.

• One handler-level capture-on test. The four handler tests only assert the capture-off value (JSON.stringify('variation-a')). §1.5 asks handler tests to cover capture on too. A single test that goes through config().invoke with createXHandler({ captureContent: true }) and asserts result.value === JSON.stringify(rawVariation) would cover the wiring from start to finish.
• Serialization question. gonfalon's flag Monitoring Traces tab groups chart series by the exact feature_flag.result.value string, matched against JSON.stringify(variation.value) (static/ld/components/feature-monitoring/hooks/useVariationChartConfig.ts:8,45-49). JS already matches with capture on. The capture-off "variation-a" fallback matches no variation, so if AI Config Monitoring uses that chart, those traces land in an unlabeled series. Do you know which UI reads result.value for AI Configs? Not blocking.

Happy to approve once the capture-on test is in.

[jeffdupont]

bda9e7b adds the end-to-end capture-on test I asked for: it goes through config().invoke with capture on and asserts the raw variation as the value. Approving.

This matches launchdarkly/ai-sdks-monorepo#113. setLdSpanAttributes writes feature_flag.result.variant only for a non-empty variation key. It writes feature_flag.result.value as JSON.stringify of the raw variation, which is how the node-server-sdk-otel TracingHook writes it. The core sets __ldValue only when handler.captureContent is true. Otherwise the value is the JSON-encoded key, and it is left off when there is neither. The event is still written only by the shared helper on the root span. The tests cover capture on (client.test.ts), capture off and unset (tracking.test.ts, for track and stream), and the case with neither (utils.test.ts). The change is additive. extractVariation gains a return field, and no public type or API report changes, so API compatibility passes.

On parity with python-ai-sdk#240, the span output agrees, with two differences in how it gets there. First, JS gates capture in the core and Python gates it in the helper. Second, a judge run from a JudgeTask writes the raw value in Python but only the key here, because the JS JudgeTask path has no raw variation. Both stay within the spec.

[jeffdupont]

Since my approval at bda9e7b: 8398f4e puts the judge variation on JudgeTask.judgeValue, and runJudge gates it with the judge's handler. That closes the deferred-judge gap I mentioned, and the structuredClone test is a good touch. 4ad0f34 gives the launchdarkly.graph root the graph variation under the process default. de89d4a routes both through captureContentFor. The main merge had no conflict resolutions.

About the red API compatibility runs: they come from 8398f4e. JudgeTask is public, so the new field changed ai-server.api.md, and the check counts any changed declaration as breaking. The rename to feat(client)! at 16:34 made it pass, and the PR is clean now. Adding an optional field doesn't break anyone, though. If you'd rather not have a BREAKING CHANGES entry in the changelog, the api-compatible label on a title without ! is the more accurate route. Your call.

One thing to settle with #190. I merged #190 onto this branch locally. It merges with no conflicts, but leaves two captureContentFor functions. The one in content.ts keeps the hard-coded false default, and tracking.ts and graph.ts still import it. The one in sdk-options.ts follows configure(), and judges.ts imports it. Handlers built with createHandler are fine, because #190 makes captureContent a getter that always returns a boolean. A plain function handler, and the graph root, would still send only the key while the default is on. A quick vitest probe on the merged tree confirms it for the plain function, and the full suite still passes, because graph.test.ts mocks content.js. Whichever of the two merges second should delete the content.ts version, point tracking.ts and graph.ts at sdk-options.ts, and use configure() in the graph test instead of the mock. A test with a plain function handler under the built-in default would keep it fixed.

Nit: executeAndTrack and executeAndStream spread the caller's variables before adding __ldValue. A capture-off handler therefore writes a caller-supplied variables.__ldValue. Overwriting or deleting it the way __ld is handled would close that.