[hsadhvani]

Why

Shadow requests let a team try a candidate AI Config variation on real traffic without serving it, and compare it on the normal AI metrics. The SDK owners asked for this to be merged as experimental (TESTING.md §0). Earlier versions (#187, #193) either added an invocation hook to core or changed config() for everyone; this one is an opt-in wrapper, so core behavior is unchanged.

Summary

Adds shadowConfig, runShadow, ShadowTask, ShadowResult and ShadowConfigArgs to @launchdarkly/ai-server/experimental. shadowConfig() evaluates the flag once with variationDetail, runs the served variation through the same code as config(), and starts the sampled shadows in the background once it succeeds (or hands them to onTask). Core gets only generic internal seams: configFromVariation behind configInternal, optional runId, extraTrackData and spanAttributes on executeAndTrack, runId on executeAndStream, and a symbol key that setLdSpanAttributes reads. The root API report does not change, and config() still calls variation().

Testing approaches

• Added tests for shadow selection by rule and fallthrough, ended, malformed, disabled and unhandled entries.
• Added tests for sticky sampling, including the cross SDK bucket values.
• Added tests that the served response, usage and events match config(), and that shadows start only after it succeeds.
• Added tests for streaming, timeouts, failures, onTask, runShadow, shutdown and usage IDs.
• Added tests for shadow event fields and root span attributes.
• Added a test that plain config() never calls variationDetail and runs no shadows.

Feature flags

N/A

---

Note

Overview
Adds experimental shadow requests on @launchdarkly/ai-server/experimental: shadowConfig() mirrors config() but, after a successful primary invoke() or stream done, runs LaunchDarkly-configured candidate variations in the background (or hands them to onTask for runShadow()). Served responses, usage, and primary events stay aligned with plain config(); shadows reuse the primary runId, emit normal $ld:ai:* events with shadow metadata, and never get tools, judges, or user-visible output.

Core changes are opt-in plumbing: configFromVariation, fetchVariation / parseVariation (reason via variationDetail only when needed), shared runId and extra tracking/span hooks on executeAndTrack, plus conversation helpers for deferred shadow runs. config() still evaluates with variation() only.

^{Reviewed by Cursor Bugbot for commit 32e0293. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit abdad41. Configure here.}

[XieX]

Comment-only review. The configFromVariation(args, evaluate) seam is a neat way to do this: the wrapper hands core its evaluation, with no subclass and no copied fields, and plain config() is unchanged. There's one bug inline, plus one difference from the Python SDK.

[XieX]

When the caller binds a conversation id around the stream() call only and iterates later, the shadow loses it. Core's stream() binds the id at call time through bindConversationId, which re-enters it only around the inner generator's next(). This outer generator runs in the iterating caller's context, so startShadows, and the shadow's spans, run unbound.

Reproduced with an AsyncLocalStorageContextManager and handlers that read the conversation key:

const gen = withConversationId('conv-1', () => shadowConfig({ ... }).stream('hi', ctx));
for await (const _ of gen) {}
// primary handler sees 'conv-1'; shadow handler sees undefined

invoke() is fine, and so is iterating inside the block. The Python SDK keeps the id on both paths.

A fix that works: capture the active OTel context when stream() is called, and start the shadows inside it, e.g. const callerContext = otelContext.active() up front and otelContext.with(callerContext, () => startShadows(...)) here. (The context parameter shadows the OTel context import in this function, hence the alias.) That also parents the shadow's spans to the caller's trace, as invoke() does. Wrapping the outer generator in bindConversationId(...) fixes only the id. A regression test like the snippet above would be worth adding.

[XieX]

Difference from the Python SDK: this done is the public one, which core yields after judges run, so stream shadows start after judges. Python starts stream shadows on the internal done, before judges. On invoke(), both start after judges. Neither delays the caller, but it'd be good for the two SDKs to agree.

[hsadhvani]

These have been fixed, and I had Claude confirm. Christie, could you take a look next week?

From Claude: Fixed Christie's comments in 32e0293, merged with main in bd10137.

• stream() captures the caller's OTel context and starts shadows inside it, with Christie's repro as a regression test.
• Stream shadows still start on the public done, after judges, which is now the rule in the spec.
• Tasks carry conversationId.
• Shadow spans skip the feature_flag event.