[andrewklatzke]

Summary

• A wildcard handler can take a providers list, so a LangChain adapter used only for Bedrock accepts config.provider.name === "Bedrock" and no other provider.
• Exact [provider, mode] still wins. Among scoped wildcards that contain the name, the shortest list wins, then the earlier registration. An unscoped * is the fallback.
• Registry identity is providesFor plus the provider set, so two scoped wildcards of the same mode can be registered together. Judges and graph selection use the same order.

Test plan

• Vitest on the client, langchain-messages, and langchain-agents suites
• Biome and tsc on the touched packages
• Integration success and failure for agent, graph, streaming, graph-streaming, langchain, langchain-messages, langchain-agents, and native-graph-langchain
• TypeScript judge example still looks for examples/judge-worker.js while only judge-worker.ts is present, so that case does not start

Made with Cursor

[andrewklatzke]

Human message:

I think this is a "nice to have" not necessarily "must do it". I'm thinking of situations where, for example, a user might want to use the Google ADK and scope it only to gemini (create_google_adk_handler(providers=['Gemini'])). Another example might be folks wanting to use langchain but only for Bedrock so they don't have to deal with boto.

You can technically achieve this now by properly including the right subset of handlers, but this makes it more obvious/colocates it with the code if you want to scope them down.

tldr; if we think this is confusing or don't want it, we can close it

[jeffdupont]

Selection is right in config(), in graph node selection, and in the three judge entry points. The registry keeps scoped wildcards apart the way the PR body says. One judge path skips the allowlist (1 below), and the registry orders handlers differently from Python #136 (2). Those two need fixing before this merges.

At 6cd4ce42, build, tsc, Biome and Sherif are clean, and yarn test gives 1177 passed. I trial-merged it onto today's main (#73, #68 and the rest) and got no conflicts. The merged tree gives 1325 passed, with typecheck and Biome clean. The #73 parameter allowlists in langchain-messages and langchain-agents handler.ts merge without trouble next to the new providers option, and the factory signatures are unchanged.

I probed selectModeHandler, the registry and the judge functions directly:

• An exact handler beats a scoped wildcard, and a scoped wildcard beats an unscoped one, whatever the registration order.
• The shortest matching list wins. On equal lengths the earlier handler wins.
• Names match exactly, so bedrock doesn't match ['Bedrock']. Exact handlers follow the same rule (client.ts:39-44).
• [], [''] and [' '] throw at createHandler, and the message mentions providers.
• If a scoped wildcard doesn't list the provider, config() throws Handler for provider OpenAI not found. That's the same message as having no handler at all, and Python gives the same string.
• Two scoped wildcards of one mode both stay registered. Registering the same set again, in either order, warns and replaces. ['Bedrock', 'Bedrock'] counts as the same identity as ['Bedrock']. compose(a, b) lets b win on a matching set and keeps different sets.
• runJudges, buildJudgeTasks and runJudge all pick the shortest list. Each falls back to a scoped agent handler with collapseMessages: true, and each skips a judge whose provider isn't in any list.

I tried 24 mutations, and 20 of them fail a test. The 4 that get through are in the inline comments.

1. The graph-level judge ignores the allowlist. graph.ts:796-807 calls runJudges without handlers, so runJudges takes the judgeHandler = handler branch and never checks the provider (judges.ts:157-158). I ran graph().invoke() with one handler, ['*', 'messages'] scoped to ['Bedrock'], and a graphJudge whose provider is Anthropic. The Bedrock-only handler ran the Anthropic judge. Adding an Anthropic-scoped handler didn't help, because the root handler still ran it. The missing handlers argument predates this PR. Before, though, it only affected which of two acceptable handlers ran. Now it lets a scoped handler run a provider it was set up to refuse. Python passes handlers=resolved_handlers on the same call (graph.py:1016-1024 and :1238-1250). Please pass handlers: judgeHandlers and add a graphJudge test with a provider outside the list.
2. Re-registering moves a handler to the end, but Python replaces it in place. registry.ts:29 filters out the old entry and pushes the new one. Python's registry.py:74 assigns into the same index. Ties go to the earlier registration, so the two SDKs now pick different handlers. As a repro, register ['Bedrock', 'Gemini'], then ['Bedrock', 'Anthropic'], then ['Bedrock', 'Gemini'] again. JS ends up with [BA, BG], and a Bedrock config runs BA. Python keeps [BG, BA] and runs BG. compose shows the same split: JS gives [aU, bB, bA], Python [bB, aU, bA]. Before this PR it didn't matter, because only one wildcard per mode could be registered. Monorepo #39 doesn't say which is right. I'd go with in place, so a replacement takes the slot of what it replaces. That needs a line in #39 §3.6 and a test in both SDKs.
3. Repeated names count toward list length. The registry key collapses ['Bedrock', 'Bedrock'] to ['Bedrock'] (registry.ts:6), but bestWildcard compares allow.length (utils.ts:92). So ['Bedrock', 'Bedrock'] ties with ['Bedrock', 'Anthropic'] and loses on registration order, even though the registry treats it as the narrower list. Python does the same (utils.py:97). Deduping in createHandler fixes this, and then #39 can say "fewest distinct names".
4. createHandler keeps the caller's array. utils.ts:60 stores the array itself, not a copy. If the caller later pushes 'OpenAI' onto it, an OpenAI config routes to that handler, and its registry identity changes too. Python copies into a tuple. Object.freeze([...new Set(providers)]) would fix 3 and 4 together.

Questions and smaller items

• The createHandler signature freezes at 1.0. providers is the fifth positional argument, so a caller writes createHandler(pf, fn, undefined, undefined, ['Bedrock']), and #39's A.14 has to explain the undefineds. Python made it keyword-only. An options object ({ stream, captureContent, providers }) is harder to get wrong, but we can't switch to one after 1.0 without a major release. I'm fine either way, but let's decide now. The name and type match Python (providers, readonly string[], and Python's Sequence[str] stored as a tuple). The selection helpers aren't exported from index.ts, which is right.
• API report. main has no API Extractor setup yet, because it's on #71's branch. I trial-merged this PR into #71 (c27005cc). There were no conflicts, but api:check fails on two lines: createHandler(..., providers?: readonly string[]) and ProviderHandler.providers?. Whichever of the two lands second needs yarn api:update.
• Whitespace. [' Bedrock'] is accepted and never matches anything. Python's docstring says surrounding whitespace is rejected, but its code accepts it too. Either both should reject name !== name.trim(), or the claim should go.
• Case. Routing is exact, which fits exact handlers and #39. The LangChain handlers lower-case the provider themselves (langchain-messages handler.ts:70 and :198), though. So a config with provider.name = 'bedrock' would take the Bedrock model path, but it won't route to ['Bedrock']. That's fine if LaunchDarkly always serves the canonical spelling, which I haven't checked.
• Exact handlers that have a list. Routing ignores the list on an exact handler, but the registry key still includes it (registry.ts:5-6). So ['OpenAI', 'messages'] with a list and without one both stay registered, and the first one wins. #39 says two exact handlers collide. Python does the same thing. I'd leave providers out of the key when providesFor[0] !== '*'.
• Calling createHandler twice on one function. createHandler changes the function in place, so two calls with different lists return the same object, holding the last list. A later unscoped call on the same function keeps the old list, because utils.ts:60 only sets the property when one is passed. captureContent already works this way. The LangChain factories build a new closure each time, so they aren't affected. A JSDoc note is enough.
• The priority comment at judges.ts:132-138 doesn't mention scoped lists. Nit.

The judge example (from the PR body)

examples/judge.ts:56 resolves ./judge-worker.js, but only examples/judge-worker.ts exists. Both files date from the initial commit (977dba8), and this PR doesn't touch examples/, so the problem already exists on main. I didn't run it.

Parity with python-ai-sdk #136

On my probe cases, the two SDKs agree on selection order, ties, case, a missing provider (undefined / None), empty and blank lists, the error strings and set identity. They differ in five places:

• re-registration position (2)
• the graph-level judge (1)
• copying the caller's list (4)
• identity of ['a,b'] vs ['a', 'b']: JS joins with commas, so these collide and one replaces the other (registry.ts:6), while Python's frozenset keeps both
• a bare string or a non-string name: Python raises ValueError mentioning providers, while JS throws a TypeError (providers.some is not a function). That's acceptable, since the TS types already rule it out.

Python also routes judges in its evaluations runner. JS has no runner, and #39's A.14 says so.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 64d94c8. Configure here.}

[jeffdupont]

Merged main in and resolved the conflicts (now at the head of aklatzke/AIC-3439/scoped-handlers):

• config() selection, findHandler and judge selection all go through selectModeHandler (exact provider → shortest scoped wildcard → unscoped *). Judges keep main's agent-mode fallback for a messages judge, and handlerFor is gone.
• Regenerated ai-server.api.md, the new langchain-agents/messages reports, and api-surface.
• AGENTS.md now describes streaming as the stream option.
• Retitled with !: createHandler moves from positional (providesFor, fn, stream?, captureContent?) to (providesFor, fn, options?), which is breaking and freezes at 1.0.

Locally, typecheck and api:check pass, and every package's tests pass (client: 1859 passed). This needs a human re-review; jeffdupont's CHANGES_REQUESTED still stands.

[jeffdupont]

Re-reviewed 3b19aba against my 10-08 review. Your 10-21 fixes resolve my comments. Two things left:

1. Conflict with main. After #159 merged, the branch conflicts only in packages/client/etc/ai-server.api.md. Please merge main, run yarn workspace @launchdarkly/ai-server api:update, and push.

2. The evaluations duplicate check ignores the provider scope. packages/client/src/evaluations.ts:314 builds the handler identity from providesFor alone:

   const identity = `${providesFor[0]}\u0000${normalizeMode(providesFor[1])}`;

   So handlers: [createLangChainHandler(llm, { providers: ['Bedrock'] }), createLangChainHandler(llm, { providers: ['Anthropic'] })] throws "Two handlers provide for provider '*' in mode 'messages'", even though the registry treats them as distinct. Python accepts the same pair: python-ai-sdk#136 keys this check on registration_key (evaluations/module.py:669). Please export registrationKey from registry.ts, use it here, and add a test with two wildcards scoped to different providers.

I'll approve once both are in. This one has to land before the 1.0 cut, because the positional createHandler change can't go in after it.

[jeffdupont]

Re-reviewed b2adf5f against my 10-27 comment. b2adf5f fixes the evaluations duplicate check: validateHandlerPool now keys on registrationKey (evaluations.ts:314), which also normalizes the mode the way Python's registration_key does. The two new tests cover both cases, two different sets and two matching sets. All the 10-08 items still hold at this head, and CI is green.

The branch conflicts with main again. Eight commits landed after your 10-27 merge, and #173 renamed createLangChainHandler to createLangChainMessagesHandler. A trial three-way merge of head, main and their merge base c6c00fc conflicts in three files: packages/client/etc/ai-server.api.md, packages/langchain-messages/etc/ai-langchain-messages.api.md and packages/langchain-messages/src/handler.ts. In handler.ts the only conflict is the langchainMessages wrapper line: keep main's name and your providers. Three more places merge without a conflict but keep the old name (inline below), so tests fail after the merge until they're renamed. After that, run yarn api:update and push.

I checked every createHandler( call in the repo at head and on main. No caller passes stream or captureContent positionally. The handler packages go through createHandlerInternal, which stays positional, and the API report matches the new options-object signature. Users who pass a stream function or captureContent positionally have to move it into { stream, captureContent }. TypeScript users get a compile error. Plain JavaScript callers don't, and their stream or captureContent is silently ignored (inline at utils.ts:60).

Parity with python-ai-sdk#136 is unchanged and holds: exact provider first, then the scoped wildcard with the shortest list of distinct names, then the earlier registration, then the unscoped *. Identity is the same, a replacement keeps its slot, and the empty/blank/padded-name error string is identical. Monorepo #39 needs two edits after #173: A.19 still says createLangChainHandler, and §3.13 says TypeScript has no evaluations runner, while A.19 (and evaluations.ts) route judges through findHandler.

I'll approve once the merge with main is in and the old LangChain name is renamed. This one has to land before the 1.0 cut.

[jeffdupont]

#173 renamed this factory to createLangChainMessagesHandler on main. A three-way merge renames the existing calls in this file but leaves your new ones at :128, :132 and :138 on createLangChainHandler, so the file no longer typechecks after the merge, and there's no textual conflict to point at it. The same old name stays in packages/langchain-messages/README.md:7 and packages/langchain-messages/agents.md:15. Please rename all five when you merge main.

[jeffdupont]

This still says to pass providers "as the fifth argument to createHandler". After 64d94c8 it's a field of the third-argument options object: createHandler(providesFor, fn, { providers: ['Bedrock'] }).

[jeffdupont]

Users who call createHandler(pf, fn, streamFn) or createHandler(pf, fn, undefined, true) from plain JavaScript don't get an error. options.stream reads undefined off the function, so stream() quietly falls back to the blocking handler, and the fourth argument is dropped. TypeScript catches both. waitForSkills handled the same kind of change with a TypeError that names the new form (skills-fdv2.ts:1757-1771). Something like if (typeof options === 'function' || arguments.length > 3) throw new TypeError('createHandler takes { stream, captureContent, providers } as its third argument') would make the 1.0 change fail loudly. Optional, but cheap to add before the signature freezes.

[jeffdupont]

Re-reviewed 04eb5a8 against my 10-28 comment. All three threads are fixed. The merge with main renames the new scoped-handler tests to createLangChainMessagesHandler, and README.md:7 and agents.md:15 now use the new name. The only remaining createLangChainHandler string is the usage-telemetry key, which main has too. AGENTS.md now describes providers as a field of the third-argument options object. createHandler throws a TypeError that names the new form when the third argument isn't an object or a fourth argument is passed (utils.ts:70-76). The four new tests cover a positional stream function, a fourth argument, non-object values and an explicit undefined. The client tests pass locally (458), and CI is green.

One heads-up for sequencing, not a change here: #190 also edits createHandler. It keeps the positional fourth captureContent, which this PR now rejects. It returns a new wrapper instead of the same function. And a re-wrap drops an earlier captureContent override, which this PR's doc comment (utils.ts:57-59) promises to keep. A trial merge of the two heads conflicts in utils.ts, types.ts, utils.test.ts and both LangChain handler.ts files. I'd land this one first and have #190 rebase onto the options object. I'll raise the details on #190.

Approving. This changes the public createHandler signature, so it needs to land before the 1.0 cut.