Repository navigation
test(client): add makeGraphTrackData to the root exports snapshot - #137
Merged
Merged
Conversation
#101 exported makeGraphTrackData from the package root but left ROOT_VALUES in skills.test.ts unchanged, so the snapshot test failed on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
XieX
marked this pull request as ready for review
October 14, 2026 21:38
Contributor
|
Thanks, this fixes the Tests job. Type check still fails because |
…Data Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
approved these changes
Oct 15, 2026
jeffdupont
left a comment
jeffdupont
left a comment
Contributor
There was a problem hiding this comment.
Re-checked at bb7f00d. The API report is regenerated with the makeGraphTrackData entry, so Type check passes now along with Tests and the rest of CI. The paired approved-names change (ai-sdks-monorepo#61) has merged. The ae-internal-missing-underscore warning matches the 4 already on main. Once this lands, main's CI (red since 4d97cfa) goes green.
Merged
XieX
added a commit
that referenced
this pull request
Oct 19, 2026
… success (#138) ## Problem With an Anthropic key that had no credits, a Claude Agents run returned `{ output: 'Credit balance is too low' }` as a successful run. `config().invoke` tracked `$ld:ai:generation:success`, a caller would show the error text to a user as model output, and the trace showed an OK `chat <synthetic>` span. The Claude Code CLI does not fail a run when an API call fails. Captured live from Agent SDK 0.3.220 with a rejected key (no credits takes the same path): - an assistant message with `model: "<synthetic>"`, `error: "authentication_failed"`, zero usage, and the error text as content - then a result with `subtype: "success"`, **`is_error: true`**, `api_error_status: 401`, `terminal_reason: "api_error"`, and the error text in `result` The handlers checked only `subtype`. The agent SDK itself treats `is_error` as the failure signal and raises after the result, but the handler stops reading at the result and never sees that. ## Fix - `assertRunSucceeded` (exported from `handler.ts`, not from the package root) fails a run when `subtype !== 'success'` or `is_error === true`, and narrows the message to `SDKResultSuccess` otherwise. Both handler paths and `toClaudeAgents` use it. - `toClaudeAgents` now gates on `message.type === 'result'` instead of `'result' in message`. The old gate also let an error-subtype result through as an empty answer. A failed root now rejects and tracks `$ld:ai:graph:invocation_failure`; a failed sub-agent fails its tool call. - An assistant message whose `model` is `<synthetic>` gets no `chat` span, joins no conversation, and adds no usage. It keys on `model`, not the `error` field, because the CLI also emits a synthetic message after a response stops at `max_output_tokens`, and that run can still succeed. - `agents.md` documents the rule and drops the stale `'result' in message` example. ## Tests Six new tests, each failing on `main` for the intended reason: an API-error result on the blocking and streaming paths, a synthetic message that shares a real response's id in a run that then succeeds, and the native graph root (API error, and `error_max_turns`) and sub-agent cases. A seventh pins that `is_error: false` stays a success. Changed existing fixtures: two `native-graph.test.ts` results yielded `{ result, usage }` with no `type`, which only worked through the old gate; they now carry `type: 'result', subtype: 'success'`. The `client` package-exports snapshot test fails on `main` independently of this change; #137 fixes it. Also checked end to end against the real CLI with a fake key and an empty `CLAUDE_CONFIG_DIR`: both paths reject with `Claude agent run ended with an error result (HTTP 401): Invalid API key · Fix external API key`, and the only span is `invoke_agent` with status ERROR. Spec: TESTING.md §2.2 "Anthropic agents handler — run outcome from the `result` message" (launchdarkly/ai-sdks-monorepo#62). Counterpart: launchdarkly/python-ai-sdk#174. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Fixes a case where Claude Agent SDK runs could **succeed** while returning API error text (e.g. “Credit balance is too low”) because the CLI emits `subtype: 'success'` with **`is_error: true`**. > > **Run outcome:** Adds exported **`assertRunSucceeded`** and uses it on blocking, streaming, and **`toClaudeAgents`** paths. Result handling now gates on **`message.type === 'result'`** instead of `'result' in message`, so error subtypes without a `result` field and API-error “success” results both **throw** (generation/graph failure tracking instead of treating error text as model output). > > **Telemetry:** Assistant messages with **`model: '<synthetic>'`** are ignored for `chat` spans, conversation history, and usage so CLI-injected error notices do not pollute traces or span content when a run later succeeds. > > Docs in **`agents.md`** and new/updated tests cover API errors, streaming, synthetic messages, and native-graph root/sub-agent failures. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f5258bb. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#101 exported
makeGraphTrackDatafrom the@launchdarkly/ai-serverroot (packages/client/src/index.ts) but didn't updateROOT_VALUESinpackages/client/src/__tests__/skills.test.ts. The "the package root exports exactly the snapshotted values" test has failed onmainsince 4d97cfa, andyarn teststops before the later workspaces run, so every branch cut frommainis red.What
'makeGraphTrackData'toROOT_VALUESin sorted order.packages/client/etc/ai-server.api.md(api:update) so the Type check job'sapi:checkpasses.The approved-names list in ai-sdks-monorepo gets the matching
- makeGraphTrackDataline (leaving the public surface, likemakeNodeTrackData/makeRunTrackData): launchdarkly/ai-sdks-monorepo#61. Its JSDoc already marks it@internalfor the native graph adapters.Testing
yarn workspace @launchdarkly/ai-server run test: 1401 passed, 11 skipped.yarn workspace @launchdarkly/ai-server api:checkpasses.🤖 Generated with Claude Code
Note
Overview
Fixes a failing root export snapshot test after
makeGraphTrackDatawas added to the@launchdarkly/ai-serverpackage entry in an earlier change.Adds
'makeGraphTrackData'to the sortedROOT_VALUESlist inskills.test.tssoObject.keys(packageIndex)matches the snapshotted exports again. The API Extractor report (ai-server.api.md) is updated to document the existing@internalmakeGraphTrackDataexport alongsidemakeNodeTrackData/makeRunTrackData.Reviewed by Cursor Bugbot for commit bb7f00d. Bugbot is set up for automated code reviews on this repo. Configure here.