Skip to content

slack-agent-eve: eve 0.63 on Sentry.eveInstrumentation - #31

Merged
sergical merged 6 commits into
mainfrom
eve-063-eve-instrumentation
Sep 24, 2026
Merged

sergical merged 6 commits into
mainfrom
eve-063-eve-instrumentation

Conversation

@sergical

@sergical sergical commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

What

Moves slack-agent-eve from eve 0.34 to eve 0.63.0 and onto the eve provider that @sentry/node 11.0.0 ships.

  • agent/instrumentation/sentry.ts wraps Sentry.eveInstrumentation(options) in defineInstrumentation and adds a flush. The SDK owns the OpenTelemetry setup; no otel() or otelIntegration() files remain, and the hand-built SentrySpanProcessor / SentrySampler / beforeSendSpan code is gone.
  • eve hooks and providers do not see Slack metadata. The Slack channel's turn.started and actions.requested events record the thread and triggering user per eve session id in agent/lib/conversation.ts. The card tools read it through ctx, so a card's destination never comes from a tool argument.
  • agent/hooks/sentry.ts sets the Slack user and a gen_ai.conversation.id tag on the isolation scope at every step. agent/lib/slack-user.ts holds the users.info display-name lookup.
  • SENTRY_AI_RECORD_INPUTS / OUTPUTS feed both dataCollection.genAI and Sentry.vercelAIIntegration({ recordInputs, recordOutputs }), because eve passes per-call recordInputs: false outside eve dev.
  • enableLogs is removed from the options (gone in SDK 11, logs are on). agent/lib/agent-name.ts and the .bak file are deleted.
  • DD_CLI_FIXTURES=1 answers every dd-cli call from agent/lib/dd-fixtures.ts (three restaurants, menus, item options, an in-memory cart, preview, checkout link). The agent, its cards and its spans run with no DoorDash account, no dd-cli and no sandbox. pnpm smoke:fixtures exercises every dd helper against the fixtures.
  • estimate_nutrition calls generateText with Output.object instead of generateObject. In ai 7.0.x only generateText publishes to the AI SDK telemetry channel that Sentry's vercelAIIntegration subscribes to, so the tool's own model call arrived as a bare http.client span with no gen_ai spans and no usage. Now it arrives as invoke_agent nutrition-estimator plus its generate_content.
  • Both READMEs describe the new layout and the measured span inventory.

Verified

pnpm typecheck && pnpm lint && pnpm build && pnpm smoke:fixtures green.

Deployed Slack conversation on mealbot.sentry.dev with DD_CLI_FIXTURES=1, project sentry-developer-experience, session wrun_41M392STD30GTTMNDNJ7J5XXGS, 2026-09-24 06:49 to 06:54 UTC:

  • Turn 1 (message event) posted the restaurant cards in the triggering thread. Turn 2 (a block_actions Pick click about 50 s later, a cold continuation on a new invocation) posted the meal cards in the same thread.
  • Every gen_ai.* span of both turns carries gen_ai.conversation.id = the session id and user.id = the Slack user. 8 invoke_agent mealbot, 8 generate_content anthropic/claude-sonnet-5, 1 invoke_agent nutrition-estimator with its generate_content anthropic/claude-haiku-4.5, 9 execute_tool spans (find_restaurants, present_restaurant_options, get_menu, 4× get_item_details, estimate_nutrition, present_meal_options).
  • Usage is counted once: 9 OpenRouter requests, 9 generate_content spans, no second gen_ai tree.

Content capture off (SENTRY_AI_RECORD_INPUTS=false, SENTRY_AI_RECORD_OUTPUTS=false) under eve dev, session wrun_01M38Z6Y0BR42QKXBKRHXAFA4: 4 gen_ai spans, gen_ai.request.messages and gen_ai.response.text both empty.

Final spans before idle: eve start, two turns, the server killed with SIGKILL one second after the second turn returned (session wrun_01M3908RCZ4HZ2RQNRGWH7KTHV). Both turns' invoke_agent and generate_content spans reached Sentry.

Known: gen_ai.agent.name stays empty on this path (getsentry/sentry-javascript#20041).

Created with Claude Code

sergical and others added 4 commits September 21, 2026 13:44
Sentry.init loads from agent/instrumentation/sentry.ts, the conversation
hook lives in agent/hooks/sentry.ts, and eve's OpenTelemetry stays off.
Verified with eve invoke under eve start and eve dev. The Slack thread,
user, and card tool handoff from the removed step.started hook have no
new home yet.

Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude <claude@anthropic.com>
…tation

# Conflicts:
#	slack-agent-eve-otel/README.md
#	slack-agent-eve-otel/agent/tools/estimate_nutrition.ts
#	slack-agent-eve-otel/package-lock.json
#	slack-agent-eve/package-lock.json
@sentry/node 11.0.0 ships an eve provider, so the app no longer carries its
own OpenTelemetry pieces. agent/instrumentation/sentry.ts wraps
Sentry.eveInstrumentation with a flush; the provider sets the eve session id
as the conversation id on every step.

eve hooks and providers do not see Slack metadata, so the Slack channel's
turn.started and actions.requested events now record the thread and user per
session in agent/lib/conversation.ts, the card tools read it through ctx,
and agent/hooks/sentry.ts sets the Slack user on the Sentry scope per step.

Drop enableLogs (removed in SDK 11, logs are on) and the stale agent-name
helper. READMEs describe the new layout and the measured span inventory.

Co-authored-by: Claude <claude@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread slack-agent-eve/agent/channels/slack.ts
Comment on lines 18 to 20
// Mentions and DMs dispatch anonymously ({auth: null}) instead of through
// eve's defaults, which attach a Slack auth context: mentions on that
// default path were dropped without a turn or a log line, while every

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The code assumes channel.state.triggeringUserId exists for Slack channels, but this is unverified. If it's missing, user attribution will silently fail for all Slack interactions.
Severity: HIGH

Suggested Fix

Verify that channel.state from the eve Slack channel contains triggeringUserId at runtime. If it does not, the logic should be updated to correctly source the user ID. Consider adding defensive code, such as logging or a fallback mechanism, to handle cases where triggeringUserId is missing from the state to prevent silent failures.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: slack-agent-eve/agent/channels/slack.ts#L18-L20

Potential issue: The code was refactored to retrieve `triggeringUserId` from
`channel.state` as part of a migration to a new event handler model. However, it is
unverified whether the `eve` Slack channel implementation actually populates
`triggeringUserId` in the durable `channel.state`. If this field is `undefined` at
runtime, the `userId` variable will be `undefined`. This would cause user attribution to
silently fail for all Slack interactions, preventing user data from being set in Sentry
and omitting the `user.id` attribute from critical log events.

Also affects:

  • slack-agent-eve/agent/lib/conversation.ts:52~64

Did we get this right? 👍 / 👎 to inform future reviews.

DD_CLI_FIXTURES=1 answers every dd-cli call from agent/lib/dd-fixtures.ts:
three restaurants, their menus and item options, an in-memory cart, a
preview and a checkout link. The agent, its cards and its Sentry spans then
run with no DoorDash account, no dd-cli binary and no Vercel Sandbox. Model
calls stay real. `pnpm smoke:fixtures` exercises every dd* helper against
the fixtures.

Co-authored-by: Claude <claude@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 64aefb1. Configure here.

lines: FixtureCartLine[];
}

const carts = new Map<string, FixtureCart>([["fx-cart-0001", { storeId: "", lines: [] }]]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixture cart state does not persist

Medium Severity

The fixture cart lives in a module-level carts Map. Eve can inline a second copy of this module into a step bundle — the same reason conversation.ts hangs its map on globalThis — and a Vercel isolate does not share that Map with the next step. After add_to_cart, preview_order and later tools can see an empty cart even though the add succeeded. The README still tells people to set DD_CLI_FIXTURES=1 on the deployment.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 64aefb1. Configure here.

In ai 7.0.x generateText publishes to the AI SDK telemetry tracing channel
that Sentry's vercelAI integration subscribes to, and generateObject does
not. The estimate_nutrition tool's own model call therefore arrived as a
bare http.client span under execute_tool, with no gen_ai spans and no usage.
Switching to generateText with Output.object keeps the structured result
and yields invoke_agent nutrition-estimator plus its generate_content span.

Co-authored-by: Claude <claude@anthropic.com>
@sergical
sergical merged commit 2234ca5 into main Sep 24, 2026
11 checks passed
@sergical
sergical deleted the eve-063-eve-instrumentation branch September 24, 2026 15:35
@linear-code

linear-code Bot commented Sep 28, 2026

Copy link
Copy Markdown

DEVEX-728

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant