Skip to content

feat: Tracker enhancements: tool-call tracking + TrackMetricsOf (AIC-2677) - #5

Closed
knfreemLD wants to merge 2 commits into
mainfrom
kfreeman/AIC-2677/tracker-enhancements
Closed

knfreemLD wants to merge 2 commits into
mainfrom
kfreeman/AIC-2677/tracker-enhancements

Conversation

@knfreemLD

@knfreemLD knfreemLD commented Jun 13, 2026 •

Copy link
Copy Markdown
Contributor

Ports the remaining non-experimental Python tracker surface; equivalents: python#112 (launchdarkly/python-server-sdk-ai#112), js-core#1332 (launchdarkly/js-core#1332)).

  • TrackToolCall/TrackToolCalls — $ld:ai:tool_call event per key (multi-fire, toolKey in payload); ToolCalls added to MetricSummary.
  • TrackMetricsOf[T] — generalized, Config-decoupled tracking (package-level func; Go has no generic methods) with new AIMetrics type (declared in the ldai package). Extractor returns *AIMetrics (nil = duration-only, Python's None); extractor panics degrade to duration-only; task panics still record duration+error.
  • TrackRequest reimplemented atop TrackMetricsOf — ⚠️ behavior change: failed requests now record duration (matching Python/Node, which both track duration on failure).

@knfreemLD
knfreemLD requested a review from jsonbailey June 13, 2026 00:00
@knfreemLD
knfreemLD requested review from a team as code owners June 13, 2026 00:00
Comment thread ldai/tracker.go Resolved
Comment thread ldai/tracker.go Resolved

@jsonbailey jsonbailey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few questions since I'm not as familiar with Go.

@knfreemLD
knfreemLD requested a review from jsonbailey June 15, 2026 19:35
@jsonbailey

Copy link
Copy Markdown
Contributor

This should be feat!: rather than feat: — it's a breaking change for the changelog/version bump.

TrackRequest and the exported ProviderResponse type shipped in the previous release (go-server-sdk/ldai v0.9.0), and this repo continues that version lineage (ldai/package_info.go is at 0.9.1). This PR changes TrackRequest's signature (func(task func(c *Config) (ProviderResponse, error)) (ProviderResponse, error) → (AIMetrics, error)) and removes the exported ProviderResponse type entirely, so code compiling against v0.9.0 will fail to build.

Marking it feat!: (with a BREAKING CHANGE: note in the body) lets release-please apply the correct bump and generate an accurate changelog. The behavior change to TrackRequest (failed requests now also record duration) is worth calling out in that note too.

Comment thread ldai/tracker.go
//
// Subsequent calls re-run the task but emit only metrics not already recorded
// on this Tracker. Call CreateTracker on the AI Config to start a new run.
func (t *Tracker) TrackRequest(task func(c *Config) (AIMetrics, error)) (AIMetrics, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question on the approach here: this changes TrackRequest's signature in place ((ProviderResponse, error) → (AIMetrics, error)) and removes the exported ProviderResponse type, which silently breaks any caller compiling against v0.9.0. Rather than redefining the existing method, would you prefer to either (a) leave the existing TrackRequest/ProviderResponse in place alongside the new TrackMetricsOf (keeps it non-breaking), or (b) remove TrackRequest entirely as an explicit breaking change (clean, and pushes callers to TrackMetricsOf)? Changing it in place is the one option that breaks callers without it being obvious. What's your preference?

@mattrmc1 mattrmc1 closed this Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants