fix: scope system-instruction change detection to a single agent - #522
Open
mittalpk wants to merge 1 commit into
Open
fix: scope system-instruction change detection to a single agent#522mittalpk wants to merge 1 commit into
mittalpk wants to merge 1 commit into
Conversation
updateSystemInstructionFlags() compared consecutive LLM spans across the entire session, with no partitioning by agent. In any multi-agent / sub-agent flow, crossing an agent boundary makes the system instruction legitimately differ (each agent has its own instruction and cache context), which incorrectly tripped the "System Instruction Performance Analysis" warning and showed a diff between two unrelated agents' prompts. Group the time-sorted LLM spans by agent (event author, falling back to the span's own attrAgentName) before comparing consecutive turns, so the comparison only ever runs within a single agent's own sequence of calls. Fixes google#462
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #462 — the "System Instruction Performance Analysis" warning in the dev UI false-positives on any multi-agent / sub-agent session, even when no system instruction was actually modified.
updateSystemInstructionFlags()inchat.component.tsflattens every LLM span across a session, sorts by time, and compares each chronologically-adjacent pair for a changed system instruction — with no partitioning by agent. In a root-agent-delegates-to-sub-agent flow, the LLM calls interleave across agents (e.g.root -> sub -> sub -> root), so two calls from different agents end up compared directly. Each agent has its own system instruction and its own cache context, so this is a category error, not a real change — independent of whether the two agents happen to share a model/provider.The fix
Group the (already time-sorted) LLM spans by agent before doing the consecutive-pair comparison, instead of comparing across the whole session. The comparison logic itself is unchanged — it's the same per-pair check, just scoped to run within each agent's own sequence of calls.
Agent identity is resolved as
eventData.get(span.attrEventId)?.author, falling back to the span's ownattrAgentName(an OTel-promoted attribute that already exists onSpanbut wasn't used anywhere in the app) when the event lookup doesn't resolve — e.g. for spans without a matching event ineventData.Testing
Added 3 unit tests for
updateSystemInstructionFlags()(previously untested):attrAgentNamefallback path, when the event has no resolvableauthor.Confirmed the first and third tests fail against the pre-fix code (
Expected true to be falsy) and pass after this change. Full test suite (646 tests) andng buildboth pass.