Repository navigation
Commit 361aa3f
committed
Make tool-set churn visible between turns
The tools array is the head of the provider's cached prompt prefix, so
any change to it re-prefills the whole request. Whether that actually
happens in real sessions is unmeasured: it shows up only as a billing and
latency spike a turn later, with nothing tying it back to a mount.
Log a digest when the set changes. Hashed, not verbatim — MCP tool
descriptions are arbitrary-length server-supplied text and do not belong
in the log stream.
This replaces an earlier attempt that sorted the array by name. That was
wrong: advertisedTools already orders deterministically, so sorting added
no stability, and an alphabetical insert can land at index 0 and
invalidate more of the prefix than appending does. A gate run carrying it
measured cache-hit rate down 3-8 points across all four eval tiers.
CL-78681 parent d9ec82c commit 361aa3f
1 file changed
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
48 | 49 | | |
49 | 50 | | |
50 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
51 | 71 | | |
52 | 72 | | |
53 | 73 | | |
| |||
508 | 528 | | |
509 | 529 | | |
510 | 530 | | |
| 531 | + | |
| 532 | + | |
511 | 533 | | |
| 534 | + | |
| 535 | + | |
512 | 536 | | |
513 | 537 | | |
514 | 538 | | |
| |||
0 commit comments