Skip to content

fix(kiro): estimate prompt cache and report Anthropic input tokens - #571

Merged
caigee-cmd merged 2 commits into
mainfrom
fix/kiro-usage-cache
Nov 20, 2026
Merged

caigee-cmd merged 2 commits into
mainfrom
fix/kiro-usage-cache

Conversation

@caigee-cmd

Copy link
Copy Markdown
Owner

Summary

  • Gateway: Anthropic Messages streams now carry input_tokens in the final message_delta (it was missing, so clients saw 0). Anthropic input_tokens now excludes cache read/creation, matching Anthropic semantics; OpenAI prompt_tokens is unchanged. Logic lives in internal/gateway/anthropic_usage.go.
  • Kiro: local prompt-cache estimator modeled on Kiro-Go cache_tracker.go (internal/providers/kiro/prompt_cache.go): per-account SHA-256 prefix fingerprints, 5m TTL, 1024/4096 (opus) minimum, 85% read cap, first request writes only, stored only after a clean stream. The estimate is scaled to Kiro's context-share input count and emitted as cache_read_tokens / cache_write_tokens; collectOutcome passes them through for non-stream.
  • Kiro: output estimate uses Kiro-Go weights (token_estimate.go); upstream token parsing accepts more usage shapes, including nested usage maps, cache sums, and total minus output (upstream_tokens.go).

Tests

  • go test ./... and go vet ./... pass.
  • New tests: anthropic_usage_test.go, prompt_cache_test.go, token_estimate_test.go, upstream_tokens_test.go.

- gateway: Anthropic input_tokens excludes cache read/creation; stream
  message_delta now carries input_tokens.
- kiro: local per-account prompt-cache estimator (prompt_cache.go),
  Kiro-Go style token estimator (token_estimate.go), broader upstream
  usage parsing (upstream_tokens.go).
- collectOutcome passes cache read/write through.
@caigee-cmd
caigee-cmd merged commit 04aafc2 into main Nov 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant