Description
Cache policies support memory and Redis backends, TTL, scope by model or API key, a cache-status response header, and tokens-saved on the usage record. But there is no Prometheus counter for cache hits, misses, or evictions in the metrics registry — saved tokens are recorded on the usage event, so hit rate can be reconstructed in the analytics UI, but it cannot be scraped, alerted on, or graphed alongside the other aisix_* series.
Requested: hit / miss / eviction counters in the Prometheus registry, labelled consistently with the existing series (policy, model, backend).
Scope note: on-demand purge is deliberately not part of this issue — that is tracked in #89.
Why
Cache hit rate is the number that justifies the cache in a cost review, and it belongs in the same scrape and alerting path as every other metric rather than only in an analytics view. Operators cannot alert on "hit rate collapsed after a prompt-template change" today.
Priority
Medium.
Prior art
Hit/miss telemetry is standard across peers; LiteLLM ships dedicated counters.
Surveyed 2026-07-15; every claim rests on a fetched docs/source page.
Description
Cache policies support memory and Redis backends, TTL, scope by model or API key, a
cache-statusresponse header, and tokens-saved on the usage record. But there is no Prometheus counter for cache hits, misses, or evictions in the metrics registry — saved tokens are recorded on the usage event, so hit rate can be reconstructed in the analytics UI, but it cannot be scraped, alerted on, or graphed alongside the otheraisix_*series.Requested: hit / miss / eviction counters in the Prometheus registry, labelled consistently with the existing series (policy, model, backend).
Scope note: on-demand purge is deliberately not part of this issue — that is tracked in #89.
Why
Cache hit rate is the number that justifies the cache in a cost review, and it belongs in the same scrape and alerting path as every other metric rather than only in an analytics view. Operators cannot alert on "hit rate collapsed after a prompt-template change" today.
Priority
Medium.
Prior art
litellm_cache_hits_metric/_misses_metric(in code, not yet in docs)cache_statuslabel onai_llm_requests_totalcacheStatuslabel on/metricsbifrost_cache_hits_total(hits only)Hit/miss telemetry is standard across peers; LiteLLM ships dedicated counters.
Surveyed 2026-07-15; every claim rests on a fetched docs/source page.