Repository navigation
Conversation
|
Docs preview: https://modelplane-docs-pr-676.vercel.app (ready once the site's Content workflow finishes) |
Collaborator
|
/review Thanks @chethanuk! |
The collector scraped Envoy's upstream retry, timeout, reset, ejection
and response-class counters but filter/modelplane dropped them because
nothing renamed them. Keep the inference gateway's copies (the
cluster-gateways also report httproute/<ns>/<deployment>/rule/0, so
require gateway=inference-gateway) with namespace and service, and map
them to modelplane_gateway_{retries,timeouts,upstream_resets,ejections,
upstream_responses}_total. Also map the gateway listeners' downstream
response classes to modelplane_gateway_requests_total.
Timeouts and resets fold several Envoy counters onto one name, so they
carry a kind label to keep the series apart.
Fixes modelplaneai#536.
Signed-off-by: ChethanUK <chethanuk@outlook.com>
chethanuk
force-pushed
the
oss/536-envoy-upstream-metrics
branch
from
November 17, 2026 04:36
ccf9a06 to
a8388e6
Compare
chethanuk
marked this pull request as ready for review
November 17, 2026 04:58
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Fixes #536.
The collector scrapes the gateway's Envoy retry, timeout, reset, ejection and response-class counters, but nothing renames them, so
filter/modelplanedrops them.This keeps the fleet gateway's copies with
namespaceandservice, the same wayenvoy_cluster_upstream_rq_activealready is, and maps them tomodelplane_gateway_retries_total,_ejections_total,_timeouts_total,_upstream_resets_totaland_upstream_responses_total. The gateway listeners' downstream response classes becomemodelplane_gateway_requests_total, with alistenerlabel so http and https don't collide.How it handles each of negz's kind e2e findings:
httproute/<ns>/<deployment>/rule/0. The new series go through_in_flight_relabels, which reads thegatewaytarget label before it clears it and keeps onlygateway=inference-gateway(collector.py:236-267). The stats listener and Istio sidecar copies are dropped too (collector.py:244,collector.py:442-443).per_try_idle_timeoutcounts attempts,timeoutcounts requests. Timeouts carrykind:request,per_tryorper_try_idle, so they're never summed by accident, and the docs say which count attempts (metrics.py:70-82,telemetry.md:95).idlereaches the caller as a 200. The model-service docs now say onlyper_try_idlerecords it (model-service.md:188-191). This replaces the Gateway retries, timeouts and ejections are collected and then dropped #536 pointer that Stop documenting metrics and merges the collector doesn't produce #639 added there.upstream_rq_xxcounts once per request by final code.upstream_responses_totalis documented that way, including the gateway's own 503 and 504 (telemetry.md:93).retry_limit_exceededmissed a 503 that used every retry, so it isn't mapped (collector.py:75-85).Resets fold two counters, so they also carry
kind.stream_resetcounts requests andconnection_closedcounts connections closed with requests in flight (telemetry.md:96). Status classes are1xxto5xx.I added table tests that run the generated relabels over fleet gateway, cluster-gateway, stats-listener and Istio series and check which are kept and with what labels (
test_collector.py:982-1233). All 289 function tests pass.I have:
RunNo nix here; ran the function tests and ruff directly.nix flake check(or./nix.sh flake check) and made sure it passes.git commit -s.