Repository navigation
Conversation
|
Docs preview: https://modelplane-docs-pr-671.vercel.app (ready once the site's Content workflow finishes) |
A MetricMapping renames every series of a metric, and the collector
sums them. Under --enable-priority-scheduling SGLang publishes its
running and waiting counts once as a total, with an empty priority,
and again per priority, so modelplane_requests_running and
modelplane_requests_waiting counted each request twice. Nothing in a
mapping could pick one of the two.
A metric in a MetricMapping now takes a match: label entries, each
either a value the label must hold or absent, and only the series
matching every entry are kept. They compile to a filter processor
ahead of transform/modelplane, because OTTL in the transform processor
can't drop a datapoint. Running first means a match selects on the
name the component emitted, before a part is extracted or the metric
renamed, and it applies to the metric wherever it appears. So a second
mapping of a name a built-in already renamed still filters it, even
though its rename does nothing. An empty label counts as absent, as it
does in Prometheus, whose receiver drops it anyway. Dollar signs in
quoted OTTL strings are now doubled, since the collector expands ${...}
in its config before OTTL parses it.
The built-in SGLang mapping keeps only the series without a priority.
It doesn't match tp_rank 0 for --enable-metrics-for-all-schedulers:
SGLang labels by global tp_rank, and under DP attention each group's
first rank publishes its own count, so tp_rank 0 alone would undercount.
The docs show a user how to match tp_rank on both counts where DP
attention isn't in use.
Fixes modelplaneai#549.
Signed-off-by: ChethanUK <chethanuk@outlook.com>
55cc494 to
8b96a8c
Compare
|
/review |
There was a problem hiding this comment.
I think the API for match needs another pass before this is ready, though compiling it to a filter that runs ahead of the transform looks right.
The open questions are about where match sits and how its entries combine. It's a field on one rename entry, but it drops series for every mapping of that from. Each entry is either a boolean or one value, with no room for other operators. Answers to those could change the schema. No maintainer has signed off on the plan in #549 yet. Settling the schema first saves polishing code that may change.
The PR says it fixes #549, but its tp_rank recipe undercounts under DP attention, so that half of the issue isn't fixed.
Reviewed by modelplane-review 22e194c. Reply, or react 👍 or 👎, on each comment to help tune it.
| putting an engine's own vocabulary into | ||
| Modelplane's. A value with no entry here is left | ||
| as the component wrote it. | ||
| match: |
There was a problem hiding this comment.
Is match in the right place? As a field on one from→to entry it reads as "which series this rename takes", but conditions() compiles it to a drop keyed only on from. So it also filters sibling entries, other mappings and the built-in. Both this description and the spec-level metrics one have to explain that, and the tp_rank recipe in the docs has the user write a rename that "does nothing" just to hold a match.
Could a spec-level list of series to keep, keyed by metric name, describe what it does more directly? The cost is a second place to name a metric.
| out: list[str] = [] | ||
| for mapping in mappings: | ||
| for m in mapping.spec.metrics: | ||
| for entry in m.match or []: |
There was a problem hiding this comment.
Is "a mapping can only narrow" the semantics we want? Each entry is its own drop condition, so matches on one from across all mappings intersect. A user mapping of sglang:num_running_reqs with priority: value "1" meets the built-in's priority absent and doesn't keep any series, and nothing reports it. design/telemetry.md names "nothing reports that a mapping matched no series" as a failure the kind exists to avoid.
Could a user's match for a from replace the built-in's rather than add to it? Or could the XRD at least say that matches on one metric combine and can only narrow?
| series without the label doesn't match. | ||
| absent: | ||
| type: boolean | ||
| enum: [true] |
There was a problem hiding this comment.
Would an operator discriminator serve better than value versus absent: true? A boolean with enum: [true] can't grow additively into "not this value", "one of these" or "present". Each would need a new sibling field and a wider CEL rule. The ModelDeployment tolerations already use operator: [Exists, Equal] (apis/modeldeployments/definition.yaml:328), and users know the same structure from matchExpressions:
match:
- label: priority
operator: DoesNotExistThe downside is one more field on the common case.
| maxLength: 63 | ||
| pattern: '^[a-zA-Z_][a-zA-Z0-9_]*$' | ||
| description: The label to match on. | ||
| value: |
There was a problem hiding this comment.
Won't value: "" drop every series of the metric? The Prometheus receiver drops an empty label, and the attribute arrives as nil. In OTTL nil != "" is true. The condition then drops series without the label as well as those with any other value. A run against otelcol-contrib 0.161.0 confirmed that my_engine_x{kind=""} and {kind="a"} both disappear. It's an easy thing to write, since PromQL selects a missing label with ="".
Could we add minLength: 1 here, or compile value: "" the same way as absent?
| under, so a second mapping's rename of that name does nothing | ||
| and the built-in stands. Its `match` still applies, because | ||
| matching runs before any rename and drops the series it doesn't | ||
| keep wherever the metric appears. Select on the `modelplane_` |
There was a problem hiding this comment.
Won't a match on a modelplane_ from silently do nothing? This sentence points users at the modelplane_ name, but filter/match runs before transform/modelplane, while the series still has the engine's name. So metric.name == "modelplane_requests_running" never holds and the filter keeps every series.
Could a CEL rule reject match on a modelplane_ from, or could the match description say it only applies to the emitted name?
| once in total and once per priority. The built-in mapping keeps only the total. With | ||
| `--enable-metrics-for-all-schedulers`, it publishes them once per tensor-parallel rank, | ||
| and a sum over `modelplane_requests_running` or `modelplane_requests_waiting` counts each | ||
| request once per rank. To keep one rank's series, give a mapping of |
There was a problem hiding this comment.
Should #549 get a follow-up rather than close with this PR? This tp_rank: "0" recipe for --enable-metrics-for-all-schedulers undercounts with DP attention, and the caveat only comes after the example. If the right series under DP attention is "each group's first rank", can match express that at all?
Description of your changes
Fixes #549.
Under
--enable-priority-schedulingSGLang publishes its running and waiting counts once as a total, with an empty priority, and again per priority. A MetricMapping renames every series of a metric, somodelplane_requests_runningandmodelplane_requests_waitingcounted each request twice.This adds
matchto a MetricMapping metric. Each entry names a label and either a value it must hold orabsent: true, and only series matching every entry are kept. The entries compile to afilter/matchprocessor that runs beforetransform/modelplane, because the transform processor can't drop a datapoint. Since it runs first, a match selects on the name the component emitted, before a part is extracted or the metric renamed. An empty label counts as absent, as in Prometheus. Match and label values are quoted with every$doubled, because the collector expands${...}anywhere in its config before OTTL sees it.The built-in SGLang mapping now keeps only the series without a priority. The issue also suggested matching
tp_rank: "0"for--enable-metrics-for-all-schedulers. I left that out: SGLang labels by global tp_rank, and under DP attention each group's first rank publishes its own count, so tp_rank 0 alone would undercount. The telemetry docs show how to match tp_rank where DP attention isn't in use.I ran otelcol-contrib 0.161.0 against a static SGLang-style
/metricswith two ranks, each publishing a total of 5 and priorities of 2 and 3. Before the changemodelplane_requests_runningkept the totals and the per-priority series. With the rendered config only the two totals (5 and 5) remain.otelcol validatepasses on the rendered config with and without mappings.I have:
RunNo nix here; ran the function tests, ruff and the docs validator directly.nix flake check(or./nix.sh flake check) and made sure it passes.git commit -s.